Subject: [dtn-interest] more on reactive fragmentation

From: "Burleigh, Scott C (312G)" <[email-masked]>
Date: Fri, 2 May 2014 22:45:49 +0000

Will, I well remember reading about how useful you found reactive fragmentation in the UK-DMC experiments, so I can understand why you now feel it is a requirement.  My thinking has developed in the opposite direction: I was a big fan of reactive fragmentation originally, but now I believe it's the wrong answer.  Here's my reasoning:
*       The problem we're trying to solve is the waste of bandwidth that is caused by needing to retransmit a large amount of data that has already been transmitted, simply because connectivity was lost part way through transmission of a single large indivisible protocol data unit (bundle).
*       Reactive fragmentation solves this problem by dynamically dividing the original large bundle at the time of connectivity loss: most or all of the portion that has already been transmitted becomes a new (fragmentary) bundle, and the residual portion that hasn't yet been transmitted likewise becomes a fragmentary bundle.
*       But reactive fragmentation is possible only when the receiving entity can somehow notify the sender of how much of the original bundle was received prior to the end of connectivity.  (Otherwise the sender has no idea of how much data it need not retransmit.)
*       So there has to be a constant stream of "I got this much" messages returned from the receiver to the sender.  Those messages constitute bandwidth consumption that to some extent offsets what you save by reactively fragmenting.  (And of course the handling of these messages also consumes processing cycles at both the sender and the receiver.)
*       The benefit (bandwidth savings) that you get from reactive fragmentation varies directly with the size of your bundles and the frequency of contact truncation.  If contact truncations are rare or if your bundles are small then the cost of enabling reactive fragmentation outweighs the benefit.
*       From the size of the "I got this much" messages and the rate at which they are returned, we can compute the minimum bandwidth savings that must result from reactive fragmentation in order for it to be cost justified.  Divide that by the estimated frequency of contact truncation and you get the minimum mean bundle size at which reactive fragmentation makes sense.  Now, if you proactively fragment bundles to never exceed that size, then reactive fragmentation is unnecessary.

That's why ION implements "just in time" proactive fragmentation, at the moment of transmission, rather than reactive transmission.  (I wrote a little about this a few days ago.)

When BP is running over TCP/IP in the terrestrial Internet, the probability of unanticipated contact truncation is extremely low, so mostly there's no point in bothering with proactive fragmentation.

When BP is running in a space link environment, it arguably ought to be running over LTP rather than TCP/IP.  In ION, LTP is driven by a plan of scheduled contacts, so again the probability of unanticipated contact truncation is low -- contact truncation happens all the time, but it's almost always anticipated.  Moreover, LTP segments outbound bundles anyway; the segments are usually pretty small, so if one gets lost due to contact truncation there's negligible waste of bandwidth.

(As an aside, there is an open work item to modify ION's LTP to proactively truncate an outbound bundle that is too large to fit within the expected remaining seconds of the current contact.  This enables part of the bundle to be transmitted in the current contact, fully utilizing the contact, and the rest may be transmitted on a subsequent contact, possibly with a different network neighbor.  And again you don't miss reactive fragmentation.)

In your UK-DMC experiments you were running BP in a space link environment over TCP/IP, and I'll admit that this is a configuration in which reactive fragmentation is a bacon-saving feature: you were sending big bundles and you had several unanticipated contact truncations.  But I'll argue that this is a configuration that we should stay away from, for this reason among others.

Scott

-----Original Message-----
From: Stephen Farrell [mailto:[email-masked]]
Sent: Friday, May 02, 2014 1:59 PM
To: Ivancic, William D. (GRC-RHN0); Burleigh, Scott C (312G); [email-masked]
Subject: Re: [dtn-interest] DTN BoF Proposal for IETF90

On 02/05/14 21:24, Ivancic, William D. (GRC-RHN0) wrote:
> Some really nice slides from Scott- seriously.  I found these while
> preparing for a talk. These do a nice job of highlight the difference
> between CFDP and DTN.
>
> http://chapters.comsoc.org/comsig/Slides/Jan2004_BurleighS_IPN_DTN.pdf
>
>  Interesting that reactive fragmentation is called out.  I was not a
> big fan originally, but not I feel it is a requirement.
>
> Has anyone done reactive fragmentation over LTP?  I was the
> beneficiary over TCP.

Not with our LTP implementation. LTPlib and the LTPCL we implemented would handle the disruption the LTP session and not tell the BPA. Not sure about ION. Handing the problem back to the bundle layer at some point would be an idea but I've not looked to see if it'd be easy or hard to code up.

> Stephen, was your experience with reactive fragmentation over TCP or
> some other convergence layer?

The N4C stuff was all TCP.

Cheers,
S.

---

From: Stephen Farrell <[email-masked]>
Date: Fri, 02 May 2014 23:54:01 +0100

An aside on the aside...

On 02/05/14 23:45, Burleigh, Scott C (312G) wrote:
> 
> (As an aside, there is an open work item to modify ION's LTP to
> proactively truncate an outbound bundle that is too large to fit
> within the expected remaining seconds of the current contact.  This
> enables part of the bundle to be transmitted in the current contact,
> fully utilizing the contact, and the rest may be transmitted on a
> subsequent contact, possibly with a different network neighbor.  And
> again you don't miss reactive fragmentation.)

LTPlib does that already and it works fine.

The benefit with reactively fragementing the bundle is that the
re-transmitted fragment(s) can take different routes. Remember in
our N4C case we really did have 55 fragments in total that took
a whole pile of routes.

S.

---

From: "Burleigh, Scott C (312G)" <[email-masked]>
Date: Fri, 2 May 2014 22:56:06 +0000

Sure.  But is there some reason that just-in-time proactive fragmentation can't achieve the same result?

Scott

-----Original Message-----
From: Stephen Farrell [mailto:[email-masked]] 
Sent: Friday, May 02, 2014 3:54 PM
To: Burleigh, Scott C (312G); [email-masked]
Subject: Re: [dtn-interest] more on reactive fragmentation

An aside on the aside...

The benefit with reactively fragementing the bundle is that the re-transmitted fragment(s) can take different routes. Remember in our N4C case we really did have 55 fragments in total that took a whole pile of routes.

S.

---

From: Elwyn Davies <[email-masked]>
Date: Sat, 03 May 2014 00:24:22 +0100

I suspect, but cannot immediately prove, that on a multiple hop DTN path
with wi-fi and highly independent node behaviour estimating the correct
chunk size for proactive fragmentation is difficult.  You would also
probably have to ensure that all your nodes were set up with the same
value or you could get some unpleasant refragmentation.

Further when using TCP on terrestrial links most of the overhead is
already built into TCP so you don't have a lot of extra overhead (the
underlying transport tells you how much has been transmitted/received).
This also gives you reliability so that FEC is not needed in most cases.
Finally, the proactive calculation doesn't mention the extra bundle
headers that are transmitted.   I am not sure how much of this is
factored iinto your actual calculation. 

Accordingly I think the trade off with the TCP CL is rather different
from the LTP/unidirectional links case.

Regards,
Elwyn
On Fri, 2014-05-02 at 22:56 +0000, Burleigh, Scott C (312G) wrote:
> _______________________________________________
> dtn-interest mailing list
> [email-masked]
> https://www.irtf.org/mailman/listinfo/dtn-interest

---

From: "Burleigh, Scott C (312G)" <[email-masked]>
Date: Sat, 3 May 2014 00:49:04 +0000

Elwyn, certainly I would not advocate proactive fragmentation anywhere you don't feel strongly that you need it; as you say, fragmentation of any type always imposes some cost (such as the extra bundle headers for all those additional bundles).  I would think that mass proactive fragmentation at the source -- as opposed to "just in time" proactive fragmentation as needed -- would always be a bad idea.

Suppose your outbound link is sending at 8 Mbps and your bundle is 4 Megabytes, and transmission of that bundle fails due to unanticipated contact truncation; okay, you have wasted 4 seconds of contact time (the 4 seconds it will take to retransmit the bundle when contact resumes).  Do you really care about 4 seconds of wasted contact time, once every 5 days?  Or once a day?  Or once an hour (which is still 99.89 %bandwidth utilization)?

On the other hand, if you're sending a 1GB image -- and you're not using CFDP (which does its own segmentation and reassembly) to send the image as a file -- then an unanticipated contact truncation costs you 1000 seconds of contact time.  If this happens once an hour then your bandwidth utilization has dropped to 97.2%, which is pretty bad; reactive fragmentation , or proactive fragmentation into smaller chunks, might be a good idea.  But if it only happens once a day, you're still getting 99.998% bandwidth utilization.  Is reactive fragmentation really worthwhile?

My point, I guess, is that this is an engineering question, not an ideology.

One point on TCP transmission: I believe the return code from a TCP socket write() tells you how much was sent, not necessarily how much was received.  I think that's why it was felt that the TCPCL specification required extra reporting from the receiving node.  But someone who has worked on that spec ought to weigh in at this point.

Scott 

-----Original Message-----
From: Elwyn Davies [mailto:[email-masked]] 
Sent: Friday, May 02, 2014 4:24 PM
To: Burleigh, Scott C (312G)
Cc: [email-masked]
Subject: Re: [dtn-interest] more on reactive fragmentation

I suspect, but cannot immediately prove, that on a multiple hop DTN path with wi-fi and highly independent node behaviour estimating the correct chunk size for proactive fragmentation is difficult.  You would also probably have to ensure that all your nodes were set up with the same value or you could get some unpleasant refragmentation.

Further when using TCP on terrestrial links most of the overhead is already built into TCP so you don't have a lot of extra overhead (the underlying transport tells you how much has been transmitted/received).

Accordingly I think the trade off with the TCP CL is rather different from the LTP/unidirectional links case.

Regards,
Elwyn

---

From: Vint Cerf <[email-masked]>
Date: Sat, 3 May 2014 02:10:45 -0400

see also
ftp://192.6.29.21/gatekeeper/pub/DEC/WRL/research-reports/WRL-TR-87.3.pdf

"fragmentation considered harmful" by Jeff Mogul and Christopher Kent

vint

On Fri, May 2, 2014 at 6:45 PM, Burleigh, Scott C (312G) <
[email-masked]> wrote:

>  Will, I well remember reading about how useful you found reactive
> fragmentation in the UK-DMC experiments, so I can understand why you now
> feel it is a requirement.  My thinking has developed in the opposite
> direction: I was a big fan of reactive fragmentation originally, but now I
> believe it's the wrong answer.  Here's my reasoning:
>
>    - The problem we're trying to solve is the waste of bandwidth that is
>    caused by needing to retransmit a large amount of data that has already
>    been transmitted, simply because connectivity was lost part way through
>    transmission of a single large indivisible protocol data unit (bundle).
>    - Reactive fragmentation solves this problem by dynamically dividing
>    the original large bundle at the time of connectivity loss: most or all of
>    the portion that has already been transmitted becomes a new (fragmentary)
>    bundle, and the residual portion that hasn't yet been transmitted likewise
>    becomes a fragmentary bundle.
>    - But reactive fragmentation is possible only when the receiving
>    entity can somehow notify the sender of how much of the original bundle was
>    received prior to the end of connectivity.  (Otherwise the sender has no
>    idea of how much data it need not retransmit.)
>    - So there has to be a constant stream of "I got this much" messages
>    returned from the receiver to the sender.  Those messages constitute
>    bandwidth consumption that to some extent offsets what you save by
>    reactively fragmenting.  (And of course the handling of these messages also
>    consumes processing cycles at both the sender and the receiver.)
>    - The benefit (bandwidth savings) that you get from reactive
>    fragmentation varies directly with the size of your bundles and the
>    frequency of contact truncation.  If contact truncations are rare or if
>    your bundles are small then the cost of enabling reactive fragmentation
>    outweighs the benefit.
>    - From the size of the "I got this much" messages and the rate at
>    which they are returned, we can compute the minimum bandwidth savings that
>    must result from reactive fragmentation in order for it to be cost
>    justified.  Divide that by the estimated frequency of contact truncation
>    and you get the minimum mean bundle size at which reactive fragmentation
>    makes sense.  Now, if you proactively fragment bundles to never exceed that
>    size, then reactive fragmentation is unnecessary.
>
>
> That's why ION implements "just in time" proactive fragmentation, at the
> moment of transmission, rather than reactive transmission.  (I wrote a
> little about this a few days ago.)
>
> When BP is running over TCP/IP in the terrestrial Internet, the
> probability of unanticipated contact truncation is extremely low, so mostly
> there's no point in bothering with proactive fragmentation.
>
> When BP is running in a space link environment, it arguably ought to be
> running over LTP rather than TCP/IP.  In ION, LTP is driven by a plan of
> scheduled contacts, so again the probability of unanticipated contact
> truncation is low -- contact truncation happens all the time, but it's
> almost always anticipated.  Moreover, LTP segments outbound bundles anyway;
> the segments are usually pretty small, so if one gets lost due to contact
> truncation there's negligible waste of bandwidth.
>
> proactively truncate an outbound bundle that is too large to fit within the
> expected remaining seconds of the current contact.  This enables part of
> the bundle to be transmitted in the current contact, fully utilizing the
> contact, and the rest may be transmitted on a subsequent contact, possibly
> with a different network neighbor.  And again you don't miss reactive
> fragmentation.)
>
> In your UK-DMC experiments you were running BP in a space link environment
> over TCP/IP, and I'll admit that this is a configuration in which reactive
> fragmentation is a bacon-saving feature: you were sending big bundles and
> you had several unanticipated contact truncations.  But I'll argue that
> this is a configuration that we should stay away from, for this reason
> among others.
>
> Scott
>
> From: Stephen Farrell [mailto:[email-masked]<[email-masked]>]
>
> Not with our LTP implementation. LTPlib and the LTPCL we implemented would
> handle the disruption the LTP session and not tell the BPA. Not sure about
> ION. Handing the problem back to the bundle layer at some point would be an
> idea but I've not looked to see if it'd be easy or hard to code up.
>

---

From: Kevin Fall <[email-masked]>
Date: Sat, 3 May 2014 08:20:05 -0400

Right.  IP fragmentation has the property that if you lose any of the
fragments [or parts of fragments] you lose the entire datagram.
Reactive fragmentation was, in part, a mitigation strategy for the
issues raised in that paper.  Proactive fragmentation is perhaps more
similar to IP-style fragmentation, but whereas IP fragmentation is
more about adapting the abstract IP datagram for carriage in a lower
layer frame of a certain size, DTN proactive fragmentation was
conceived to be more about filling up the (temporarily available)
bandwidth-delay product opportunities available from scheduled
connectivity.

- Kevin

On Sat, May 3, 2014 at 2:10 AM, Vint Cerf <[email-masked]> wrote:
> On Fri, May 2, 2014 at 6:45 PM, Burleigh, Scott C (312G)
> <[email-masked]> wrote:
>>
>> The problem we're trying to solve is the waste of bandwidth that is caused
>> by needing to retransmit a large amount of data that has already been
>> transmitted, simply because connectivity was lost part way through
>> Reactive fragmentation solves this problem by dynamically dividing the
>> original large bundle at the time of connectivity loss: most or all of the
>> portion that has already been transmitted becomes a new (fragmentary)
>> But reactive fragmentation is possible only when the receiving entity can
>> somehow notify the sender of how much of the original bundle was received
>> prior to the end of connectivity.  (Otherwise the sender has no idea of how
>> much data it need not retransmit.)
>> So there has to be a constant stream of "I got this much" messages
>> The benefit (bandwidth savings) that you get from reactive fragmentation
>> varies directly with the size of your bundles and the frequency of contact
>> truncation.  If contact truncations are rare or if your bundles are small
>> then the cost of enabling reactive fragmentation outweighs the benefit.
>> From the size of the "I got this much" messages and the rate at which they
>> are returned, we can compute the minimum bandwidth savings that must result
>> from reactive fragmentation in order for it to be cost justified.  Divide
>> that by the estimated frequency of contact truncation and you get the
>> minimum mean bundle size at which reactive fragmentation makes sense.  Now,
>> if you proactively fragment bundles to never exceed that size, then reactive
>> fragmentation is unnecessary.
>>
>>
>> expected remaining seconds of the current contact.  This enables part of the
>> bundle to be transmitted in the current contact, fully utilizing the
>> you had several unanticipated contact truncations.  But I'll argue that this
>> is a configuration that we should stay away from, for this reason among
>> others.
>>
>> Scott
>>

---

From: <[email-masked]>
Date: Sat, 3 May 2014 12:43:00 +0000

Proactive fragmentation is more akin to correct MTU use (provided by pre-existing knowledge or MTU discovery).

MTU tells you how big the opportunity to transmit it, you fill it up.

Lloyd Wood
http://sat-net.com/L.Wood/dtn
________________________________________
From: dtn-interest <[email-masked]> on behalf of Kevin Fall <[email-masked]>
Sent: Saturday, 3 May 2014 10:20 PM
To: Vint Cerf
Cc: [email-masked]
Subject: Re: [dtn-interest] more on reactive fragmentation

- Kevin

---

From: "Ivancic, William D. (GRC-RHN0)" <[email-masked]>
Date: Sun, 4 May 2014 12:34:08 +0000

>>> fragmentation in the UK-DMC experiments, ......
>>>

Scott.  The reactive fragmentation occurred over the terrestrial network
between, I think Hawaii (but it might have been Alaska) and Glenn Research
Center.  Saratoga was used on the Space/Ground link.

In Stephen's case reactive fragmentation also occurred on terrestrial
links.

I don't know what was flaky with our terrestrial links.  We did not have
the resources to repeat the experiments or investigate properly.  It may
have been related to VPN rekeying or something like that.

Will

---

From: Joerg Ott <[email-masked]>
Date: Sun, 04 May 2014 18:26:03 +0300

Hi Scott,

> Elwyn, certainly I would not advocate proactive fragmentation anywhere you don't feel strongly that you need it; as you say, fragmentation of any type always imposes some cost (such as the extra bu…

you would try to avoid this at the source, indeed, unless you want to
have individually authenticated fragments.  Then, you will have to
something at the source (which could be done at the application layer
by sending smaller bundles in the first place).

> On the other hand, if you're sending a 1GB image -- and you're not using CFDP (which does its own segmentation and reassembly) to send the image as a file -- then an unanticipated contact truncatio…

This is indeed a good question.  We came to the conclusion that it _may_
be as long as you don't fragment arbitrarily but still adhere to some
boundaries (even if this comes at some minor loss).  Our interest came
from opportunistic contacts (e.g., via WLAN ad-hoc links or access
points) in which cases you have no up-front knowledge if a contact
is going to last 10s or 1min or 1h.  Reactive fragmentation helps
making short contacts useful that might otherwise be totally wasted.

> One point on TCP transmission: I believe the return code from a TCP socket write() tells you how much was sent, not necessarily how much was received.  I think that's why it was felt that the TCPCL…

This is correct.  And knowing what was received by the receiver
kernel (and possibly ACKed) isn't enough, because you don't know
if the receiver side DTN daemon actually read and processed the
those bytes.  There is no way around the TCPCL or other higher layer
acknowledgements.

Jörg

---

From: "Burleigh, Scott C (312G)" <[email-masked]>
Date: Sun, 4 May 2014 15:46:16 +0000

Jörg, I think even if you want your fragments individually authenticated you don't have to do fragmentation at the source.  The BAB mechanism is applied on a hop-by-hop basis, at the moment the sender transmits the bundle.  So if you fragment just-in-time, at any point along the end-to-end path, you have got a fragmentary bundle that can acquire an attached BAB immediately before transmission.  ION does this, for example.

You raise a good point about utilization of contacts of unknown but possibly very short duration, but again I think just-in-time proactive fragmentation gives you the same advantage at lower cost.

Suppose your outbound data rate on an opportunistic link is 8 Mbps (just for convenience) and you know the link could be lost at any moment.  If you proactively fragment all your bundles to a maximum size of 1 MB (at the moment of transmission) then in the worst case you lose 1 second's worth of the opportunistic contact: if the contact length is 10s, you can get at least 9 fragmentary bundles across it without relying on TCPCL acknowledgments at all.  And because the fragmentation was not reactive, you can easily attach authentication blocks to each fragment.

Scott

-----Original Message-----
From: Joerg Ott [mailto:[email-masked]] 
Sent: Sunday, May 04, 2014 8:26 AM
To: Burleigh, Scott C (312G); [email-masked]
Subject: Re: [dtn-interest] more on reactive fragmentation

Hi Scott,

> Elwyn, certainly I would not advocate proactive fragmentation anywhere you don't feel strongly that you need it; as you say, fragmentation of any type always imposes some cost (such as the extra bu…

you would try to avoid this at the source, indeed, unless you want to have individually authenticated fragments.  Then, you will have to something at the source (which could be done at the application layer by sending smaller bundles in the first place).

> On the other hand, if you're sending a 1GB image -- and you're not using CFDP (which does its own segmentation and reassembly) to send the image as a file -- then an unanticipated contact truncatio…

This is indeed a good question.  We came to the conclusion that it _may_ be as long as you don't fragment arbitrarily but still adhere to some boundaries (even if this comes at some minor loss).  Our interest came from opportunistic contacts (e.g., via WLAN ad-hoc links or access
points) in which cases you have no up-front knowledge if a contact is going to last 10s or 1min or 1h.  Reactive fragmentation helps making short contacts useful that might otherwise be totally wasted.

> One point on TCP transmission: I believe the return code from a TCP socket write() tells you how much was sent, not necessarily how much was received.  I think that's why it was felt that the TCPCL…

This is correct.  And knowing what was received by the receiver kernel (and possibly ACKed) isn't enough, because you don't know if the receiver side DTN daemon actually read and processed the those bytes.  There is no way around the TCPCL or other higher layer acknowledgements.

Jörg