Subject: [dtn-interest] Flow Control Testing
From: Gombo Bayartsog <[email-masked]>
To: [email-masked]
Date: Wed, 22 Sep 2004 13:21:42 -0400
List-Id: Delay Tolerant Networking Interest List <dtn-interest.mailman.dtnrg.org>

I am a student at Yorktown High School and I was wondering if I would
be able to do some testing for IPN and use it as my Science Fair
Project. I was wondering if I could do some testing involving Flow
Control. Where would I get some information about flow control and
what kind of testing there is involving flow control

From: "Daniel Caughran" <[email-masked]>
To: <[email-masked]>
Date: Tue, 21 Sep 2004 21:29:23 -0400
Importance: Normal
Subject: [dtn-interest] Science Project
Sender: [email-masked]

I am working on a science project involving the IPN protocols. Can anybody
tell me what they simulate and what data it can give me. Also it would be
appreciated if anyone had an idea of a variable to test. Please email me at
[email-masked].

-Daniel Caughran

From: "Robert C. Durst" <[email-masked]>
To: "'Daniel Caughran'" <[email-masked]>
Cc: <[email-masked]>
Date: Mon, 20 Sep 2004 13:00:27 -0400
Organization: The MITRE Corporation
Subject: [dtn-interest] RE: [ipnsig-discuss] Science Project
Sender: [email-masked]

Daniel,

Most of the Interplanetary Internet work is now being done under the
auspices of the Internet Research Task Force's (www.irtf.org) Delay Tolerant
Networking Research Group (http://www.irtf.org/charters/dtnrg.html,
www.dtnrg.org).  The IRTF is the research arm of the Internet Society,
whereas the Internet Engineering Task Force (www.ietf.org) is the
engineering/standardization arm.  The former IRTF Interplanetary Internet
Research Group (http://www.irtf.org/historical/ipnrg.html) has essentially
been retired under the IRTF, and the work transferred to the DTNRG.  At the
DTNRG, you can find information about many ongoing applications of the
architecture and protocols, along with a document describing an
interplanetary transfer example.  The DTNRG site normally has instructions
to access the current code base, which all are free to download, but due to
a recent system compromise, this feature is not active right now.  We hope
that will be fixed soon.  There's a very old version in a tarball on the web
site, but you'd do better waiting for the CVS server to be made available
again.  

In the mean time, there are a few documents that you should consider looking
at:  The DTN Architecture document, the Bundle protocol specification, and
the Interplanetary Internet example bundle transfer, all available from the
dtnrg site.  Mr. Scott Burleigh, of the Jet Propulsion Laboratory
([email-masked]) is implementing a version of the bundle
protocol specifically for use on interplanetary spacecraft.  There is a
companion protocol to provide reliability over very long-distances (e.g.,
interplanetary), called the Licklider Transport Protocol, or LTP.  This
protocol sits underneath the bundle protocol when it operates over
interplanetary distances to provide the reliability that the bundle protocol
expects.  The specification for LTP is also available from the dtnrg site.  

Good luck with your project!  Active discussion about the DTN architecture
and protocols occurs on the dtn-interest mailing list copied above.  We
welcome your assistance and look forward to integrating your contribution of
time, talent, and equipment to the DTN work and to the IPN in particular.

Best regards,
Bob Durst
Senior Principal Networking and Communications Engineer
The MITRE Corporation
7515 Colshire Drive, M/S H440
McLean, VA 22102
[email-masked]

-----Original Message-----
From: [email-masked]
[mailto:[email-masked]] On Behalf Of Daniel
Caughran
Sent: Saturday, September 18, 2004 4:17 PM
To: [email-masked]
Subject: [ipnsig-discuss] Science Project

My name is Daniel Caughran; I am a high school student at Yorktown High
School in Arlington, VA. I was advised by my computer science to do a
science project involving the Inter Planetary Network and contribute to the
ongoing research. I am working with a team and have multiple computers that
can be put in multiple locations. Could anyone help me find a beta version
of the protocols or help me setting up the software? Please email me at
[email-masked].

_______________________________________________
Ipnsig-discuss mailing list
[email-masked]
http://mailman.isoc.org/mailman/listinfo/ipnsig-discuss

Date: Tue, 14 Sep 2004 09:19:29 +0200
To: [email-masked]
From: "Magnus Lindholm" <[email-masked]>
Subject: [dtn-interest] Bundles source code availability
Sender: [email-masked]

Is there any news on the availability of the lates bundles source code? 
The CVS repository at code.dtnrg.org is still down. Is there an alternative
CVS server, available to the public?

Regards

Magnus Lindholm
Developer Engineer
SNC project

Date: Wed, 08 Sep 2004 08:36:42 -0700
To: [email-masked]
From: Scott Burleigh <[email-masked]>
Subject: [dtn-interest] small protocol tweaks
Sender: [email-masked]

Now that the latest Internet Draft on the Bundling protocol has been 
posted, I'd like to propose a couple of minor revisions before we start it 
on the RFC track.  They mainly have to do with the Class Of Service flags 
in the primary header.

First I would like to reserve only two -- rather than three -- bits of the 
Class Of Service Flags byte for priority.  We currently only have three 
levels of priority identified in the service interface (bulk, normal, 
expedited), and these three mutually exclusive values could be encoded in 
two bits.

Second, I'd like to add one more level of priority to the service 
interface: emergency.  A bundle with priority set to "emergency" would be 
forwarded in preference even to one that was "expedited", but in addition 
it could displace previously accepted bundles that are currently queued for 
forwarding, even if custody had been accepted for those bundles.  That is, 
you would never discard an emergency bundle due to congestion (not enough 
unused buffer space); you would instead free up sufficient buffer space by 
discarding bundles, exactly as if their TTLs had been reduced to 
zero.  Obviously abuse of this class of service would have to be controlled 
somehow, but we've got that problem with the other classes of service as 
well.  At some point we need to confront that issue but in the meantime I 
think it's important to provide this capability, if only as a potential 
mechanism for addressing congestion issues by changing routing tables.

This would leave us with four levels of priority, which could still be 
encoded in two bits of the Class Of Service Flags byte as follows:
	00	-	bulk
	01	-	normal
	10	-	expedited
	11	-	emergency

I would propose to use the newly freed-up bit of the Class Of Service Flags 
byte as an additional (fifth) Delivery Record Request Flag: request 
reporting of deletion.  We have status flags in the Status Report for "TTL 
expired" and "bundle found to be inauthentic", and we've got a Time Of 
Deletion in the Status Report, but we don't have any general way of 
requesting the return of this information.  Currently the only way we can 
trigger return of this stuff is by requesting custody transfer reporting: 
refusal to take custody for some reason would trigger this kind of report, 
but that only applies to custodial bundles.  If we want to provide a 
general bundle tracing capability then I think we have to offer a way to 
request reporting on bundle deletion as well as bundle reception, 
forwarding, and final delivery.

Finally, we don't currently have any way to say in the Status Report that a 
bundle (custodial or otherwise) was deleted for some reason other than TTL 
expiration and failure to pass authentication check.  I suggest that we use 
bits 1-3 [the bits masked by 01110000] of the Status Flags byte of the 
Status Report as a deletion reason code, reassigning the current TTL 
expiration and inauthenticity flags and the unused 0x40 bit.  The values of 
this deletion reason code could be aligned with the custodial signal reason 
codes (which we could beef up a little at the same time):

	000	-	bundle not deleted
	001	-	TTL expired
	002	-	unused
	003	-	unused
	004	-	depleted storage
	005	-	no known route to destination from here
	006	-	no timely contact with next node on route
	007	-	bundle was found to be inauthentic [add this to the custodial signal 
reason codes as well]

I don't think this would do too much damage to existing implementations, 
and I think it would give us substantially more useful bundle tracing 
capabilities.  Any thoughts?

Scott

Date: Wed, 08 Sep 2004 15:28:29 +0100
From: Stephen Farrell <[email-masked]>
To: Scott Burleigh <[email-masked]>
Cc: [email-masked]
Subject: Re: [dtn-interest] extension headers in bundle protocol
Sender: [email-masked]

Scott Burleigh wrote:
> The latest edition of the Bundling spec makes some
> provision for this, I think, in the Bundle Authentication header (and 
> there's also a separate Payload Security header for assuring the 
> authenticity and/or integrity of the payload).  Would you want to look 
> that stuff over and comment?

That's me told then:-)

Sure I'll take a peek when I get a chance and get back then.

Stephen.

Date: Tue, 07 Sep 2004 17:14:43 -0400
From: Keith Scott <[email-masked]>
To: [email-masked]
Organization: The MITRE Corporation
Subject: [dtn-interest] FW: I-D ACTION:draft-irtf-dtnrg-bundle-spec-02.txt
Sender: [email-masked]

This is a multi-part message in MIME format.

--Boundary_(ID_zQ+OObWDpWq/P+RjlTg8+w)
Content-type: text/plain; charset=us-ascii
Content-transfer-encoding: 7BIT

> -----Original Message-----
> From: [email-masked] 
> [mailto:[email-masked]] On Behalf Of 
> [email-masked]
> Sent: Tuesday, September 07, 2004 2:57 PM
> To: [email-masked]; [email-masked]
> Subject: I-D ACTION:draft-irtf-dtnrg-bundle-spec-02.txt
> 
> A New Internet-Draft is available from the on-line 
> Internet-Drafts directories.
> 
> 
> 	Title		: Bundle Protocol Specification
> 	Author(s)	: K. Scott, S. Burleigh
> 	Filename	: draft-irtf-dtnrg-bundle-spec-02.txt
> 	Pages		: 40
> 	Date		: 2004-9-7
> 	
> This document describes the end-to-end protocol, header 
> formats, and abstract service description for the exchange of 
> messages (bundles) in Delay Tolerant Networking (DTN).
> 
> A URL for this Internet-Draft is:
> http://www.ietf.org/internet-drafts/draft-irtf-dtnrg-bundle-sp
> ec-02.txt
> 
> To remove yourself from the I-D Announcement list, send a 
> message to [email-masked] with the word 
> unsubscribe in the body of the message.  
> You can also visit https://www1.ietf.org/mailman/listinfo/I-D-announce
> to change your subscription settings.
> 
> 
> Internet-Drafts are also available by anonymous FTP. Login 
> with the username "anonymous" and a password of your e-mail 
> address. After logging in, type "cd internet-drafts" and then
> 	"get draft-irtf-dtnrg-bundle-spec-02.txt".
> 
> A list of Internet-Drafts directories can be found in 
> http://www.ietf.org/shadow.html or 
> ftp://ftp.ietf.org/ietf/1shadow-sites.txt
> 
> 
> Internet-Drafts can also be obtained by e-mail.
> 
> Send a message to:
> 	[email-masked].
> In the body type:
> 	"FILE /internet-drafts/draft-irtf-dtnrg-bundle-spec-02.txt".
> 	
> NOTE:	The mail server at ietf.org can return the document in
> 	MIME-encoded form by using the "mpack" utility.  To use this
> 	feature, insert the command "ENCODING mime" before the "FILE"
> 	command.  To decode the response(s), you will need "munpack" or
> 	a MIME-compliant mail reader.  Different MIME-compliant 
> mail readers
> 	exhibit different behavior, especially when dealing with
> 	"multipart" MIME messages (i.e. documents which have been split
> 	up into multiple messages), so check your local documentation on
> 	how to manipulate these messages.
> 		
> 		
> Below is the data which will enable a MIME compliant mail reader
> implementation to automatically retrieve the ASCII version of the
> Internet-Draft.
> 

--Boundary_(ID_zQ+OObWDpWq/P+RjlTg8+w)
Content-type: Message/External-body; name=draft-irtf-dtnrg-bundle-spec-02.txt
Content-transfer-encoding: 7bit
Content-disposition: attachment; filename=draft-irtf-dtnrg-bundle-spec-02.txt

Content-Type: text/plain
Content-ID: <[email-masked]>

--Boundary_(ID_zQ+OObWDpWq/P+RjlTg8+w)--

Date: Tue, 07 Sep 2004 09:28:37 -0700
To: [email-masked]
From: Scott Burleigh <[email-masked]>
Subject: Re: [dtn-interest] extension headers in bundle protocol
Sender: [email-masked]

At 11:17 AM 9/6/2004 +0100, Stephen Farrell wrote:

>Scott, Michael,
>
>So you're saying that we don't require end-to-end data
>integrity for any of the non payload bytes for non buffer-able
>(e.g. v. v. big) bundles? (The problem of course being that
>if the MAC is a header, and is supposed to cover the entire
>bundle, then I need to buffer up all the payload bytes before
>I can send out the MAC header.)

Hi, Stephen.  I don't think anybody's saying that end-to-end data integrity 
for the non-payload bytes of large bundles is unimportant.  Rather, we're 
saying that the reactive fragmentation mechanism -- which we think in some 
case we'll need in order to get large bundles to flow at all -- makes it 
difficult to accommodate bundle trailers.  If there's some other way to 
assure end-to-end data integrity for the non-payload bytes of large bundles 
without requiring trailers then that would be preferable.  The latest 
edition of the Bundling spec makes some provision for this, I think, in the 
Bundle Authentication header (and there's also a separate Payload Security 
header for assuring the authenticity and/or integrity of the 
payload).  Would you want to look that stuff over and comment?

Scott 

Date: Mon, 06 Sep 2004 11:17:36 +0100
From: Stephen Farrell <[email-masked]>
To: Michael Demmer <[email-masked]>, Scott Burleigh <[email-masked]>
Cc: [email-masked]
Subject: Re: [dtn-interest] extension headers in bundle protocol
Sender: [email-masked]

Scott, Michael,

So you're saying that we don't require end-to-end data
integrity for any of the non payload bytes for non buffer-able
(e.g. v. v. big) bundles? (The problem of course being that
if the MAC is a header, and is supposed to cover the entire
bundle, then I need to buffer up all the payload bytes before
I can send out the MAC header.)

Seems a bit strange, but I guess I might be able to buy it,
although I'd have to be concerned that whatever mechanism
is used to ack custody (you can see I haven't read the
drafts recently;-), is, if piggy-backed on a such a large
bundle, then maybe open to spoofing and other attacks which
would seem on the face of it to be bad (assuming that
typical behaviour is to not MAC too-big bundles).

Maybe someone ought to look through the docs with a view
to analysing such cases? (And I might yet volunteer, but
not this month I'm afraid),

Stephen.

Michael Demmer wrote:
>>Another point on this, now that I think of it: we commonly say we fragment 
>>bundles but of course we don't, any more than IP fragments datagrams.  Just 
>>as IP fragments only the content of datagrams, wrapping the two content 
>>fragments in two different but related IP datagrams with complete headers, 
>>so does Bundling fragment only the payload of a bundle, wrapping it in two 
>>different but related bundles with complete headers.  Including some 
>>non-payload bundle data in one fragmentary bundle but not in the other 
>>seems to break that model in a way that might turn out to be fairly 
>>profound.  I don't think we want to go there unless there is an extremely 
>>compelling reason, and as best I can recall nobody has identified any such 
>>requirement in the past six years.
> 
> 
> Agreed wholeheartedly.
> 
> -m
> _______________________________________________
> dtn-interest mailing list
> [email-masked]
> http://mailman.dtnrg.org/mailman/listinfo/dtn-interest
> 

Date: Fri, 3 Sep 2004 16:07:50 -0700
From: Michael Demmer <[email-masked]>
To: Scott Burleigh <[email-masked]>
Cc: [email-masked]
Subject: Re: [dtn-interest] bundle overwrite feature
Mail-Followup-To: Scott Burleigh <[email-masked]>, [email-masked]
Sender: [email-masked]

> I agree that a flow label in the extension header isn't strictly
> necessary, but I would be inclined to go with it anyway.  Although
> it's certainly true that a receiver might identify itself by
> multiple tuples (DTN endpoint IDs), that's a fairly coarse-grained
> mechanism for flow identification. I think it's roughly analogous to
> opening a new socket on a different IP address for the same
> (multi-homed) host, rather than on a different port, because the
> entire "admin part" of an endpoint ID is presented to intra-regional
> route computation algorithms: the two flows would be routed
> separately, and in some implementations that might affect
> performance.

That does make some assumptions about how the intra-regional routing
works, i.e. it may be possible in some implementations to aggregate
for routing purposes, admin ids that correspond to the same host
endpoint, making the "demux string" more like a port than another
address. But now I'm making some implementation assumptions.

> Also the receiving application would have to register for each flow
> independently, possibly increasing application complexity. 

That's not necessarily true. In the DTN1 reference implementation, a
registration defines a prefix rule for matching incoming bundles,
which would let an application cover a set of tuples with a single
registration. This isn't explicitly mentioned in either the
architecture or bundle protocol documents, but I might be inclined to
include some abstract notion of a "endpoint pattern", used for both
registrations and within the routing state to match sets of tuples.

> Tuples are kind of heavyweight in DTN; I really think it would be
> useful at least to offer flow labeling as an option.

With all the above said though, I agree with you and that we should
add a field to the header and provisions in the protocol for the flow
id. The cost is only a few bytes (maybe only one, but probably two),
and still leaves open the possibility of using separate tuple strings.

-m

Date: Fri, 03 Sep 2004 15:28:10 -0700
To: [email-masked]
From: Scott Burleigh <[email-masked]>
Subject: Re: [dtn-interest] bundle overwrite feature
Sender: [email-masked]

At 03:07 PM 9/3/2004 -0700, Michael Demmer wrote:

> > >By marking a bundle for overwriting, you in effect create a name
> > >for a flow of bundles.  In your proposal the name consists of just
> > >the (source,destination) pair.  This brings up the question of how
> > >fine-grained the naming scheme you suggest is (what is the
> > >definition of a "source" and a "destination"?) and how secure it is
> > >(how easily can source and destination be spoofed?).  Allowing
> > >overwriting in the network raises a security concern at the bundle
> > >agent, since overwriting a bundle has the effect of denying service
> > >to the sender of the deleted bundle, and this is something that
> > >cannot be corrected by the receiver in the absence of an end-to-end
> > >protocol.
> >
> > I think this is a good point.  One of the many things I like about
> > Michael's option 2 is that it gives us the flexibility to add, for
> > example, a flow label to the overwrite extension header.  This would
> > be in addition to the queue count, which limits the number of
> > bundles in the named flow (a subset of the traffic between this
> > source and destination) to retain in the queue, and the overwrite
> > identifier rule that indicates how to determine which one(s) to
> > jettison when the limit is exceeded.
>
>I was originally leaning towards including a flow identifier in the
>header. I dropped it because I think it's redundant, since the
>sender/receiver can communicate using more than one unique pair of
>source/destination tuples. In essence, the sender could tack a unique
>flow identifier into the admin portion of the source or destination
>tuple, thus isolating the deletion rules for multiple bundle flows.

I agree that a flow label in the extension header isn't strictly necessary, 
but I would be inclined to go with it anyway.  Although it's certainly true 
that a receiver might identify itself by multiple tuples (DTN endpoint 
IDs), that's a fairly coarse-grained mechanism for flow identification. I 
think it's roughly analogous to opening a new socket on a different IP 
address for the same (multi-homed) host, rather than on a different port, 
because the entire "admin part" of an endpoint ID is presented to 
intra-regional route computation algorithms: the two flows would be routed 
separately, and in some implementations that might affect 
performance.  Also the receiving application would have to register for 
each flow independently, possibly increasing application 
complexity.  Tuples are kind of heavyweight in DTN; I really think it would 
be useful at least to offer flow labeling as an option.

Scott 

Date: Fri, 3 Sep 2004 15:07:51 -0700
From: Michael Demmer <[email-masked]>
To: Scott Burleigh <[email-masked]>
Cc: [email-masked]
Subject: Re: [dtn-interest] bundle overwrite feature
Mail-Followup-To: Scott Burleigh <[email-masked]>, [email-masked]
Sender: [email-masked]

> >By marking a bundle for overwriting, you in effect create a name
> >for a flow of bundles.  In your proposal the name consists of just
> >the (source,destination) pair.  This brings up the question of how
> >fine-grained the naming scheme you suggest is (what is the
> >definition of a "source" and a "destination"?) and how secure it is
> >(how easily can source and destination be spoofed?).  Allowing
> >overwriting in the network raises a security concern at the bundle
> >agent, since overwriting a bundle has the effect of denying service
> >to the sender of the deleted bundle, and this is something that
> >cannot be corrected by the receiver in the absence of an end-to-end
> >protocol.
>
> I think this is a good point.  One of the many things I like about
> Michael's option 2 is that it gives us the flexibility to add, for
> example, a flow label to the overwrite extension header.  This would
> be in addition to the queue count, which limits the number of
> bundles in the named flow (a subset of the traffic between this
> source and destination) to retain in the queue, and the overwrite
> identifier rule that indicates how to determine which one(s) to
> jettison when the limit is exceeded.

I was originally leaning towards including a flow identifier in the
header. I dropped it because I think it's redundant, since the
sender/receiver can communicate using more than one unique pair of
source/destination tuples. In essence, the sender could tack a unique
flow identifier into the admin portion of the source or destination
tuple, thus isolating the deletion rules for multiple bundle flows.

To address Micah's questions a bit more specifically, this should
allow all the flexibility you'd need for naming flows. As to the
security question, I'm inclined to wave my hands a bit since the
overall question of security in DTN is still fairly unexplored.

Intuitively, if the DTN authentication system is able to verify that
the source address marked in a tuple truly does correspond to the
sender, then there "shouldn't" be any security or spoofing issues in
this new proposal. But I'm not a security person, so I can't claim to
stand firmly behind this statement.

> Well, only if what is being sent on the flow is new versions of the
> same data.  I think the proposed scheme would tolerate heterogeneous
> transmission on a given flow, which could be very useful for some
> applications.

In fact, as I mentioned in my examples, essentially the main goal of
this is to allow the network to drop data bundles that are no longer
relevant to anyone in the network, i.e. yesterday's weather forecast
is only useful if you don't have today's forecast. So there's no
reason for the sender to hold onto a copy of old bundles.

> If the sender is isolated while all of this transmission is going
> on, then all of these bundles must indeed be "kept" in the outbound
> bundles queue of the sender's own bundle agent.  Though I suppose we
> could control resource consumption by saying that the overwrite
> extension header authorizes this source agent itself to do the same
> "squashing" -- at the source -- that downstream forwarders do.
> Could be very handy technique, in fact.

My assumption throughout is that the source bundle agent would act as
any other agent wrt this feature. Again, this allows the sender
application to just send appropriately marked bundles periodically and
not worry about managing queues anywhere in the network, including at
the source agent itself.

> >The need to keep multiple copies might be solved if some other
> >storage operations could be applied to the data as it is stored at
> >the bundle agent, namely read and write.  If the sender could read
> >the data back, then they wouldn't have to maintain their own copy,
> >and if they could modify it then perhaps the update wouldn't have
> >to be resent in full.  In fact, the data could be accessed and
> >updates made by network actors other than the original sender, if
> >they knew the name of the flow.  Security concerns would have to be
> >addressed, and locking between sender and bundle agent would have
> >to be a part of any such scheme to ensure that data did not get
> >transmitted while an update was in progress.

Personally, I'm less inclined to think of this feature as a storage
management feature as it is a network QoS metric, offering the network
routers information as to how to drop packets.

> A really interesting perspective, Micah, but I'm not fully persuaded
> that it's useful to transform DTN from an overlay network protocol
> suite into a content distribution service.  Surely a content
> distribution service -- a network application -- could use DTN to
> accomplish its distribution of content, enabling distributed
> management of stored data in a delay-tolerant network.  Conversely,
> DTN could surely use a network storage service as an underlying
> mechanism enabling transmission over a "regional" network.  But it
> seems to me that there is a fair amount of network protocol
> functionality that a storage service ought not to be required to
> provide, and likewise a fair amount of data management functionality
> that a network protocol ought not to be required to provide.

I agree with Scott that I think this would better off separated, but
Micah's ideas for leveraging a shared underlying storage
infrastructure to improve efficiency are certainly interesting ones.

-m

Date: Fri, 03 Sep 2004 14:19:52 -0700
To: [email-masked]
From: Scott Burleigh <[email-masked]>
Subject: Re: [dtn-interest] bundle overwrite feature
Sender: [email-masked]

At 05:10 PM 9/3/2004 -0400, Micah Beck wrote:
>Hi Scott!
>
>----- Original Message -----
>From: "Scott Burleigh" <[email-masked]>
>To: <[email-masked]>
>Sent: Friday, September 03, 2004 4:05 PM
>Subject: Re: [dtn-interest] bundle overwrite feature
>...
>
> > >Putting an overwrite bit into DTN is an interesting and minimal first
>step,
> > >but it's easy to think of further steps one could take in the direction
>of
> > >exposing the nature of DTN as a storage-based service.  By adding a
> > >parameter that at least temporarily inhibits the deletion of a bundle
>when
> > >custody transfer occurs, DTN could be transformed into a content
> > >distribution service that publishes data at bundle agents along the path
> > >from sender to reciever, making it accessible by name as described above.
> > >Loose source routing with storage directives could then turn DTN into a
> > >powerful mechanism for distributed management of stored data.  So much of
> > >the necessary infrastructure is already there.
> >
> > A really interesting perspective, Micah, but I'm not fully persuaded that
> > it's useful to transform DTN from an overlay network protocol suite into a
> > content distribution service.  Surely a content distribution service -- a
> > network application -- could use DTN to accomplish its distribution of
> > content, enabling distributed management of stored data in a
>delay-tolerant
> > network.  Conversely, DTN could surely use a network storage service as an
> > underlying mechanism enabling transmission over a "regional" network.  But
> > it seems to me that there is a fair amount of network protocol
> > functionality that a storage service ought not to be required to provide,
> > and likewise a fair amount of data management functionality that a network
> > protocol ought not to be required to provide.
>
>Thanks for pointing this out.  I agree that DTN and Content Distribution
>functionality are probably better implemented seperately, but sharing access
>to common services and resources.  I had simplified the scenario, folding
>the two together, in order to shorten the presentation.
>
>DTN and Content Distribution might use a single, interoperable storage
>protocol, and storage infrastructure might actually be shared to avoid
>unnecessary copying and replication of data and unnecessary fragmentation
>(balkanization?) of storage resources.  In addition, there might be a value
>to having a common scheme for naming flows of bundles in DTN and channels of
>data published through a Content Distribution Network, and perhaps user
>authentication and authorization mechanisms required to implement features
>such as bundle overwriting and channel management.  (These are just
>off-the-cuff examples.)  The idea would be to look for commonality, and to
>take advantage of it in implementing services that could be ultimately be
>exposed through heterogeneous network services (eg DTN, Content
>Distribution, Distributed File Systems, ...)

I agree, a very reasonable strategy to explore.

Scott 

From: "Micah Beck" <[email-masked]>
To: <[email-masked]>
Subject: Re: [dtn-interest] bundle overwrite feature
Date: Fri, 3 Sep 2004 17:10:45 -0400
Sender: [email-masked]

Hi Scott!

----- Original Message ----- 
From: "Scott Burleigh" <[email-masked]>
To: <[email-masked]>
Sent: Friday, September 03, 2004 4:05 PM
Subject: Re: [dtn-interest] bundle overwrite feature
...

> >Now, let's broaden our thinking a little bit, to a case where the sender
is
> >isolated for long periods of time, and is sending truely large amounts of
> >data.  In that case, in order to make use of the overwrite facility you
> >suggest, the sender must keep a copy of the data in order to be able to
> >update it and resend a new version.
>
> Well, only if what is being sent on the flow is new versions of the same
> data.  I think the proposed scheme would tolerate heterogeneous
> transmission on a given flow, which could be very useful for some
applications.
>
> >This is true even if the copy stored in
> >the bundle agent is accessible to the sender through the local IP
network.
> >The updated bundle must also be resent in its entirety.
>
> Only if you think of it as an updated bundle.  The more general model is
> that a series of bundles are being sent, and the content (as well as the
> headers, obviously) of any one bundle in the flow might be entirely
> different from that of the one before it.

Point taken - my scenario was a special case, but I believe there are many
other situations in which exposing the fact that queued bundes reside in
storage and can be operated through read and write rather than simply being
kept or deleted could allow useful optimizations.  If anyone wants more
examples, I'm willing to cook some up.
...

> >Putting an overwrite bit into DTN is an interesting and minimal first
step,
> >but it's easy to think of further steps one could take in the direction
of
> >exposing the nature of DTN as a storage-based service.  By adding a
> >parameter that at least temporarily inhibits the deletion of a bundle
when
> >custody transfer occurs, DTN could be transformed into a content
> >distribution service that publishes data at bundle agents along the path
> >from sender to reciever, making it accessible by name as described above.
> >Loose source routing with storage directives could then turn DTN into a
> >powerful mechanism for distributed management of stored data.  So much of
> >the necessary infrastructure is already there.
>
> A really interesting perspective, Micah, but I'm not fully persuaded that
> it's useful to transform DTN from an overlay network protocol suite into a
> content distribution service.  Surely a content distribution service -- a
> network application -- could use DTN to accomplish its distribution of
> content, enabling distributed management of stored data in a
delay-tolerant
> network.  Conversely, DTN could surely use a network storage service as an
> underlying mechanism enabling transmission over a "regional" network.  But
> it seems to me that there is a fair amount of network protocol
> functionality that a storage service ought not to be required to provide,
> and likewise a fair amount of data management functionality that a network
> protocol ought not to be required to provide.

Thanks for pointing this out.  I agree that DTN and Content Distribution
functionality are probably better implemented seperately, but sharing access
to common services and resources.  I had simplified the scenario, folding
the two together, in order to shorten the presentation.

DTN and Content Distribution might use a single, interoperable storage
protocol, and storage infrastructure might actually be shared to avoid
unnecessary copying and replication of data and unnecessary fragmentation
(balkanization?) of storage resources.  In addition, there might be a value
to having a common scheme for naming flows of bundles in DTN and channels of
data published through a Content Distribution Network, and perhaps user
authentication and authorization mechanisms required to implement features
such as bundle overwriting and channel management.  (These are just
off-the-cuff examples.)  The idea would be to look for commonality, and to
take advantage of it in implementing services that could be ultimately be
exposed through heterogeneous network services (eg DTN, Content
Distribution, Distributed File Systems, ...)

> Scott
/micah

Date: Fri, 3 Sep 2004 13:12:51 -0700
From: Michael Demmer <[email-masked]>
To: Scott Burleigh <[email-masked]>
Cc: [email-masked]
Subject: Re: [dtn-interest] extension headers in bundle protocol
Mail-Followup-To: Scott Burleigh <[email-masked]>, [email-masked]
Sender: [email-masked]

> Another point on this, now that I think of it: we commonly say we fragment 
> bundles but of course we don't, any more than IP fragments datagrams.  Just 
> as IP fragments only the content of datagrams, wrapping the two content 
> fragments in two different but related IP datagrams with complete headers, 
> so does Bundling fragment only the payload of a bundle, wrapping it in two 
> different but related bundles with complete headers.  Including some 
> non-payload bundle data in one fragmentary bundle but not in the other 
> seems to break that model in a way that might turn out to be fairly 
> profound.  I don't think we want to go there unless there is an extremely 
> compelling reason, and as best I can recall nobody has identified any such 
> requirement in the past six years.

Agreed wholeheartedly.

-m

Date: Fri, 03 Sep 2004 13:05:52 -0700
To: [email-masked]
From: Scott Burleigh <[email-masked]>
Subject: Re: [dtn-interest] bundle overwrite feature
Sender: [email-masked]

At 11:49 AM 9/2/2004 -0400, Micah Beck wrote:
>By marking a bundle for overwriting, you in effect create a name for a flow
>of bundles.  In your proposal the name consists of just the
>(source,destination) pair.  This brings up the question of how fine-grained
>the naming scheme you suggest is (what is the definition of a "source" and a
>"destination"?) and how secure it is (how easily can source and destination
>be spoofed?).  Allowing overwriting in the network raises a security concern
>at the bundle agent, since overwriting a bundle has the effect of denying
>service to the sender of the deleted bundle, and this is something that
>cannot be corrected by the receiver in the absence of an end-to-end
>protocol.

I think this is a good point.  One of the many things I like about 
Michael's option 2 is that it gives us the flexibility to add, for example, 
a flow label to the overwrite extension header.  This would be in addition 
to the queue count, which limits the number of bundles in the named flow (a 
subset of the traffic between this source and destination) to retain in the 
queue, and the overwrite identifier rule that indicates how to determine 
which one(s) to jettison when the limit is exceeded.

>Let's assume that there is some naming scheme, either the one you propose or
>another one, that is fine-grained enough to allow multiple applications at a
>single node to share the network without having to worry about overwriting
>each other's bundles, and secure enough to preserve the integrity of the
>network.  Your proposal allows applications to make use of the storage
>resources and the forwarding services of the bundle agent in a way that is
>somewhat more general that simple bundle delivery, and it can avoid wasting
>bandwidth by delivering data that is out of date befiore it arrives.  It
>makes use of the fact that bundles are stored at bundle agents and therefore
>a storage operation can be applied there, namely deletion.
>
>Now, let's broaden our thinking a little bit, to a case where the sender is
>isolated for long periods of time, and is sending truely large amounts of
>data.  In that case, in order to make use of the overwrite facility you
>suggest, the sender must keep a copy of the data in order to be able to
>update it and resend a new version.

Well, only if what is being sent on the flow is new versions of the same 
data.  I think the proposed scheme would tolerate heterogeneous 
transmission on a given flow, which could be very useful for some applications.

>This is true even if the copy stored in
>the bundle agent is accessible to the sender through the local IP network.
>The updated bundle must also be resent in its entirety.

Only if you think of it as an updated bundle.  The more general model is 
that a series of bundles are being sent, and the content (as well as the 
headers, obviously) of any one bundle in the flow might be entirely 
different from that of the one before it.

If the sender is isolated while all of this transmission is going on, then 
all of these bundles must indeed be "kept" in the outbound bundles queue of 
the sender's own bundle agent.  Though I suppose we could control resource 
consumption by saying that the overwrite extension header authorizes this 
source agent itself to do the same "squashing" -- at the source -- that 
downstream forwarders do.  Could be very handy technique, in fact.

>The need to keep multiple copies might be solved if some other storage
>operations could be applied to the data as it is stored at the bundle agent,
>namely read and write.  If the sender could read the data back, then they
>wouldn't have to maintain their own copy, and if they could modify it then
>perhaps the update wouldn't have to be resent in full.  In fact, the data
>could be accessed and updates made by network actors other than the original
>sender, if they knew the name of the flow.  Security concerns would have to
>be addressed, and locking between sender and bundle agent would have to be a
>part of any such scheme to ensure that data did not get transmitted while an
>update was in progress.
>
>Turning the bundle agent into some kind of a storage server might allow
>further optimizations, but there is an obvious problem: once a connection is
>made and custody transfer takes place, the bundle agent will delete its copy
>of the data, and the sender will be left without a copy.  One way to deal
>with this is to for both the sender and the bundle agent to make common use
>of a storage infrastructure that allows the sender to exercise more control
>over the duration of storage.  Kevin suggested the possibility of using a
>file or database service, which then requires that all senders and the
>bundle agent itself be clients of that service.  I advocate the use of a
>more minimal, generic storage service such as the Internet Backplane
>Protocol, that may allow looser coupling between the network components,
>with the goal of enabling greater scalability.
>
>Putting an overwrite bit into DTN is an interesting and minimal first step,
>but it's easy to think of further steps one could take in the direction of
>exposing the nature of DTN as a storage-based service.  By adding a
>parameter that at least temporarily inhibits the deletion of a bundle when
>custody transfer occurs, DTN could be transformed into a content
>distribution service that publishes data at bundle agents along the path
>from sender to reciever, making it accessible by name as described above.
>Loose source routing with storage directives could then turn DTN into a
>powerful mechanism for distributed management of stored data.  So much of
>the necessary infrastructure is already there.

A really interesting perspective, Micah, but I'm not fully persuaded that 
it's useful to transform DTN from an overlay network protocol suite into a 
content distribution service.  Surely a content distribution service -- a 
network application -- could use DTN to accomplish its distribution of 
content, enabling distributed management of stored data in a delay-tolerant 
network.  Conversely, DTN could surely use a network storage service as an 
underlying mechanism enabling transmission over a "regional" network.  But 
it seems to me that there is a fair amount of network protocol 
functionality that a storage service ought not to be required to provide, 
and likewise a fair amount of data management functionality that a network 
protocol ought not to be required to provide.

Scott

>----- Original Message -----
>From: "Michael Demmer" <[email-masked]>
>To: <[email-masked]>
>Sent: Wednesday, September 01, 2004 3:25 PM
>Subject: [dtn-interest] bundle overwrite feature
>
>
> >
> > One idea that I've floated in various discussions and I think would
> > merit from wider dtn-interest exploration is that of a new "bundle
> > queue overwrite" feature. (In discussions, this has most often been
> > referred to as "squashing")
> >
> > The motivating scenario is that of a dtn application that periodically
> > transmits data over an intermittent network, where each bunde is a
> > more "recent" update to a particular information source and therefore
> > subsumes previous transmissions. Simple examples include weather
> > forecasts, commodity or stock prices, broadcast web page updates,
> > etc. In all these cases, the sender's primary goal is that the most
> > up-to-date information is received by the destination. Given this,
> > interior nodes in the network can (and should) proactively delete
> > older queued bundles to make the most efficient use of possibly
> > constrained network links in delivering the most recent transmission.
> >
> > The specifics of the proposal involve adding a new class-of-service to
> > the bundling specification (and possibly to the dtn architecture
> > internet draft), optionally marking a bundle for overwrite processing.
> > When a bundle with the overwrite class set is received by a dtn node,
> > that node will then scan though any queued bundles, and if it finds
> > one or more that a) have the overwrite bit set and b) have the same
> > source and destination tuples as the newly arrived bundle, then the
> > dtn node should replace the "older" bundle with the arrival,
> > discarding the older bundle.
> >
> > It is important to note that this feature is just a performance
> > optimization; it is obviously possible that a particular flow of
> > bundles will never encounter a situation in which two bundles are
> > coresident in a node's queue, and therefore never engage this
> > feature. This also implies that not all dtn nodes be required to
> > implement the feature for correctness.
> >
> > One question that remains (and for which input would be appreciated)
> > is the specification of how an interior node should decide which
> > bundles to delete. I can envision two possiblilities:
> >
> > 1) The simplest requires just adding the single class of service bit,
> >    and uses the sender timestamp field to determine which bundles are
> >    to be deleted.
> >
> >    The deletion rules are straightforward: for any two bundles with
> >    the overwrite bit set and matching source/destination tuples, the
> >    one with the older (smaller) timestamp is deleted.
> >
> > 2) A slightly more complicated, but more flexible, approach is to add
> >    an extension header (see my previous message on the subject of
> >    extension headers) to enable the overwrite feature. The header data
> >    would define a one byte "queue count" and an 8 byte "overwrite
> >    identifier". Both would be supplied by the application along with
> >    the bundle payload data.
> >
> >    The deletion rules would then specify that an internal node keep up
> >    to the specified count of bundles in the queue, and use the
> >    specified identifier to determine which bundle should be
> >    deleted. The advantage of this approach is that it allows more
> >    application flexibility in controlling the deletion rules.
> >
> >    My personal instinct is to implement this approach, allowing the
> >    feature to be implemented entirely as an extension header in the
> >    protocol (and therefore not defining a new class of service bit in
> >    the bundle specification).
> >
> > A couple final notes: Custody does not come into play here, and an
> > internal node should be free to overwrite bundles even if it has taken
> > custody of them. The logic behind this is that it is at the
> > applications behest that the deletion occur, therefore custody can be
> > safely ignored. Also, the overwrite should only be performed for full
> > (i.e. non-fragmented) bundles.
> >
> > Thoughts and comments are appreciated.
> >
> > -mike

Date: Fri, 03 Sep 2004 12:19:25 -0700
To: [email-masked]
From: Scott Burleigh <[email-masked]>
Subject: Re: [dtn-interest] extension headers in bundle protocol
Sender: [email-masked]

At 11:32 AM 9/3/2004 -0700, Michael Demmer wrote:

> > Currently, no; the payload header has to be the last one.  The point
> > of this is to enable reactive fragmentation: if you receive any of
> > the payload, then you are guaranteed to have received all of the
> > other headers -- so you are guaranteed to know everything you might
> > need to know in order to divide the payload between two related
> > fragmentary bundles.  LTP need not do after-the-fact fragmentation,
> > so it is not similarly constrained.
>
>That's true. Although I can see advantages to having support for
>"footers". By definition, they would arrive as part of the last bundle
>fragment, which would be fine for things like end-to-end MACs or other
>info that makes sense in the context of the reassembled bundle.

Another point on this, now that I think of it: we commonly say we fragment 
bundles but of course we don't, any more than IP fragments datagrams.  Just 
as IP fragments only the content of datagrams, wrapping the two content 
fragments in two different but related IP datagrams with complete headers, 
so does Bundling fragment only the payload of a bundle, wrapping it in two 
different but related bundles with complete headers.  Including some 
non-payload bundle data in one fragmentary bundle but not in the other 
seems to break that model in a way that might turn out to be fairly 
profound.  I don't think we want to go there unless there is an extremely 
compelling reason, and as best I can recall nobody has identified any such 
requirement in the past six years.

Scott 

Date: Fri, 03 Sep 2004 11:45:11 -0700
To: [email-masked]
From: Scott Burleigh <[email-masked]>
Subject: Re: [dtn-interest] extension headers in bundle protocol
Sender: [email-masked]

At 11:32 AM 9/3/2004 -0700, Michael Demmer wrote:

> > Currently, no; the payload header has to be the last one.  The point
> > of this is to enable reactive fragmentation: if you receive any of
> > the payload, then you are guaranteed to have received all of the
> > other headers -- so you are guaranteed to know everything you might
> > need to know in order to divide the payload between two related
> > fragmentary bundles.  LTP need not do after-the-fact fragmentation,
> > so it is not similarly constrained.
>
>That's true. Although I can see advantages to having support for
>"footers". By definition, they would arrive as part of the last bundle
>fragment, which would be fine for things like end-to-end MACs or other
>info that makes sense in the context of the reassembled bundle.

Sure, no problem if you can guarantee that the payload header and all 
subsequent headers are of value to (and will be used by) only the final 
recipient of the reassembled bundle.

I think that kind of constraint invites big problems, myself, but we could 
impose it if there's a strong sense that bundle "trailers" are really 
needed.  My preference would be to say that, since this information is only 
of interest and value to the final recipient, it should simply be appended 
to the payload by the application (or some common service that is exposed 
to applications).

Scott 

Date: Fri, 3 Sep 2004 11:32:35 -0700
From: Michael Demmer <[email-masked]>
To: Scott Burleigh <[email-masked]>
Cc: [email-masked]
Subject: Re: [dtn-interest] extension headers in bundle protocol
Mail-Followup-To: Scott Burleigh <[email-masked]>, [email-masked]
Sender: [email-masked]

> Currently, no; the payload header has to be the last one.  The point
> of this is to enable reactive fragmentation: if you receive any of
> the payload, then you are guaranteed to have received all of the
> other headers -- so you are guaranteed to know everything you might
> need to know in order to divide the payload between two related
> fragmentary bundles.  LTP need not do after-the-fact fragmentation,
> so it is not similarly constrained.

That's true. Although I can see advantages to having support for
"footers". By definition, they would arrive as part of the last bundle
fragment, which would be fine for things like end-to-end MACs or other
info that makes sense in the context of the reassembled bundle.

-m

Date: Fri, 03 Sep 2004 11:21:42 -0700
To: [email-masked]
From: Scott Burleigh <[email-masked]>
Subject: Re: [dtn-interest] extension headers in bundle protocol
Sender: [email-masked]

At 09:34 AM 9/2/2004 -0700, Michael Demmer wrote:

> > Why limit all future extensions to 32-bit values? As well as
> > those you've proposed, I'd suggest allowing (in whatever way,
> > I'm not too pushed) a variable length header - after all if
> > these are for experimenting then you can expect all sorts of
> > stuff to go in there.
>
>I guess my message wasn't clear... In fact, all of these are variable
>length headers, the difference being how many bytes are needed to
>encode the _length_ of the header data.
>
>A better picture would have been:
>
>EXT1:
>+----------------+----------------+----------------+----------------+
>| Next Header    | Extension Type | Length (1 byte)| Data...        |
>+----------------+----------------+----------------+----------------|
>
> > Only other thought is that with LTP we also allowed for
> > extension values at end of segments, specifically to
> > support things like MACs and signatures - (avoids having
> > to buffer up entire segments whilst doing calculations -
> > this is done in lots of other protocols btw). I'd imagine
> > a similar "at the end" extensibility mechanism might also
> > be useful. And again, such crypto artefacts tend to be
> > longer than 32-bits.
>
>afik, this should still be supported. i.e. you could put your
>extension header after the payload header and its associated data.

Currently, no; the payload header has to be the last one.  The point of 
this is to enable reactive fragmentation: if you receive any of the 
payload, then you are guaranteed to have received all of the other headers 
-- so you are guaranteed to know everything you might need to know in order 
to divide the payload between two related fragmentary bundles.  LTP need 
not do after-the-fact fragmentation, so it is not similarly constrained.

Scott 

Date: Thu, 02 Sep 2004 17:40:56 +0100
From: Stephen Farrell <[email-masked]>
To: Michael Demmer <[email-masked]>
Cc: [email-masked]
Subject: Re: [dtn-interest] extension headers in bundle protocol
Sender: [email-masked]

Grand so. Sounds like a great idea then.

Cheers,
Stephen.

Michael Demmer wrote:
>>Why limit all future extensions to 32-bit values? As well as
>>those you've proposed, I'd suggest allowing (in whatever way,
>>I'm not too pushed) a variable length header - after all if
>>these are for experimenting then you can expect all sorts of
>>stuff to go in there.
> 
> 
> I guess my message wasn't clear... In fact, all of these are variable
> length headers, the difference being how many bytes are needed to
> encode the _length_ of the header data.
> 
> A better picture would have been:
> 
> EXT1:
> +----------------+----------------+----------------+----------------+
> | Next Header    | Extension Type | Length (1 byte)| Data...        |
> +----------------+----------------+----------------+----------------|
> 
> 
>>Only other thought is that with LTP we also allowed for
>>extension values at end of segments, specifically to
>>support things like MACs and signatures - (avoids having
>>to buffer up entire segments whilst doing calculations -
>>this is done in lots of other protocols btw). I'd imagine
>>a similar "at the end" extensibility mechanism might also
>>be useful. And again, such crypto artefacts tend to be
>>longer than 32-bits.
> 
> 
> afik, this should still be supported. i.e. you could put your
> extension header after the payload header and its associated data.
> 
> -mike
> 

Date: Thu, 2 Sep 2004 09:34:49 -0700
From: Michael Demmer <[email-masked]>
To: Stephen Farrell <[email-masked]>
Cc: [email-masked]
Subject: Re: [dtn-interest] extension headers in bundle protocol
Mail-Followup-To: Stephen Farrell <[email-masked]>, [email-masked]
Sender: [email-masked]

> Why limit all future extensions to 32-bit values? As well as
> those you've proposed, I'd suggest allowing (in whatever way,
> I'm not too pushed) a variable length header - after all if
> these are for experimenting then you can expect all sorts of
> stuff to go in there.

I guess my message wasn't clear... In fact, all of these are variable
length headers, the difference being how many bytes are needed to
encode the _length_ of the header data.

A better picture would have been:

EXT1:
+----------------+----------------+----------------+----------------+
| Next Header    | Extension Type | Length (1 byte)| Data...        |
+----------------+----------------+----------------+----------------|

> Only other thought is that with LTP we also allowed for
> extension values at end of segments, specifically to
> support things like MACs and signatures - (avoids having
> to buffer up entire segments whilst doing calculations -
> this is done in lots of other protocols btw). I'd imagine
> a similar "at the end" extensibility mechanism might also
> be useful. And again, such crypto artefacts tend to be
> longer than 32-bits.

afik, this should still be supported. i.e. you could put your
extension header after the payload header and its associated data.

-mike

Date: Thu, 02 Sep 2004 17:21:46 +0100
From: Stephen Farrell <[email-masked]>
To: Michael Demmer <[email-masked]>
Cc: [email-masked]
Subject: Re: [dtn-interest] extension headers in bundle protocol
Sender: [email-masked]

Hi,

Why limit all future extensions to 32-bit values? As well as
those you've proposed, I'd suggest allowing (in whatever way,
I'm not too pushed) a variable length header - after all if
these are for experimenting then you can expect all sorts of
stuff to go in there.

Only other thought is that with LTP we also allowed for
extension values at end of segments, specifically to
support things like MACs and signatures - (avoids having
to buffer up entire segments whilst doing calculations -
this is done in lots of other protocols btw). I'd imagine
a similar "at the end" extensibility mechanism might also
be useful. And again, such crypto artefacts tend to be
longer than 32-bits.

For a concrete example - someone might want to use a
non-standard hash algorithm for with a signature/MAC
scheme (hash algorithm attacks being in topical
nowadays;-). In such cases I'd like to put a ciphersuite
and perhaps keyId in the header and the signature/MAC
value at the back.

Regards,
Stephen.

Michael Demmer wrote:

> I'd like to propose that we add a facility for extensible headers to
> the bundle protocol specification. Currently, the specification
> assumes that all compliant implementations interpret and parse all the
> header type codes, even if the header represents an optional
> feature. As a side effect of this, there is no generic framework for
> encoding the length of headers, as for each header type, the length is
> either inferrable from the format of the header itself or encoded in a
> header-specific manner.
> 
> I would like to add support to the protocol for the development of
> experimental features do not require implementation by all bundling
> nodes in the interior of the network, but may require header
> information that is interpreted by participating nodes. To this end,
> these features shouldn't require a new revision of the protocol to
> support their header information, but rather need some means by which
> "old" nodes can ignore the feature's headers, while "new" nodes parse
> and interpret the headers.
> 
> Therefore, I'd like to add three new type codes to the specification:
> EXT1, EXT2, and EXT4. Each will include one byte for the next header
> type, one byte for the extension typecode, and one, two, or four
> bytes, respectively. Graphically:
> 
> EXT1:
> +----------------+----------------+----------------+
> | Next Header    | Extension Type | Length (1 byte)|
> +----------------+----------------+----------------+
> 
> EXT2:
> +----------------+----------------+----------------+----------------+
> | Next Header    | Extension Type | Length (2 bytes)                |
> +----------------+----------------+----------------+----------------+
> 
> EXT4:
> +----------------+----------------+--------+-------+--------+-------+
> | Next Header    | Extension Type | Length (4 bytes) ...            |
> +----------------+----------------+--------+-------+--------+-------+
> 
> All compliant bundle implementations must be able to parse the
> extension header formats, and given that the header length is
> explicit, the node can skip over an unimplemented header and parse the
> next header.
> 
> Comments are of course requested.
> 
> -mike
> _______________________________________________
> dtn-interest mailing list
> [email-masked]
> http://mailman.dtnrg.org/mailman/listinfo/dtn-interest
> 

From: "Micah Beck" <[email-masked]>
To: "Michael Demmer" <[email-masked]>, <[email-masked]>
Subject: Re: [dtn-interest] bundle overwrite feature
Date: Thu, 2 Sep 2004 11:49:18 -0400
Sender: [email-masked]

Mike,

I hope you won't mind if I hotwire your train of thought and take it on a
little joyride.  Let's see, I think it's blue wire to blue wire....

By marking a bundle for overwriting, you in effect create a name for a flow
of bundles.  In your proposal the name consists of just the
(source,destination) pair.  This brings up the question of how fine-grained
the naming scheme you suggest is (what is the definition of a "source" and a
"destination"?) and how secure it is (how easily can source and destination
be spoofed?).  Allowing overwriting in the network raises a security concern
at the bundle agent, since overwriting a bundle has the effect of denying
service to the sender of the deleted bundle, and this is something that
cannot be corrected by the receiver in the absence of an end-to-end
protocol.

Let's assume that there is some naming scheme, either the one you propose or
another one, that is fine-grained enough to allow multiple applications at a
single node to share the network without having to worry about overwriting
each other's bundles, and secure enough to preserve the integrity of the
network.  Your proposal allows applications to make use of the storage
resources and the forwarding services of the bundle agent in a way that is
somewhat more general that simple bundle delivery, and it can avoid wasting
bandwidth by delivering data that is out of date befiore it arrives.  It
makes use of the fact that bundles are stored at bundle agents and therefore
a storage operation can be applied there, namely deletion.

Now, let's broaden our thinking a little bit, to a case where the sender is
isolated for long periods of time, and is sending truely large amounts of
data.  In that case, in order to make use of the overwrite facility you
suggest, the sender must keep a copy of the data in order to be able to
update it and resend a new version.  This is true even if the copy stored in
the bundle agent is accessible to the sender through the local IP network.
The updated bundle must also be resent in its entirety.

The need to keep multiple copies might be solved if some other storage
operations could be applied to the data as it is stored at the bundle agent,
namely read and write.  If the sender could read the data back, then they
wouldn't have to maintain their own copy, and if they could modify it then
perhaps the update wouldn't have to be resent in full.  In fact, the data
could be accessed and updates made by network actors other than the original
sender, if they knew the name of the flow.  Security concerns would have to
be addressed, and locking between sender and bundle agent would have to be a
part of any such scheme to ensure that data did not get transmitted while an
update was in progress.

Turning the bundle agent into some kind of a storage server might allow
further optimizations, but there is an obvious problem: once a connection is
made and custody transfer takes place, the bundle agent will delete its copy
of the data, and the sender will be left without a copy.  One way to deal
with this is to for both the sender and the bundle agent to make common use
of a storage infrastructure that allows the sender to exercise more control
over the duration of storage.  Kevin suggested the possibility of using a
file or database service, which then requires that all senders and the
bundle agent itself be clients of that service.  I advocate the use of a
more minimal, generic storage service such as the Internet Backplane
Protocol, that may allow looser coupling between the network components,
with the goal of enabling greater scalability.

Putting an overwrite bit into DTN is an interesting and minimal first step,
but it's easy to think of further steps one could take in the direction of
exposing the nature of DTN as a storage-based service.  By adding a
parameter that at least temporarily inhibits the deletion of a bundle when
custody transfer occurs, DTN could be transformed into a content
distribution service that publishes data at bundle agents along the path
from sender to reciever, making it accessible by name as described above.
Loose source routing with storage directives could then turn DTN into a
powerful mechanism for distributed management of stored data.  So much of
the necessary infrastructure is already there.

OK, I'm done with my joyride.  I hope no one checks the odometer...
/micah

----- Original Message ----- 
From: "Michael Demmer" <[email-masked]>
To: <[email-masked]>
Sent: Wednesday, September 01, 2004 3:25 PM
Subject: [dtn-interest] bundle overwrite feature

>
> One idea that I've floated in various discussions and I think would
> merit from wider dtn-interest exploration is that of a new "bundle
> queue overwrite" feature. (In discussions, this has most often been
> referred to as "squashing")
>
> The motivating scenario is that of a dtn application that periodically
> transmits data over an intermittent network, where each bunde is a
> more "recent" update to a particular information source and therefore
> subsumes previous transmissions. Simple examples include weather
> forecasts, commodity or stock prices, broadcast web page updates,
> etc. In all these cases, the sender's primary goal is that the most
> up-to-date information is received by the destination. Given this,
> interior nodes in the network can (and should) proactively delete
> older queued bundles to make the most efficient use of possibly
> constrained network links in delivering the most recent transmission.
>
> The specifics of the proposal involve adding a new class-of-service to
> the bundling specification (and possibly to the dtn architecture
> internet draft), optionally marking a bundle for overwrite processing.
> When a bundle with the overwrite class set is received by a dtn node,
> that node will then scan though any queued bundles, and if it finds
> one or more that a) have the overwrite bit set and b) have the same
> source and destination tuples as the newly arrived bundle, then the
> dtn node should replace the "older" bundle with the arrival,
> discarding the older bundle.
>
> It is important to note that this feature is just a performance
> optimization; it is obviously possible that a particular flow of
> bundles will never encounter a situation in which two bundles are
> coresident in a node's queue, and therefore never engage this
> feature. This also implies that not all dtn nodes be required to
> implement the feature for correctness.
>
> One question that remains (and for which input would be appreciated)
> is the specification of how an interior node should decide which
> bundles to delete. I can envision two possiblilities:
>
> 1) The simplest requires just adding the single class of service bit,
>    and uses the sender timestamp field to determine which bundles are
>    to be deleted.
>
>    The deletion rules are straightforward: for any two bundles with
>    the overwrite bit set and matching source/destination tuples, the
>    one with the older (smaller) timestamp is deleted.
>
> 2) A slightly more complicated, but more flexible, approach is to add
>    an extension header (see my previous message on the subject of
>    extension headers) to enable the overwrite feature. The header data
>    would define a one byte "queue count" and an 8 byte "overwrite
>    identifier". Both would be supplied by the application along with
>    the bundle payload data.
>
>    The deletion rules would then specify that an internal node keep up
>    to the specified count of bundles in the queue, and use the
>    specified identifier to determine which bundle should be
>    deleted. The advantage of this approach is that it allows more
>    application flexibility in controlling the deletion rules.
>
>    My personal instinct is to implement this approach, allowing the
>    feature to be implemented entirely as an extension header in the
>    protocol (and therefore not defining a new class of service bit in
>    the bundle specification).
>
> A couple final notes: Custody does not come into play here, and an
> internal node should be free to overwrite bundles even if it has taken
> custody of them. The logic behind this is that it is at the
> applications behest that the deletion occur, therefore custody can be
> safely ignored. Also, the overwrite should only be performed for full
> (i.e. non-fragmented) bundles.
>
> Thoughts and comments are appreciated.
>
> -mike
> _______________________________________________
> dtn-interest mailing list
> [email-masked]
> http://mailman.dtnrg.org/mailman/listinfo/dtn-interest
>

Date: Wed, 1 Sep 2004 12:25:51 -0700
From: Michael Demmer <[email-masked]>
To: [email-masked]
Mail-Followup-To: [email-masked]
Subject: [dtn-interest] bundle overwrite feature
Sender: [email-masked]

One idea that I've floated in various discussions and I think would
merit from wider dtn-interest exploration is that of a new "bundle
queue overwrite" feature. (In discussions, this has most often been
referred to as "squashing")

The motivating scenario is that of a dtn application that periodically
transmits data over an intermittent network, where each bunde is a
more "recent" update to a particular information source and therefore
subsumes previous transmissions. Simple examples include weather
forecasts, commodity or stock prices, broadcast web page updates,
etc. In all these cases, the sender's primary goal is that the most
up-to-date information is received by the destination. Given this,
interior nodes in the network can (and should) proactively delete
older queued bundles to make the most efficient use of possibly
constrained network links in delivering the most recent transmission.

The specifics of the proposal involve adding a new class-of-service to
the bundling specification (and possibly to the dtn architecture
internet draft), optionally marking a bundle for overwrite processing.
When a bundle with the overwrite class set is received by a dtn node,
that node will then scan though any queued bundles, and if it finds
one or more that a) have the overwrite bit set and b) have the same
source and destination tuples as the newly arrived bundle, then the
dtn node should replace the "older" bundle with the arrival,
discarding the older bundle.

It is important to note that this feature is just a performance
optimization; it is obviously possible that a particular flow of
bundles will never encounter a situation in which two bundles are
coresident in a node's queue, and therefore never engage this
feature. This also implies that not all dtn nodes be required to
implement the feature for correctness.

One question that remains (and for which input would be appreciated)
is the specification of how an interior node should decide which
bundles to delete. I can envision two possiblilities:

1) The simplest requires just adding the single class of service bit,
   and uses the sender timestamp field to determine which bundles are
   to be deleted. 

   The deletion rules are straightforward: for any two bundles with
   the overwrite bit set and matching source/destination tuples, the
   one with the older (smaller) timestamp is deleted.

2) A slightly more complicated, but more flexible, approach is to add
   an extension header (see my previous message on the subject of
   extension headers) to enable the overwrite feature. The header data
   would define a one byte "queue count" and an 8 byte "overwrite
   identifier". Both would be supplied by the application along with
   the bundle payload data.

   The deletion rules would then specify that an internal node keep up
   to the specified count of bundles in the queue, and use the
   specified identifier to determine which bundle should be
   deleted. The advantage of this approach is that it allows more
   application flexibility in controlling the deletion rules.

   My personal instinct is to implement this approach, allowing the
   feature to be implemented entirely as an extension header in the
   protocol (and therefore not defining a new class of service bit in
   the bundle specification).

A couple final notes: Custody does not come into play here, and an
internal node should be free to overwrite bundles even if it has taken
custody of them. The logic behind this is that it is at the
applications behest that the deletion occur, therefore custody can be
safely ignored. Also, the overwrite should only be performed for full
(i.e. non-fragmented) bundles.

Thoughts and comments are appreciated.

-mike