Subject: two shim6-esd-00 comments
From: Pekka Savola <[email-masked]>
To: [email-masked]
Date: Fri, 31 Mar 2006 12:32:24 +0300 (EEST)

Hi,

Two smaller comments, probably already made, on 
draft-nordmark-shim6-esd-00.txt:

1) this obviously requires control over the reverse tree, which is by 
no means given for generic shim6 audience, though it might be more 
popular with sites which want to employ rewriting.

       The shim performs the identifier to locator lookup very similarly
       to normal IPv6 reverse lookups (form a query name based on the
       nibbles in reverse order and append ip6.arpa), but it queries for
       SRV records.

2) In the latter part of the example below, AFAICS, are you able to 
use the Sent Locator immediately?  This could be part of a 3rd party 
[bombing] attack?  Does it need to be probed first or is this 
sufficiently secure as it is?

       B processes the I1 message as specified in [7] to generate a R1
       message.  In addition, it copies the content of the Sent Locator
       Pair option into a Received Locator Pair option.  Host B must
       decide whether it should send the R1 message to the IP source
       address of the R1 message, or send it to the potentially different
       Sender Locator in the Sent Locator Pair option in the I1 message.
       Once B has made this decision, it puts the addresses, in this
       example <B1, A2> in the IPv6 header as well as into a Sent Locator
       Pair option.

-- 
Pekka Savola                 "You each name yourselves king, yet the
Netcore Oy                    kingdom bleeds."
Systems. Networks. Security. -- George R.R. Martin: A Clash of Kings

Delivery-date: Fri, 31 Mar 2006 09:28:21 +0000
Date: Fri, 31 Mar 2006 12:28:09 +0300 (EEST)
From: Pekka Savola <[email-masked]>
To: [email-masked]
Subject: transparent addrsel policy adjustment for outbound TE

Hi,

Reading the extended shim design draft, in particular the discussion 
on app modifications to do SRV lookup triggered the following thought.

I'm pretty sure someone must have run this thought experiment before, 
so pointers would be useful if so.

When applications perform DNS lookups and get multiple responses, the 
_resolver libraries_ could, based on transparent (to the app) SRV 
lookups or policy database, "weigh" the getaddrinfo responses given to 
the applications.  That is, because the apps by default try the 
addresses in the order they get them from getaddrinfo, instead of 
returning the records in round-robin fashion, the resolver could very 
well return certain addresses first (e.g.,) 90% of the time, some 
others 10%.  (The obvious other address destination selection criteria 
should be applied first.)

This would not have any negative impact on the application as all the 
addresses would still be there but the ordering would just be modified 
based on preferences, though running transparent SRV lookups could 
incur delays etc. if it's not done in parallel.

This could be very effective means for outbound TE decisions without a 
need to touch applications at all.

This doesn't really help with inbound TE though.  (One could add 
similar function the site's authoritative DNS server, and unmodified 
resolvers might comply with that policy, but caching DNS servers would 
mess this up.)

One could imagine that a part of inbound TE (for sessions which 
originate at the site) could be handled with slightly similar source 
address selection policies, but this doesn't help with inbound TE for 
traffic originated from the Internet (but you could add the SRV 
records or whatever if you care about this).

-- 
Pekka Savola                 "You each name yourselves king, yet the
Netcore Oy                    kingdom bleeds."
Systems. Networks. Security. -- George R.R. Martin: A Clash of Kings

Delivery-date: Fri, 31 Mar 2006 09:12:51 +0000
Date: Fri, 31 Mar 2006 12:11:40 +0300 (EEST)
From: Pekka Savola <[email-masked]>
To: [email-masked]
Subject: Re: SHIM6 Working Group Last Call on draft-ietf-shim6-proto-04.txt

On Fri, 24 Mar 2006, Geoff Huston wrote:
>        This note starts the WG Last Call for comments on
>        draft-ietf-shim6-proto-04.txt, "Level 3 multi-homing shim
>        protocol".

I read the spec on the way back from the IETF.  In general, it was a 
rather fluent read.  What I most kept thinking through the read were 
security implications which weren't described or referred to from the 
body of the draft, but many of those were indeed mentioned in the 
security considerations.  Maybe some forward pointers could have 
helped.

Other than that, there were a couple of issues described below.

I think the most fundamental one was that I'm not sure whether the 
4-way handshake is useful or even necessary, given that we don't even 
want to establish the context before the session is already up (which 
would mitigate DoS concerns).  On the other hand, if we already need 
to prepare for an Extended Shim Design....

substantial
-----------

1) On IP multicast: it's not clear whether the text only applies to
Any-Source-Multicast, rather than also to SSM.  The IP addresses do matter
with SSM, as the hosts specify from which IP address they wish to receive
the stream (and the multicast routing then goes to that IP address's
designated router to set up the forwarding state).  With SSM, changing IP
addresses at the sender end would imply that all the receivers will need
to redo the MLD and PIM signalling.  With ASM, this happens automatically.

    While in theory it would be possible to apply the shim re-mapping of
    the IP address fields between ULIDs and locators, the fact that all
    the multicast receivers would need to know the mapping to perform,
    makes such an approach difficult in practice.  Thus it makes sense to
    have multicast ULPs operate directly on locators and not use the
    shim.  This is quite a natural fit for protocols which use RTP [14],
    since RTP already has an explicit identifier in the form of the SSRC
    field in the RTP headers.  Thus the actual IP address fields are not
    important to the application.

2) carrying the context tag opens the avenue for time-shifting attacks
(e.g., data insertion into a session) after an attacker has moved from
on-path to off-path ?

    o  Every control message of the shim6 protocol, past the context
       establishment, carry the context tag assigned to the particular
       context.  This implies that an attacker needs to discover that
       context tag before being able to spoof any shim6 control message.
       Such discovery probably requires to be along the path in order to
       be sniff the context tag value.  The result is that through this
       technique, the shim6 protocol is protected against off-path
       attackers.

3) The spec IMHO needs to say more about application framing impacts, as a
result of adding the shim to some packets (well, we have been rather vague
about this in the past, RFC4038 sec 5.4.2).  TCP-like protocols are not a
big problem because the stack can just adjust to whatever value is
appropriate, but protocol such as UDP where the user needs to specify this
are trickier.  You should probably describe the required modifications
to the implementations fragmentation handling code.

Triggered by this text, but there was also discussion in 11.1 as well as
section 15, but the more appropriate place for discussing it
should be a new section 11.2 ?

    Layering the fragmentation header above the multihoming shim makes
    reassembly robust in the case that there is broken multi-path routing
    which results in using different paths, hence potentially different
    source locators, for different fragments.  Thus, effectively the
    multihoming shim layer is placed between the IP endpoint sublayer,
    which handles fragmentation, reassembly, and IPsec, and the IP
    routing sublayer, which selects which next hop and interface to use
    for sending out packets.

4) There seem to be a number of minor attacks, and/or non-issues or residual
threats which might be good to spell out better in the security
considerations sections.

  4.1) The only hash input that an attacker could not affect is the secret
value, and even that is static.  As the responder nonce is basically
incremented based on the shim exchanges of the responder, the attacker can
predict that as well.  With this kind of information,
(after some tries, and possibly other ways of obtaining knowledge how long
the host has been up) a) the attacker should be able to figure out the
secret as well, and b) given that the attacker can choose so many bits in
the hash function input, it should be reasonable to find hash function
collisions as well (especially with weaker hash functions).

That said, both of these issues may be no-op's because the value of the
information protected is not very high.  But these considerations should be
spelled out.

  4.2) what about the scenario where the initiator wishes to establish shim6
sessions when the other party isn't (yet) willing to do so? (Think about
large content providers.)  What are the graceful mechanisms to achieve this?

  4.3) Following from 4.2), you really don't need the 4-way handshake under
normal circumstances.  When you wish to establish the shim6 state, the peer
has already proven its desire for genuine communication by opening the TCP
connection and using it for some time.  If it wanted to attack, it wouldn't
be using SHIM at all, just TCP.  So, you might be able to simplify the SHIM
exchange under normal situations.  On the other hand, the responders need
the mechanisms to deal with the cases where they don't want to establish
shim6 state yet (e.g., before the TCP connection is even established, or
whatever), or want to use a secure SHIM exchange variant.

  4.4) 47-bit context tags are the most troublesome issue to me.  You should
be more explicit what the effects of guessing (or knowing, from being on the
path) the tag are, 'cause disruption' is a bit vague.  The problem with
these is that there is that from the network security manager's perspective,
there is no way to do any kind of filtering on the received packets -- IP
addresses, port numbers, etc. lose their relevancy when protecting hosts
from the big bad world.  Some may even consider this a feature...

semi-substantial (not sure if these are real issues or not)
----------------

               ------------- R1 --------------->
        Left old context
        in ESTABLISHED

==> at this point, I kept thinking, "so.. isn't there any timeout for
contexts in ESTABLISHED state..?  Does it stay there forever?"

    Next the host verifies that the Responder Nonce is a recent one, and
    that the Responder Validator option matches the validator the host
    would have computed for the ULID, locators, responder nonce, and FII.

==> how do you define 'a recent one' ?

    If the host receives an ICMP error with "payload type unknown" (type
    4, code 1) and the included packet is the I1 message it just sent,
    then this is a more reliable indication that the peer ULID does not
    implement shim6.  Again, in this case, the host should remember to
    not try again to establish a context with that ULID.  Such negative
    caching should retained for at most ICMP_HOLDDOWN_TIME, which should
    be significantly longer than the previous case.

==> you should probably apply the same policy for certain other ICMP errors,
at least type 1 code 1 "Communication with destination administratively
prohibited", implying the protocol may be filtered at the firewall.

semi-editorial
--------------

    Next Header:   NO_NXT_HDR (59).

==> I'd make this 'Normally...' or the like, so that it would be clearer
that it might be OK for future extensions (if any) to also put in some other
value if they wanted to.

    The following options are defined for this message:

==> but this is not an exhaustive list.  Do you already specify somehow how
you will treat yet-to-be-specified options?  How do you parse the options
that are not on the lists?

    All of the TLV parameters have a length (including Type and Length
    fields) which is a multiple of 8 bytes.  When needed, padding MUST be
    added to the end of the parameter so that the total length becomes a
    multiple of 8 bytes.  This rule ensures proper alignment of data.  If
    padding is added, the Length field MUST NOT include the padding.  Any
    added padding bytes MUST be zeroed by the sender, and their values
    SHOULD NOT be checked by the receiver.

==> I think this specification of padding adding is sub-optimal: why do you
say "the Length field MUST NOT include the padding" at all?  Do you want to
cover the scenario where the length field says X, but there is Y (>>X)
amount of extra padding?  Specifying that padding up to 8 bytes can be
stated in much simpler terms.

7.3  Normal context establishment

    The normal context establishment consists of a 4 message exchange in
    the order of I1, R1, I2, R2 as can be seen in Figure 24.

         Initiator                          Responder

          IDLE                               IDLE
               ------------- I1 -------------->
          I1-SENT
               <------------ R1 ---------------
                                             IDLE
               ------------- I2 -------------->

==> at this point, I wondered, "how can the responder stay idle?  it needs
to remember _something_", so maybe a forward pointer to the security
discussion (i.e., a global secret value) could be useful.

    We assume that all the incoming packets that trigger the generation
    of an R1bis message contain a locator pair (in the address fields of
    the IPv6 header) and a Context Tag.

==> there is no need to "assume" anything, as this should already happen due
to the nature of shim6.  Reword e.g., "The locator pair is extraced from the
source and destination IP addresses" or whatever.

The Update Request message can include
    just a Locator List option, to convey the new set of locators (which
    requires a CGA signature option as well), just a Locator Preferences
    option, or both a new Locator List and new Locator Preferences.

==> The CGA signature options is only applicable w/ new locators when you
use non-HBA CGA addresses, right?

   Should there be no response, the retransmissions continue forever.
    The binary exponential backoff stops at MAX_UPDATE_TIMEOUT.

==> "forever" is a rather harsh way to say that.  "until the shim state is
discarded" or something?

   I1_RETRIES_MAX

==> you didn't specify the constant...

editorial
---------

                    Level 3 multihoming shim protocol

==> how about "Layer 3 Multihoming Shim Protocol for IPv6" ?

    The SHIM6 protocol is a layer 3 shim for providing locator agility
    below the transport protocols, so that multihoming can be provided
    for IPv6 with failover and load sharing properties, without assuming
    that a multihomed site will have a provider independent IPv6 address
    prefix which is announced in the global IPv6 routing table.

==> the latter part is not important for this spec, so almost the same thing
would be to say:

    The SHIM6 protocol is a layer 3 shim for providing locator agility
    below the transport protocols, so that multihoming can be provided
    for IPv6 with failover and load sharing properties, without assuming
    that a multihomed site will have a globally routable provider
    independent IPv6 address prefix.

...

    The reachability detection and failure detection, including how a new

==> remove the first "detection" as unnecessary.

    o  Do not require an extra roundtrip up front to setup shim specific
       state.

==> remove "Do" for style consistency?

But it is not a goal to try to make communication
    survive a renumbering event (which causes all the locators of a host
    to change to a new set of locators).  This proposal does not attempt
    to solve the, perhaps related, problem of host mobility.

==> s/proposal/proposal also/ ?

    Worst case we could end up with two separate hosts using the same
    ULID while both of them are communicating with the same host.

==> s/Worst/In the worst/

    The proposal uses an multihoming shim layer within the IP layer,

==> s/an/a/

    2), there is a choice whether the shim applies inside the tunnel or
    outside the tunnel, which effects the location of the shim6 header.

==> s/effects/affects/

    The responder can choose exactly what input uses to compute the
    validator,

==> s/uses/is used/

7.1  Uniqness of Context Tags

==> s/Uniq/Unique/

         Host A                             Host B

          IDLE                               IDLE
               -\
          I1-SENT---\
                     ---\                  /---
                         --- I1 ---\   /---  I1-SENT
                                    ---\
                        /--- I1 ---/    ---\
                   /---                     -->
               <---

==> could this look nicer if the diagrams were drawn with more
straight lines, like:

                 IDLE                          IDLE
                      -----------.
                                  \      .-----
                                   I1   I1
                                     \ /
                                      X
                                     / \
                                        '----->
...

    If the host receives an ICMP error with "payload type unknown" (type
    4, code 1) and the included packet is the I1 message it just sent,

==> in a couple of places, the error name is actually "Unrecognized Next
Header type encountered".

    If any of the above verifications fails, then the host silently
    discard the message and it has completed the I2 processing.

==> s/discard/discards/

    o  The HBA technique [7] for verifying the locators to prevent an
       attacker from redirecting the packet stream to somewhere else.

    o  Requiring a Reachability Probe+Reply before a new locator is used

==> if you want consistent style, I'd s/for verifying/verifies/,
s/Requiring/Requires/ (not sure about this one...)

    specification.  Special thanks to the careful comments from Geoff
    Houston and Shinta Sugimoto on earlier versions of this draft.

==> s/Houston/Huston/

Appendix C.  Change Log

==> add a note to the rfc-editor to remove this upon publication (it would
actually be best if this was the last appendix, so the appendix numbers
wouldn't get shuffled when this gets removed.)

-- 
Pekka Savola                 "You each name yourselves king, yet the
Netcore Oy                    kingdom bleeds."
Systems. Networks. Security. -- George R.R. Martin: A Clash of Kings

Delivery-date: Thu, 30 Mar 2006 20:06:40 +0000
Date: Thu, 30 Mar 2006 15:05:19 -0500 (EST)
From: Scott Leibrand <[email-masked]>
To: marcelo bagnulo braun <[email-masked]>
cc: [email-masked]
Subject: Re: Extending shim6 to use multiple locator pairs simultaneously

On 03/28/06 at 9:41am +0300, marcelo bagnulo braun <[email-masked]> wro=
te:

> El 27/03/2006, a las 22:28, Scott Leibrand escribi=F3:
>
> > Would the impact on the failure detection mechanism be eliminated by
> > having TCP initiate a new forked context for each additional path it
> > wanted to use?
>
> yes this seems a clever idea indeed
>
> >   That way, each context would consider only a single path as "active"
> > at any one time.  Upon my initial re-read of
> > draft-ietf-shim6-reach-detect-01.txt, that seems it would be
> > sufficient, but you'd likely know better than I if there are any other
> > issues to worry about...
>
> well, forked contexts behave like two separate contexts.
> This basically means that there is not a conscience that they are both
> being used for sending data for the same communication. This means for
> instance that both contexts may end-up using the same locator pair
> after a failure i guess (I am not sure how would a forked context would
> rehome a communication after a failure in the selected locator pair has
> been detected, but i guess that the default behaviour would be to
> explore and find an alternative path, in which case, two different
> forked context may end up using the same locator pair). this basically
> means that we would end up running the failure detection protocol over
> the same locator pair, for two forked context that are actually being
> used for the same communication, which seems to be at least suboptimal.
> In addition, after a failure, the desired behaviour would probably be
> that the different forked contexts are rehomed to different locator
> pairs in order to still benefit from multipath. However, since the shim
> has no conscience that the multipath is being used, it won't be able to
> actually take this into account when selecting the new locator pairs.

Yes, I agree that in a failure scenario you have the possibility of
collapsing both contexts onto the same path.  Since such a situation
provides little benefit, it would be in the ULP's interest to be notified
of a failure & rehoming event, so that it could stop using the forked
context or possibly tell the shim to try to find another locator pair that
works and isn't already in use.

> Bottom line is, that context forking is a tool to be used by ulp that
> want to have control of the locators. So, because of that, ULP need to
> be more involved in the decision and this does not work so
> automatically, especially when decision such as selecting alternative
> locators are required.

Yes, I agree.  The shim is responsible for restoring reachability, and the
ULP is responsible for telling the shim if it wants anything done
differently to meet the ULP's specific needs.  As long as the API supports
that, we're good.

> However, this may well be a starting point for this and optimizations
> can be worked out for this case. For instance, to include this type of
> considerations in the locator selection process,

I would envision the ULP telling the shim about its locator pair
preferences for each context, and the shim simply taking that at face
value (and not try to infer anything about the "best" locator pair from
the other forked context).

> or as Iljitsch suggested a long time ago, to allow that a single
> keepalive/probe message is used to test all the contexts that are using
> the same locator pair.

That might be appropriate, but it would seem that you'd want to keep the
contexts conceptually separate, and just let implementors take advantage
of overlap as long as it exists.  This reminds me of shared memory, where
multiple processes can have read access to the same chunk of code in
memory, but if one of them tries to write to it, the OS makes a new copy
to allow that process to have its own copy to play with.

-Scott

Delivery-date: Thu, 30 Mar 2006 18:43:35 +0000
Date: Thu, 30 Mar 2006 13:42:57 -0500 (EST)
From: Scott Leibrand <[email-masked]>
To: marcelo bagnulo braun <[email-masked]>
cc: [email-masked]
Subject: Re: Shim6 failure recovery after garbage collection

On 03/28/06 at 9:06am +0300, marcelo bagnulo braun <[email-masked]> wro=
te:

> El 27/03/2006, a las 23:54, Scott Leibrand escribi=F3:
>
> > I expressed the opinion that it would be nice if a heavily loaded
> > server could aggressively garbage collect shim6 state after initial
> > context establishment, and rely on the client to perform failure and
> > reachability detection and initiate context re-establishment if a
> > failure is detected.
>
> This point was proposed by Iljitsch in the Manchester design team
> meeting and the protocol was kind of designed to support this feature

Good, good.

> > I was also concerned whether REAP (as defined in
> > draft-ietf-shim6-failure-detection-03.txt) would require frequent
> > context re-establishment if one side were to garbage collect.
> > However, if I assume that TCP ACKs are considered "payload" packets in
> > the context of resetting the REAP keepalive timer, then that is not an
> > issue, because that timer will only be started upon sending a TCP data
> > packet, and will be reset by that data packet's ACK.
>
> yes, this i my understanding. TCP ACKs do reset the REAP timer
>
> > In any event, I think that draft-ietf-shim6-failure-detection-03.txt
> > needs to better define "payload packet" to differentiate between data
> > packets (which IMO should both set and reset the keepalive timer) and
> > ACK packets (which IMO should only reset the keepalive timer, not
> > start it, or it will cause unnecessary context re-establishment).
>
> I am not sure about this point...
> I mean, as i understand it, payload packets for the shim are any packet
> generated by any ULP, it doesn't matter if these contain ULP signalling
> information or actual user data information. As far as the shim is
> concerned any packet generated by a layer that is above the shim are
> payload packets.
> I mean, i don't think the shim should try to understand ULP.

I can see the virtue in that approach, particularly in terms of layer
separation...

> So, for me a TCP ACK is just another payload packet that is processed
> as any other, i.e. set and reset timer
>
> Do you see any problems with this behaviour?

Yes, I do.  Say you have a one-way data transfer, from a busy server that
aggressively garbage-collects shim6 context to a multihomed shim6 client.
The client at some point establishes shim6 context, which the server
promptly discards.  The client requests a bunch of data, which the server
sends.  The last data packet from the server clears the keepalive timer,
but it is restarted again when the client ACKs the last packet.  Since the
server has no more data to send, the keepalive timer expires.  This prompts
the client to send a shim6 keepalive, which in turn generates an R1bis
message and re-establishes the context.  IMO this unnecessary context
re-establishment could pose a problem for sites who can't afford any
additional state on their servers, especially if it happens repeatedly and
prevents the server from garbage-collecting the state a second time.

> OTOH, it may make sense to explicitly state in the draft what a payload
> packet is...

Yeah, I think we need to at least talk about it there, as it seemed rather
ambiguous to me.

-Scott

Delivery-date: Thu, 30 Mar 2006 16:47:07 +0000
Date: Thu, 30 Mar 2006 11:46:49 -0500 (EST)
From: Scott Leibrand <[email-masked]>
To: marcelo bagnulo braun <[email-masked]>
cc: [email-masked]
Subject: Re: Shim6 proxies

On 03/28/06 at 8:57am +0300, marcelo bagnulo braun <[email-masked]> wro=
te:

> Hi Scott,
>
> This topic has been discussed several times on this ml.

Cool.

> My understanding of this topic is that there can be basically two
> flavors of proxy-like devices:
> - on one hand, a proxy that basically performs locator selection on
> behalf of the host. In this case, the security functions are still end
> to end, but the locator selection is performed by another device,
> likely a router, that has more information about available routes
> and/or is managed by the network admins. This is basically what is
> described in draft-nordmark-shim6-esd-00.

That's not the impression I got.  It looks to me like
draft-nordmark-shim6-esd-00 provides similar functionality, in that the
routers can provide hints to the shim6 hosts that certain locators would
be preferred, but they don't actually perform the locator selection in any
direct manner...

> This seems an architecturally clean approach, since the locator
> selection function can be cleanly off loaded from the host itself, as
> long as the security remains e2e.

Perhaps, but I'm not sure what benefits a locator-selecting proxy would
have over simply rewriting the source locators in each direction, thereby
providing hints to the shim6-capable hosts of what they should do.

>  - on the other hand, there is the full proxy. this was discussed
> several times (most of times by Iljitsch). In this case, there is a
> proxy that establishes shim contexts on behalf of a non-shim capable
> node. In this case, the proxy performs all the functions of the shim.

This is more what I was thinking.

> Now, this seems to be an useful tool, especially for facilitating the
> deployment and providing some benefits of the shim to legacy hosts.
> However there seems to be certain questions that need to be answered in
> this scenario. For instance, does the host has one or many addresses? i
> mean is the host aware that it has multiple addresses or not? depending
> on the assumption here, different problems appear. Let's start by this
> point... what did you have in mind for this?

If you're going to do a full proxy, you have to go all the way IMO.  That
means that for whatever locators the end hosts use, whether they have
multiple locators are not, have to be assumed to be fixed for the session,
just like ULIDs.  The shim6 proxy would then intercept all shim6 control
traffic to that IP, and perform the shim functions on behalf of the host.
It would have a bunch of its own locators, which would make up the locator
set.  It could also include the ULID as one of those locators, and
intercept traffic to that IP with shim6 headers, or I suppose it could
treat the host's IP as a non-routable identifier for shim6 purposes and
just use its own locators in the locator set.  Either way, the proxy would
process all shim6-tagged traffic for the host, de-shim it as normal, and
then pass the traffic along to the host's IP instead of passing it up to
the ULP.

Does that sound reasonable?  What problems do you see with such a setup?
The most obvious one is that if you use the hosts's actual IP as one of
the locators, you risk missing some shim6-tagged traffic if there is more
than one route to the host.  That could be alleviated by only using local
locators in the locator set, and treating the host's IP as an unroutable
locator in shim6.  If you did that, I guess there's the same problem I
complained about in Erik's esd draft, which is that the host at the other
end is forced to shim6-tag all packets and maintain shim6 state for the
duration of the session.

-Scott

> El 27/03/2006, a las 22:54, Scott Leibrand escribi=F3:
>
> > Has there been any consideration of the interaction of the proposed
> > shim6 protocol with a potential shim6 proxy?  I'm thinking it would be
> > quite useful to be able to place a device on-path near one end of a
> > TCP connection (i.e. on the default router), and have the device
> > perform shim6 functions (and possibly other multihoming duties) on
> > behalf of one or more host(s) behind it.
> >
> > -Scott
> >

Delivery-date: Thu, 30 Mar 2006 16:32:02 +0000
Date: Thu, 30 Mar 2006 11:30:53 -0500 (EST)
From: Scott Leibrand <[email-masked]>
To: Erik Nordmark <[email-masked]>
cc: shim6 <[email-masked]>
Subject: Re: [Fwd: I-D ACTION:draft-nordmark-shim6-esd-00.txt]

Erik,

I just finally got time to read this draft and your discussion about it.
I have some comments relating to the source locator rewriting support,
particularly as it relates to minimizing the need for non-multihomed (or
PI-multihomed)  hosts to keep shim context.

The draft says:

  The support for this rewriting consists of:

   o  Introducing a new Sent Locator Pair option, which the receiver or
      a packet can use to see how the routers rewrote the locators in a
      packet sent towards it.  The receiver also "echos" this
      information to its peer using the next option.

   o  Introducing a new Received Locator Pair option, which is a echo of
      the content of a received Sent Locator Pair option.

Which I think is reasonable and a beneficial extension...  But it also
says:

   o  The hosts SHOULD include a shim6 extension header for ULP payloads
      where the sending shim does not rewrite the locators, in order to
      allow the routers to rewrite the locators.

   o  To maximize the utility of the locator rewriting, we should avoid
      the use of deferred context establishment.  If the first packet
      sent is an I1 packet, and the above SHOULD is honored, then the
      routers are free to rewrite the locators in all the packets that
      are exchanged between a pair of shim6 hosts.  (If we are also
      doing complete identifier/locator split, then there is no deferred
      context establishment, thus this doesn't add any additional
      costs.)

I disagree with this.  If a rewriting-capable host, which may or may not
have a rewriting router on the path, started up communication observing
these SHOULDs, then the host it's talking to, which may be a very busy
non-multihomed (or PI-multihomed) server, would have to keep full shim6
context and would be unable to discard it during communication due to the
presence of shim6 extension headers on every packet.  This would be true
even if no source-rewriting routers were deployed, and no one gained any
benefit from the capability.

IMO a better approach would be to continue using deferred context
establishment (except in the identifier/locator split case, of course),
and initially only put the Sent Locator Pair option packets that already
have shim6 headers (shim6 control packets and any packets sent with
non-ULID locators).  Then, *if* source locator rewriting is done by a
router on *those* packets, you can start putting shim6 extension headers
on all ULP payload packets.

My reasoning for wanting to make the changes above is that IMO there will
be a lot more users who want to minimize shim6 state on their servers than
who want to do shim6 rewriting in their routers, at least initially.  If
we start requiring all packets between shim6-capable hosts to have shim6
extension headers, that requires that all shim6-capable hosts keep full
shim6 state for all communications with other shim6-capable hosts, which
in turn means that operators of large servers will turn off shim6
entirely, and it will be of very limited utility due to very limited
deployment.

-Scott

On 03/01/06 at 11:45am -0800, Erik Nordmark <[email-masked]> wrote:

>
>
> -------- Original Message --------
> Subject: I-D ACTION:draft-nordmark-shim6-esd-00.txt
> Date: Tue, 28 Feb 2006 15:50:02 -0500
> From: [email-masked]
> Reply-To: [email-masked]
> To: [email-masked]
>
> A New Internet-Draft is available from the on-line Internet-Drafts
> directories.
>
>
> 	Title		: Extended Shim6 Design for ID/loc split and
>                            Traffic Engineering
> 	Author(s)	: E. Nordmark
> 	Filename	: draft-nordmark-shim6-esd-00.txt
> 	Pages		: 25
> 	Date		: 2006-2-28
>
>     The Shim6 protocol provides for locator agility while satisfying the
>     'first, do no harm' security requirements.  This document outlines
>     some extensions to Shim6 that in addition provides complete
>     separation between identifiers and locators, and allows routers to
>     rewrite the locators in the shim6 packets as a way to provide traffic
>     engineering information to the hosts.
>
>     The document also outlines a simple extension to allow shim6, with a
>     CGA upper-layer ID, to operate using IPv4 addresses as locators.
>
>     The purpose of this outline is to stimulate discussions.
>
> A URL for this Internet-Draft is:
> http://www.ietf.org/internet-drafts/draft-nordmark-shim6-esd-00.txt
>
> To remove yourself from the I-D Announcement list, send a message to
> [email-masked] with the word unsubscribe in the body of
> the message.
> You can also visit https://www1.ietf.org/mailman/listinfo/I-D-announce
> to change your subscription settings.
>
>
> Internet-Drafts are also available by anonymous FTP. Login with the username
> "anonymous" and a password of your e-mail address. After logging in,
> type "cd internet-drafts" and then
> 	"get draft-nordmark-shim6-esd-00.txt".
>
> A list of Internet-Drafts directories can be found in
> http://www.ietf.org/shadow.html
> or ftp://ftp.ietf.org/ietf/1shadow-sites.txt
>
>
> Internet-Drafts can also be obtained by e-mail.
>
> Send a message to:
> 	[email-masked].
> In the body type:
> 	"FILE /internet-drafts/draft-nordmark-shim6-esd-00.txt".
>
> NOTE:	The mail server at ietf.org can return the document in
> 	MIME-encoded form by using the "mpack" utility.  To use this
> 	feature, insert the command "ENCODING mime" before the "FILE"
> 	command.  To decode the response(s), you will need "munpack" or
> 	a MIME-compliant mail reader.  Different MIME-compliant mail readers
> 	exhibit different behavior, especially when dealing with
> 	"multipart" MIME messages (i.e. documents which have been split
> 	up into multiple messages), so check your local documentation on
> 	how to manipulate these messages.
>
>
> Below is the data which will enable a MIME compliant mail reader
> implementation to automatically retrieve the ASCII version of the
> Internet-Draft.
>
>

Delivery-date: Tue, 28 Mar 2006 08:38:01 +0000
Date: Tue, 28 Mar 2006 17:37:09 +0900
From: Shinta Sugimoto <[email-masked]>
To: marcelo bagnulo braun <[email-masked]>
Subject: Re: mip6 over shim6 (was: Re: shim - mip interaction)
Cc: "Henderson, Thomas R" <[email-masked]>, "shim6-wg" <[email-masked]>

Hi Marcelo,

Please find my comments inline.  Note that I am not trying to push
MIPv6 over SHIM6, but trying to clarify if it really does make sense
or not :)

On Mon, 27 Mar 2006 13:18:32 +0300
marcelo bagnulo braun <[email-masked]> wrote:

> Hi Shinta,
> 
> El 24/03/2006, a las 15:52, Shinta Sugimoto escribis:
> 
> > Hi,
> >
> > In relation to the discussion on interaction between SHIM6
> > and MIPv6, let me try to address how should they work cooperatively.
> > There are mainly two scenarios: SHIM6 runs over MIPv6, and MIPv6
> > runs over SHIM6.  The former case is well addressed in
> > draft-bagnulo-shim6-mip-00.txt, in which MIPv6 home address(es) are
> > treated as locator from SHIM6 perspective.  In the latter case,
> > it should be possible to run MIPv6 over SHIM6 keeping MIPv6
> > completely unaware of underlying shim layer.
> 
> yes, but i think that it would make sense to define whether the shim is 
> above or below the shim once and for all, so that we figure out how the 
> stack is actually ordered, right?
> 
> Personally i think that at this point it makes sense to lay shim above 
> mip, in order to support the most interesting scenarios.

I agree that SHIM6 over MIPv6 is certainly an interesting scenario
and its technically feasible.  On the other hand, I would also like to
understand what about the case in MIPv6 over SHIM6.  When it comes to
feasibility, I understand how it may work.  Now I am trying to see
if its really meaningful/useful, but after reading your response
and some exploration, I came to think that SHIM6 over MIPv6 is more
preferable and provides more benefits than the other way around.

> >  Let me expand a
> > little bit more on the latter case:
> >
> > 1. Shimming HA-MN bi-directional tunnel
> >
> > In order to do this, MN and HA ought to establish SHIM6 context
> > with ULID pair of MN's CoA and HA's address.  MIPv6 is completely
> > unaware of the presence of shim layer and it just operates normally.
> > The packet which appears on the wire should look like below.  The
> > HA will first demux the shimmed packet based on the associated
> > context.  Decapsulation of IP-in-IP tunneled packet will follow.
> >
> 
> this case is useful indeed.
> However, in this particular case, i am not sure that this is strictly 
> the case where MIP lays on top of the shim.
> I mean, in this case we have tunnel interface, so basically we have the 
> IP layer twice. So as i see it, shim can also be twice.

Yes.  But as the ULID pair does not match in the first try (as the context
database is looked up with ULID<CoA_MN, HA>).

> The particular 
> case that you are considering is when the shim of the IP layer of the 
> tunnel interface is working. So it is below the MIP, since the MIP is 
> associated to the other IP layer.

Yes.

> We can see it from a different p.o.v. if we consider that it would 
> possible in this case to also have a shimed session between the CN and 
> the MN, so and additional shim header associated to the upper IP layer 
> would be also in place.

Right.

> 
> So, my point in this case is:
> - SHIM is above MIP
> - each IP layer may have a shim sublayer within
> - in the case of tunneled interfaces, we have two IP layers and for 
> each of these IP layers, we can have a shim layer.
> - however, in all the cases, inside a given IP layer, shim is always 
> above MIP
> 
> makes sense?

Yes, finally.

> > shimmed packet from MN to CN:
> >
> > 	[Outer IPv6 Header]
> > 		src=L(MN), dst=L(HA)
> > 	[SHIM6 Ext Header]
> > 	[Inner IPv6 Header]
> > 		src=MN_HoA, dst=CN
> > 	[Payload]
> >
> >
> > 2a. Shimming RO path between MN and CN
> 
> > In MIPv6, it is possible for MN and CN to optimize the routing
> > path between the nodes by utilizing IPv6 extension headers
> > (Home Address Destination Option and Routing Header Type 2)
> > specific for MIPv6.  Semantically the route optimized path can
> > be regarded as a virtual tunnel between the nodes although the
> > mechanism of delivering IP packet is different in each direction.
> > ULID pair should be MN's CoA and CN's address.
> >
> 
> right this could be possible imho, but i am not sure if it would 
> provide additional benefits than the case where shim is above mip.
> I mean, in this case, you can only provide fault tolerance against 
> failures in the currently used CoA.

You are right.  The case I described above provides fault tolerance
against failures which may occur between the path (CN and MN's CoA)
in a limited way.

In comparison to the SHIM base RO mode (in case of SHIM6 over MIPv6),
the problem is that application referral to identify the MN (HoA)
has nothing to do with the ULID stored in the associated context.
It may not be easy for application to access the context via SHIM6
API.  So, establishing context based on ULID<MN's CoA, CN> may not
be useful.

> failure in the path with the HoA 
> are sill affecting the communication both in the BT mode and the RO 
> mode (because of the hoT/HoTI exchange)

Right.

> So, even if i can see that some additional fault tolerance is achieved, 
> i think that the case where shim is above mip provides more benefits...

Yes, understood.

> > shimmed packet from MN to CN:
> >
> > 	[IPv6 Header]
> > 		src=L(MN), dst=L(CN)
> > 	[SHIM6 Ext Header]
> > 	[DSTOPT]
> > 		HAO=MN_HoA
> >
> > shimmed packet from CN to MN:
> >
> > 	[IPv6 Header]
> > 		src=L(CN), dst=L(MN)
> > 	[SHIM6 Ext Header]
> > 	[RTHDR2]
> > 		addr=MN_HoA
> >
> >
> > 2b. Shimming RO path between MN and CN
> >
> > If the CN happens to be a mobile too (no matter if it is served
> > by the same HA of the MN), CN and MN may establish RO path
> > in slightly more complicated way.  In order to avoid confusion
> > let MN and CN be denoted  as MN0 and MN1, respectively.  Assuming
> > that MN0 and MN1 mutually established MIPv6 corresponding binding
> > (MN0 stores binding of MN1, and MN1 stores binding of MN0),
> > in order to shim RO-path between MN0 and MN1, they need to
> > establish a SHIM6 context based on ULID pair of their CoAs.
> >
> > shimmed packet from MN0 to MN1:
> >
> > 	[IPv6 Header]
> > 		src=L(MN0), dst=L(MN1)
> > 	[SHIM6 Ext Header]
> > 	[RTHDR2]
> > 		addr=MN1_HoA
> > 	[DSTOPT]
> > 		HAO=MN0_HoA
> >
> > Note that packet format provided in above scenarios 2a and 2b
> > are slightly contradict with what is stated in Section 4.6 of
> > shim6-proto-04 saying that shim6 extension header should
> > appear after any routing header.  But RTHDR2 can be treated as
> > an exceptional case as it is meant to be processed only by the
> > final destination, IMO.
> >
> > There are some requirement and issues in above scenarios:
> >
> > First of all, it should be noted that the above scenarios are
> > directly related to the work being done in the Monami6 WG.
> 
> I am not sure why do you think so... MONAMI is chartered to do very 
> specific work, in particluar related to multiple CoA registration and 
> flow policy
> No work is chartered for fault tolerance, failure detection, multiple 
> HoA support, which is what is being provided here

Ok, I see.

> 
> > Duplication of the work should be avoided from IETF perspective,
> > but at first the mechanism how it (MIPv6 over SHIM6) works
> > should be clearly understood, IMO.
> >
> 
> agree
> 
> > One requirement in above scenarios is that MIPv6 CoA should be CGA.
> > Otherwise there would be another security mechanism required to
> > prevent reflection attacks (as we assume that MN changes CoA
> > frequently).  In the scenario where SHIM6 runs over MIPv6,
> > HBA would help to assure legitimate locator update among the
> > the locator set (=MIPv6 HoAs).  But not in the scenarios described
> > above.
> >
> 
> agree
> however, there seem to be the underlying assumption that in the case 
> that shim is layered above mip, the shim can not be exposed to the 
> CoAs. While this make sense, there is not yet defined if this is the 
> best alternative. I mean, allowwing the shim to be aware of the coas 
> may provide additional benefits, like some form of shim based ro mode

Right.  I came to understand what you mean by shim based RO mode.
In SHIM6 over MIPv6 scenario, MN and CN can just establish ULID-pair
context and which can be used for RO by using Lp(MN) as MN's CoA.
I agree that it's efficient and presumably useful.

> > Besides, I see an issue how the ULID pair can be determined by
> > SHIM6 who has no knowledge on MIPv6 binding information.  As MN's
> > local address namely CoA is unpredictable, it may be hard for SHIM6
> > to determine when and with which address should the ULID pair
> > context should be established/updated in order to shimming the MIPv6
> > tunnel.
> >
> 
> I am not sure i understand your point here: are you considering ULID 
> selection or locator selection? are you considering the case when a 
> node is selecting its owm addresses or when it is selecting the peer's 
> addresses?

I meant selection of the ULID of MN.  So, on CN side, Lp(peer).

> 
> All these cases are quite different, but there is some support for this 
> in RFC3484 (for ULID selection), and the flag field in the Locator 
> preference option has a temporary flag to tell the peer that a given 
> address is a CoA.
> Additional guidance when selection local locators may be required though

That makes sense.  I will look into more on Locator Preference Option
in the proto-04 document.  Thanks for your clarification.

Regards,
Shinta

Delivery-date: Tue, 28 Mar 2006 06:41:45 +0000
Cc: [email-masked]
From: marcelo bagnulo braun <[email-masked]>
Subject: Re: Extending shim6 to use multiple locator pairs simultaneously
Date: Tue, 28 Mar 2006 09:41:41 +0300
To: Scott Leibrand <[email-masked]>

Hi Scott,

El 27/03/2006, a las 22:28, Scott Leibrand escribi=F3:

> On 03/21/06 at 11:25am -0500, Scott Leibrand <[email-masked]>=20=

> wrote:
>
>> On 03/21/06 at 10:08am -0600, marcelo bagnulo braun=20
>> <[email-masked]> wrote:
>>
>>> El 20/03/2006, a las 22:51, Scott Leibrand escribi=F3:
>>
>>> i guess that we need to understand the impact of this in the shim6=20=

>>> base
>>> protocol and the failure detection mechanism, since both of them are
>>> currently designed to use a single locator pair to exchange traffic.
>>> I don't think that there would be much trouble with supporting this,
>>> though
>>
>> Yes, definitely.  I've read the I-D's, but I should probably re-read=20=

>> them
>> with this in mind and see if I can better understand the impact...
>
> Marcelo,
>
> Would the impact on the failure detection mechanism be eliminated by
> having TCP initiate a new forked context for each additional path it
> wanted to use?

yes this seems a clever idea indeed

>   That way, each context would consider only a single path
> as "active" at any one time.  Upon my initial re-read of
> draft-ietf-shim6-reach-detect-01.txt, that seems it would be=20
> sufficient,
> but you'd likely know better than I if there are any other issues to=20=

> worry
> about...
>

well, forked contexts behave like two separate contexts.
This basically means that there is not a conscience that they are both=20=

being used for sending data for the same communication. This means for=20=

instance that both contexts may end-up using the same locator pair=20
after a failure i guess (I am not sure how would a forked context would=20=

rehome a communication after a failure in the selected locator pair has=20=

been detected, but i guess that the default behaviour would be to=20
explore and find an alternative path, in which case, two different=20
forked context may end up using the same locator pair). this basically=20=

means that we would end up running the failure detection protocol over=20=

the same locator pair, for two forked context that are actually being=20
used for the same communication, which seems to be at least suboptimal.
In addition, after a failure, the desired behaviour would probably be=20
that the different forked contexts are rehomed to different loccator=20
pairs in order to still benefit from multipath. However, since the shim=20=

has no conscience that the multipath is being used, it won't be able to=20=

actually take this into account when selecting the new locator pairs.

Bottom line is, that context forking is a tool to be used by ulp that=20
want to have control of the locators. So, because of that, ULP need to=20=

be more involved in the decision and this does not work so=20
automatically, especially when decision such as selecting alternative=20
locators are required.
However, this may well be a starting point for this and optimizations=20
can be worked out for this case. For instance, to include this type of=20=

considerations in the locator selection process, or as Iljitsch=20
suggested a long time ago, to allow that a single keepalive/probe=20
message is used to test all the contexts that are using the same=20
locator pair.

Regards, marcelo

> Thanks,
> Scott
>

Delivery-date: Tue, 28 Mar 2006 06:06:17 +0000
Cc: [email-masked]
From: marcelo bagnulo braun <[email-masked]>
Subject: Re: Shim6 failure recovery after garbage collection
Date: Tue, 28 Mar 2006 09:06:17 +0300
To: Scott Leibrand <[email-masked]>

Hi Scott,

El 27/03/2006, a las 23:54, Scott Leibrand escribi=F3:

> In a private discussion at IETF in Dallas, I was discussing with=20
> someone
> the impact on a content provider's server of implementing shim6.  I
> expressed the opinion that it would be nice if a heavily loaded server
> could aggressively garbage collect shim6 state after initial context
> establishment, and rely on the client to perform failure and=20
> reachability
> detection and initiate context re-establishment if a failure is=20
> detected.
>

Exactly...

This point was proposed by Iljitsch in the Manchester design team=20
meeting and the protocol was kind of designed to support this feature

> If such behavior is supported by the protocol, we're much more likely=20=

> to
> be able to convince potential implementors to turn on basic shim6=20
> support
> by default, with the understanding that implementors can aggressively
> discard shim6 context state (and not initiate shim6 context=20
> negotiation)
> if the implementation doesn't have multiple locators or otherwise=20
> doesn't
> need the capabilities shim6 provides.  This would (properly, IMO) push=20=

> the
> responsibility for context tracking, failure detection and =
reachability
> exploration to the multihomed host that stands to benefit most from=20
> shim6
> and wants to run it in the first place.
>
> My understanding after re-reading of the protocol, particularly the
> R1bis/I2bis message exchange sections, is that context =
re-establishment
> can occur (within approx. 2 RTTs of failure detection) even if the
> original ULID-ULID connection has failed and one side has discarded =
the
> context state.  Does that match others' understanding of the process?
>

exactly

> I was also concerned whether REAP (as defined in
> draft-ietf-shim6-failure-detection-03.txt) would require frequent=20
> context
> re-establishment if one side were to garbage collect.  However, if I
> assume that TCP ACKs are considered "payload" packets in the context =
of
> resetting the REAP keepalive timer, then that is not an issue, because
> that timer will only be started upon sending a TCP data packet, and=20
> will
> be reset by that data packet's ACK.
>

yes, this i my understanding. TCP ACKs do reset the REAP timer

> In any event, I think that draft-ietf-shim6-failure-detection-03.txt=20=

> needs
> to better define "payload packet" to differentiate between data =
packets
> (which IMO should both set and reset the keepalive timer) and ACK=20
> packets
> (which IMO should only reset the keepalive timer, not start it, or it=20=

> will
> cause unnecessary context re-establishment).
>

I am not sure about this point...
I mean, as i understand it, payload packets for the shim are any packet=20=

generated by any ULP, it doesn't matter if these contain ULP signalling=20=

information or actual user data information. As far as the shim is=20
concerned any packet generated by a layer that is above the shim are=20
payload packets.
I mean, i don't think the shim should try to understand ULP.

So, for me a TCP ACK is just another payload packet that is processed=20
as any other, i.e. set and reset timer

Do you see any problems with this behaviour?

OTOH, it may make sense to explicitly state in the draft what a payload=20=

packet is...

Regards, marcelo

> -Scott
>

Delivery-date: Tue, 28 Mar 2006 05:57:55 +0000
Cc: [email-masked]
From: marcelo bagnulo braun <[email-masked]>
Subject: Re: Shim6 proxies
Date: Tue, 28 Mar 2006 08:57:06 +0300
To: Scott Leibrand <[email-masked]>

Hi Scott,

This topic has been discussed several times on this ml.

My understanding of this topic is that there can be basically two=20
flavors of proxy-like devices:
- on one hand, a proxy that basically performs locator selection on=20
behalf of the host. In this case, the security functions are still end=20=

to end, but the locator selection is performed by another device,=20
likely a router, that has more information about available routes=20
and/or is managed by the network admins. This is basically what is=20
described in draft-nordmark-shim6-esd-00. This seems an architecturally=20=

clean approach, since the locator selection function can be cleanly off=20=

loaded from the host itself, as long as the security remains e2e.
- on the other hand, there is the full proxy. this was discussed=20
several times (most of times by Iljitsch). In this case, there is a=20
proxy that establishes shim contexts on behalf of a non-shim capable=20
node. In this case, the proxy performs all the functions of the   shim.=20=

Now, this seems to be an useful tool, especially for facilitating the=20
deployment and providing some benefits of the shim to legacy hosts.=20
However there seems to be certain questions that need to be answered in=20=

this scenario. For instance, does the host has one or many addresses? i=20=

mean is the host aware that it has multiple addresses or not? depending=20=

on the assumption here, different problems appear. Let's start by this=20=

point... what did you have in mind for this?

Regards, marcelo

El 27/03/2006, a las 22:54, Scott Leibrand escribi=F3:

> Has there been any consideration of the interaction of the proposed=20
> shim6
> protocol with a potential shim6 proxy?  I'm thinking it would be quite
> useful to be able to place a device on-path near one end of a TCP
> connection (i.e. on the default router), and have the device perform=20=

> shim6
> functions (and possibly other multihoming duties) on behalf of one or=20=

> more
> host(s) behind it.
>
> -Scott
>

Delivery-date: Mon, 27 Mar 2006 20:55:08 +0000
Date: Mon, 27 Mar 2006 15:54:33 -0500 (EST)
From: Scott Leibrand <[email-masked]>
To: [email-masked]
Subject: Shim6 failure recovery after garbage collection

In a private discussion at IETF in Dallas, I was discussing with someone
the impact on a content provider's server of implementing shim6.  I
expressed the opinion that it would be nice if a heavily loaded server
could aggressively garbage collect shim6 state after initial context
establishment, and rely on the client to perform failure and reachability
detection and initiate context re-establishment if a failure is detected.

If such behavior is supported by the protocol, we're much more likely to
be able to convince potential implementors to turn on basic shim6 support
by default, with the understanding that implementors can aggressively
discard shim6 context state (and not initiate shim6 context negotiation)
if the implementation doesn't have multiple locators or otherwise doesn't
need the capabilities shim6 provides.  This would (properly, IMO) push the
responsibility for context tracking, failure detection and reachability
exploration to the multihomed host that stands to benefit most from shim6
and wants to run it in the first place.

My understanding after re-reading of the protocol, particularly the
R1bis/I2bis message exchange sections, is that context re-establishment
can occur (within approx. 2 RTTs of failure detection) even if the
original ULID-ULID connection has failed and one side has discarded the
context state.  Does that match others' understanding of the process?

I was also concerned whether REAP (as defined in
draft-ietf-shim6-failure-detection-03.txt) would require frequent context
re-establishment if one side were to garbage collect.  However, if I
assume that TCP ACKs are considered "payload" packets in the context of
resetting the REAP keepalive timer, then that is not an issue, because
that timer will only be started upon sending a TCP data packet, and will
be reset by that data packet's ACK.

In any event, I think that draft-ietf-shim6-failure-detection-03.txt needs
to better define "payload packet" to differentiate between data packets
(which IMO should both set and reset the keepalive timer) and ACK packets
(which IMO should only reset the keepalive timer, not start it, or it will
cause unnecessary context re-establishment).

-Scott

Delivery-date: Mon, 27 Mar 2006 19:54:12 +0000
Date: Mon, 27 Mar 2006 14:54:01 -0500 (EST)
From: Scott Leibrand <[email-masked]>
To: [email-masked]
Subject: Shim6 proxies

Has there been any consideration of the interaction of the proposed shim6
protocol with a potential shim6 proxy?  I'm thinking it would be quite
useful to be able to place a device on-path near one end of a TCP
connection (i.e. on the default router), and have the device perform shim6
functions (and possibly other multihoming duties) on behalf of one or more
host(s) behind it.

-Scott

Delivery-date: Mon, 27 Mar 2006 19:29:23 +0000
Date: Mon, 27 Mar 2006 14:28:51 -0500 (EST)
From: Scott Leibrand <[email-masked]>
To: marcelo bagnulo braun <[email-masked]>
cc: [email-masked]
Subject: Re: Extending shim6 to use multiple locator pairs simultaneously

On 03/21/06 at 11:25am -0500, Scott Leibrand <[email-masked]> wrote=
:

> On 03/21/06 at 10:08am -0600, marcelo bagnulo braun <[email-masked]> =
wrote:
>
> > El 20/03/2006, a las 22:51, Scott Leibrand escribi=F3:
>
> > i guess that we need to understand the impact of this in the shim6 base
> > protocol and the failure detection mechanism, since both of them are
> > currently designed to use a single locator pair to exchange traffic.
> > I don't think that there would be much trouble with supporting this,
> > though
>
> Yes, definitely.  I've read the I-D's, but I should probably re-read them
> with this in mind and see if I can better understand the impact...

Marcelo,

Would the impact on the failure detection mechanism be eliminated by
having TCP initiate a new forked context for each additional path it
wanted to use?  That way, each context would consider only a single path
as "active" at any one time.  Upon my initial re-read of
draft-ietf-shim6-reach-detect-01.txt, that seems it would be sufficient,
but you'd likely know better than I if there are any other issues to worry
about...

Thanks,
Scott

Delivery-date: Mon, 27 Mar 2006 18:11:23 +0000
Date: Mon, 27 Mar 2006 13:11:15 -0500 (EST)
From: Scott Leibrand <[email-masked]>
To: Erik Nordmark <[email-masked]>
cc: marcelo bagnulo braun <[email-masked]>, [email-masked]
Subject: Re: Extending shim6 to use multiple locator pairs simultaneously

On 03/22/06 at 9:18am -0800, Erik Nordmark <[email-masked]> wrote:

> Scott,
>
> This is an interesting idea. I think that the hard parts about this fall
> squarely in the transport protocol,

Yes, probably.

> and that the shim might not need to change at all (You might just need
> to have the transport tell the shim to do forked contexts, and that way
> the shim will automatically verify that the two paths are working.)

If this is the case, great!

I just re-read draft-ietf-shim6-proto-04.txt with this in mind, and I
can't find any reason why this couldn't work, but am not sure it would,
either.

The way I understand forking, the ULP tells the shim, via the
as-yet-undeveloped API, that it wants to fork its context for some reason.
Since the shim operates at layer 3, it doesn't really care why the forking
is occurring, or what traffic goes over each forked context.  It simply
forks the context at the sending end, changes the locators in use if
directed to do so by the API, and then sends each context's packets over
the appropriate locator pair.

At the receiving end, AFAICT the shim is aware of the forking, but when it
de-shims the packets and sends them up to the ULP, it erases all traces of
the forking.  Is that correct?  If so, that satisfies the requirement for
multipath TCP that the receiving TCP/IP stack be shim6 capable, but not be
required to have any knowledge of the multiple paths the packets it
receives are taking.

> I suspect there are other issues than congestion control.
> For instance, you will have some skew between the different paths,
> causing packets to arrive at a different order than they were sent. This
> will cause the receiving TCP to send duplicate ACKs, and after the
> sender has received 3 duplicate ACKs, it will do a fast retransmit.
> Those types of algorithms would need to be make robust against packets
> arriving out of order.

Yes, I had thought of that as well.  For this reason I've decided it's
unwise to attempt any multipath at layer 3, and better to have TCP manage
the multipath using context forking and the shim6 API.  TCP multipath
would probably require SACK, and of course would require modifications to
the multipath sender's fast retransmit algorithm as you describe.

> In terms of thinking about this, it might make sense to start with SCTP,
> since SCTP already supports tracking congestion information for each
> path. Then you can ask yourself what would be needed if both paths were
> used as the same time. Once we know what to do, then applying it to TCP
> would be conceptually simple (even though it might be hard in practice.)
> Note that SCTP only uses one path at a time, and only switches after
> detecting a failure. So AFAIK the SCTP folks didn't explore what would
> happen is both paths are used concurrently.

This part is probably getting OT for this list, but...

I just read the specs on SCTP, and it looks like they only specify one
very limited form of multipath: when congestion occurs and retransmission
is required, "the endpoint should attempt to pick the most divergent
source-destination pair from the original source-destination pair to which
the packet was transmitted," so as to not be held back by (or exacerbate)
the congestion that caused the retransmission in the first place.

According to "SCTP for Beginners" at
http://tdrwww.exp-math.uni-essen.de/inhalt/forschung/sctp_fb/sctp_data.html,
"For each destination address (i.e. each possible path) a discrete set of
flow and congestion control parameters is kept, such that from the point
of view of the network, an SCTP association with a number of paths may
behave similarly as the same number of TCP connections."

So it would seem similar behavior in multipath TCP would be warranted,
along with a SACK prerequisite and additional logic to process duplicate
ACKs and avoid fast retransmit when they're caused by different RTTs and
don't indicate loss.

Thanks,
Scott

Delivery-date: Mon, 27 Mar 2006 18:11:16 +0000
Date: Mon, 27 Mar 2006 13:10:23 -0500 (EST)
From: Scott Leibrand <[email-masked]>
To: Iljitsch van Beijnum <[email-masked]>
cc: marcelo bagnulo braun <[email-masked]>, [email-masked]
Subject: Re: Extending shim6 to use multiple locator pairs simultaneously

On 03/21/06 at 7:02pm +0100, Iljitsch van Beijnum <[email-masked]> wrote:

> On 21-mrt-2006, at 17:25, Scott Leibrand wrote:
>
> > In some mental modeling I've done so far, the biggest problem I see in
> > not using TCP congestion control on each locator pair is in deciding
> > how much traffic to send over each link.
>
> I've actually worked on some stuff where we load balanced a bunch of
> traffic over several TCP sessions towards different destinations
> (where it didn't matter which part of the aggregate traffic ended up
> where), and the results were interesting. When you get the hang of
> it, it works pretty well, though. The key issue is that you feed data
> to the right session/path. If you don't, you run the risk of being
> limited by the slowest one, and have performance that's even worse
> than not bonding the paths.
>
> When I think about this, this may not be all that easy if only the
> sending side knows about what we're doing. You probably need to use
> selective acknowledgments and you may still trigger fast retransmit
> often and take TCP processing at the receiver off of the fast path.

I'd certainly want to make SACK a prerequisite for starting up multipath,
and IMO the multipath sender would definitely need to have his duplicate
acknowledgement processing modified in order to only trigger fast
retransmit when appropriate.

I'm not sure what you mean by "take TCP processing at the receiver off of
the fast path": can you elaborate on that?

> And you'd have to be really careful to make sure that the unmodified
> side doesn't see confusing RTT estimates.

True.  I would think that in order to minimize that for the unmodified
(non-multipath) side, you'd want to use a single locator pair (probably
the one the unmodified side is using) for sending all acks, and only use
the second pair for transmitting multipath data.  That would ensure a
single round-trip path for data from the unmodified (non-multipath) sender
and acks back from the modified (multipath) receiver.

-Scott

Delivery-date: Mon, 27 Mar 2006 10:19:32 +0000
Cc: "Henderson, Thomas R" <[email-masked]>, "shim6-wg" <[email-masked]>
From: marcelo bagnulo braun <[email-masked]>
Subject: Re: mip6 over shim6 (was: Re: shim - mip interaction)
Date: Mon, 27 Mar 2006 13:18:32 +0300
To: Shinta Sugimoto <[email-masked]>

Hi Shinta,

El 24/03/2006, a las 15:52, Shinta Sugimoto escribi=F3:

> Hi,
>
> In relation to the discussion on interaction between SHIM6
> and MIPv6, let me try to address how should they work cooperatively.
> There are mainly two scenarios: SHIM6 runs over MIPv6, and MIPv6
> runs over SHIM6.  The former case is well addressed in
> draft-bagnulo-shim6-mip-00.txt, in which MIPv6 home address(es) are
> treated as locator from SHIM6 perspective.  In the latter case,
> it should be possible to run MIPv6 over SHIM6 keeping MIPv6
> completely unaware of underlying shim layer.

yes, but i think that it would make sense to define whether the shim is=20=

above or below the shim once and for all, so that we figure out how the=20=

stack is actually ordered, right?

Personally i think that at this point it makes sense to lay shim above=20=

mip, in order to support the most interesting scenarios.

>  Let me expand a
> little bit more on the latter case:
>
> 1. Shimming HA-MN bi-directional tunnel
>
> In order to do this, MN and HA ought to establish SHIM6 context
> with ULID pair of MN's CoA and HA's address.  MIPv6 is completely
> unaware of the presence of shim layer and it just operates normally.
> The packet which appears on the wire should look like below.  The
> HA will first demux the shimmed packet based on the associated
> context.  Decapsulation of IP-in-IP tunneled packet will follow.
>

this case is useful indeed.
However, in this particular case, i am not sure that this is strictly=20
the case where MIP lays on top of the shim.
I mean, in this case we have tunnel interface, so basically we have the=20=

IP layer twice. So as i see it, shim can also be twice. The particular=20=

case that you are considering is when the shim of the IP layer of the=20
tunnel interface is working. So it is below the MIP, since the MIP is=20
associated to the other IP layer.

We can see it from a different p.o.v. if we consider that it would=20
possible in this case to also have a shimed session between the CN and=20=

the MN, so and additional shim header associated to the upper IP layer=20=

would be also in place.

So, my point in this case is:
- SHIM is above MIP
- each IP layer may have a shim sublayer within
- in the case of tunneled interfaces, we have two IP layers and for=20
each of these IP layers, we can have a shim layer.
- however, in all the cases, inside a given IP layer, shim is always=20
above MIP

makes sense?

> shimmed packet from MN to CN:
>
> 	[Outer IPv6 Header]
> 		src=3DL(MN), dst=3DL(HA)
> 	[SHIM6 Ext Header]
> 	[Inner IPv6 Header]
> 		src=3DMN_HoA, dst=3DCN
> 	[Payload]
>
>
> 2a. Shimming RO path between MN and CN

> In MIPv6, it is possible for MN and CN to optimize the routing
> path between the nodes by utilizing IPv6 extension headers
> (Home Address Destination Option and Routing Header Type 2)
> specific for MIPv6.  Semantically the route optimized path can
> be regarded as a virtual tunnel between the nodes although the
> mechanism of delivering IP packet is different in each direction.
> ULID pair should be MN's CoA and CN's address.
>

right this could be possible imho, but i am not sure if it would=20
provide additional benefits than the case where shim is above mip.
I mean, in this case, you can only provide fault tolerance against=20
failures in the currently used CoA. failure in the path with the HoA=20
are sill affecting the communication both in the BT mode and the RO=20
mode (because of the hoT/HoTI exchange)

So, even if i can see that some additional fault tolerance is achieved,=20=

i think that the case where shim is above mip provides more benefits...

> shimmed packet from MN to CN:
>
> 	[IPv6 Header]
> 		src=3DL(MN), dst=3DL(CN)
> 	[SHIM6 Ext Header]
> 	[DSTOPT]
> 		HAO=3DMN_HoA
>
> shimmed packet from CN to MN:
>
> 	[IPv6 Header]
> 		src=3DL(CN), dst=3DL(MN)
> 	[SHIM6 Ext Header]
> 	[RTHDR2]
> 		addr=3DMN_HoA
>
>
> 2b. Shimming RO path between MN and CN
>
> If the CN happens to be a mobile too (no matter if it is served
> by the same HA of the MN), CN and MN may establish RO path
> in slightly more complicated way.  In order to avoid confusion
> let MN and CN be denoted  as MN0 and MN1, respectively.  Assuming
> that MN0 and MN1 mutually established MIPv6 corresponding binding
> (MN0 stores binding of MN1, and MN1 stores binding of MN0),
> in order to shim RO-path between MN0 and MN1, they need to
> establish a SHIM6 context based on ULID pair of their CoAs.
>
> shimmed packet from MN0 to MN1:
>
> 	[IPv6 Header]
> 		src=3DL(MN0), dst=3DL(MN1)
> 	[SHIM6 Ext Header]
> 	[RTHDR2]
> 		addr=3DMN1_HoA
> 	[DSTOPT]
> 		HAO=3DMN0_HoA
>
> Note that packet format provided in above scenarios 2a and 2b
> are slightly contradict with what is stated in Section 4.6 of
> shim6-proto-04 saying that shim6 extension header should
> appear after any routing header.  But RTHDR2 can be treated as
> an exceptional case as it is meant to be processed only by the
> final destination, IMO.
>
> There are some requirement and issues in above scenarios:
>
> First of all, it should be noted that the above scenarios are
> directly related to the work being done in the Monami6 WG.

I am not sure why do you think so... MONAMI is chartered to do very=20
specific work, in particluar related to multiple CoA registration and=20
flow policy
No work is chartered for fault tolerance, failure detection, multiple=20
HoA support, which is what is being provided here

> Duplication of the work should be avoided from IETF perspective,
> but at first the mechanism how it (MIPv6 over SHIM6) works
> should be clearly understood, IMO.
>

agree

> One requirement in above scenarios is that MIPv6 CoA should be CGA.
> Otherwise there would be another security mechanism required to
> prevent reflection attacks (as we assume that MN changes CoA
> frequently).  In the scenario where SHIM6 runs over MIPv6,
> HBA would help to assure legitimate locator update among the
> the locator set (=3DMIPv6 HoAs).  But not in the scenarios described
> above.
>

agree
however, there seem to be the underlying assumption that in the case=20
that shim is layered above mip, the shim can not be exposed to the=20
CoAs. While this make sense, there is not yet defined if this is the=20
best alternative. I mean, allowwing the shim to be aware of the coas=20
may provide additional benefits, like some form of shim based ro mode

> Besides, I see an issue how the ULID pair can be determined by
> SHIM6 who has no knowledge on MIPv6 binding information.  As MN's
> local address namely CoA is unpredictable, it may be hard for SHIM6
> to determine when and with which address should the ULID pair
> context should be established/updated in order to shimming the MIPv6
> tunnel.
>

I am not sure i understand your point here: are you considering ULID=20
selection or locator selection? are you considering the case when a=20
node is selecting its owm addresses or when it is selecting the peer's=20=

addresses?

All these cases are quite different, but there is some support for this=20=

in RFC3484 (for ULID selection), and the flag field in the Locator=20
preference option has a temporary flag to tell the peer that a given=20
address is a CoA.
Additional guidance when selection local locators may be required though

Regards, marcelo

> Any comments ?
>
>
> Regards,
> Shinta
>
>

Delivery-date: Fri, 24 Mar 2006 23:31:22 +0000
Content-class: urn:content-classes:message
Subject: RE: Extending shim6 to use multiple locator pairs simultaneously
Date: Sat, 25 Mar 2006 01:31:11 +0200
From: <[email-masked]>
To: <[email-masked]>, <[email-masked]>
Cc: <[email-masked]>, <[email-masked]>, <[email-masked]>

Vijay,

>> > For SCTP + SHIM (which Marcelo & I had a hallway chat about - and=20
>> > maybe should write-up), there could be confusion about what the=20
>> > 'path' is when combined.  I think the SHIM API should have a=20
>> > mechanism to signal up to the transport layer that the SHIM layer=20
>> > has re-homed so that the transport layer could decide what should
be=20
>> > done - if it thinks slow-start should be run, etc.
>>
>> I don't understand how your comment relates to Scott's idea to run=20
>> multiple paths in parallel. You seem to be talking about=20
>shim and sctp=20
>> interaction, which wasn't what I and Scott was discussing.
>>
>> If you are talking about how SCTP and SHIM6 would interact,=20
>I think we=20
>> want to start from the point of SCTP setting the DONTSHIM option, so=20
>> that the shim and sctp aren't used for the same communication.
>
>I agree with this. we discussed this briefly when we were=20
>having the shim6 + mip6 discussion. you would be better off=20
>turning shim6 off when you use SCTP.

That was exactly my point.

John

Delivery-date: Fri, 24 Mar 2006 23:30:00 +0000
Content-class: urn:content-classes:message
Subject: RE: Extending shim6 to use multiple locator pairs simultaneously
Date: Sat, 25 Mar 2006 01:28:58 +0200
From: <[email-masked]>
To: <[email-masked]>
Cc: <[email-masked]>, <[email-masked]>, <[email-masked]>

Erik,

I think I answered this before, but let me reply again.

>> For SCTP + SHIM (which Marcelo & I had a hallway chat about - and=20
>> maybe should write-up), there could be confusion about what=20
>the 'path'=20
>> is when combined.  I think the SHIM API should have a mechanism to=20
>> signal up to the transport layer that the SHIM layer has re-homed so=20
>> that the transport layer could decide what should be done - if it=20
>> thinks slow-start should be run, etc.
>
>I don't understand how your comment relates to Scott's idea to=20
>run multiple paths in parallel. You seem to be talking about=20
>shim and sctp interaction, which wasn't what I and Scott was=20
>discussing.

My mistake - I should have changed the subject line, as I was
responding to a comment.

>If you are talking about how SCTP and SHIM6 would interact, I=20
>think we want to start from the point of SCTP setting the=20
>DONTSHIM option, so that the shim and sctp aren't used for the=20
>same communication.

I agree, but I think explaining why, perhaps in the Applicability
Statement would be good.

>Past that, it makes sense to study if there are more=20
>interesting ways for sctp and the shim to share the=20
>multihoming work. Potentially this can provide a more secure=20
>way that sctp's addip functionality. But I don't have a good=20
>idea on how such a split should be done.
>Anybody interested in looking into this?

Yes, I am. I'll look at sending some text.

John

Delivery-date: Fri, 24 Mar 2006 19:57:24 +0000
Date: Fri, 24 Mar 2006 11:56:34 -0800
From: "Vijay Devarapallli" <[email-masked]>
To: "Erik Nordmark" <[email-masked]>
Subject: Re: Extending shim6 to use multiple locator pairs simultaneously
Cc: [email-masked], [email-masked], [email-masked],  [email-masked]

Delivery-date: Fri, 24 Mar 2006 16:32:51 +0000
Date: Thu, 23 Mar 2006 07:47:41 -0800
From: Erik Nordmark <[email-masked]>
To: [email-masked]
CC: [email-masked], [email-masked], [email-masked]
Subject: Re: Extending shim6 to use multiple locator pairs simultaneously

[email-masked] wrote:
> Eric,
> 
> For SCTP + SHIM (which Marcelo & I had a hallway chat about - and maybe
> should write-up), there could be confusion about what the 'path' is
> when combined.  I think the SHIM API should have a mechanism to signal
> up to the transport layer that the SHIM layer has re-homed so that the
> transport layer could decide what should be done - if it thinks
> slow-start
> should be run, etc. 

I don't understand how your comment relates to Scott's idea to run 
multiple paths in parallel. You seem to be talking about shim and sctp 
interaction, which wasn't what I and Scott was discussing.

If you are talking about how SCTP and SHIM6 would interact, I think we 
want to start from the point of SCTP setting the DONTSHIM option, so 
that the shim and sctp aren't used for the same communication.

Past that, it makes sense to study if there are more interesting ways 
for sctp and the shim to share the multihoming work. Potentially this 
can provide a more secure way that sctp's addip functionality. But I 
don't have a good idea on how such a split should be done.
Anybody interested in looking into this?

    Erik

Delivery-date: Fri, 24 Mar 2006 13:53:25 +0000
Date: Fri, 24 Mar 2006 07:52:13 -0600
From: Shinta Sugimoto <[email-masked]>
To: marcelo bagnulo braun <[email-masked]>
Subject: mip6 over shim6 (was: Re: shim - mip interaction)
Cc: "Henderson, Thomas R" <[email-masked]>, "shim6-wg" <[email-masked]>

Hi,

In relation to the discussion on interaction between SHIM6
and MIPv6, let me try to address how should they work cooperatively.
There are mainly two scenarios: SHIM6 runs over MIPv6, and MIPv6
runs over SHIM6.  The former case is well addressed in
draft-bagnulo-shim6-mip-00.txt, in which MIPv6 home address(es) are
treated as locator from SHIM6 perspective.  In the latter case,
it should be possible to run MIPv6 over SHIM6 keeping MIPv6
completely unaware of underlying shim layer.  Let me expand a
little bit more on the latter case:

1. Shimming HA-MN bi-directional tunnel

In order to do this, MN and HA ought to establish SHIM6 context
with ULID pair of MN's CoA and HA's address.  MIPv6 is completely
unaware of the presence of shim layer and it just operates normally.
The packet which appears on the wire should look like below.  The
HA will first demux the shimmed packet based on the associated
context.  Decapsulation of IP-in-IP tunneled packet will follow.

shimmed packet from MN to CN:

	[Outer IPv6 Header]
		src=L(MN), dst=L(HA)
	[SHIM6 Ext Header]
	[Inner IPv6 Header]
		src=MN_HoA, dst=CN
	[Payload]

2a. Shimming RO path between MN and CN

In MIPv6, it is possible for MN and CN to optimize the routing
path between the nodes by utilizing IPv6 extension headers
(Home Address Destination Option and Routing Header Type 2)
specific for MIPv6.  Semantically the route optimized path can
be regarded as a virtual tunnel between the nodes although the
mechanism of delivering IP packet is different in each direction.
ULID pair should be MN's CoA and CN's address.

shimmed packet from MN to CN:

	[IPv6 Header]
		src=L(MN), dst=L(CN)
	[SHIM6 Ext Header]
	[DSTOPT]
		HAO=MN_HoA

shimmed packet from CN to MN:

	[IPv6 Header]
		src=L(CN), dst=L(MN)
	[SHIM6 Ext Header]
	[RTHDR2]
		addr=MN_HoA

2b. Shimming RO path between MN and CN

If the CN happens to be a mobile too (no matter if it is served
by the same HA of the MN), CN and MN may establish RO path
in slightly more complicated way.  In order to avoid confusion
let MN and CN be denoted  as MN0 and MN1, respectively.  Assuming
that MN0 and MN1 mutually established MIPv6 corresponding binding
(MN0 stores binding of MN1, and MN1 stores binding of MN0),
in order to shim RO-path between MN0 and MN1, they need to
establish a SHIM6 context based on ULID pair of their CoAs.

shimmed packet from MN0 to MN1:

	[IPv6 Header]
		src=L(MN0), dst=L(MN1)
	[SHIM6 Ext Header]
	[RTHDR2]
		addr=MN1_HoA
	[DSTOPT]
		HAO=MN0_HoA

Note that packet format provided in above scenarios 2a and 2b
are slightly contradict with what is stated in Section 4.6 of
shim6-proto-04 saying that shim6 extension header should
appear after any routing header.  But RTHDR2 can be treated as
an exceptional case as it is meant to be processed only by the
final destination, IMO.

There are some requirement and issues in above scenarios:

First of all, it should be noted that the above scenarios are
directly related to the work being done in the Monami6 WG.
Duplication of the work should be avoided from IETF perspective,
but at first the mechanism how it (MIPv6 over SHIM6) works
should be clearly understood, IMO.

One requirement in above scenarios is that MIPv6 CoA should be CGA.
Otherwise there would be another security mechanism required to
prevent reflection attacks (as we assume that MN changes CoA
frequently).  In the scenario where SHIM6 runs over MIPv6,
HBA would help to assure legitimate locator update among the
the locator set (=MIPv6 HoAs).  But not in the scenarios described
above.

Besides, I see an issue how the ULID pair can be determined by
SHIM6 who has no knowledge on MIPv6 binding information.  As MN's
local address namely CoA is unpredictable, it may be hard for SHIM6
to determine when and with which address should the ULID pair
context should be established/updated in order to shimming the MIPv6
tunnel.

Any comments ?

Regards,
Shinta

Delivery-date: Thu, 23 Mar 2006 16:31:45 +0000
From: arifumi <[email-masked]>
Subject: Re: Extending shim6 to use multiple locator pairs simultaneously
Date: Fri, 24 Mar 2006 01:31:22 +0900
To: [email-masked]

Hi folks,

>
> In terms of thinking about this, it might make sense to start with  
> SCTP, since SCTP already supports tracking congestion information  
> for each path. Then you can ask yourself what would be needed if  
> both paths were used as the same time. Once we know what to do,  
> then applying it to TCP would be conceptually simple (even though  
> it might be hard in practice.)
> Note that SCTP only uses one path at a time, and only switches  
> after detecting a failure. So AFAIK the SCTP folks didn't explore  
> what would happen is both paths are used concurrently.
>
>    Erik

The SCTP RFC doesn't support concurrent multiple path, but there are  
not a single researches in this area. CMT-SCTP proposed by PET in  
Univ. of Delaware is one of those researches and looks successful.

http://www.eecis.udel.edu/~amer/PEL/poc/index.html

As Eric suggests, it seems to me not a good idea to use multiple  
paths concurrently without any modifications to transport protocols.  
CMT-SCTP is just those modifications to SCTP.

Delivery-date: Thu, 23 Mar 2006 16:18:52 +0000
Content-class: urn:content-classes:message
Subject: RE: Path-switch-triggered congestion (Was: RE: Extending shim6 to use multiple locator pairs simultaneously)
Date: Thu, 23 Mar 2006 18:17:56 +0200
From: <[email-masked]>
To: <[email-masked]>, <[email-masked]>
Cc: <[email-masked]>, <[email-masked]>, <[email-masked]>

Erik,

>> IMO this is not a show-stopping problem, as TCP will throttle back=20
>> fairly quickly, and the impact should be limited to=20
>approximately the=20
>> depth of the slow link's buffer.  That's not to say it's not worth=20
>> addressing (in the TSVWG), but it doesn't seem to me like something=20
>> that should hold up shim6...
>
>My take is that shim6 should provide a notification mechanism=20
>that the transports can use to be told when there is a locator=20
>pair change.
>What the transports do when they are notified is presumably=20
>something that folks in the transport area can discuss, thus=20
>it isn't a show-stopper for shim6.

FYI - I agree.

John

Delivery-date: Thu, 23 Mar 2006 16:18:22 +0000
Content-class: urn:content-classes:message
Subject: RE: Extending shim6 to use multiple locator pairs simultaneously
Date: Thu, 23 Mar 2006 18:17:05 +0200
From: <[email-masked]>
To: <[email-masked]>
Cc: <[email-masked]>, <[email-masked]>, <[email-masked]>

Erik,

>> For SCTP + SHIM (which Marcelo & I had a hallway chat about - and=20
>> maybe should write-up), there could be confusion about what the
'path'=20
>> is when combined.  I think the SHIM API should have a mechanism to=20
>> signal up to the transport layer that the SHIM layer has re-homed so=20
>> that the transport layer could decide what should be done - if it=20
>> thinks slow-start should be run, etc.
>
>I don't understand how your comment relates to Scott's idea to=20
>run multiple paths in parallel. You seem to be talking about=20
>shim and sctp interaction, which wasn't what I and Scott was=20
>discussing.

Sorry - I wasn't responding to the subject line topic - I should
have changed the subject line.

>If you are talking about how SCTP and SHIM6 would interact, I=20
>think we want to start from the point of SCTP setting the=20
>DONTSHIM option, so that the shim and sctp aren't used for the=20
>same communication.

Yes, that would be the idea.

>Past that, it makes sense to study if there are more=20
>interesting ways for sctp and the shim to share the=20
>multihoming work. Potentially this can provide a more secure=20
>way that sctp's addip functionality. But I don't have a good=20
>idea on how such a split should be done.
>Anybody interested in looking into this?

I was meaning that we'd be looking into this, and writing up=20
some ideas.=20

John

Delivery-date: Thu, 23 Mar 2006 15:58:44 +0000
Date: Thu, 23 Mar 2006 07:55:40 -0800
From: Erik Nordmark <[email-masked]>
To: Scott Leibrand <[email-masked]>
CC: [email-masked], [email-masked], [email-masked], [email-masked]
Subject: Re: Path-switch-triggered congestion (Was: RE: Extending shim6 to use multiple locator pairs simultaneously)

Scott Leibrand wrote:

> IMO this is not a show-stopping problem, as TCP will throttle back fairly
> quickly, and the impact should be limited to approximately the depth of
> the slow link's buffer.  That's not to say it's not worth addressing (in
> the TSVWG), but it doesn't seem to me like something that should hold up
> shim6...

My take is that shim6 should provide a notification mechanism that the 
transports can use to be told when there is a locator pair change.
What the transports do when they are notified is presumably something 
that folks in the transport area can discuss, thus it isn't a 
show-stopper for shim6.

    Erik

Delivery-date: Thu, 23 Mar 2006 15:30:28 +0000
Date: Fri, 24 Mar 2006 02:29:51 +1100
To: [email-masked]
From: Geoff Huston <[email-masked]>
Subject: SHIM6 Working Group Last Call on draft-ietf-shim6-proto-04.txt
Cc: kurt Erik Lindqvist <[email-masked]>

Folks,

         This note starts the WG Last Call for comments on
         draft-ietf-shim6-proto-04.txt, "Level 3 multi-homing shim
         protocol".

         It can be found at:
         ftp://ftp.ietf.org/internet-drafts/draft-ietf-shim6-proto-04.txt

         The document is intended to be submitted by the Working Group
         to the IESG for publication as a Proposed Standard.

         Please review the document carefully, and send your feedback to the
         SHIM6 list.  Please also indicate in your response whether or not
         you believe that this document is ready to go to the IESG.

         This Last Call will end on Friday 14 Apr 2006 at 0800 AEST
         (UTC+10).

         Thanks,

               Geoff & Kurtis

----------------------------------------------------------

Additional comments to this Last Call:

o The three week period for the WG Last Call is to allow some leeway for
   post-IETF recovery!

o We also note that we have an outstanding topic of compatibility with
   Mobile IPv6 and SHIM6 as discussed in the Working Group meeting this
   week. This last call has been generated on the basis of verbal advice
   provided by the document's editors that this matter has been discussed
   with Mobile IPv6 folk and that there are no serious issues of
   incompatibility anticipated. I expect a note to this effect from Marcelo
   Bagnulo summarizing the outcomes of this discussion to be posted to the
   SHIM6 WG during this Last Call period.

o We have consulted with the technical advisors to this Working Group and
   they have indicated that are happy for us to proceed with this last call at
   this time.

Delivery-date: Thu, 23 Mar 2006 14:28:24 +0000
Date: Thu, 23 Mar 2006 09:28:01 -0500 (EST)
From: Scott Leibrand <[email-masked]>
To: [email-masked]
cc: [email-masked], [email-masked], [email-masked],  [email-masked]
Subject: RE: Path-switch-triggered congestion (Was: RE: Extending shim6 to use multiple locator pairs simultaneously)

On 03/23/06 at 3:45pm +0200, [email-masked] wrote:

> I think you are seriously missing the point. Congestion control is
> used to protect the network.  It isn't about link bandwidth, per-say.
>
> TCP uses slow-start to probe the network to ensure it doesn't start
> sending at a higher rate than the network can support.  Link knowledge
> is not enough.  There is on-going work in TCP to try to fix this, so
> that a new TCP connection could start sending at a higher rate, if
> there is some path knowledge, Quick Start for TCP and IP.
> http://www.ietf.org/internet-drafts/draft-ietf-tsvwg-quickstart-02.txt

Sure.  And if TCP wants to do Quick Start or anything else on a rehoming
event, I'd say that's fine.  I just don't think that we can say what
higher-layer protocols MUST do in response to a lower-layer rehoming
event.  The whole point of shim6 is that it's transparent to upper layers
for compatibility.

> But think about it - if there is a rehoming event because one network
> path became unavailable, there's a possibility that many hosts will
> rehome. If they rehome to a network that is already congested and
> continue sending at a high rate, there will be severe network
> congestion, which might cause major network problems.

Sure, but that's the case today as well.  If I'm running an IGP with two
unequal-bandwidth links, and the high-bandwidth link goes down, all the
traffic from all those hosts will rehome to the slower link.  This will
cause congestion until congestion avoidance halves the sending rate enough
times to sufficiently lower throughput.  I don't think a shim6 rehoming
event is any different, so I don't think any special action is required to
avoid violating the "do no harm" principle.

> I see the only options to be:
>
> 1) after rehoming, TCP will need to run slow-start, in order to probe
>    the network.
> 2) have an alternate mechanism, like Quick Start, in order to quickly
>    determine the proper sending rate after a rehoming event.

I think those are optimizations.  Since TCP isn't necessarily aware of a
rehoming event (unless it knows how to talk to shim6 with the API), I
think that simply letting congestion avoidance deal with the new path is a
viable option as well.

-Scott

> PS - there is some work that has been looking at modifying TCP to deal
> better with different kinds of connectivity changes:
>
> https://datatracker.ietf.org/public/idindex.cgi?command=id_detail&id=120
> 91
> https://datatracker.ietf.org/public/idindex.cgi?command=id_detail&id=101
> 65
>
>
>
> >-----Original Message-----
> >From: ext Iljitsch van Beijnum [mailto:[email-masked]]
> >Sent: 23 March, 2006 09:57
> >To: Scott Leibrand
> >Cc: Loughney John (Nokia-NRC/Helsinki); [email-masked];
> >[email-masked]; [email-masked]
> >Subject: Re: Path-switch-triggered congestion (Was: RE:
> >Extending shim6 to use multiple locator pairs simultaneously)
> >
> >On 22-mrt-2006, at 23:30, Scott Leibrand wrote:
> >
> >>> Current behavior is that TCP doesn't really take into
> >account radical
> >>> chances in link capacity.  There is some on-going work in TSVWG on
> >>> how to handle extreme congestion events, but this isn't really
> >>> something solved.
> >
> >> But how much of a problem is this in reality?  Say you are
> >> transmitting at 1Gbps (1,000,000 kbps), and get switched to a 28.8
> >> kbps modem link.
> >> Since each packet lost halves the sending rate, and you've reduced
> >> your throughput by a factor of approximately 2^15, it should take
> >> approximately
> >> 15 dropped packets to throttle back the sending rate sufficiently.
> >
> >That's with congestion avoidance, which kicks in for
> >individual lost packets. If you lose a lot of them you're in
> >slow start.
> >
> >But you're not considering delay and window size. If you're
> >doing 1 Gbps over a somewhat long distance (let's say 40 ms
> >round trip delay) then you need a ~ 5 MB window. If you were
> >to pump such a large window into 28 kbps you'd saturate that
> >link for 23 minutes...
> >
> >Even assuming a regular 64k window (which would limit your
> >datatransfer with 40 ms RTT to 13 Mbps) you'd be saturating
> >that 28k link for 18 seconds.
> >
> >> IMO this is not a show-stopping problem, as TCP will throttle back
> >> fairly quickly, and the impact should be limited to
> >approximately the
> >> depth of the slow link's buffer.  That's not to say it's not worth
> >> addressing (in the TSVWG), but it doesn't seem to me like something
> >> that should hold up shim6...
> >
> >Well, let's just do some experimenting when we have implementations.
> >
>
>

Delivery-date: Thu, 23 Mar 2006 14:24:03 +0000
Content-class: urn:content-classes:message
Subject: RE: Path-switch-triggered congestion (Was: RE: Extending shim6 to use multiple locator pairs simultaneously)
Date: Thu, 23 Mar 2006 16:23:43 +0200
From: <[email-masked]>
To: <[email-masked]>, <[email-masked]>
Cc: <[email-masked]>, <[email-masked]>, <[email-masked]>

>> Well, let's just do some experimenting when we have implementations.
>
>Agreed.  In parallel, we should probably put into the API a=20
>way to tell Layer 4 that we've rehomed, so that TCP can do=20
>whatever it thinks best in that situation.

I strongly agree. I think this should be the default behavior -
optimizations can
come later. =20

Joh

Delivery-date: Thu, 23 Mar 2006 14:21:57 +0000
Date: Thu, 23 Mar 2006 09:21:29 -0500 (EST)
From: Scott Leibrand <[email-masked]>
To: Iljitsch van Beijnum <[email-masked]>
cc: [email-masked], [email-masked], [email-masked],  [email-masked]
Subject: Re: Path-switch-triggered congestion (Was: RE: Extending shim6 to use multiple locator pairs simultaneously)

On 03/23/06 at 8:56am +0100, Iljitsch van Beijnum <[email-masked]> wrote:

> On 22-mrt-2006, at 23:30, Scott Leibrand wrote:
>
> > But how much of a problem is this in reality?  Say you are
> > transmitting at 1Gbps (1,000,000 kbps), and get switched to a 28.8
> > kbps modem link. Since each packet lost halves the sending rate, and
> > you've reduced your throughput by a factor of approximately 2^15, it
> > should take approximately 15 dropped packets to throttle back the
> > sending rate sufficiently.
>
> That's with congestion avoidance, which kicks in for individual lost
> packets. If you lose a lot of them you're in slow start.
>
> But you're not considering delay and window size. If you're doing 1
> Gbps over a somewhat long distance (let's say 40 ms round trip delay)
> then you need a ~ 5 MB window. If you were to pump such a large
> window into 28 kbps you'd saturate that link for 23 minutes...

Sure I am.  No one buffers 23 minutes of data, so all the extra would
simply be dropped, and the buffer would then drain at line rate.

> Even assuming a regular 64k window (which would limit your
> datatransfer with 40 ms RTT to 13 Mbps) you'd be saturating that 28k
> link for 18 seconds.

Possibly, if the buffer(s) is/are that deep.  However, if that is the
case, then you would also be able to buffer 18 seconds of data simply by
running multiple TCP sessions across the slow link.  It doesn't require a
rehoming event to fill buffers, because the only way TCP will slow down is
when the network stops buffering and starts dropping packets.

> > IMO this is not a show-stopping problem, as TCP will throttle back
> > fairly quickly, and the impact should be limited to approximately the
> > depth of the slow link's buffer.  That's not to say it's not worth
> > addressing (in the TSVWG), but it doesn't seem to me like something
> > that should hold up shim6...
>
> Well, let's just do some experimenting when we have implementations.

Agreed.  In parallel, we should probably put into the API a way to tell
Layer 4 that we've rehomed, so that TCP can do whatever it thinks best in
that situation.

-Scott

Delivery-date: Thu, 23 Mar 2006 13:45:59 +0000
Content-class: urn:content-classes:message
Subject: RE: Path-switch-triggered congestion (Was: RE: Extending shim6 to use multiple locator pairs simultaneously)
Date: Thu, 23 Mar 2006 15:45:06 +0200
From: <[email-masked]>
To: <[email-masked]>, <[email-masked]>
Cc: <[email-masked]>, <[email-masked]>, <[email-masked]>

Guys,

I think you are seriously missing the point. Congestion control is
used to protect the network.  It isn't about link bandwidth, per-say.

TCP uses slow-start to probe the network to ensure it doesn't start
sending at a higher rate than the network can support.  Link knowledge
is not enough.  There is on-going work in TCP to try to fix this, so
that a new TCP connection could start sending at a higher rate, if
there is some path knowledge, Quick Start for TCP and IP.
http://www.ietf.org/internet-drafts/draft-ietf-tsvwg-quickstart-02.txt

But think about it - if there is a rehoming event because one network
path
became unavailable, there's a possibility that many hosts will rehome.
If they rehome to a network that is already congested and continue=20
sending at a high rate, there will be severe network congestion, which
might
cause major network problems.

I see the only options to be:

1) after rehoming, TCP will need to run slow-start, in order to probe=20
   the network.
2) have an alternate mechanism, like Quick Start, in order to quickly=20
   determine the proper sending rate after a rehoming event.

John

PS - there is some work that has been looking at modifying TCP to deal
better=20
with different kinds of connectivity changes:

https://datatracker.ietf.org/public/idindex.cgi?command=3Did_detail&id=3D=
120
91
https://datatracker.ietf.org/public/idindex.cgi?command=3Did_detail&id=3D=
101
65

>-----Original Message-----
>From: ext Iljitsch van Beijnum [mailto:[email-masked]]=20
>Sent: 23 March, 2006 09:57
>To: Scott Leibrand
>Cc: Loughney John (Nokia-NRC/Helsinki); [email-masked];=20
>[email-masked]; [email-masked]
>Subject: Re: Path-switch-triggered congestion (Was: RE:=20
>Extending shim6 to use multiple locator pairs simultaneously)
>
>On 22-mrt-2006, at 23:30, Scott Leibrand wrote:
>
>>> Current behavior is that TCP doesn't really take into=20
>account radical=20
>>> chances in link capacity.  There is some on-going work in TSVWG on=20
>>> how to handle extreme congestion events, but this isn't really=20
>>> something solved.
>
>> But how much of a problem is this in reality?  Say you are=20
>> transmitting at 1Gbps (1,000,000 kbps), and get switched to a 28.8=20
>> kbps modem link.
>> Since each packet lost halves the sending rate, and you've reduced=20
>> your throughput by a factor of approximately 2^15, it should take=20
>> approximately
>> 15 dropped packets to throttle back the sending rate sufficiently.
>
>That's with congestion avoidance, which kicks in for=20
>individual lost packets. If you lose a lot of them you're in=20
>slow start.
>
>But you're not considering delay and window size. If you're=20
>doing 1 Gbps over a somewhat long distance (let's say 40 ms=20
>round trip delay) then you need a ~ 5 MB window. If you were=20
>to pump such a large window into 28 kbps you'd saturate that=20
>link for 23 minutes...
>
>Even assuming a regular 64k window (which would limit your=20
>datatransfer with 40 ms RTT to 13 Mbps) you'd be saturating=20
>that 28k link for 18 seconds.
>
>> IMO this is not a show-stopping problem, as TCP will throttle back=20
>> fairly quickly, and the impact should be limited to=20
>approximately the=20
>> depth of the slow link's buffer.  That's not to say it's not worth=20
>> addressing (in the TSVWG), but it doesn't seem to me like something=20
>> that should hold up shim6...
>
>Well, let's just do some experimenting when we have implementations.
>

Delivery-date: Thu, 23 Mar 2006 07:58:52 +0000
Cc: [email-masked], [email-masked], [email-masked], [email-masked]
From: Iljitsch van Beijnum <[email-masked]>
Subject: Re: Path-switch-triggered congestion (Was: RE: Extending shim6 to use multiple locator pairs simultaneously)
Date: Thu, 23 Mar 2006 08:56:35 +0100
To: Scott Leibrand <[email-masked]>

On 22-mrt-2006, at 23:30, Scott Leibrand wrote:

>> Current behavior is that TCP doesn't really take into account radical
>> chances in link capacity.  There is some on-going work in TSVWG on  
>> how
>> to handle extreme congestion events, but this isn't really something
>> solved.

> But how much of a problem is this in reality?  Say you are  
> transmitting at
> 1Gbps (1,000,000 kbps), and get switched to a 28.8 kbps modem link.
> Since each packet lost halves the sending rate, and you've reduced  
> your
> throughput by a factor of approximately 2^15, it should take  
> approximately
> 15 dropped packets to throttle back the sending rate sufficiently.

That's with congestion avoidance, which kicks in for individual lost  
packets. If you lose a lot of them you're in slow start.

But you're not considering delay and window size. If you're doing 1  
Gbps over a somewhat long distance (let's say 40 ms round trip delay)  
then you need a ~ 5 MB window. If you were to pump such a large  
window into 28 kbps you'd saturate that link for 23 minutes...

Even assuming a regular 64k window (which would limit your  
datatransfer with 40 ms RTT to 13 Mbps) you'd be saturating that 28k  
link for 18 seconds.

> IMO this is not a show-stopping problem, as TCP will throttle back  
> fairly
> quickly, and the impact should be limited to approximately the  
> depth of
> the slow link's buffer.  That's not to say it's not worth  
> addressing (in
> the TSVWG), but it doesn't seem to me like something that should  
> hold up
> shim6...

Well, let's just do some experimenting when we have implementations.

Delivery-date: Thu, 23 Mar 2006 03:51:50 +0000
Content-class: urn:content-classes:message
Subject: RE: [Fwd: I-D ACTION:draft-nordmark-shim6-esd-00.txt]
Date: Wed, 22 Mar 2006 19:51:11 -0800
From: "Henderson, Thomas R" <[email-masked]>
To: "Erik Nordmark" <[email-masked]>
Cc: "shim6" <[email-masked]>

=20

> -----Original Message-----
> From: Erik Nordmark [mailto:[email-masked]]=20
> Sent: Wednesday, March 22, 2006 9:10 AM
> To: Henderson, Thomas R
> Cc: shim6
> Subject: Re: [Fwd: I-D ACTION:draft-nordmark-shim6-esd-00.txt]
>=20
> Henderson, Thomas R wrote:
> > As I briefly mentioned today, there has been some=20
> complementary work in
> > the HIP RG that discusses the handling of non-routable=20
> identifiers in
> > legacy applications:
> >=20
> http://www.ietf.org/internet-drafts/draft-henderson-hip-applic
> ations-02.
> > txt,=20
> > the main differences being the use of KHI (now ORCHIDs) in=20
> HIP instead
> > of CGAs. =20
>=20
> Is there an orchid draft? (I'm curious what might have changed other=20
> than the name.)

http://www.ietf.org/internet-drafts/draft-laganier-ipv6-khi-01.txt
(see section 9.1 for changes)

>=20
> > Until recently, the HIP drafts defined a "Type 2" HIT with=20
> the property
> > that the upper 64 bits contained support for two levels of=20
> hierarchical
> > naming (enabling reverse resolution), with the lower bits=20
> being drawn
> > from a hash of the public key, but this HIT type was=20
> dropped due to lack
> > of interest last year:
> > http://www1.ietf.org/mail-archive/web/hipsec/current/msg01519.html
> > It was also felt by some that 64 bits of hash was insufficient to
> > protect the binding between HIT and public key.
>=20
> I can understand the 64 bit concern for HIP, since HIP is=20
> securing the=20
> payload. Hence the comparison is with the strength that IKE=20
> can provide.
>=20
> But shim6 is only preventing redirection attacks; if one cares about=20
> payload protection one would run IPsec, TLS, etc above shim6.
> For the redirection threats, 64 bits is probably plenty.

Also, there were concerns about possible collisions of long-lived
identifiers (HITs) and the birthday paradox; this is discussed in the
HIP architecture draft.

Tom

Delivery-date: Thu, 23 Mar 2006 02:36:08 +0000
Content-class: urn:content-classes:message
Subject: RE: Extending shim6 to use multiple locator pairs simultaneously
Date: Wed, 22 Mar 2006 23:48:17 +0200
From: <[email-masked]>
To: <[email-masked]>, <[email-masked]>
Cc: <[email-masked]>, <[email-masked]>

Eric,

For SCTP + SHIM (which Marcelo & I had a hallway chat about - and maybe
should write-up), there could be confusion about what the 'path' is
when combined.  I think the SHIM API should have a mechanism to signal
up to the transport layer that the SHIM layer has re-homed so that the
transport layer could decide what should be done - if it thinks
slow-start
should be run, etc.=20

John

>-----Original Message-----
>From: [email-masked] [mailto:[email-masked]] On=20
>Behalf Of ext Erik Nordmark
>Sent: 22 March, 2006 19:19
>To: Scott Leibrand
>Cc: marcelo bagnulo braun; [email-masked]
>Subject: Re: Extending shim6 to use multiple locator pairs=20
>simultaneously
>
>
>Scott,
>
>This is an interesting idea. I think that the hard parts about=20
>this fall squarely in the transport protocol, and that the=20
>shim might not need to change at all (You might just need to=20
>have the transport tell the shim to do forked contexts, and=20
>that way the shim will automatically verify that the two paths=20
>are working.)
>
>Scott Leibrand wrote:
>
>> Ok, that's certainly worth study.  In some mental modeling I've done=20
>> so far, the biggest problem I see in not using TCP=20
>congestion control=20
>> on each locator pair is in deciding how much traffic to send=20
>over each=20
>> link.  I suppose you could use both links equally, which would allow=20
>> you to use (# of links) * (the bandwidth of the slower link) rather=20
>> than the sum of the bandwidths of all the links.  IOW, as=20
>soon as one=20
>> link experiences congestion, the sending rate would be=20
>halved over all links.
>
>I suspect there are other issues than congestion control.
>For instance, you will have some skew between the different=20
>paths, causing packets to arrive at a different order than=20
>they were sent. This will cause the receiving TCP to send=20
>duplicate ACKs, and after the sender has received 3 duplicate=20
>ACKs, it will do a fast retransmit.=20
>Those types of algorithms would need to be make robust against=20
>packets arriving out of order.
>
>In terms of thinking about this, it might make sense to start=20
>with SCTP, since SCTP already supports tracking congestion=20
>information for each path. Then you can ask yourself what=20
>would be needed if both paths were used as the same time. Once=20
>we know what to do, then applying it to TCP would be=20
>conceptually simple (even though it might be hard in=20
>practice.) Note that SCTP only uses one path at a time, and=20
>only switches after detecting a failure. So AFAIK the SCTP=20
>folks didn't explore what would happen is both paths are used=20
>concurrently.
>
>    Erik
>
>
>
>

Delivery-date: Wed, 22 Mar 2006 22:30:49 +0000
Date: Wed, 22 Mar 2006 17:30:12 -0500 (EST)
From: Scott Leibrand <[email-masked]>
To: [email-masked]
cc: [email-masked], [email-masked], [email-masked],  [email-masked]
Subject: Path-switch-triggered congestion (Was: RE: Extending shim6 to use multiple locator pairs simultaneously)

On 03/23/06 at 12:01am +0200, [email-masked] wrote:

> Erik,
>
> >The two reasons I remember being stated for why slow-start
> >isn't necessary the obvious answer are:
> >  - the transport protocol needs to be robust against the case when the
> >    route in the network changes from a high to a low capacity one.
> >  - Not all changes are for the worse; when going from a low to a high
> >    capacity path then there is no need to make things go slower.
>
> Current behavior is that TCP doesn't really take into account radical
> chances in link capacity.  There is some on-going work in TSVWG on how
> to handle extreme congestion events, but this isn't really something
> solved.

But how much of a problem is this in reality?  Say you are transmitting at
1Gbps (1,000,000 kbps), and get switched to a 28.8 kbps modem link.
Since each packet lost halves the sending rate, and you've reduced your
throughput by a factor of approximately 2^15, it should take approximately
15 dropped packets to throttle back the sending rate sufficiently.

I understand the concern about it taking a long time to drain a modem
link's buffer, though: I've managed to buffer about 10 seconds of data on
my parents' modem link.  :)

Does anyone here have a good understanding of how existing TCP
implementations set their retransmission timer?  Is is usually/always less
than 2 * the average RTT?  If so, I suspect that in the case of a switch
from a fast link to a deep-buffering modem link, the retransmission timer
on the sender will expire well before enough packets drain the buffer for
the receiver to start sending duplicate ACKs.

IMO this is not a show-stopping problem, as TCP will throttle back fairly
quickly, and the impact should be limited to approximately the depth of
the slow link's buffer.  That's not to say it's not worth addressing (in
the TSVWG), but it doesn't seem to me like something that should hold up
shim6...

-Scott

Delivery-date: Wed, 22 Mar 2006 22:02:47 +0000
Content-class: urn:content-classes:message
Subject: RE: Extending shim6 to use multiple locator pairs simultaneously
Date: Thu, 23 Mar 2006 00:01:01 +0200
From: <[email-masked]>
To: <[email-masked]>, <[email-masked]>
Cc: <[email-masked]>, <[email-masked]>, <[email-masked]>

Erik,

>The two reasons I remember being stated for why slow-start=20
>isn't necessary the obvious answer are:
>  - the transport protocol needs to be robust against the case when the
>    route in the network changes from a high to a low capacity one.
>  - Not all changes are for the worse; when going from a low to a high
>    capacity path then there is no need to make things go slower.

Current behavior is that TCP doesn't really take into account radical
chances
in link capacity.  There is some on-going work in TSVWG on how to handle
extreme congestion events, but this isn't really something solved. =20

John

Delivery-date: Wed, 22 Mar 2006 19:43:07 +0000
Cc: marcelo bagnulo braun <[email-masked]>, Scott Leibrand <[email-masked]>, [email-masked]
From: Iljitsch van Beijnum <[email-masked]>
Subject: Re: Extending shim6 to use multiple locator pairs simultaneously
Date: Wed, 22 Mar 2006 20:41:59 +0100
To: Erik Nordmark <[email-masked]>

On 22-mrt-2006, at 18:21, Erik Nordmark wrote:

>> We need to figure that out for regular shim stuff anyway, though.  
>> When a gigE link goes down and you get to rehome to 28 kbps GPRS  
>> link you REALLY don't want to do congestion control the hard  
>> way... It has been suggested in the past that a rehoming should  
>> trigger slow start, but I remember someone disagreeing with this,  
>> don't remember why.

> The two reasons I remember being stated for why slow-start isn't  
> necessary the obvious answer are:
>  - the transport protocol needs to be robust against the case when the
>    route in the network changes from a high to a low capacity one.
>  - Not all changes are for the worse; when going from a low to a high
>    capacity path then there is no need to make things go slower.

Well, I do agree with the second point but I still worry about the  
situation where rehoming is from a fast path to a much slower one,  
resulting in massive congestion. The trouble is that buffers on slow  
interfaces can take seconds to empty, so filling them all the way up  
and letting the packets drop where they may is not a very good  
solution. Add to that the potential for trouble when many hosts do  
the same thing at the same time...

But we're going to do a reachability check before the rehoming, which  
is important in two ways:

- we get an idea about the round trip time
- while we wait for this the session isn't rehomed yet so all packets  
sent are lost and very likely TCP will go in slow start anyway

So this may all work out without us having to do anything. Still,  
congesion control is absolutely vital so we absolutely have to get  
this right or we'll "break the internet".

Delivery-date: Wed, 22 Mar 2006 17:23:42 +0000
Date: Wed, 22 Mar 2006 09:21:47 -0800
From: Erik Nordmark <[email-masked]>
To: Iljitsch van Beijnum <[email-masked]>
CC: marcelo bagnulo braun <[email-masked]>, Scott Leibrand <[email-masked]>, [email-masked]
Subject: Re: Extending shim6 to use multiple locator pairs simultaneously

Iljitsch van Beijnum wrote:

> We need to figure that out for regular shim stuff anyway, though. When a 
> gigE link goes down and you get to rehome to 28 kbps GPRS link you 
> REALLY don't want to do congestion control the hard way... It has been 
> suggested in the past that a rehoming should trigger slow start, but I 
> remember someone disagreeing with this, don't remember why.

The two reasons I remember being stated for why slow-start isn't 
necessary the obvious answer are:
  - the transport protocol needs to be robust against the case when the
    route in the network changes from a high to a low capacity one.
  - Not all changes are for the worse; when going from a low to a high
    capacity path then there is no need to make things go slower.

   Erik

Delivery-date: Wed, 22 Mar 2006 17:20:51 +0000
Date: Wed, 22 Mar 2006 09:18:51 -0800
From: Erik Nordmark <[email-masked]>
To: Scott Leibrand <[email-masked]>
CC: marcelo bagnulo braun <[email-masked]>, [email-masked]
Subject: Re: Extending shim6 to use multiple locator pairs simultaneously

Scott,

This is an interesting idea. I think that the hard parts about this fall 
squarely in the transport protocol, and that the shim might not need to 
change at all (You might just need to have the transport tell the shim 
to do forked contexts, and that way the shim will automatically verify 
that the two paths are working.)

Scott Leibrand wrote:

> Ok, that's certainly worth study.  In some mental modeling I've done so
> far, the biggest problem I see in not using TCP congestion control on each
> locator pair is in deciding how much traffic to send over each link.  I
> suppose you could use both links equally, which would allow you to use (#
> of links) * (the bandwidth of the slower link) rather than the sum of the
> bandwidths of all the links.  IOW, as soon as one link experiences
> congestion, the sending rate would be halved over all links.

I suspect there are other issues than congestion control.
For instance, you will have some skew between the different paths, 
causing packets to arrive at a different order than they were sent. This 
will cause the receiving TCP to send duplicate ACKs, and after the 
sender has received 3 duplicate ACKs, it will do a fast retransmit. 
Those types of algorithms would need to be make robust against packets 
arriving out of order.

In terms of thinking about this, it might make sense to start with SCTP, 
since SCTP already supports tracking congestion information for each 
path. Then you can ask yourself what would be needed if both paths were 
used as the same time. Once we know what to do, then applying it to TCP 
would be conceptually simple (even though it might be hard in practice.)
Note that SCTP only uses one path at a time, and only switches after 
detecting a failure. So AFAIK the SCTP folks didn't explore what would 
happen is both paths are used concurrently.

    Erik

Delivery-date: Wed, 22 Mar 2006 17:11:57 +0000
Date: Wed, 22 Mar 2006 09:09:46 -0800
From: Erik Nordmark <[email-masked]>
To: "Henderson, Thomas R" <[email-masked]>
CC: shim6 <[email-masked]>
Subject: Re: [Fwd: I-D ACTION:draft-nordmark-shim6-esd-00.txt]

Henderson, Thomas R wrote:
> As I briefly mentioned today, there has been some complementary work in
> the HIP RG that discusses the handling of non-routable identifiers in
> legacy applications:
> http://www.ietf.org/internet-drafts/draft-henderson-hip-applications-02.
> txt, 
> the main differences being the use of KHI (now ORCHIDs) in HIP instead
> of CGAs.  

Is there an orchid draft? (I'm curious what might have changed other 
than the name.)

> Until recently, the HIP drafts defined a "Type 2" HIT with the property
> that the upper 64 bits contained support for two levels of hierarchical
> naming (enabling reverse resolution), with the lower bits being drawn
> from a hash of the public key, but this HIT type was dropped due to lack
> of interest last year:
> http://www1.ietf.org/mail-archive/web/hipsec/current/msg01519.html
> It was also felt by some that 64 bits of hash was insufficient to
> protect the binding between HIT and public key.

I can understand the 64 bit concern for HIP, since HIP is securing the 
payload. Hence the comparison is with the strength that IKE can provide.

But shim6 is only preventing redirection attacks; if one cares about 
payload protection one would run IPsec, TLS, etc above shim6.
For the redirection threats, 64 bits is probably plenty.

    Erik

Delivery-date: Wed, 22 Mar 2006 05:02:11 +0000
Date: Wed, 22 Mar 2006 16:00:21 +1100
To: [email-masked]
From: Geoff Huston <[email-masked]>
Subject: SHIM6 WG notes from IETF 65

Hi,

heres the notes taken by Nevil Brownlee of the SHIM6 meeting - I'll make 
these the minutes if there are no objections...

thanks to Nevil for this

regards,

    Geoff

==============

Site Multihoming by IPv6 Intermediation WG (SHIM6)

Monday, March 20, 2006
1740-1840 Afternoon Session III
1850-1950 Afternoon Session IV
===============================

CHAIRS: Geoff Huston <[email-masked]>
         Kurtis Lindqvist <[email-masked]>

AGENDA

1. Agenda Bashing, Victimization of scribe
                                     [5 minutes]

     The agenda is broken into 2 sections. Agenda items 2 - 6 address
     topics related to the current set of drafts and necessary steps to
     progress these documents through the WG.  Agenda items 7 - 9 are a
     followup to the current consideration of (loosely phrased) "TE"
     considerations for possible extensions to Shim6 framework, and the
     related aspect of re-chartering this WG to study this topic.

2. Review of Document Status and WG Roadmap
    Chairs

       The question to the working group is whether to adopt
       draft-huitema-shim6-ingress-filtering-00 as a WG document

                                     [5 minutes]

                                     noone has read this document
                                     ** to the mailing list!

3. Review of Shim6 Protocol Specification
    draft-ietf-shim6-proto-03.txt  (-04 to be submitted)
    Erik Nordmark

      The question to the working group following a review of the
      document updates is whether the WG is ready for a WG Last Call on
      this document.  [15 minutes]

4. Review of Shim6 Failure Detection Specification
    draft-ietf-shim6-failure-detection-03.txt
    Jari Arkko
    Illitsch vav Beijnum

      The question to the working group following a review of the
      document updates is whether the WG is ready for a WG Last Call on
      this document.  [10 minutes]

5. Privacy Analysis for the SHIM6 protocol
    draft-bagnulo-shim6-privacy-00
    Marcelo Bagnulo

      Do the privacy issues identified in the draft need to be
      addressed? i.e. do we need to define some extensions to address
      the privacy concerns identified in the draft?  [15 minutes]

6. Default Address Selection in RFC3484
    draft-arifumi-ipv6-policy-dist-00.txt
    draft-fujisaki-dhc-addr-select-opt-01.txt
    Arifumi Matsumoto

      This draft raises some issues about default address selection of
      RFC 3484 and proposes some solutions towards them. (This is an
      informational briefing to the WG.)  [10 minutes]

7. Report on IAB BOFs on multi-homing
    Dave Meyer
                                     [10 minutes]

8. Extended Shim6 Design for ID/loc split and
    Traffic Engineering
    draft-nordmark-shim6-esd-00.txt
    Erik Nordmark
                                     [35 minutes]

9. WG Discussion on Re-Chartering
    Chairs

                                     [15 minutes]

Recommended Reading:
   a) active wg documents to be discussed at the meeting
       draft-ietf-shim6-proto-03
       draft-ietf-shim6-failure-detection-03
   b) other documents whose status will be considered
       draft-huitema-shim6-ingress-filtering-00.txt
       draft-bagnulo-shim6-privacy-00
       draft-arifumi-ipv6-policy-dist-00.txt
       draft-fujisaki-dhc-addr-select-opt-01.txt
       draft-nordmark-shim6-esd-00a.txt
   c) other shim6 documents that may be referred to
       draft-ietf-shim6-arch-00.txt
       draft-iietf-shim6-applicability-00.txt
       draft-ietf-shim6-applicability-00.txt

NOTES
=====

Notes by Nevil Brownlee

Intro by Kurt

Agenda bashing by Geoff

Overview: Doc Status, Road Map:
  3 core docs, IP-stack-centric view of shim6
  one i-d with ADs, waiting on other two.
  What do we need to do to complete these base specs?
  IAB BOF report on multihoming; site-based policies, TE
    shim6 may want to know details of transport(s)
  WG next steps

  Ingress filtering draft (Christian Huitema).  Should that be a WG doc?
  It is part of the wg charter.
  No-one has read it.
  Take to mailing list.

Part I:

Erik Nordmark: Core Spec draft, -04 changes
  03 to 04.  Mainly editorial.
    no IPv6 NATs now an explicit assumption.
    text ordering reworked
    Locator pereference options: requirement that any with element length >3
      must have flags, priority, ..
    retrans rules for I2bis (ocpied from I2 rules)
    fixed security hole where single message could cause CT (peer) to be
    updated.  Now require a 3way handshake.
    Context Recovery
  02 to 03
    Context recovery redone
    expanded context tag, 32 to 47 bits
    specified how context recovery and forked contexts work together
    picked initial retransmit times fir I1 and I2, 4s.  binary exp backoff.
    Require R1/R2bus verifiers be usable for some minimum time, initiator
    knows how long it is safe to retransmit I2 before going back to sending
    I1.
    Message type codes 0-63 don't generate R1bis messsages, 64-127 will
    lots of editorial changes

  Is it now ready for last call?

  Comments:
    Jim Bound: very political.  Thinks its OK, but we (or someone)
      should look at other options.
    Geoff response - shim6 follows multi6, it's not the only way.
    Jim: this is a lot of work to implement, could break things,
      doesn't see customer pressure for it.
    Geoff: Erik isn't saying "put it into production s/w" The WG question is
    "is this draft ready for last call?"
    Jim: Yes. Architecturally its a good thing.
    Thomas Narten: not trying to push shim6 as 'the' approach.  Who do we
      want to implement this?
    Geoff: WIDE Kame project is one example.  But not ready yet for
      companies to cimmit to making production code.
    Thomas: Expermental?  Plus app stmt saying "will eventually hit
      standards track"  Goal is to have it production-ready in, say,
      2 years?
    Dave Thaler: what's relationship betw shim6 / mobileIP6 / hip ?
      supports 'experimental'
    Jari Akko: need to look at this
    Dave: want community consensus on relationship between these.
      Should people implement all three?
    Kurt: chairs/ADs should try to make a doc on this.  Dave will
      co-author.
    Brian Carpenter: reminder about *proposed* standards
    Marcelo Bagnulo: mobileIP and HIP interactions.  What do you want to do?
      why would you use shim and hip together?
    Dave: want to know which scenarious are likely
    Jari: need to know how this works with exising stuff, not neccessary
      with new
    Margaret Waserman: Have a stack doing shim6, need to understand its
      interactions with others.  Which one turns off what??
      Group is chartered to make a stds track doc, and started with 32
      variations!
    Pekka Nikkander: draft on hip and mobileip exists
      pkt formagt for shim6 and hip is roughly the same, could implement
      them in same s/w module.
    Erik: can you run them on the same host? Yes.
      Are there cases when you need to use both together??
      Charter addresses optimisations.  Doc handwaves on this.

Jari Arko: Exploration
  redesign of exploration protocol
  -03 being considered for last call
  issues: termination states, needs more detail

    Erik: it doesn't, by itself, terminate
      Have exp terminate at time we throw away the context;
      fate sharing makes things simpler

   Timers:
     Erik: comparison with TCP timer settings.  Don't know whether
       probe packets will work, be aggressive, but not too aggressive
     Geoff: what is sensible limit to exponential backoff?

   Deprecated addresses (Marcelo)
     draft recoimmends moving away from deprecated addresses.
   Need to update the draft, and get more reviews
   John Loughney: has read draft, will senmd comments

   Geoff: will proceed to WG Last Call after next rev

Marcelo: Privacy Analysis
  Scenarios
    ULID-pair conext, estab exchange
    Packets with the payload header
    Update messages
    keepalive & probe msgs
    Draft doesn;t go into solution space
      Do we care about privacy?
    Erik: haven't written down: be careful with rfc 3041 temprary addresses,
      etc.  Need to write these down.  Hiding doesn;t seem to achieve much.
    Dave: never combine temp and non-temp address, 3041bis ??
    ?: 3041bis currently stalled

  Geoff: does anyone think that this should be part of the shim6
    applicability doc?  Agreed to fold that into the applicability doc.

Extensions to 3484 are currently with ADs rather than this WG.
They will decide what to do about further updates/revisions.

Arifumi Matsumoto: 3484 & shim
  initial contact must succeed if we use deferred shim context setup.
    if dest host isn't shim-enabled, you can;t reach it!
  initial contact failure reasons
    "you can't implement a large SO with ULA"
  Address selection policy forinitial contact
  Redundancy for initial contact - an app should cycle dst & src
  TE for initial contact
  3484 is useable, not enough for redundancy and TE
    No discussion

Dave Meyer
  Notes from Operator Workshops
  NANOG35  end-system complexity, DNS latency, inbound TE
      current multihoming is site based, not host based.  This is a big
      change.
      hosts and routers may not be managed by same group of people
    IPv6 routing and addressing architectures
      still an open problem
  APRICOT06
  Will continue the series, RIPE 52 in Turkey
    Scot Lybrand: present this at NANOG?  yes
    Jim Bound:  This is good, want to see more.
       need to get managters along to those meetings too!
    ?: go one-by-one to operators.

Part II:

Erik: Is shim6 a brick wall, or part of an interesting architecture?
    What's missing?   We need to determine whether shin6 could reduce
    pressure for IPv6 PI space
    ID-locator separation
    Unreachable ULID format - only know how to do scalable lookups
    from a hierarchically allocated 'name'
    Could do lookups in DNS (as an example)
    ULIDs could be in a reverse lookup, e.g. ip6.arpa
    get back SRV records
    Traffic Engineering
    Static and Dynamic TE
    Locator rewriting by routers

    Jim Bound:  Hasn't read draft yet, but - CGA isn't happening in the
        market, it has IPR problems, it breaks IPsec, ...

    Erik agrees it could break e2e encryption.  Receiver will see
        the IPv6 packet.
    Jim understands
    Leslie: if path changes, might inject different addresses -
         will shim6 switch back?  would end nodes have to know what the
         routers would inject??  No, user app would never see the shim6
         addresses

    IPv4 addresses as locators
    Conclusion: non-routabloe ULIDs doesn't place any new reqs on shim6
    mechanism.  If we want to use router rewriting, define it now.

    Dave: remarks about hip and CGA.  hip has many of the same issues.
      (lots of) convergence is good.
    Jim: Now wants to go to Last Call for shim6.
      Maybe we should just use identifiers (rather than addresses)?
      That would make shim same as hip.
      Erik likes having both adress-based and identity-based (?) IDs
      Geoff: recycling names is expensive, uniqueness is not cheap.
    Pekka: IPv4 - was IPv4 a header extension?
      Erik: not decided, no work done yet.
    Tom Henderson: hip research group has looked at what happens if
      you use non-routable identifiers.  Ongoing.

Geoff: we will explore these extensions Erik talked about.
  Request WG to start writing some drafts about an API.

Delivery-date: Tue, 21 Mar 2006 19:07:23 +0000
Date: Tue, 21 Mar 2006 14:06:21 -0500 (EST)
From: Scott Leibrand <[email-masked]>
To: [email-masked]
cc: [email-masked]
Subject: RE: Extending shim6 to use multiple locator pairs simultaneously

(Cc'ing the list on a private reply: hope that's OK...)

Ah.  In that case, then yes, I would agree that an implementation that
uses "TCP shim" that interacts with shim6 and accomplishes the same thing
as I described would be just as good.  In fact, I would go so far as to
say what they say for state machines: there's nothing to say you have to
do it this way, as long as the observed behavior is the same.

-Scott

On 03/21/06 at 10:22am -0800, Tony Li <[email-masked]> wrote:

>
> Sorry, I was assuming shim6 as part of the equation, for exactly the
> reasons that you suggest.
>
> T
>
>
> > -----Original Message-----
> > From: Scott Leibrand [mailto:[email-masked]]
> > Sent: Tuesday, March 21, 2006 10:16 AM
> > To: [email-masked]
> > Cc: 'marcelo bagnulo braun'; [email-masked]
> > Subject: RE: Extending shim6 to use multiple locator pairs
> > simultaneously
> >
> > On 03/21/06 at 10:08am -0800, Tony Li <[email-masked]> wrote:
> >
> > > Here's another thought that's worth what you paid for it:
> > >
> > > Instead, create a single TCP connection for each link that
> > you want to
> > > use.  Create a muxing layer on top of these connections
> > that distributes
> > > data across the connections in a possibly uneven manner, filling the
> > > buffers of each connection.  This layer would require some
> > additional
> > > framing and sequence numbering.  On the receiver side,
> > demux using the
> > > framing and then reorder the arrived data.  Now, you're
> > using the full
> > > bandwidth of all of the available links, and saturating
> > them up to their
> > > full TCP performance level.
> >
> > Sounds like BitTorrent.  :)
> >
> > The problem I see with this is in creating the multiple TCP
> > connections
> > for the multiple locator pairs.  Without some sort of signalling (aka
> > shim6 context establishment) the two hosts can't know each others'
> > locators.  Without that knowledge, you don't know if you're talking to
> > multiple locators on the same host, or multiple different
> > hosts proving
> > the same service (if you get the multiple IPs via DNS, for example).
> >
> > So I see the incorporation with shim6 as a better fit than trying to
> > re-create many of shim6's features at layer 4.
> >
> > -Scott
> >
>
>
>

Delivery-date: Tue, 21 Mar 2006 18:15:57 +0000
Date: Tue, 21 Mar 2006 13:15:44 -0500 (EST)
From: Scott Leibrand <[email-masked]>
To: [email-masked]
cc: 'marcelo bagnulo braun' <[email-masked]>, [email-masked]
Subject: RE: Extending shim6 to use multiple locator pairs simultaneously

On 03/21/06 at 10:08am -0800, Tony Li <[email-masked]> wrote:

> Here's another thought that's worth what you paid for it:
>
> Instead, create a single TCP connection for each link that you want to
> use.  Create a muxing layer on top of these connections that distributes
> data across the connections in a possibly uneven manner, filling the
> buffers of each connection.  This layer would require some additional
> framing and sequence numbering.  On the receiver side, demux using the
> framing and then reorder the arrived data.  Now, you're using the full
> bandwidth of all of the available links, and saturating them up to their
> full TCP performance level.

Sounds like BitTorrent.  :)

The problem I see with this is in creating the multiple TCP connections
for the multiple locator pairs.  Without some sort of signalling (aka
shim6 context establishment) the two hosts can't know each others'
locators.  Without that knowledge, you don't know if you're talking to
multiple locators on the same host, or multiple different hosts proving
the same service (if you get the multiple IPs via DNS, for example).

So I see the incorporation with shim6 as a better fit than trying to
re-create many of shim6's features at layer 4.

-Scott

Delivery-date: Tue, 21 Mar 2006 18:10:06 +0000
Reply-To: <[email-masked]>
From: "Tony Li" <[email-masked]>
To: "'Scott Leibrand'" <[email-masked]>, "'marcelo bagnulo braun'" <[email-masked]>
Cc: <[email-masked]>
Subject: RE: Extending shim6 to use multiple locator pairs simultaneously
Date: Tue, 21 Mar 2006 10:08:31 -0800

> Ok, that's certainly worth study.  In some mental modeling 
> I've done so
> far, the biggest problem I see in not using TCP congestion 
> control on each
> locator pair is in deciding how much traffic to send over 
> each link.  I
> suppose you could use both links equally, which would allow 
> you to use (#
> of links) * (the bandwidth of the slower link) rather than 
> the sum of the
> bandwidths of all the links.  IOW, as soon as one link experiences
> congestion, the sending rate would be halved over all links.

Here's another thought that's worth what you paid for it:

Instead, create a single TCP connection for each link that you want to
use.  Create a muxing layer on top of these connections that distributes
data across the connections in a possibly uneven manner, filling the
buffers of each connection.  This layer would require some additional
framing and sequence numbering.  On the receiver side, demux using the
framing and then reorder the arrived data.  Now, you're using the full
bandwidth of all of the available links, and saturating them up to their
full TCP performance level.

Regards,
Tony

Delivery-date: Tue, 21 Mar 2006 18:03:17 +0000
Cc: marcelo bagnulo braun <[email-masked]>, [email-masked]
From: Iljitsch van Beijnum <[email-masked]>
Subject: Re: Extending shim6 to use multiple locator pairs simultaneously
Date: Tue, 21 Mar 2006 19:02:29 +0100
To: Scott Leibrand <[email-masked]>

On 21-mrt-2006, at 17:25, Scott Leibrand wrote:

> In some mental modeling I've done so
> far, the biggest problem I see in not using TCP congestion control  
> on each
> locator pair is in deciding how much traffic to send over each link.

I've actually worked on some stuff where we load balanced a bunch of  
traffic over several TCP sessions towards different destinations  
(where it didn't matter which part of the aggregate traffic ended up  
where), and the results were interesting. When you get the hang of  
it, it works pretty well, though. The key issue is that you feed data  
to the right session/path. If you don't, you run the risk of being  
limited by the slowest one, and have performance that's even worse  
than not bonding the paths.

When I think about this, this may not be all that easy if only the  
sending side knows about what we're doing. You probably need to use  
selective acknowledgments and you may still trigger fast retransmit  
often and take TCP processing at the receiver off of the fast path.

And you'd have to be really careful to make sure that the unmodified  
side doesn't see confusing RTT estimates.

> I suppose you could use both links equally, which would allow you  
> to use (#
> of links) * (the bandwidth of the slower link) rather than the sum  
> of the
> bandwidths of all the links.

But that's exactly what you don't want. What if you have a 10 Mbit  
link and a 100 Mbit link? Then using 10 Mbit over both gives you 20  
total, while the fast link alone can give you 100. What you really  
want is 110.

Delivery-date: Tue, 21 Mar 2006 17:52:59 +0000
Cc: Scott Leibrand <[email-masked]>, [email-masked]
From: Iljitsch van Beijnum <[email-masked]>
Subject: Re: Extending shim6 to use multiple locator pairs simultaneously
Date: Tue, 21 Mar 2006 18:52:17 +0100
To: marcelo bagnulo braun <[email-masked]>

On 21-mrt-2006, at 17:08, marcelo bagnulo braun wrote:

>> Now say that he
>> has implemented some shim6 extensions that allow him to  
>> simultaneously
>> send packets with the shim6 context tag over the second locator  
>> pair while
>> continuing to send packets over the ULID-ULID pair.  These  
>> extensions also
>> include modifications to his TCP stack such that he can independently
>> track the congestion windows (and run the TCP congestion avoidance
>> algorithm) for each locator pair.

> But if you need to change both the shim and TCP perhaps this is too  
> many changes...

As long as the changes can be made to one end of the communication  
independently this shouldn't be much of a problem.

> First i would like to understand what would be the effect of using  
> multiple locator pairs  in order to use multiple parallel paths  
> without modifying TCP. I guess that this would be nasty for TCP,  
> but i would like to make quantify the results of such approach. I  
> mean, having the shim to support this wihtout requiring  
> modifications to TCP would make this is much more attractive i guess

How are you going to handle congestion issues?

We need to figure that out for regular shim stuff anyway, though.  
When a gigE link goes down and you get to rehome to 28 kbps GPRS link  
you REALLY don't want to do congestion control the hard way... It has  
been suggested in the past that a rehoming should trigger slow start,  
but I remember someone disagreeing with this, don't remember why.

We may even want to go so far as to drop packets at the source in the  
shim to enforce this...

> i guess that we need to understand the impact of this in the shim6  
> base protocol and the failure detection mechanismm, since both of  
> them are currently designed to use a single locator pair to  
> exchange traffic.

I think this would mostly be an API issue: TCP would have to be able  
to tell the shim to test extra paths even if there is no failure, and  
tell the shim which packets should go over which path. Essentially,  
the upper layer would be making the decisions and the shim would only  
be used as a bag of tools to do what the ULP wants to happen.

>> If
>> not, should I start writing up an Internet-Draft on it?  I brought  
>> this up
>> in the shim6 Jabber room today, and was told that this might be  
>> logical
>> followup work to the base shim6 spec.

Blame me for that.  :-)

>> Do others agree with that
>> assessment?  (This is my first IETF, so if anyone has any other  
>> newbie
>> advice, feel free to speak up.)

You may want to review the shim6 and multi6 archives, but those are  
big...

Do you think the issues are clear enough to be able to write a draft  
at this point? And of course you may want to check with some TCP  
people...

Delivery-date: Tue, 21 Mar 2006 16:57:52 +0000
Cc: [email-masked]
From: marcelo bagnulo braun <[email-masked]>
Subject: Re: Extending shim6 to use multiple locator pairs simultaneously
Date: Tue, 21 Mar 2006 10:46:59 -0600
To: Scott Leibrand <[email-masked]>

El 21/03/2006, a las 10:25, Scott Leibrand escribi=F3:
>
>> First i would like to understand what would be the effect of using
>> multiple locator pairs  in order to use multiple parallel paths=20
>> without
>> modifying TCP. I guess that this would be nasty for TCP, but i would
>> like to make quantify the results of such approach. I mean, having =
the
>> shim to support this without requiring modifications to TCP would =
make
>> this is much more attractive i guess
>
> Ok, that's certainly worth study.  In some mental modeling I've done =
so
> far, the biggest problem I see in not using TCP congestion control on=20=

> each
> locator pair is in deciding how much traffic to send over each link.  =
I
> suppose you could use both links equally, which would allow you to use=20=

> (#
> of links) * (the bandwidth of the slower link) rather than the sum of=20=

> the
> bandwidths of all the links.  IOW, as soon as one link experiences
> congestion, the sending rate would be halved over all links.

well, you have the preference option that would allow to the receiver=20
to inform about how he woudl preffer to distribute incoming load, which=20=

could possibly reflect the access bandwidth that the receiver has, but=20=

this of course does not reflect the badwidth of the path nor congestion=20=

issues.
having congestion window information would be indeed useful for this

But i see two levels of issues here:
- on one hand we have the issue of whether using this multiple path=20
extension wihtout any TCP mofiffication would result in problems for=20
TCP and even degraded performance. I mean it may be the case that=20
becuase of reordering, using the shim to exchamge packets through=20
multiple paths simultaneously results in worse perfomance than using a=20=

single path. If this is the case, then it does not make sense to use=20
this without upgrading TCP
- on the other hand it may be the case that using the shim to use=20
multiple paths for the same communication simultaneously result in=20
ehanced performance without any TCP modification but it can indeed=20
perform better if TCP is modified or at least, as you mention there is=20=

a richer information exchange between TCP and the shim, like congestion=20=

window information exchnge

I guess that the fist step would be to clear out the first issue and=20
then move on

regards, marcelo

>
>>> This would allow the sending host to fully utilize the available
>>> bandwidth on both his links for a single TCP session, which =
otherwise
>>> would be impossible (without bonding the two links with some sort of
>>> per-packet load balancing, which you can do for outbound, but not =
for
>>> inbound with two different providers).  It would require some
>>> modifications to the sending host in order to keep track of
>>> everything, but the receiving host need only do ordinary shim6 and=20=

>>> ack
>>> packets as they came in.
>>
>> i guess that we need to understand the impact of this in the shim6=20
>> base
>> protocol and the failure detection mechanism, since both of them are
>> currently designed to use a single locator pair to exchange traffic.
>> I don't think that there would be much trouble with supporting this,
>> though
>
> Yes, definitely.  I've read the I-D's, but I should probably re-read=20=

> them
> with this in mind and see if I can better understand the impact...
>
>>>  Since the shim would be in place, the recipient's TCP stack would =
be
>>> completely unaware of the fact that his packets are coming over two
>>> different paths.
>>>
>>> Can anyone think of any show-stopper reasons why this wouldn't work?
>>
>> I would like to understand how badly this behave if we use it without
>> updating TCP
>
> Ok.  Let me do some quick modeling and confirm that my mental model=20
> above
> seems to be correct.  I'll also have to consider whether out-of-order
> packet arrival will be much of a problem...
>
> Thanks,
> Scott
>

Delivery-date: Tue, 21 Mar 2006 16:32:40 +0000
Cc: "shim6-wg" <[email-masked]>
From: marcelo bagnulo braun <[email-masked]>
Subject: Re: shim - mip interaction
Date: Tue, 21 Mar 2006 10:32:30 -0600
To: "Henderson, Thomas R" <[email-masked]>

El 21/03/2006, a las 10:24, Henderson, Thomas R escribi=F3:

>
>
>> -----Original Message-----
>> From: marcelo bagnulo braun [mailto:[email-masked]]
>> Sent: Tuesday, March 21, 2006 7:34 AM
>> To: Henderson, Thomas R
>> Cc: shim6-wg
>> Subject: Re: shim - mip interaction
>>
>> Hi Tom
>>
>> thanks for reading the draft and commenting
>>
>> The idea is to include the new CoAs in the locator set of the
>> peer for
>> the established shim6 context using the UPDATE request
>> message and CGA
>> validation
>>
>> Hope this answers your question
>
> Thanks, it does.  Also, the monami6 WG seems to be studying the
> interaction between multihoming and mipv6, although they explicitly
> state that they haven't yet considered the shim6 work:
> http://www.ietf.org/internet-drafts/draft-ietf-monami6-mipv6-analysis=20=

> -00
> .txt
>

well, i guess that the work monami6 is doing in the solution space is =20=

very focused in a limited set of tools i.e. multiple coas registration =20=

and policy about how to use this multiple coas.
In particular, multiple HoAs support or fault tolerance i.e. how to =20
detect failures in the paths and explore alternative paths is out of =20
the scope of their work

regards, marcelo

> Tom
>

Delivery-date: Tue, 21 Mar 2006 16:25:33 +0000
Date: Tue, 21 Mar 2006 11:25:26 -0500 (EST)
From: Scott Leibrand <[email-masked]>
To: marcelo bagnulo braun <[email-masked]>
cc: [email-masked]
Subject: Re: Extending shim6 to use multiple locator pairs simultaneously

On 03/21/06 at 10:08am -0600, marcelo bagnulo braun <[email-masked]> wr=
ote:

> This is an interesting extension of shim6 indeed...

Thanks.

> some comments below...

Ditto.

> El 20/03/2006, a las 22:51, Scott Leibrand escribi=F3:
>
> > Say you have two shim6-capable hosts, and at least one of them has two
> > transit connections (say DSL + Cable) with two corresponding locators.
> > Now say the two hosts establish a connection, start sending traffic,
> > and establish shim6 state.  Now say that the multihomed host has lots
> > of data to send, and is maxing out the default (ULID-ULID) path.  Now
> > say that he has implemented some shim6 extensions that allow him to
> > simultaneously send packets with the shim6 context tag over the second
> > locator pair while continuing to send packets over the ULID-ULID pair.
> > These extensions also include modifications to his TCP stack such that
> > he can independently track the congestion windows (and run the TCP
> > congestion avoidance algorithm) for each locator pair.
>
> But if you need to change both the shim and TCP perhaps this is too
> many changes...

Perhaps.  But as others have pointed out privately, TCP and IP are usually
implemented together as TCP/IP.  So for implementors this would probably
be less of a problem than for the IETF, where IP and TCP are under the
purview of different WG's (and even Areas?).  Perhaps we could specify a
shim6-only implementation, and simply point out that implementors may make
further optimizations to TCP to improve performance if they wish...

> First i would like to understand what would be the effect of using
> multiple locator pairs  in order to use multiple parallel paths without
> modifying TCP. I guess that this would be nasty for TCP, but i would
> like to make quantify the results of such approach. I mean, having the
> shim to support this without requiring modifications to TCP would make
> this is much more attractive i guess

Ok, that's certainly worth study.  In some mental modeling I've done so
far, the biggest problem I see in not using TCP congestion control on each
locator pair is in deciding how much traffic to send over each link.  I
suppose you could use both links equally, which would allow you to use (#
of links) * (the bandwidth of the slower link) rather than the sum of the
bandwidths of all the links.  IOW, as soon as one link experiences
congestion, the sending rate would be halved over all links.

> > This would allow the sending host to fully utilize the available
> > bandwidth on both his links for a single TCP session, which otherwise
> > would be impossible (without bonding the two links with some sort of
> > per-packet load balancing, which you can do for outbound, but not for
> > inbound with two different providers).  It would require some
> > modifications to the sending host in order to keep track of
> > everything, but the receiving host need only do ordinary shim6 and ack
> > packets as they came in.
>
> i guess that we need to understand the impact of this in the shim6 base
> protocol and the failure detection mechanism, since both of them are
> currently designed to use a single locator pair to exchange traffic.
> I don't think that there would be much trouble with supporting this,
> though

Yes, definitely.  I've read the I-D's, but I should probably re-read them
with this in mind and see if I can better understand the impact...

> >  Since the shim would be in place, the recipient's TCP stack would be
> > completely unaware of the fact that his packets are coming over two
> > different paths.
> >
> > Can anyone think of any show-stopper reasons why this wouldn't work?
>
> I would like to understand how badly this behave if we use it without
> updating TCP

Ok.  Let me do some quick modeling and confirm that my mental model above
seems to be correct.  I'll also have to consider whether out-of-order
packet arrival will be much of a problem...

Thanks,
Scott

Delivery-date: Tue, 21 Mar 2006 16:25:11 +0000
Content-class: urn:content-classes:message
Subject: RE: shim - mip interaction
Date: Tue, 21 Mar 2006 08:24:52 -0800
From: "Henderson, Thomas R" <[email-masked]>
To: "marcelo bagnulo braun" <[email-masked]>
Cc: "shim6-wg" <[email-masked]>

=20

> -----Original Message-----
> From: marcelo bagnulo braun [mailto:[email-masked]]=20
> Sent: Tuesday, March 21, 2006 7:34 AM
> To: Henderson, Thomas R
> Cc: shim6-wg
> Subject: Re: shim - mip interaction
>=20
> Hi Tom
>=20
> thanks for reading the draft and commenting
>=20
> The idea is to include the new CoAs in the locator set of the=20
> peer for=20
> the established shim6 context using the UPDATE request=20
> message and CGA=20
> validation
>=20
> Hope this answers your question

Thanks, it does.  Also, the monami6 WG seems to be studying the
interaction between multihoming and mipv6, although they explicitly
state that they haven't yet considered the shim6 work:
http://www.ietf.org/internet-drafts/draft-ietf-monami6-mipv6-analysis-00
.txt

Tom

Delivery-date: Tue, 21 Mar 2006 16:08:41 +0000
Cc: [email-masked]
From: marcelo bagnulo braun <[email-masked]>
Subject: Re: Extending shim6 to use multiple locator pairs simultaneously
Date: Tue, 21 Mar 2006 10:08:30 -0600
To: Scott Leibrand <[email-masked]>

Hi Scott,

This is an interesting extension of shim6 indeed...

some comments below...
El 20/03/2006, a las 22:51, Scott Leibrand escribi=F3:

> I recently had an idea that I thought I should bring up here (then
> possibly write up as an I-D):
>
> Say you have two shim6-capable hosts, and at least one of them has two
> transit connections (say DSL + Cable) with two corresponding locators.
> Now say the two hosts establish a connection, start sending traffic,=20=

> and
> establish shim6 state.  Now say that the multihomed host has lots of=20=

> data
> to send, and is maxing out the default (ULID-ULID) path.  Now say that=20=

> he
> has implemented some shim6 extensions that allow him to simultaneously
> send packets with the shim6 context tag over the second locator pair=20=

> while
> continuing to send packets over the ULID-ULID pair.  These extensions=20=

> also
> include modifications to his TCP stack such that he can independently
> track the congestion windows (and run the TCP congestion avoidance
> algorithm) for each locator pair.
>

But if you need to change both the shim and TCP perhaps this is too=20
many changes...

First i would like to understand what would be the effect of using=20
multiple locator pairs  in order to use multiple parallel paths without=20=

modifying TCP. I guess that this would be nasty for TCP, but i would=20
like to make quantify the results of such approach. I mean, having the=20=

shim to support this wihtout requiring modifications to TCP would make=20=

this is much more attractive i guess

> This would allow the sending host to fully utilize the available=20
> bandwidth
> on both his links for a single TCP session, which otherwise would be
> impossible (without bonding the two links with some sort of per-packet
> load balancing, which you can do for outbound, but not for inbound =
with
> two different providers).  It would require some modifications to the
> sending host in order to keep track of everything, but the receiving=20=

> host
> need only do ordinary shim6 and ack packets as they came in.

i guess that we need to understand the impact of this in the shim6 base=20=

protocol and the failure detection mechanismm, since both of them are=20
currently designed to use a single locator pair to exchange traffic.
I don't think that there would be much trouble with supporting this,=20
though

>  Since the
> shim would be in place, the recipient's TCP stack would be completely
> unaware of the fact that his packets are coming over two different=20
> paths.
>
> Can anyone think of any show-stopper reasons why this wouldn't work?

I would like to understand how badly this behave if we use it without=20
updating TCP

regards, marcelo

>   If
> not, should I start writing up an Internet-Draft on it?  I brought=20
> this up
> in the shim6 Jabber room today, and was told that this might be =
logical
> followup work to the base shim6 spec.  Do others agree with that
> assessment?  (This is my first IETF, so if anyone has any other newbie
> advice, feel free to speak up.)
>
> If I'm not missing anything big, and this is indeed workable, I think=20=

> this
> could be a big enough feature to help push shim6 adoption onto a wide
> installed base.
>
> Thanks,
> Scott
>

Delivery-date: Tue, 21 Mar 2006 15:34:11 +0000
Cc: "shim6-wg" <[email-masked]>
From: marcelo bagnulo braun <[email-masked]>
Subject: Re: shim - mip interaction
Date: Tue, 21 Mar 2006 09:33:41 -0600
To: "Henderson, Thomas R" <[email-masked]>

Hi Tom

thanks for reading the draft and commenting

The idea is to include the new CoAs in the locator set of the peer for=20=

the established shim6 context using the UPDATE request message and CGA=20=

validation

Hope this answers your question

regards, marcelo

El 21/03/2006, a las 2:02, Henderson, Thomas R escribi=F3:

> Marcelo,
> I didn't understand the part about the CoAs being included in the
> locator set-- you don't necessarily know future CoAs when you =
establish
> context, or are you talking about changing contexts when your locator
> set changes?
>
> Thanks,
> Tom
>
>> -----Original Message-----
>> From: marcelo bagnulo braun [mailto:[email-masked]]
>> Sent: Monday, March 20, 2006 4:35 PM
>> To: shim6-wg
>> Subject: shim - mip interaction
>>
>> Some months ago, Erik and I wrote a draft about shim-mip interaction.
>> It is no longer available in the ID directory, but you can still find
>> it in:
>>
>> http://www.watersprings.org/pub/id/draft-bagnulo-shim6-mip-00.txt
>>
>> Regards, marcelo
>>
>>
>>
>

Delivery-date: Tue, 21 Mar 2006 13:42:04 +0000
Date: Tue, 21 Mar 2006 08:41:02 -0500 (EST)
From: Scott Leibrand <[email-masked]>
To: [email-masked]
cc: [email-masked]
Subject: RE: Extending shim6 to use multiple locator pairs simultaneously

Yeah.  The main motivation on my part for doing an RFC would be that if
I'm a multihomed user seeking to benefit from this, I would want the
server(s) I'm downloading from to support it as well, so they could send
me traffic over both my links.  If I just implement it on my own host, I
would only be able to upload with both links, not download.

-Scott

On 03/20/06 at 11:31pm -0800, Tony Li <[email-masked]> wrote:

>
> Scott,
>
> I think that would indeed work.  Further, as that is entirely contained
> within a single implementation and doesn't require any protocol changes,
> I don't think it requires an RFC.  Not that one wouldn't be welcome, of
> course.
>
> Regards,
> Tony
>
>
> > -----Original Message-----
> > From: [email-masked] [mailto:[email-masked]] On
> > Behalf Of Scott Leibrand
> > Sent: Monday, March 20, 2006 8:51 PM
> > To: [email-masked]
> > Subject: Extending shim6 to use multiple locator pairs simultaneously
> >
> > I recently had an idea that I thought I should bring up here (then
> > possibly write up as an I-D):
> >
> > Say you have two shim6-capable hosts, and at least one of them has two
> > transit connections (say DSL + Cable) with two corresponding locators.
> > Now say the two hosts establish a connection, start sending
> > traffic, and
> > establish shim6 state.  Now say that the multihomed host has
> > lots of data
> > to send, and is maxing out the default (ULID-ULID) path.  Now
> > say that he
> > has implemented some shim6 extensions that allow him to simultaneously
> > send packets with the shim6 context tag over the second
> > locator pair while
> > continuing to send packets over the ULID-ULID pair.  These
> > extensions also
> > include modifications to his TCP stack such that he can independently
> > track the congestion windows (and run the TCP congestion avoidance
> > algorithm) for each locator pair.
> >
> > This would allow the sending host to fully utilize the
> > available bandwidth
> > on both his links for a single TCP session, which otherwise would be
> > impossible (without bonding the two links with some sort of per-packet
> > load balancing, which you can do for outbound, but not for
> > inbound with
> > two different providers).  It would require some modifications to the
> > sending host in order to keep track of everything, but the
> > receiving host
> > need only do ordinary shim6 and ack packets as they came in.
> > Since the
> > shim would be in place, the recipient's TCP stack would be completely
> > unaware of the fact that his packets are coming over two
> > different paths.
> >
> > Can anyone think of any show-stopper reasons why this
> > wouldn't work?  If
> > not, should I start writing up an Internet-Draft on it?  I
> > brought this up
> > in the shim6 Jabber room today, and was told that this might
> > be logical
> > followup work to the base shim6 spec.  Do others agree with that
> > assessment?  (This is my first IETF, so if anyone has any other newbie
> > advice, feel free to speak up.)
> >
> > If I'm not missing anything big, and this is indeed workable,
> > I think this
> > could be a big enough feature to help push shim6 adoption onto a wide
> > installed base.
> >
> > Thanks,
> > Scott
> >
> >
>
>
>
>

Delivery-date: Tue, 21 Mar 2006 08:02:15 +0000
Content-class: urn:content-classes:message
Subject: RE: shim - mip interaction
Date: Tue, 21 Mar 2006 00:02:05 -0800
From: "Henderson, Thomas R" <[email-masked]>
To: "marcelo bagnulo braun" <[email-masked]>, "shim6-wg" <[email-masked]>

Marcelo,
I didn't understand the part about the CoAs being included in the
locator set-- you don't necessarily know future CoAs when you establish
context, or are you talking about changing contexts when your locator
set changes?

Thanks,
Tom=20

> -----Original Message-----
> From: marcelo bagnulo braun [mailto:[email-masked]]=20
> Sent: Monday, March 20, 2006 4:35 PM
> To: shim6-wg
> Subject: shim - mip interaction
>=20
> Some months ago, Erik and I wrote a draft about shim-mip interaction.
> It is no longer available in the ID directory, but you can still find=20
> it in:
>=20
> http://www.watersprings.org/pub/id/draft-bagnulo-shim6-mip-00.txt
>=20
> Regards, marcelo
>=20
>=20
>=20

Delivery-date: Tue, 21 Mar 2006 07:53:18 +0000
Content-class: urn:content-classes:message
Subject: RE: [Fwd: I-D ACTION:draft-nordmark-shim6-esd-00.txt]
Date: Mon, 20 Mar 2006 23:53:00 -0800
From: "Henderson, Thomas R" <[email-masked]>
To: "Erik Nordmark" <[email-masked]>, "shim6" <[email-masked]>

As I briefly mentioned today, there has been some complementary work in
the HIP RG that discusses the handling of non-routable identifiers in
legacy applications:
http://www.ietf.org/internet-drafts/draft-henderson-hip-applications-02.
txt,=20
the main differences being the use of KHI (now ORCHIDs) in HIP instead
of CGAs. =20

Until recently, the HIP drafts defined a "Type 2" HIT with the property
that the upper 64 bits contained support for two levels of hierarchical
naming (enabling reverse resolution), with the lower bits being drawn
from a hash of the public key, but this HIT type was dropped due to lack
of interest last year:
http://www1.ietf.org/mail-archive/web/hipsec/current/msg01519.html
It was also felt by some that 64 bits of hash was insufficient to
protect the binding between HIT and public key.

Tom

Delivery-date: Tue, 21 Mar 2006 07:33:48 +0000
Reply-To: <[email-masked]>
From: "Tony Li" <[email-masked]>
To: "'Scott Leibrand'" <[email-masked]>, <[email-masked]>
Subject: RE: Extending shim6 to use multiple locator pairs simultaneously
Date: Mon, 20 Mar 2006 23:31:36 -0800

Scott,

I think that would indeed work.  Further, as that is entirely contained
within a single implementation and doesn't require any protocol changes,
I don't think it requires an RFC.  Not that one wouldn't be welcome, of
course.

Regards,
Tony

> -----Original Message-----
> From: [email-masked] [mailto:[email-masked]] On 
> Behalf Of Scott Leibrand
> Sent: Monday, March 20, 2006 8:51 PM
> To: [email-masked]
> Subject: Extending shim6 to use multiple locator pairs simultaneously
> 
> I recently had an idea that I thought I should bring up here (then
> possibly write up as an I-D):
> 
> Say you have two shim6-capable hosts, and at least one of them has two
> transit connections (say DSL + Cable) with two corresponding locators.
> Now say the two hosts establish a connection, start sending 
> traffic, and
> establish shim6 state.  Now say that the multihomed host has 
> lots of data
> to send, and is maxing out the default (ULID-ULID) path.  Now 
> say that he
> has implemented some shim6 extensions that allow him to simultaneously
> send packets with the shim6 context tag over the second 
> locator pair while
> continuing to send packets over the ULID-ULID pair.  These 
> extensions also
> include modifications to his TCP stack such that he can independently
> track the congestion windows (and run the TCP congestion avoidance
> algorithm) for each locator pair.
> 
> This would allow the sending host to fully utilize the 
> available bandwidth
> on both his links for a single TCP session, which otherwise would be
> impossible (without bonding the two links with some sort of per-packet
> load balancing, which you can do for outbound, but not for 
> inbound with
> two different providers).  It would require some modifications to the
> sending host in order to keep track of everything, but the 
> receiving host
> need only do ordinary shim6 and ack packets as they came in.  
> Since the
> shim would be in place, the recipient's TCP stack would be completely
> unaware of the fact that his packets are coming over two 
> different paths.
> 
> Can anyone think of any show-stopper reasons why this 
> wouldn't work?  If
> not, should I start writing up an Internet-Draft on it?  I 
> brought this up
> in the shim6 Jabber room today, and was told that this might 
> be logical
> followup work to the base shim6 spec.  Do others agree with that
> assessment?  (This is my first IETF, so if anyone has any other newbie
> advice, feel free to speak up.)
> 
> If I'm not missing anything big, and this is indeed workable, 
> I think this
> could be a big enough feature to help push shim6 adoption onto a wide
> installed base.
> 
> Thanks,
> Scott
> 
> 

Delivery-date: Tue, 21 Mar 2006 04:52:08 +0000
Date: Mon, 20 Mar 2006 23:51:20 -0500 (EST)
From: Scott Leibrand <[email-masked]>
To: [email-masked]
Subject: Extending shim6 to use multiple locator pairs simultaneously

I recently had an idea that I thought I should bring up here (then
possibly write up as an I-D):

Say you have two shim6-capable hosts, and at least one of them has two
transit connections (say DSL + Cable) with two corresponding locators.
Now say the two hosts establish a connection, start sending traffic, and
establish shim6 state.  Now say that the multihomed host has lots of data
to send, and is maxing out the default (ULID-ULID) path.  Now say that he
has implemented some shim6 extensions that allow him to simultaneously
send packets with the shim6 context tag over the second locator pair while
continuing to send packets over the ULID-ULID pair.  These extensions also
include modifications to his TCP stack such that he can independently
track the congestion windows (and run the TCP congestion avoidance
algorithm) for each locator pair.

This would allow the sending host to fully utilize the available bandwidth
on both his links for a single TCP session, which otherwise would be
impossible (without bonding the two links with some sort of per-packet
load balancing, which you can do for outbound, but not for inbound with
two different providers).  It would require some modifications to the
sending host in order to keep track of everything, but the receiving host
need only do ordinary shim6 and ack packets as they came in.  Since the
shim would be in place, the recipient's TCP stack would be completely
unaware of the fact that his packets are coming over two different paths.

Can anyone think of any show-stopper reasons why this wouldn't work?  If
not, should I start writing up an Internet-Draft on it?  I brought this up
in the shim6 Jabber room today, and was told that this might be logical
followup work to the base shim6 spec.  Do others agree with that
assessment?  (This is my first IETF, so if anyone has any other newbie
advice, feel free to speak up.)

If I'm not missing anything big, and this is indeed workable, I think this
could be a big enough feature to help push shim6 adoption onto a wide
installed base.

Thanks,
Scott

Delivery-date: Tue, 21 Mar 2006 00:42:04 +0000
Content-class: urn:content-classes:message
Subject: RE: shim - mip interaction
Date: Tue, 21 Mar 2006 02:41:36 +0200
From: <[email-masked]>
To: <[email-masked]>, <[email-masked]>

Marcelo,

I think some of this content would be useful to put into the
applicability=20
statement document.  My feeling is that SHIM6 would feel a lot less
'experimental'
if there was some analysis written down on how SHIM6 interacts with
other IPv6
features (including MIPv6).

John=20

>-----Original Message-----
>From: [email-masked] [mailto:[email-masked]] On=20
>Behalf Of ext marcelo bagnulo braun
>Sent: 21 March, 2006 02:35
>To: shim6-wg
>Subject: shim - mip interaction
>
>Some months ago, Erik and I wrote a draft about shim-mip interaction.
>It is no longer available in the ID directory, but you can=20
>still find it in:
>
>http://www.watersprings.org/pub/id/draft-bagnulo-shim6-mip-00.txt
>
>Regards, marcelo
>
>
>

Delivery-date: Tue, 21 Mar 2006 00:36:18 +0000
To: shim6-wg <[email-masked]>
From: marcelo bagnulo braun <[email-masked]>
Subject: shim - mip interaction
Date: Mon, 20 Mar 2006 18:34:40 -0600

Some months ago, Erik and I wrote a draft about shim-mip interaction.
It is no longer available in the ID directory, but you can still find 
it in:

http://www.watersprings.org/pub/id/draft-bagnulo-shim6-mip-00.txt

Regards, marcelo

Delivery-date: Wed, 15 Mar 2006 09:38:09 +0000
Subject: Re: TE & SHIM6 (was Re: comments on draft-ietf-shim6-proto-03
From: Jeroen Massar <[email-masked]>
To: Erik Nordmark <[email-masked]>
Cc: shim6-wg <[email-masked]>
Organization: Unfix
Date: Wed, 15 Mar 2006 10:37:23 +0100

--=-QQMmaKvgqVSAdbW0uSNR
Content-Type: text/plain
Content-Transfer-Encoding: quoted-printable

On Tue, 2006-03-14 at 14:22 -0800, Erik Nordmark wrote:
> Iljitsch van Beijnum wrote:
[..big snip..]
> > The latter would probably be desireable for sites that want to optimize=
=20
> > for TE or want to balance incoming sessions over different hosts.
>=20
> I'm having a hard time seeing how this can be done without introducing=20
> several additional SRV lookups; being able to express all these in a=20
> single SRV lookup would be hard. (Hmm, unless we take the=20
> _ip.www.example.com hack above and hack it some more by overloading the=20
> returned port number field to have semantics like the above. At that=20
> point in time it might be better to introduce a new DNS RR type.)

For that matter, why isn't there an AAAA/A which looks somewhat like:
(PAAAA/PA =3D Prioritized AAAA/A)

host.example.org PAAAA 100 2001:db8:1000::1
                 PAAAA 200 2001:db8:2000::2
                 PAAAA 300 2001:db8:3000::3

Thus giving preference to the ::1 address, if that fails, try ::2 etc.
This would allow one to specify their 'primary' provider and their
backups. One could also mimic SRV completely adding the balancing
options. I actually guess that in many implementations the following
will work already though:

host.example.org SRV 100 50 0 2001:db8:1000::1
                 SRV 100 50 0 2001:db8:2000::2
                 SRV 200 20 0 2001:db8:3000::3

Just like folks inserting IP addresses directly in MX's. Most tools use
getaddrinfo() or gethostaddr() or similar to look it up and these
recognize the IP address and use that directly.

Greets,
 Jeroen

--=-QQMmaKvgqVSAdbW0uSNR
Content-Type: application/pgp-signature; name=signature.asc
Content-Description: This is a digitally signed message part

--=-QQMmaKvgqVSAdbW0uSNR--

Delivery-date: Tue, 14 Mar 2006 22:25:22 +0000
Date: Tue, 14 Mar 2006 14:22:56 -0800
From: Erik Nordmark <[email-masked]>
To: Iljitsch van Beijnum <[email-masked]>
CC: shim6-wg <[email-masked]>
Subject: Re: TE & SHIM6 (was Re: comments on draft-ietf-shim6-proto-03

[Sorry for the delay in responding]

Iljitsch van Beijnum wrote:

>> Or next hdr = IPPROTO_SHIM6.
> 
> I don't find this very suitable. If we're going to send many shimmed 
> packets, it's more important than ever that we omit the shim header 
> whenever possible.

I know packets don't have infinite size, but if we were willing to have
the IP header grow 20 bytes to make it possible to give an IPv6 address
to every cell phone and light switch, then perhaps we can allow
ourselves to have the header grow 8 bytes if this enable the routing
system to scale.

> Apart from that, using the source address to signal 
> that the source address may be changed is much cleaner. It also has the 
> advantage that we can now borrow some bits  to make the process easier. 
> What we can do is have shim6 capable hosts emit a "source rewriting 
> information request". That would be a packet addressed to a shim6 
> correspondent that has the magic prefix in the source address that 
> triggers source address rewriting, and an additional bit combination 
> that tells the router to send back a list of prefixes it will use to 
> rewrite. The host can then make sure that the correspondent knows to 
> expect packets with these source addresses.

To me this has some downsides. For one there is the deployment
constraint that if there isn't a rewriting router, then things just
fail. Thus the hosts need to behave different when there is no rewriting
exit router.
I don't know enough about current router implementation considerations
to say whether having some bit pattern trigger sending an information
back to the source is hard or easy, but it seems like uneccessary
complication for the router. The fact that it rewrote the packet
provides the necessary information.

> I've been thinking about something similar for traffic engineering ever 
> since my message yesterday where I mentioned A6 records. The problem is 
> that it's far from inconceivable that at some point, a disconnect forms 
> between the info in the DNS and the actual state of the network. The way 
> I see it, we have four ways to convey TE related info:
> 
> 1. out of band end-to-end: this would be stuff in the DNS
> 2. out of band hop-by-hop: BGP is like this
> 3. in-band end-to-end: measured timing and packet loss information
> 4. in-band hop-by-hop: feedback from routers

This is quite a useful categorization.

> The problem is that 2. needs aggregation to scale. 3. and 4. need to 
> have contact with the correspondent already, so it's useless in some 
> cases, like in the case where we want one or more backup addresses that 
> are only tried if the primary addresses don't work. The only way to 
> convey this is with 1. We can either reuse SRV records for individual 
> services for this, which has the advantage that it's already available 
> today, but the disadvantage that this mechanism isn't really used and it 
> needs to be supported on an application-by-application basis. 
> Alternatively, we can do some magic in the resolver library to make this 
> happen.

If one could define SRV to be for the "IP" service, then one might not
have to modify the applications. Thus we'd have SRV records for names like
	_ip.www.example.com
and the port number field wouldn't be used.
With such a hack one can make getaddrinfo() look for such things before
looking for AAAA and A records, without any application impact.

> But this doesn't really make it possible to react to traffic engineering 
> events in anything close to real time, if at all (DNS may not be 
> accessible by people who need to do TE.) The thing is, BGP isn't all 
> that great for this either: with current multihoming, you can't engineer 
> traffic such that link 1 gets the first 10 megabits, then everything 
> between 10 and 15 goes to link 2 and if there's more than 15 Mbit it's 
> balanced over the two links in a 2:1 ratio. (Believe me, this doesn't 
> stop people from asking.)
> 
> But an in-band hop-by-hop TE mechanism would allow exactly this. The way 
> it would work is that routers are configured to provide feedback for 
> packets with a shim header, if necessary. This feedback would be in the 
> form of entries that go into the address selection policy table. The 
> site egress router would probably want to inform hosts about which 
> source addresses go well with certain destination prefixes. All routers 
> between the source and destination (including the site exit router) 
> would signal back "this prefix (which would be the prefix that the 
> destination for the packets falls into) preference value XXX". To avoid 
> trouble, the preference value shouldn't be allowed to completely 
> override locally configured info.

Isn't locator rewriting an easier and more scalable way to accomplish
the dynamic TE? If need to express what source prefixes to with what
destination prefixes there is potentially a

> Last but not least: it's probably useful to use SRV records (if we're 
> going to use those anyway) to tell hosts that:
> 
> - they shouldn't initiate shim6 (because the other end wants to control 
> when this happens or shim6 isn't supported)
> - they should defer shim6 negotiation as per local policy
> - they should do shim6 negotiation before starting any sessions
> 
> The latter would probably be desireable for sites that want to optimize 
> for TE or want to balance incoming sessions over different hosts.

I'm having a hard time seeing how this can be done without introducing 
several additional SRV lookups; being able to express all these in a 
single SRV lookup would be hard. (Hmm, unless we take the 
_ip.www.example.com hack above and hack it some more by overloading the 
returned port number field to have semantics like the above. At that 
point in time it might be better to introduce a new DNS RR type.)

    Erik

Delivery-date: Fri, 10 Mar 2006 15:14:11 +0000
Cc: shim6-wg <[email-masked]>
From: Iljitsch van Beijnum <[email-masked]>
Subject: Re: shim6 @ NANOG   (forwarded note from John Payne) (fwd)
Date: Fri, 10 Mar 2006 16:13:41 +0100
To: Igor Gashinsky <[email-masked]>

On 10-mrt-2006, at 12:53, Igor Gashinsky wrote:

> I'm specificly talking about the case where the client is PA, and  
> server
> is PI. So, how do you get the server to realize that the the  
> client, which
> connected from ip-uunet, is also ip-cogent (which is something that  
> BGP
> knows in v4, and is very important for outbound TE), and do so without
> adding an enormous amount of state/latency/overhead?

Simple: you don't. And when you say this is possible in v4 with BGP,  
you mean that this is possible with v4 in BGP if the _client_ is  
multihomed with PI space. If this means you're saying that for large  
content networks, it's a requirement that all multihomed clients  
should have PI space, then I'm afraid nothing we can do is going to  
make you very happy... Also note that if they weren't multihomed,  
part of them would still try to reach you over your expensive  
connection, so this is a fact of life in general and doesn't have  
much to do with multihoming.

A very basic problem is that committing to the initial source/dest  
address choice happens very early in the process, and almost  
unavoidably before there is any communication. This means that it's  
not possible to determine information that depends on the specific  
path properties for the different paths between the two hosts in  
question. This is a roundabout way of saying that it's very hard if  
not impossible to make a client connect to a server over a link  
that's cheaper for the server if part of the internet _can_ connect  
over that cheaper link while another part of the internet can't or  
can't cheaply. (I.e., if the server peers with some networks while  
the rest of the world is reachable over more expensive transit: the  
clients have no way of knowing whether a certain path to the server  
is over peering or over transit as seen from the server.)

So I think it's probably best to assume that this specific problem  
will be dealt with in routing rather than end-to-end. Note that this  
doesn't necessarily require the use of PI: a server could also take  
address space from two or more ISPs and announce their more specifics  
to their peers, so regardless of the destination chosen by a client,  
if the server is reachable over peering, this will happen because of  
the more specifics.

Delivery-date: Fri, 10 Mar 2006 14:24:27 +0000
Cc: shim6-wg <[email-masked]>
From: marcelo bagnulo braun <[email-masked]>
Subject: Re: shim6 @ NANOG   (forwarded note from John Payne) (fwd)
Date: Fri, 10 Mar 2006 16:23:42 +0200
To: Igor Gashinsky <[email-masked]>

Hi Igor,

El 10/03/2006, a las 13:53, Igor Gashinsky escribi=F3:

> Hey Marcelo,
>
> 	Comments in-line...
>
> :: > No, it can not. Cogent, being a SFI peer, will only advertise=20
> *their
> :: > customer* blocks, which the UUnet address will not be a part of=20=

> (but
> :: > ip-cogent will be, but the server doesn't know that the 2 are=20
> equivalent).
> :: > Therefore, since I only see this user from the ip-uunet, unless I=20=

> somehow
> :: > know that this guy is also ip-cogent, I can only respond to him=20=

> via full
> :: > transit (ie expensive) providers, in this case uunet. If the
> :: > servers/network/whatnot was aware that locator1 =3D {ip-uunet,=20
> ip-cogent}
> :: > and then able to equate the two, *then* we could make a correct =
TE
> :: > decision somewhere (the where is still to be determined, but at=20=

> least now
> :: > we are capable of it).
> :: >
> :: > Ah, I see the confusion here.. so, in my example, I have no=20
> issues about
> :: > the inbound path (client->server), my issues is with TE on the=20
> *outbound*
> :: > path (server->client). Since I only see customer blocks from SFI=20=

> peers,
> :: > the sheer fact that the client picked ip-uunet has dictated that=20=

> I would
> :: > need to answer to him via ip-uunet, which means I have to do so=20=

> over the
> :: > uunet link, whereas in ipv4, I see the entire map, and I'm aware=20=

> that
> :: > locator1 is behind both uunet and cogent, and I can chose to take=20=

> the
> :: > cogent path.
> :: >
> ::
> :: In the PI case, the client don't have a way to know which address=20=

> prefix it
> :: should preffer for the source address. But i guess this the same=20
> that in the
> :: v4 case...
>
> No, in the v4 world they have only 1 address. They could chose to send=20=

> it
> out any link they want, but it's still 1 address.
>

but here you are assuming that all end-sites that are multihomed get to=20=

announce their address block in the DFZ, which seems too much imho.
I mean, in v4 there are some big multihomed sites that have their own=20
prefix announced, but there are also small sites that are multihomed=20
that they use NAT and PA addresses from each of their providers. I mean=20=

assume that a soho has a dsl access and a CATV access, do you think it=20=

will be able to announce its prefix (which in many cases will consist=20
in a single public IP address) through BGP?

> :: assuming the the client does have PA addresses (I mean, i am not
> :: assuming that all multihomed clients can get their PI block, since=20=

> this does
> :: not seems sustainable in the future, right?
> :: I know that this is just an example.
> :: But my point is that: i agree that things with the shim and PA=20
> addressing are
> :: different than in the PI BGP world
> :: But this doesn't necessarily imply that you cannot achieve similar
> :: behaviours, (probably using different tools)
>
> I'm specificly talking about the case where the client is PA, and=20
> server
> is PI. So, how do you get the server to realize that the the client,=20=

> which
> connected from ip-uunet, is also ip-cogent (which is something that =
BGP
> knows in v4, and is very important for outbound TE), and do so without
> adding an enormous amount of state/latency/overhead?
>

I agree that in the case that the client is PA and the server is PI you=20=

will need to establish the shim context and rehome the communication=20
after, which requires state.

However, in the case that i was considering before was when both the=20
server and the client have PA addresses. In this case, the client will=20=

prefer the common ISP addresses thanks to RFC3484

Anyway i guess we understand each other now and i agree that in the=20
scenario that you were considering, TE would require state.

>
...
> :: So i guess you got a point there, but i would say that this is yet=20=

> another
> :: concequence of using PA addressing rather than PI for multihoming
>
> Yes, but who deemed it as an "acceptable" concequence?
>
> I guess my point is that while trade-offs will have to be made in =
order
> to accomplish the goal, and some people might call them "acceptable",
> people should remember that in the end, those tradeoffs will be=20
> concidered
> against the current model, and quite frankly, unless *every* TE=20
> mechanism
> which we use today (not neceserily the tools, but the policy itself)=20=

> can
> be implemented in a manner which will scale, and not induce undue=20
> overhead on
> the servers, I'm not going to be implementing shim6, and from all my
> conversations with quite a few of my peers in the content provider/xSP
> world, they won't be either. So, if you want people to adopt it, =
please
> build it with *no less* useful functionality then what we currently=20
> have.
>

Well, i guess it is not really about what is acceptable but rather=20
about what is possible... i mean

In v4 there are two ways of multihoming, announcing the multihomed site=20=

prefix through BGP or acquiring multiple prefixes from each of the=20
providers (and probably using NAT)
In the second case it is the same in v4 or in v6 (only that since there=20=

is shim protocol for v4, there is no failure protection for established=20=

communications). In this configuration either in v4 or in v6 you don't=20=

have the TE capabilities that you would have in the PI case.
I would argue that the second form of multihoming will become=20
predominant since the amount of small sites becoming multihomed with=20
the widespread adoption of DSL CATV and this technologies for small=20
sites will encourage those (who don't have access to premium services=20
of the ISP) to get fault tolerance through becoming multihomed.

So i would argue that the problem is not shim specific and it is not=20
even v6 specific, but it is simply a concequence of cheap technologies=20=

that are becoming available and that enable lots of small users to=20
multihome...

As these technologies prevail, you need alternative approaches to BGP=20
injection to support them. Those alternative approaches may not provide=20=

the same features, i agree. The goal is to try to make them as good as=20=

possible so that they can provide as similar features as the currently=20=

available

I mean, we do agree that it is not possible to inject a Prefix into the=20=

dfz by each soho site with two dsl lines (neither in v4 nor in v6),=20
right?

Regards, marcelo

> -igor
>

Delivery-date: Fri, 10 Mar 2006 11:55:23 +0000
Date: Fri, 10 Mar 2006 06:53:38 -0500 (EST)
From: Igor Gashinsky <[email-masked]>
To: marcelo bagnulo braun <[email-masked]>
cc: shim6-wg <[email-masked]>
Subject: Re: shim6 @ NANOG   (forwarded note from John Payne) (fwd)

Hey Marcelo,

	Comments in-line...

:: > No, it can not. Cogent, being a SFI peer, will only advertise *their
:: > customer* blocks, which the UUnet address will not be a part of (but
:: > ip-cogent will be, but the server doesn't know that the 2 are equivalent).
:: > Therefore, since I only see this user from the ip-uunet, unless I somehow
:: > know that this guy is also ip-cogent, I can only respond to him via full
:: > transit (ie expensive) providers, in this case uunet. If the
:: > servers/network/whatnot was aware that locator1 = {ip-uunet, ip-cogent}
:: > and then able to equate the two, *then* we could make a correct TE
:: > decision somewhere (the where is still to be determined, but at least now
:: > we are capable of it).
:: > 
:: > Ah, I see the confusion here.. so, in my example, I have no issues about
:: > the inbound path (client->server), my issues is with TE on the *outbound*
:: > path (server->client). Since I only see customer blocks from SFI peers,
:: > the sheer fact that the client picked ip-uunet has dictated that I would
:: > need to answer to him via ip-uunet, which means I have to do so over the
:: > uunet link, whereas in ipv4, I see the entire map, and I'm aware that
:: > locator1 is behind both uunet and cogent, and I can chose to take the
:: > cogent path.
:: > 
:: 
:: In the PI case, the client don't have a way to know which address prefix it
:: should preffer for the source address. But i guess this the same that in the
:: v4 case... 

No, in the v4 world they have only 1 address. They could chose to send it 
out any link they want, but it's still 1 address.

:: assuming the the client does have PA addresses (I mean, i am not
:: assuming that all multihomed clients can get their PI block, since this does
:: not seems sustainable in the future, right?
:: I know that this is just an example.
:: But my point is that: i agree that things with the shim and PA addressing are
:: different than in the PI BGP world
:: But this doesn't necessarily imply that you cannot achieve similar
:: behaviours, (probably using different tools)

I'm specificly talking about the case where the client is PA, and server 
is PI. So, how do you get the server to realize that the the client, which 
connected from ip-uunet, is also ip-cogent (which is something that BGP 
knows in v4, and is very important for outbound TE), and do so without 
adding an enormous amount of state/latency/overhead?

:: > Again, see above, just because S prefers cogent, doesn't mean that it's
:: > aware that this customer is reachable via cogent, and until he is aware,
:: 
:: but he is aware since all the client locators are published in the DNS, 
:: right?

Are you suggesting that we do a DNS lookup for every client connection? I 
think you'll find that it's an unacceptabnle performance penalty for even 
a medium-size provider, not to mention a large-volume one.

:: I guess that the point here is that the one that initiates the communication
:: is the one that decides which locators to use (at least until the shim comes
:: into play)
:: 
:: So, yes for communications initiated by the client, it would be the client
:: the one that selects the locators, hence the providers
:: But as i noted above, the default configuration today, would make the client
:: to preffer the same ISP, which is the desired behaviour

But my point is that this breaks TE. If the server doesn't know that 
IP-uunet and ip-cogent are all same client, then the server (or more 
specificly the hosting/content provider) can not make a proper TE 
decision (because they don't know all the link(s) that client is reachable 
on, and therefore can't make a decision consistent with their policies).

:: I guess that this usage of SRV information shows the underlying problem that
:: the fact of having addresses tightly coupled with the provider would allow
:: attackers to launch DoS attacks to a defined access link, independently of
:: the attackers location. The case you mention is even worse because they use
:: the SRV information in order to identify which is the weaker link
:: 
:: So i guess you got a point there, but i would say that this is yet another
:: concequence of using PA addressing rather than PI for multihoming

Yes, but who deemed it as an "acceptable" concequence?

I guess my point is that while trade-offs will have to be made in order 
to accomplish the goal, and some people might call them "acceptable", 
people should remember that in the end, those tradeoffs will be concidered 
against the current model, and quite frankly, unless *every* TE mechanism 
which we use today (not neceserily the tools, but the policy itself) can 
be implemented in a manner which will scale, and not induce undue overhead on 
the servers, I'm not going to be implementing shim6, and from all my 
conversations with quite a few of my peers in the content provider/xSP 
world, they won't be either. So, if you want people to adopt it, please 
build it with *no less* useful functionality then what we currently have.

-igor

Delivery-date: Thu, 09 Mar 2006 09:11:05 +0000
Date: Thu, 9 Mar 2006 04:10:46 -0500 (EST)
From: Igor Gashinsky <[email-masked]>
To: marcelo bagnulo braun <[email-masked]>
cc: shim6-wg <[email-masked]>
Subject: Re: shim6 @ NANOG   (forwarded note from John Payne) (fwd)

Hey Marcello,

	comments are in-line...

:: > This makes a lot of sense, provided this happens under the hood of the
:: > application (ie web-browser in this case). So, right now, for example, if
:: > a client is pulling down a web page, gets the html, and and in the middle
:: > of downloading the .gif/jpg his session dies (ie TCP RST), the jpg that
:: > the client was in the middle of x-fering will get that ugly red "X".
:: > (most browsers, right now, will not re-try to get the object again, and
:: > will just show it as unavailable). This issue is deemed important enough
:: > that most large content providers are spending an inordinate amount of
:: > money on loadbalancer with active session sync to try to prevent that
:: > from happening in the even of a loadbalancer fail-over. So, if
:: > application behavior could be changed to say "if shim6 fail-over is
:: > possible, and connection just died (for any definition of die), then
:: > attempt re-establish connection through the shim, and then to re-get the
:: > failed object", that would go a long way in making this kind of fail-over
:: > better.
:: > 
:: 
:: but, i have an additional question about this point. the point is, if the
:: application is the one that will determine that there is a problem and will
:: ask the shim to establish a context (which is ok and no problem here)
:: wouldn't the application be better off simply retrying with alternative
:: locators by itself, rather then asking the shim to do it?

And therein lies the problem -- the applications *don't* do this 
(although, yes, I agree, in a perfect world the app developers should 
handle this, but they don't, and I don't really see software quality and 
practices getting any better, which is a whole other debate), which is 
why if you want shim6 to work, you need to handle this inside the shim, 
and not rely on the applications to do it. The application should just be 
a dumb thing that goes "connect" and the shim should handle everything 
for it (connection, redundancy, failover/convergence, everything), much 
like BGP does it today -- transparently to the app.

:: > The difference with shim6, instead of v4 is that in v4 world, the
:: > connection wouldn't die, it would just hang for the duration of
:: > convergence (provided  convergence is fast enough, which normally it is),
:: > and then continue on it's merry way with new tcp windows. In Shim6, if
:: > the client[ip1]-server connection goes down, re-establishing to
:: > client[ip2]-server would not be "hitless" (ie session would die), and to
:: > solve that problem we are back at either keeping an inordinate amount of
:: > state on the webservers (which is not very realistic), a shift in the
:: > way people write applications (which, in my opinion is preferred, but a
:: > *very* hard problem to solve), or to somehow figure out how to hide this
:: > in the stack with minimal performance hit (let's say sub 1% memory hit)
:: > when you have 30k+ simultaneous connections per server...
:: 
:: well if you use the shim approach that you suggest above, the server does not
:: have to store any shim state while things are doing fine and if a client
:: detects a problem it can trigger the creation of the shim context from the
:: client to the server. At this point, the server will need some shim state,
:: but only for those connections that have failed (of course if one of the
:: links to the server went down, then all the clients connecting through that
:: link will attempt to create a shim state)
:: 
:: I guess that this could be a reasonable trade-off between state in the server
:: and response time when outages occur

Ah, but if the server does not keep state (or at least is unaware of all 
of client's locators), doing outbound TE becomes *very* hard (see 
below)...

:: > : : I fail to understand the example the you are presenting here...
:: > : : 
:: > : : are you considering the case where both the client and the server 
:: > are both
:: > : : multihomed to Cognet and UUnet?
:: > : : something like
:: > : : 
:: > : : UUnet
:: > : : /     \
:: > : : C       S
:: > : : \     /
:: > : : Cognet
:: > 
:: > Yes, but now imagine the the "C" in this case is a client using shim6 with
:: > multiple IP's, and the server is in IPv6 PI space. Also, if it wasn't in
:: > PI space, the connection to the server *can* be influenced via SRV
:: > (although
:: > that's trying to shoehorn DNS into where perhaps it shouldn't go -- since
:: > now the DNS server needs to be aware of link-state in the network to
:: > determine if the UUnet/Cogent connections are even up, and for a
:: > sufficiently large "S", that could be 10's, or even 100's of links, which
:: > presents a very interesting scaling problem for DNS.. (even more
:: > interesting is that most large content providers are actually in the
:: > 1000's, and that's why they can get PI space -- they are effectively (at
:: > least) a tier-2 ISP). But, back to the example at hand.. so, for the
:: > sake of this example, let's say that the UUnet port is $20/Mbps, and
:: > the Cogent port is a SFI (free) peer. So, the client (with ips of
:: > IP-uunet and IP-cogent) picks IP-uunet (because they want to use their
:: > UUnet connection outbound) to initiate a connection to the server, the
:: > problem now comes from the fact that the server, when replying to the
:: > client is unaware that IP-cogent IP is associated with the client,
:: > (since the shim layer has not kicked in on initial connect) and will have
:: > to send traffic through the very expensive UUnet port.
:: 
:: that i don't follow
:: 
:: suppose that the server has v6 PI addresses, which for very big sites makes
:: sense imho
:: 
:: The server can send traffic with destiantion address belonging to UUNet
:: through Cognet, right? I mean i am assuming that UUNet and Cognet have
:: connectivity that is not through S

No, it can not. Cogent, being a SFI peer, will only advertise *their 
customer* blocks, which the UUnet address will not be a part of (but 
ip-cogent will be, but the server doesn't know that the 2 are equivalent). 
Therefore, since I only see this user from the ip-uunet, unless I somehow 
know that this guy is also ip-cogent, I can only respond to him via full 
transit (ie expensive) providers, in this case uunet. If the 
servers/network/whatnot was aware that locator1 = {ip-uunet, ip-cogent} 
and then able to equate the two, *then* we could make a correct TE
decision somewhere (the where is still to be determined, but at least now 
we are capable of it).

:: I mean, The client can choose to use the IP from UUNet (that is his choice
:: and he has the right to do so, because he is paying for it) This choice,
:: affects the ISP used to get _to_ the client and it shouldn't determine the
:: ISP used to get to the Server
:: 
:: So in this case the traffic would flow:
:: From the client to the Internet through UUNet
:: From the internet to the Server through Cognet
:: 
:: agree?

Ah, I see the confusion here.. so, in my example, I have no issues about 
the inbound path (client->server), my issues is with TE on the *outbound* 
path (server->client). Since I only see customer blocks from SFI peers, 
the sheer fact that the client picked ip-uunet has dictated that I would 
need to answer to him via ip-uunet, which means I have to do so over the 
uunet link, whereas in ipv4, I see the entire map, and I'm aware that 
locator1 is behind both uunet and cogent, and I can chose to take the 
cogent path.

:: > With v4, on the
:: > other hand, the router was aware that Client is reachable via both Cogent
:: > and UUnet, and could have had a localpref configured that would just say
:: > "anything reachable over cogent, use cogent". One way to fix that would be
:: > to do a shim6 init in the 3way handshake, but the problem then becomes
:: > that *every* "S" would have to have a complete routing table, and
:: > basically, perform the logic that is done in today's routers.
:: 
:: why is that?
:: 
:: I mean if S prefers Cognet, all he has to do is:
:: - In the PI case, route its outgoing packet through Cognet and do the same v4
:: bgp magic to direct incoming packet through cognet

Again, see above, just because S prefers cogent, doesn't mean that it's 
aware that this customer is reachable via cogent, and until he is aware, 
he can't make the proper decision (I think our disconnect is that you 
assume that people have full routes from everybody, which for networks 
closer to "the core" is absolutely not the case).

:: but the problem with the DNS that you have considered above is about having
:: to achieve that the DNS publish information that reflect the state of the
:: links.
:: This seems indeed very dificult, especially becuase of cached information and
:: so on. But as far as i know, no one is proposing this. The idea is to use SRV
:: records to express policy, and a not very dynamic one. I mean, you can
:: express that like 30% of the communications needs to use a given address and
:: the others the other address and so on, but the idea is not to allow the DNS
:: to reflect the state of the network
:: Actually, it may happen that some of the addresses in the DNS are down. In
:: this case, the idea is to let the hosts to detect this an retry using
:: alternative addresses. whether this retrial is visible or not to the apps, is
:: still an open issue

I see where you are going here, and it's not a bad idea, but there is a 
fundamental security problem problem here -- the SRV records would put the 
responsibility for following them onto the client, and well, who would 
trust them? Imagine a particularly smart worm writer who will look up his 
targets SRV records, and then decide to put all the load onto where his 
target doesn't want it, creating a very effective attack vector. 

:: So, you are considering here the case where policy is changed according to
:: the state of the network, right?
:: So that BGP information is used as feedback to the TE decision, is that
:: correct?
:: Is this possible today? how is it done? could you provide an example of how
:: you use this dynamic TE setting?

I have a feeling that we have a different definition of "policy". So, to 
me, there are 2 types of policies -- inbound policy and outbound policy. 
For outbound policy, BGP feedback is always used: if I can reach prefix X 
through an SFI link, and a paid link, the SFI prefixes are local-pref'd 
higher, and traffic will go out that way. If the SFI link goes down, or if 
the end-users link to the SFI peer goes down, the prefix is withdrawn on 
that peering session, and I will chose the next-best way (according to my 
policy) to get to the end-user. Inbound policy is even more interesting -- 
advertise my routes to ISP X, Y, and Z, and send communities to X to treat 
my routes as peer routes (ie advertise to customers only), communities to 
Y to bring my route-preference down to be below his peers (so only 
"active" if he doesn't see *any* other way to get to me), and to Z send 
them as-is. This way, I know that all customers of ISP X who send traffic 
to me will go through X will go thorough uplink-X, and *all other* traffic 
will come in through uplink-Z. If uplink-Z goes down, all other traffic 
will come through uplink-Y, and if uplink-X goes down, that traffic will 
re-route to uplink-Z, unless it's down, and it will go via uplink-Y. The 
inbound policies get even more interesting when you meet the peer in 
multiple places (which you usually do), and exchange MEDs with them to try 
to influence what routes they send to where, all depending on the state 
of the network, and which internal paths you or them have available at 
this time. 

So, perhaps the answer is that for people with SFI, they should get PI 
space, and that would probably take care of the majority of inbound policy 
issues (hell, you can easily make an argument that everyone who peers is 
an ISP), but that still doesn't solve the problem of lack of visibility 
for outbound TE.

:: yes, this is an option and it is nice because you actually get not only to
:: detect which ones are actually working but also to pick the fastest one. But
:: clearly there is the cost of the additional SYNs you send that is basically
:: overhead... would you be willing to pay for this multiple SYNs?

I would, yes, as long as the client, upon getting all the syn-acks, only 
picks 1 of them to ack again (ie i'm fine with doing 3-4x more syn 
cookies, as long as that's *all* i'm doing, since servers don't begin to 
keep track of state until the final ack w/ syn cookies, and most servers 
are capable of generating *lots* of syn cookies these days..)

Hope this helps,
-igor

Delivery-date: Thu, 09 Mar 2006 08:37:36 +0000
To: shim6-wg <[email-masked]>
From: marcelo bagnulo braun <[email-masked]>
Subject: TE goals
Date: Thu, 9 Mar 2006 10:36:46 +0200

Hi,

After all the discussion about TE and shim, it seems
that the first thing that is needed is to identify a
set of TE requirements for shim. In order to do that,
i guess that we could start but identifying the goals
that the current multihomed sites are after when they
use the TE engineering capabilities. By this i mean
the high level goals that they are trying to achieve,
as opposed to the actual configurations that they use
to achieve them.

It is very important to keep in mind that the focus
here is TE for multihomed end sites, as opposed to TE
for providers.

I have tried to make a list of the goals that have
been brought up to the list and expressed in some
other documents

Goal 1: Use the "best"/"shortest" path to get to a
         given destination

Goal 2: Use one link as primary and the other backup

Goal 3: Use one link for exchanging traffic between
         the sites (peering) and the other link for
         exchanging traffic with the rest of the
         Internet (transit)

Goal 4: Use one link for peering and backup and the
         other link for transit

Goal 5: Load spreading across the different links.
         An important aspect here is granularity i.e.
         how much granularity is available to express
         the percentage of traffic routed through each
         link

Are there other goals that multihomed end-sites
performing TE are commonly after?

Regards, marcelo

Delivery-date: Wed, 08 Mar 2006 14:55:59 +0000
Cc: shim6-wg <[email-masked]>, [email-masked]
From: marcelo bagnulo braun <[email-masked]>
Subject: Re: API for SHIM
Date: Wed, 8 Mar 2006 16:55:39 +0200
To: [email-masked]

Hi Wesley,

El 08/03/2006, a las 16:39, Wesley Eddy escribi=F3:

> On Wed, Mar 08, 2006 at 04:16:10PM +0200, marcelo bagnulo braun wrote:
>>
>> Others??
>>
>
> In addition to the interface between the shim and application, it is
> imperative that the interface between shim and transport protocol be
> considered.  At the time when the shim decides to failover, the
> transport protocol should be kicked so that it can reinitialize the
> state associated with estimation of various path properties =
(congestion
> window, RTT, path MTU, etc).  Research has indicated that not doing
> this can be problematic both for the end host and the network [1].
>

indeed, good point

I guess the communication between the shim and transport should be =20
bi-directional, since i guess that the shim would benefit from knowing =20=

that the connections that are using certain locator pair are working.

I am not sure if with the current failure detection mechanisms we can =20=

benefit from ULP positive feedback). I mean, what would be ULP positive =20=

feedback be useful for?

regards, marcelo

> This has been discussed in TSVWG and TCPM, and is described in at =
least
> a couple of internet drafts (in the process of merging):
> https://datatracker.ietf.org/public/idindex.cgi?=20
> command=3Did_detail&id=3D12091
> https://datatracker.ietf.org/public/idindex.cgi?=20
> command=3Did_detail&id=3D10165
>
> -Wes
>
> [1] =20
> http://gltrs.grc.nasa.gov/cgi-bin/GLTRS/browse.pl?2005/CR-2005=20
> -213838.html
>
> --=20
> Wesley M. Eddy
> Verizon Federal Network Systems

Delivery-date: Wed, 08 Mar 2006 14:43:06 +0000
Date: Wed, 8 Mar 2006 09:39:09 -0500
From: Wesley Eddy <[email-masked]>
To: marcelo bagnulo braun <[email-masked]>
Cc: shim6-wg <[email-masked]>, [email-masked]
Subject: Re: API for SHIM
Reply-To: [email-masked]

--gatW/ieO32f1wygP
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable

On Wed, Mar 08, 2006 at 04:16:10PM +0200, marcelo bagnulo braun wrote:
>=20
> Others??
>=20

In addition to the interface between the shim and application, it is
imperative that the interface between shim and transport protocol be
considered.  At the time when the shim decides to failover, the
transport protocol should be kicked so that it can reinitialize the
state associated with estimation of various path properties (congestion
window, RTT, path MTU, etc).  Research has indicated that not doing
this can be problematic both for the end host and the network [1].

This has been discussed in TSVWG and TCPM, and is described in at least
a couple of internet drafts (in the process of merging):
https://datatracker.ietf.org/public/idindex.cgi?command=3Did_detail&id=3D12=
091
https://datatracker.ietf.org/public/idindex.cgi?command=3Did_detail&id=3D10=
165

-Wes

[1] http://gltrs.grc.nasa.gov/cgi-bin/GLTRS/browse.pl?2005/CR-2005-213838.h=
tml

--=20
Wesley M. Eddy
Verizon Federal Network Systems

--gatW/ieO32f1wygP
Content-Type: application/pgp-signature
Content-Disposition: inline

--gatW/ieO32f1wygP--

Delivery-date: Wed, 08 Mar 2006 14:16:52 +0000
To: shim6-wg <[email-masked]>
From: marcelo bagnulo braun <[email-masked]>
Subject: API for SHIM
Date: Wed, 8 Mar 2006 16:16:10 +0200

Hi,

I guess that one of the issues that are open for discussion is the API 
for shim. In order to continue the discussion on this, I have tried to 
identify the potentials options for an extended API for SHIM that have 
been brought up in the list

Here it is an initial version of the list from the stuff that i recall 
that have been discussed over the years here.

- The application should be able to tell the shim to
initiate a communication with a given peer and use
the shim for that.
There are several cases for this:
   - The connect-by-name case, where the application
     indicates that it wants to initiate a communication
     with a FQDN and the resolver obtain different IP
     addresses for that FQDN and tries to use them to
     establish the communication. Not sure if the shim
     needs to be involved for this
   - The connect to an ULID case. In this case the
     application decides to establish a communication
     with a given ULID and the shim may be required
     if the ULID is unreachable/unroutable. The shim
     will attempt to establish a context using the
     ULID selected by the application and using
     alternative locators. The question is how does
     the shim retrieves the alternative locators?
     The options for this may be to cache information
     obtained by the DNS (i.e. when the query for a
     FQDN returns multiple addresses, these are passed
     to the shim as a hint about alternative locators)
     The other option would be to use the reverse DNS,
     as suggested in draft-nordmark-shim6-esd-00.
   - In addition, the application may want to inform
     about locators that are not working, or in
     particular that the ULID is not reachable. This
     may be useful in the case that the application
     is triggering the shim context establishment
     because of a failure in an ongoing communication.

- The application should be able to inform the shim
   about the timeout values for detecting a failure,
   for sending keepalives, for starting the exploration
   procedure. In particular, the application should be
   able to suppress the keepalives

- The application should be able to inform the shim
   if hot stand-by is desired, so that alternative paths
   are being probed when the current locator pair is
   working.

- The application should be able to tell how aggressive
   the alternative path exploration procedure is desired
   i.e. the number of alternative path that are probed
   in parallel

- The application should be able to request not using
   the shim for this communication

- Forking related options. In particular, the
   application should be able to request the creation
   of a Forked Instance for a context, and should be
   able to set the preferences for that forked instance,
   the same as a regular context.

- The application should be able to inform that an outage
   has occurred. Also it should be able to inform that the
   performance of the current locator pair is not
   satisfactory any more and that the shim should launch
   exploration and an alternative locator pair that works
   better (for some definition of better that may be
   included in the request.

- The application should be able to select a locator pair.
   A particular case for this, is that application should
   be able to tell the shim that it prefers using the ULID
   pair as locator pair as long as they are working.

- The application should be capable of obtaining the
   locator sets, local and remote, available for a shim
   context. In addition, the application should be able to
   obtain the preferences for that locators as well as other
   information, such whether the locator is reachable or not.

- The application should be able to set the preferences for
   the locators, local and remote one and also to the
   preferences of the local locators that will be passed to
   the peer.

- Interaction between RFC3484 and the shim. In this case,
   the information available in the shim could be used to
   influence RFC3484 selection procedure, so that
   unreachable locators are avoided.

Others??

Regards, marcelo

Delivery-date: Wed, 08 Mar 2006 12:09:36 +0000
Cc: shim6-wg <[email-masked]>
From: marcelo bagnulo braun <[email-masked]>
Subject: Re: shim6 @ NANOG   (forwarded note from John Payne) (fwd)
Date: Wed, 8 Mar 2006 14:08:49 +0200
To: Igor Gashinsky <[email-masked]>

El 08/03/2006, a las 10:28, Igor Gashinsky escribi=F3:

> Hi Marcelo,
>
> 	My comments are in-line... sorry for the late reply, but I've =
been
> traveling too much lately...
>
> :: El 01/03/2006, a las 10:10, Igor Gashinsky escribi=F3:
> :: So the effort for this case imho is putted in enabling the capacity=20=

> or
> :: establishing new sessions after an outage rather than in preserving
> :: established connections, do you think this makes any sense to you
>
> This makes a lot of sense, provided this happens under the hood of the
> application (ie web-browser in this case). So, right now, for example,=20=

> if
> a client is pulling down a web page, gets the html, and and in the=20
> middle
> of downloading the .gif/jpg his session dies (ie TCP RST), the jpg =
that
> the client was in the middle of x-fering will get that ugly red "X".
> (most browsers, right now, will not re-try to get the object again, =
and
> will just show it as unavailable). This issue is deemed important=20
> enough
> that most large content providers are spending an inordinate amount of
> money on loadbalancer with active session sync to try to prevent that
> from happening in the even of a loadbalancer fail-over. So, if
> application behavior could be changed to say "if shim6 fail-over is
> possible, and connection just died (for any definition of die), then
> attempt re-establish connection through the shim, and then to re-get=20=

> the
> failed object", that would go a long way in making this kind of=20
> fail-over
> better.
>

This is possible with the shim6 protocol, since it supports unreachable=20=

ulids when establishing the shim context, so i guess this would be ok.=20=

Probably a couple of elements that are needed, like an extended API to=20=

allow the apps to tell this to the shim (you probably want also to=20
inform the shim which locator is not working), and the shim needs to=20
remember the alternative locators obtained from the DNS even if there=20
is not shim context yet, in order to have a clue about which=20
alternative address to use (the other other option is to perform a=20
reverse lookup for retrieving those... see the threat with Erik for=20
more about this point). But in any case, i think all this issues are=20
easily solvable

but, i have an additional question about this point. the point is, if=20
the application is the one that will determine that there is a problem=20=

and will ask the shim to establish a context (which is ok and no=20
problem here) wouldn't the application be better off simply retrying=20
with alternative locators by itself, rather then asking the shim to do=20=

it?

> The difference with shim6, instead of v4 is that in v4 world, the
> connection wouldn't die, it would just hang for the duration of
> convergence (provided  convergence is fast enough, which normally it=20=

> is),
> and then continue on it's merry way with new tcp windows. In Shim6, if
> the client[ip1]-server connection goes down, re-establishing to
> client[ip2]-server would not be "hitless" (ie session would die), and=20=

> to
> solve that problem we are back at either keeping an inordinate amount=20=

> of
> state on the webservers (which is not very realistic), a shift in the
> way people write applications (which, in my opinion is preferred, but =
a
> *very* hard problem to solve), or to somehow figure out how to hide=20
> this
> in the stack with minimal performance hit (let's say sub 1% memory =
hit)
> when you have 30k+ simultaneous connections per server...

well if you use the shim approach that you suggest above, the server=20
does not have to store any shim state while things are doing fine and=20
if a client detects a problem it can trigger the creation of the shim=20
context from the client to the server. At this point, the server will=20
need some shim state, but only for those connections that have failed=20
(of course if one of the links to the server went down, then all the=20
clients connecting through that link will attempt to create a shim=20
state)

I guess that this could be a reasonable trade-off between state in the=20=

server and response time when outages occur

>
> :: > 3) While TE has been discussed at length already, but it is=20
> something
> :: > which is absolutely required for a content provider to deploy=20
> shim6. There
> :: > has been quite a bit of talk about what TE is used for, but it=20
> seems that
> :: > few people recognize it as a way of expressing =
"business/financial
> :: > policies". For example, in the v4 world, the (multi-homed)=20
> end-user maybe
> :: > visible via both a *paid* Transit path (say UUNET), and a *free*=20=

> peering
> :: > link (say Cogent), and I would wager that most content providers=20=

> would
> :: > choose the free link (even if performance on that link is (not=20
> hugely)
> :: > worse). That capability all but disappears in the v6 world if the=20=

> Client
> :: > ID was sourced from their UUnet ip address (since that's who they=20=

> chose
> :: > to use for outbound traffic), and the (web) server does not know=20=

> that
> :: > that locator also corresponds to a Cogent IP (which they can=20
> reach for
> :: > free).
> ::
> :: I fail to understand the example the you are presenting here...
> ::
> :: are you considering the case where both the client and the server=20=

> are both
> :: multihomed to Cognet and UUnet?
> :: something like
> ::
> :: UUnet
> :: /     \
> :: C       S
> :: \     /
> :: Cognet
>
> Yes, but now imagine the the "C" in this case is a client using shim6=20=

> with
> multiple IP's, and the server is in IPv6 PI space. Also, if it wasn't=20=

> in
> PI space, the connection to the server *can* be influenced via SRV=20
> (although
> that's trying to shoehorn DNS into where perhaps it shouldn't go --=20
> since
> now the DNS server needs to be aware of link-state in the network to
> determine if the UUnet/Cogent connections are even up, and for a
> sufficiently large "S", that could be 10's, or even 100's of links,=20
> which
> presents a very interesting scaling problem for DNS.. (even more
> interesting is that most large content providers are actually in the
> 1000's, and that's why they can get PI space -- they are effectively=20=

> (at
> least) a tier-2 ISP). But, back to the example at hand.. so, for the
> sake of this example, let's say that the UUnet port is $20/Mbps, and
> the Cogent port is a SFI (free) peer. So, the client (with ips of
> IP-uunet and IP-cogent) picks IP-uunet (because they want to use their
> UUnet connection outbound) to initiate a connection to the server, the
> problem now comes from the fact that the server, when replying to the
> client is unaware that IP-cogent IP is associated with the client,
> (since the shim layer has not kicked in on initial connect) and will=20=

> have
> to send traffic through the very expensive UUnet port.

that i don't follow

suppose that the server has v6 PI addresses, which for very big sites=20
makes sense imho

The server can send traffic with destiantion address belonging to UUNet=20=

through Cognet, right? I mean i am assuming that UUNet and Cognet have=20=

connectivity that is not through S

I mean, The client can choose to use the IP from UUNet (that is his=20
choice and he has the right to do so, because he is paying for it) This=20=

choice, affects the ISP used to get _to_ the client and it shouldn't=20
determine the ISP used to get to the Server

So in this case the traffic would flow:
 =46rom the client to the Internet through UUNet
 =46rom the internet to the Server through Cognet

agree?

Now the problem is when the server also has PA blocks

In this case, the destiantion address selected by the client will=20
determine the ISP of the server

Without shim, the server don't have many options, basically what he=20
could do is to use the DNS to prioritize the Cognet addresses.
With the shim, the server can rehome any communication that is using=20
UUnet addresses to Cognet and start using Cognet locators. This of=20
course does not prevent the client to keep on using the UUnet=20
destination addresses. In this case, the server can inform the client=20
about his preferences using a shim protocol option, but even in this=20
case the client can prefer other than what is expressed by S in the=20
preferences. In any case, in this model, each can always choose the=20
path used to send packets. I guess that in IPv4 is somehow different=20
because the decision belongs to the intermediate ASes, which are to=20
ones that can select which path to use (note that in this case, is not=20=

S who is in charge to select the incoming path neither)

>  With v4, on the
> other hand, the router was aware that Client is reachable via both=20
> Cogent
> and UUnet, and could have had a localpref configured that would just=20=

> say
> "anything reachable over cogent, use cogent". One way to fix that=20
> would be
> to do a shim6 init in the 3way handshake, but the problem then becomes
> that *every* "S" would have to have a complete routing table, and
> basically, perform the logic that is done in today's routers.

why is that?

I mean if S prefers Cognet, all he has to do is:
- In the PI case, route its outgoing packet through Cognet and do the=20
same v4 bgp magic to direct incoming packet through cognet
- In the PA case, always use cognet addresses and try to convince the=20
clients to  use the server's IP address of the cognet prefix (through=20
SRV and/or shim preferences)

> Obviously
> running Zebra w/ full routes on a server is a non-trivial performance=20=

> hit,
> and multiplied that out by the number of servers, and it gets
> very expensive, very fast. All to re-gain capabilities we have right=20=

> now
> in ipv4 for free...
>
> Now, of course, the "so called easy" answer would be "let's introduce =
a
> routing policy middleware box that would handle that part". That box=20=

> would
> have the full routing tables, the site policies, and when queried with
> "I'm server X, and this is the client and all his locators, which one=20=

> do I
> use?" would spit back an answer to that server that would be a fully
> informed decision, and the TE problem becomes mostly solved. I say
>

but there seems to be two different problems here (at least :-)
- one: which are the TE capabilities available with the PA addressing=20
model + the shim tool. This is what this can be done in this case.
- second: who is in control of these capabilties and how are they=20
managed i.e. who controls the policy and who manages the devices that=20
are in control of the policy. Is it possible to have a centralized=20
policy management? is it possible to enforce the usage of policy (at=20
least within the multihomed site)?

I guess that before we were considering the problem one and now the=20
second one...

This server idea that you are considering was presented by Cedric de=20
Launois in a work called NAROS a while ago

The other option is what we are discussing below about using a=20
DHCP/RAdv option to distribute the policy information among the hosts

The other option is to move to a scheme based on rewriting source=20
prefixes

Or a combination of those

> "mostly", because now there are these pesky issues of a) do I trust=20
> that
> the server is going to obey by this decision (either hacked, or is a=20=

> box
> outside of my administrative control, yet is within the scope of my
> network control); b) how do transit ISP's "influence" that decision =
(at
> some point I cross their network, and they should be able to control=20=

> how
> the packets are flowing through their network; c) how do I verify that
> their "influencing" doesn't negate mine, and is legitimate; d) how =
much
> "lag" does it introduce into every session establishment, and is it
> acceptable; d) can this proxy scale to the number of queries fired at=20=

> it,
> and the real-time computations that would have to happen on each one
> (since we can't precompute the answers); and finally is it *really*=20
> more
> cost-effective then doing all this in routers.
>
> So far, I'd rather pay for bigger routers...
>
> :: I mean in this case, the selection of the server provider is=20
> determined by
> :: the server's address not by the client address, right?
> :: The server can influence such decision using SRV records in the=20
> DNS, but not
> :: sure yet if this is the case you are considering
>
> See above about difficulties of scaling DNS to meet this goal...
>

but the problem with the DNS that you have considered above is about=20
having to achieve that the DNS publish information that reflect the=20
state of the links.
This seems indeed very dificult, especially becuase of cached=20
information and so on. But as far as i know, no one is proposing this.=20=

The idea is to use SRV records to express policy, and a not very=20
dynamic one. I mean, you can express that like 30% of the=20
communications needs to use a given address and the others the other=20
address and so on, but the idea is not to allow the DNS to reflect the=20=

state of the network
Actually, it may happen that some of the addresses in the DNS are down.=20=

In this case, the idea is to let the hosts to detect this an retry=20
using alternative addresses. whether this retrial is visible or not to=20=

the apps, is still an open issue

> :: > This change alone would add millions to the bw bills of said
> :: > content providers, and well, reduce the likelyhood of adoption of=20=

> the
> :: > protocol by them. Now, if the shim6 init takes place in the 3way
> :: > handshake process, then the servers "somewhat" know what all=20
> possible
> :: > paths to reach that locator are, but then would need some sort of=20=

> a
> :: > policy server telling them who to talk to on what ip, and that's=20=

> something
> :: > which will not simply scale for 100K+ machines.
> :: >
> ::
> :: I am not sure i understand the scaling problem here
> :: Suppose that you are using a DHCP option for distributing the SHIM6
> :: preferences of the RFC3484 policy table, are you saying that DHCP=20=

> does not
> :: scale for 100K+ machines? or is there something else other than=20
> DHCP that
>
> Well, first, show me a content provider who thinks that dhcp scales=20
> for a
> datacenter (other then initial pxeboot/kickstart/jumpstart, whatever),=20=

> but
> that aside, running zebra/quagga + synchronizing policy updates among
> 100K+ machines simply does not scale (operationally).
>

So, you are considering here the case where policy is changed according=20=

to the state of the network, right?
So that BGP information is used as feedback to the TE decision, is that=20=

correct?
Is this possible today? how is it done? could you provide an example of=20=

how you use this dynamic TE setting?

> :: > 4) As has also been discussed before, the initial connect time=20
> has to be
> :: > *very* low. Anything that takes longer then 4-5 seconds the=20
> end-users have
> :: > a funny way of clicking "stop" in their browser, deeming that "X=20=

> is down,
> :: > let me try Y", which is usually not a very acceptable scenario=20
> :-) So,
> :: > whatever methodology we use to do the initial set-up has to=20
> account for
> :: > that, and be able to get a connection that is actually starting=20=

> to do
> :: > something in under 2 seconds, along with figuring out which=20
> sourceIP and
> :: > destIP pairs actually can talk to each other.
> ::
> :: As i mentioned above, we are working in other mechanisms than the=20=

> shim6
> :: protocol itself that can be used for establishing new communication=20=

> through
> :: outages.
> ::
> :: you can find some work in this area in
> ::
> ::=20
> ftp://ftp.rfc-editor.org/in-notes/internet-drafts/draft-bagnulo-ipv6-
> :: rfc3484-update-00.txt
>
> It's a fairly good idea of negotiating which SRC and DEST ips to pick,=20=

> but
> it has to happen *fast* (ie sub 2 seconds), or the end-users will lose
> patience, and declare the site dead. Perhaps racing SYNs?

yes, this is an option and it is nice because you actually get not only=20=

to detect which ones are actually working but also to pick the fastest=20=

one. But clearly there is the cost of the additional SYNs you send that=20=

is basically overhead... would you be willing to pay for this multiple=20=

SYNs?

>
> Now, I'm not saying that all these problems can't be solved for people=20=

> to
> consider shim6 a viable solution, but so far, they aren't solved, and
> until they are, I just don't see recommending to my employer to take=20=

> shim6
> seriously,

I may well agree with you here, but remeber that we are still defining=20=

the protocol :-)

I guess the point here is how we can manage to provide a solution that=20=

fits the site's requirements, hence your feedback is very valuable

Regards, marcelo

>  since it seems like all it's going to do is to move the costs
> elsewhere, and quite possibly increase them quite a bit in the=20
> process...
>
> -igor=

Delivery-date: Wed, 08 Mar 2006 08:37:13 +0000
Cc: shim6 <[email-masked]>
From: marcelo bagnulo braun <[email-masked]>
Subject: Re: [Fwd: I-D ACTION:draft-nordmark-shim6-esd-00.txt]
Date: Wed, 8 Mar 2006 10:37:02 +0200
To: Erik Nordmark <[email-masked]>

El 07/03/2006, a las 21:07, Erik Nordmark escribi=F3:

> marcelo bagnulo braun wrote:
>
>> Perhaps an option could be to get the identifiers and the locator=20
>> from the direct DNS when the fqdn is resolved AND also perform the=20
>> reverse mapping from the ID to the locator set, to verify that the=20
>> information is there.
>> So, when the fqdn is resolved, the host has all the information and=20=

>> can proceed with the communication establishment. But also, if after=20=

>> performing the reverse lookup it discovers that it is not properly=20
>> populated, some kind of error message can be returned, so that the=20
>> admin of the zone is informed.
>> In this way, you can periodically verify the reverse zone, but reduce=20=

>> the cost involved in the verification operation. I mean there are two=20=

>> costs involved in the verification: the packet overhead due to the=20
>> query and the added latency of waiting to obtain the locator=20
>> information. IMHO the second one is the most problematic. With this=20=

>> option we would be eliminating the second one and just keeping the=20
>> first one.
>
> The above might work, but not clear what happens when there are slight=20=

> differences in the two sets.
>
> But when I thought about this a while back, I also ended up with some=20=

> concerns about authority and security to speak for the ID->locator=20
> mapping.
>
> For instance, if one application looks up www.foo.example and finds=20
> that the ID is X and the locators are L1, L2, L3. TCP gets X as the=20
> ULID, and the X->{L1, L2, L3} is installed in the shim layer.
>

i am not sure about that...
i mean i don't think that locators learned through the DNS should be=20
included in Ls(peer) for a shim context. The security of the binding=20
between the ULID and the locator set is quite different in the case=20
where it is secured with HBA/CGA and when it is learned through the DNS

So imho the information learned from the DNS is just hints about likely=20=

locators available for a ULID

So, afaiu, it should work as follows:

a)- The case of deferred setup:

- Host A (with a single IP, IPA) looks up in the DNS for host B. It=20
obtains IPB1, IPB2, and IPB3.
(Suppose that at this point in time there is no shim6 context for none=20=

of possible ULID pairs)
- Host A selects one of them and attempts to establish a communication.=20=

If it fails, it will try with another ULID/locator pair, until it=20
succeeds. When it succeeds, it initiates the communication.
- At some in point in time one of the parties decides to establish a=20
shim context for the ulid pair being used, let's say IPA and IPB2. The=20=

locator set of host B has no relation at all with the information in=20
the DNS, since it is obtained through the shim protocol. Suppose that=20
the locator set for host B is IPB2, IPB3 and IPB4

- suppose that some time later, another application in host A tries to=20=

establish a communication with www.foo.com and the DNS returns IPB2,=20
IPB5 and IPB6.
- Now, if the application in host A selects IPB2 as the ULID for this=20
communication, then the existing shim context will be used and the=20
locators that were returned by the DNS and were not in Ls(peer) for the=20=

existent context will be not included as locators
- If another address, IPB5 or IPB6 are used, then there is no shim=20
context and it is like the previous case.

b) the case of non-reachable ids

In this case the shim context is established up-front

- An application in Host A wants to initiate a communication with=20
www.foo.com
- Host A obtains form the DNS ULIDB1, LocB2 and LocB3
- the resolver returns ULIDB1 to the app and passes the information=20
about ULIDB1 and LocB1 and LocB2 to the shim (how this is done in not=20
in the current spec, but anyway)
- then the app initiates a communication with ULIDB1
- when the shim receives an initiation of communication with ULIDB1, it=20=

will send the I1 packet using an alternative locator, LocB1. But this=20
locator should only be considered as tentative, not as belonging to=20
Ls(peer) (yet)
- after the R1, and I2 exchange, when the R2 packet, it will contain=20
the real Ls(peer) verified with HBA/CGA. If LocB1 and/or LocB2 are not=20=

in there, then they should not be included in Ls(peer) imho

- Suppose now that another application in host A wants to initiate a=20
communication with www.boo.com. and that the results form the DNS are=20
ULIDB1, LocB3 and that LocB3 is not included in Ls(peer) of the=20
existent context for IPA1 and ULIDB1
- In this case, the resolved will pass ULIDB1 to the app and ULIDB1 and=20=

LocB3 to the shim, but the shim in this case already has a context for=20=

ULIDB1 and IPA1
- The app will initiate a communication with ULIDB1 and when it passes=20=

the packet to the shim, the shim will use the existent context imho,=20
and use the existent Ls(peer) even if it does not contain the locators=20=

returned by the DNS. IMHO this is ok, becuase the information secured=20
through CGA/HBA should be preferred over the one retrieved through the=20=

DNS
- The strange case would be what if all the locators in Ls(peer) are=20
unreachable right now, would it make sense to try with the new ones=20
obtained from the DNS? imho it may be good to try, but just as a hint=20
and not sending data packets until HBA/CGA and return reachability=20
verifications are performed for this address

Do you think this would address the case you mention next or i am=20
missing the case you were considering?

> A bit later some other application on the host looks up=20
> www.bar.example and finds that the ID is also X, but that the locator=20=

> set is different.
> This could be a case of both domains being hosted on the same server,=20=

> or it could be a security attack.
>
> Thus at a minimum, when there is a a conflict like this, one could=20
> have to check the reverse map. But what happens should the reverse=20
> lookup fail (NXDOMAIN) or time out when we do the deferred lookup?
> Should we fail the original communication (to www.foo.example)? Fail=20=

> both the original and the new attempt (to www.bar.example).
>
> Having it fail immediately might be more predictable.
> So there are some difficult tradeoffs.
>

(i am dropping the NAT topic for now... ok?)

regards, marcelo=

Delivery-date: Wed, 08 Mar 2006 08:29:13 +0000
Date: Wed, 8 Mar 2006 03:28:31 -0500 (EST)
From: Igor Gashinsky <[email-masked]>
To: marcelo bagnulo braun <[email-masked]>
cc: shim6-wg <[email-masked]>
Subject: Re: shim6 @ NANOG   (forwarded note from John Payne) (fwd)

  This message is in MIME format.  The first part should be readable text,
  while the remaining parts are likely unreadable without MIME-aware tools.

---19390912-2124999778-1141300446=:11230
Content-Type: TEXT/PLAIN; CHARSET=X-UNKNOWN
Content-Transfer-Encoding: QUOTED-PRINTABLE
Content-ID: <[email-masked]>

Hi Marcelo,

=09My comments are in-line... sorry for the late reply, but I've been=20
traveling too much lately...

:: El 01/03/2006, a las 10:10, Igor Gashinsky escribi=F3:
:: So the effort for this case imho is putted in enabling the capacity or
:: establishing new sessions after an outage rather than in preserving
:: established connections, do you think this makes any sense to you

This makes a lot of sense, provided this happens under the hood of the=20
application (ie web-browser in this case). So, right now, for example, if=
=20
a client is pulling down a web page, gets the html, and and in the middle=
=20
of downloading the .gif/jpg his session dies (ie TCP RST), the jpg that=20
the client was in the middle of x-fering will get that ugly red "X".=20
(most browsers, right now, will not re-try to get the object again, and=20
will just show it as unavailable). This issue is deemed important enough=20
that most large content providers are spending an inordinate amount of=20
money on loadbalancer with active session sync to try to prevent that=20
from happening in the even of a loadbalancer fail-over. So, if=20
application behavior could be changed to say "if shim6 fail-over is=20
possible, and connection just died (for any definition of die), then=20
attempt re-establish connection through the shim, and then to re-get the=20
failed object", that would go a long way in making this kind of fail-over=
=20
better.=20

The difference with shim6, instead of v4 is that in v4 world, the=20
connection wouldn't die, it would just hang for the duration of=20
convergence (provided  convergence is fast enough, which normally it is),=
=20
and then continue on it's merry way with new tcp windows. In Shim6, if=20
the client[ip1]-server connection goes down, re-establishing to=20
client[ip2]-server would not be "hitless" (ie session would die), and to=20
solve that problem we are back at either keeping an inordinate amount of=20
state on the webservers (which is not very realistic), a shift in the=20
way people write applications (which, in my opinion is preferred, but a=20
*very* hard problem to solve), or to somehow figure out how to hide this=20
in the stack with minimal performance hit (let's say sub 1% memory hit)=20
when you have 30k+ simultaneous connections per server...

:: > 3) While TE has been discussed at length already, but it is something
:: > which is absolutely required for a content provider to deploy shim6. T=
here
:: > has been quite a bit of talk about what TE is used for, but it seems t=
hat
:: > few people recognize it as a way of expressing "business/financial
:: > policies". For example, in the v4 world, the (multi-homed) end-user ma=
ybe
:: > visible via both a *paid* Transit path (say UUNET), and a *free* peeri=
ng
:: > link (say Cogent), and I would wager that most content providers would
:: > choose the free link (even if performance on that link is (not hugely)
:: > worse). That capability all but disappears in the v6 world if the Clie=
nt
:: > ID was sourced from their UUnet ip address (since that's who they chos=
e
:: > to use for outbound traffic), and the (web) server does not know that
:: > that locator also corresponds to a Cogent IP (which they can reach for
:: > free).
::=20
:: I fail to understand the example the you are presenting here...
::=20
:: are you considering the case where both the client and the server are bo=
th
:: multihomed to Cognet and UUnet?
:: something like
::=20
:: UUnet
:: /     \
:: C       S
:: \     /
:: Cognet

Yes, but now imagine the the "C" in this case is a client using shim6 with=
=20
multiple IP's, and the server is in IPv6 PI space. Also, if it wasn't in=20
PI space, the connection to the server *can* be influenced via SRV (althoug=
h=20
that's trying to shoehorn DNS into where perhaps it shouldn't go -- since=
=20
now the DNS server needs to be aware of link-state in the network to=20
determine if the UUnet/Cogent connections are even up, and for a=20
sufficiently large "S", that could be 10's, or even 100's of links, which=
=20
presents a very interesting scaling problem for DNS.. (even more=20
interesting is that most large content providers are actually in the=20
1000's, and that's why they can get PI space -- they are effectively (at=20
least) a tier-2 ISP). But, back to the example at hand.. so, for the=20
sake of this example, let's say that the UUnet port is $20/Mbps, and=20
the Cogent port is a SFI (free) peer. So, the client (with ips of=20
IP-uunet and IP-cogent) picks IP-uunet (because they want to use their=20
UUnet connection outbound) to initiate a connection to the server, the=20
problem now comes from the fact that the server, when replying to the=20
client is unaware that IP-cogent IP is associated with the client,=20
(since the shim layer has not kicked in on initial connect) and will have=
=20
to send traffic through the very expensive UUnet port. With v4, on the=20
other hand, the router was aware that Client is reachable via both Cogent=
=20
and UUnet, and could have had a localpref configured that would just say=20
"anything reachable over cogent, use cogent". One way to fix that would be=
=20
to do a shim6 init in the 3way handshake, but the problem then becomes=20
that *every* "S" would have to have a complete routing table, and=20
basically, perform the logic that is done in today's routers. Obviously=20
running Zebra w/ full routes on a server is a non-trivial performance hit,=
=20
and multiplied that out by the number of servers, and it gets=20
very expensive, very fast. All to re-gain capabilities we have right now=20
in ipv4 for free...

Now, of course, the "so called easy" answer would be "let's introduce a=20
routing policy middleware box that would handle that part". That box would=
=20
have the full routing tables, the site policies, and when queried with=20
"I'm server X, and this is the client and all his locators, which one do I=
=20
use?" would spit back an answer to that server that would be a fully=20
informed decision, and the TE problem becomes mostly solved. I say=20
"mostly", because now there are these pesky issues of a) do I trust that=20
the server is going to obey by this decision (either hacked, or is a box=20
outside of my administrative control, yet is within the scope of my=20
network control); b) how do transit ISP's "influence" that decision (at=20
some point I cross their network, and they should be able to control how=20
the packets are flowing through their network; c) how do I verify that=20
their "influencing" doesn't negate mine, and is legitimate; d) how much=20
"lag" does it introduce into every session establishment, and is it=20
acceptable; d) can this proxy scale to the number of queries fired at it,=
=20
and the real-time computations that would have to happen on each one=20
(since we can't precompute the answers); and finally is it *really* more=20
cost-effective then doing all this in routers.=20

So far, I'd rather pay for bigger routers...

:: I mean in this case, the selection of the server provider is determined =
by
:: the server's address not by the client address, right?
:: The server can influence such decision using SRV records in the DNS, but=
 not
:: sure yet if this is the case you are considering

See above about difficulties of scaling DNS to meet this goal...

:: > This change alone would add millions to the bw bills of said
:: > content providers, and well, reduce the likelyhood of adoption of the
:: > protocol by them. Now, if the shim6 init takes place in the 3way
:: > handshake process, then the servers "somewhat" know what all possible
:: > paths to reach that locator are, but then would need some sort of a
:: > policy server telling them who to talk to on what ip, and that's somet=
hing
:: > which will not simply scale for 100K+ machines.
:: >=20
::=20
:: I am not sure i understand the scaling problem here
:: Suppose that you are using a DHCP option for distributing the SHIM6
:: preferences of the RFC3484 policy table, are you saying that DHCP does n=
ot
:: scale for 100K+ machines? or is there something else other than DHCP tha=
t

Well, first, show me a content provider who thinks that dhcp scales for a=
=20
datacenter (other then initial pxeboot/kickstart/jumpstart, whatever), but=
=20
that aside, running zebra/quagga + synchronizing policy updates among=20
100K+ machines simply does not scale (operationally).

:: > 4) As has also been discussed before, the initial connect time has to =
be
:: > *very* low. Anything that takes longer then 4-5 seconds the end-users =
have
:: > a funny way of clicking "stop" in their browser, deeming that "X is do=
wn,
:: > let me try Y", which is usually not a very acceptable scenario :-) So,
:: > whatever methodology we use to do the initial set-up has to account fo=
r
:: > that, and be able to get a connection that is actually starting to do
:: > something in under 2 seconds, along with figuring out which sourceIP a=
nd
:: > destIP pairs actually can talk to each other.
::=20
:: As i mentioned above, we are working in other mechanisms than the shim6
:: protocol itself that can be used for establishing new communication thro=
ugh
:: outages.
::=20
:: you can find some work in this area in
::=20
:: ftp://ftp.rfc-editor.org/in-notes/internet-drafts/draft-bagnulo-ipv6-
:: rfc3484-update-00.txt

It's a fairly good idea of negotiating which SRC and DEST ips to pick, but=
=20
it has to happen *fast* (ie sub 2 seconds), or the end-users will lose=20
patience, and declare the site dead. Perhaps racing SYNs?

Now, I'm not saying that all these problems can't be solved for people to=
=20
consider shim6 a viable solution, but so far, they aren't solved, and=20
until they are, I just don't see recommending to my employer to take shim6=
=20
seriously, since it seems like all it's going to do is to move the costs=20
elsewhere, and quite possibly increase them quite a bit in the process...=
=20

-igor
---19390912-2124999778-1141300446=:11230--

Delivery-date: Tue, 07 Mar 2006 21:33:24 +0000
Date: Wed, 08 Mar 2006 08:32:46 +1100
To: [email-masked]
From: Geoff Huston <[email-masked]>
Subject: I-D ACTION:draft-ietf-shim6-proto-04.txt 

A New Internet-Draft is available from the on-line Internet-Drafts directories.
This draft is a work item of the Site Multihoming by IPv6 Intermediation 
Working Group of the IETF.

         Title           : Level 3 multihoming shim protocol
         Author(s)       : M. Bagnulo, E. Nordmark
         Filename        : draft-ietf-shim6-proto-04.txt
         Pages           : 121
         Date            : 2006-3-7

The SHIM6 protocol is a layer 3 shim for providing locator agility
    below the transport protocols, so that multihoming can be provided
    for IPv6 with failover and load sharing properties, without assuming
    that a multihomed site will have a provider independent IPv6 address
    prefix which is announced in the global IPv6 routing table.  The
    hosts in a site which has multiple provider allocated IPv6 address
    prefixes, will use the shim6 protocol specified in this document to
    setup state with peer hosts, so that the state can later be used to
    failover to a different locator pair, should the original one stop
    working.

A URL for this Internet-Draft is:
http://www.ietf.org/internet-drafts/draft-ietf-shim6-proto-04.txt

Delivery-date: Tue, 07 Mar 2006 19:49:56 +0000
Date: Tue, 07 Mar 2006 11:07:38 -0800
From: Erik Nordmark <[email-masked]>
To: marcelo bagnulo braun <[email-masked]>
CC: shim6 <[email-masked]>
Subject: Re: [Fwd: I-D ACTION:draft-nordmark-shim6-esd-00.txt]

marcelo bagnulo braun wrote:

> Perhaps an option could be to get the identifiers and the locator from 
> the direct DNS when the fqdn is resolved AND also perform the reverse 
> mapping from the ID to the locator set, to verify that the information 
> is there.
> So, when the fqdn is resolved, the host has all the information and can 
> proceed with the communication establishment. But also, if after 
> performing the reverse lookup it discovers that it is not properly 
> populated, some kind of error message can be returned, so that the admin 
> of the zone is informed.
> In this way, you can periodically verify the reverse zone, but reduce 
> the cost involved in the verification operation. I mean there are two 
> costs involved in the verification: the packet overhead due to the query 
> and the added latency of waiting to obtain the locator information. IMHO 
> the second one is the most problematic. With this option we would be 
> eliminating the second one and just keeping the first one.

The above might work, but not clear what happens when there are slight 
differences in the two sets.

But when I thought about this a while back, I also ended up with some 
concerns about authority and security to speak for the ID->locator mapping.

For instance, if one application looks up www.foo.example and finds that 
the ID is X and the locators are L1, L2, L3. TCP gets X as the ULID, and 
the X->{L1, L2, L3} is installed in the shim layer.

A bit later some other application on the host looks up www.bar.example 
and finds that the ID is also X, but that the locator set is different.
This could be a case of both domains being hosted on the same server, or 
it could be a security attack.

Thus at a minimum, when there is a a conflict like this, one could have 
to check the reverse map. But what happens should the reverse lookup 
fail (NXDOMAIN) or time out when we do the deferred lookup?
Should we fail the original communication (to www.foo.example)? Fail 
both the original and the new attempt (to www.bar.example).

Having it fail immediately might be more predictable.
So there are some difficult tradeoffs.

> agree but i guess that it is easy to determine that in the NAT case (in 
> the general case). I mean, if the other end sees a public address and we 
> are using a private address, then the private address is unreachable 
> from the peer and must be substituted with the public one received in 
> the option.

If not sure it is easy.
Take a multihomed host with two NICs and 2 prefixes on each.
The host has 4 locators; L1 and L2 on NICa and L3, L4 on NICb.

It finds out that when it is sending using L1 as the source, the 
receiver sees the packets coming from L1'.
Thus this mean that L1' is a replacement for L1? Probably (at least when 
talking to that destination). But does it also replace L2? I have no idea.
Should it have any impact on the locators on the second NIC.

> If both v4 addresses received are public, then this is TE and both are ok.

How can you tell if an IP address is public or not? While checking for 
the 3 RFC 1918 ranges might capture 90%, I think there is NATting going 
on even when the address space behind the NAT is global IPv4 address space.

> (The problem is that now some very big sites are starting to use public 
> addresses when they run out of private addresses 
> (http://www.arin.net/policy/proposals/2004_3.html) which would break the 
> simplicity of the approach. But even in this case, the local host should 
> be able to know which are the addresses reserved for private use and 
> apply the same heuristic mentioned above

Another reason why you can't *syntactically* check whether an IPv4 
address is NATted or not.

Doesn't the remote host need to know as well?

> yes, but shim is supposed to be able to deal with failures and the NAT 
> loosing its state can be seen as a failure. So, the shim will be able to 
> recover from this and even recreate the NAT state. Probably we need some 
> additional logic to be smart enough to identify that what is going on is 
> that the nat is loosing state and increase the keepalive frequency.
> I mean, in the case of the shim, the communication can be recovered from 
> a nat loosing state and the state in the nat can be easily recovered.

Yes, there is definitely a lot of extra work needed to make this a full 
fledged NAT traversal mechanism. Which is why I don't want to spend time 
on it.

> So what we need is a locator selection algorithm that is likely to take 
> into account the same issues that RFC3484 does, but with different 
> trade-offs being made, right?

Right.

   Erik

Delivery-date: Tue, 07 Mar 2006 08:19:21 +0000
Cc: shim6 <[email-masked]>
From: marcelo bagnulo braun <[email-masked]>
Subject: Re: [Fwd: I-D ACTION:draft-nordmark-shim6-esd-00.txt]
Date: Tue, 7 Mar 2006 10:18:23 +0200
To: Erik Nordmark <[email-masked]>

El 07/03/2006, a las 2:11, Erik Nordmark escribi=F3:

> marcelo bagnulo braun wrote:
>
>> I agree that we should care about robustness
>> But my point is that if someone is doing shim and it is using non=20
>> routable ids, then he probably cares about robustness and so he=20
>> should know that he must properly populate the reverse DNS.
>
> They might think they have populated it correctly, but if it "never"
> gets used there is no easy way for them to tell.
>
> I think some careful judgment is needed here, so ensure that it isn't=20=

> only during failures in combination with other events (such as=20
> referrals or long-lived application sessions), that the ID->locators=20=

> would be looked up in the DNS.
>
> Taking a car analogy. Folks are reasonably confident that the breaks=20=

> on the car works, because they use them when slowing down, and when=20
> stopping at intersections.
> If the breaks where only used for emergencies (such as somebody=20
> jumping into the street just in front of you, or the car in front=20
> suddenly slamming its breaks), then it would be too late to discover=20=

> you were out of break fluid.
>

right, but you probably don't want to periodically crash into things to=20=

verify the airbag... ;-)

I can see your point and i agree that this needs to be taken into=20
account.

Perhaps an option could be to get the identifiers and the locator from=20=

the direct DNS when the fqdn is resolved AND also perform the reverse=20
mapping from the ID to the locator set, to verify that the information=20=

is there.
So, when the fqdn is resolved, the host has all the information and can=20=

proceed with the communication establishment. But also, if after=20
performing the reverse lookup it discovers that it is not properly=20
populated, some kind of error message can be returned, so that the=20
admin of the zone is informed.
In this way, you can periodically verify the reverse zone, but reduce=20
the cost involved in the verification operation. I mean there are two=20
costs involved in the verification: the packet overhead due to the=20
query and the added latency of waiting to obtain the locator=20
information. IMHO the second one is the most problematic. With this=20
option we would be eliminating the second one and just keeping the=20
first one.

>
>>> We can encode the address type in different ways. One way would be=20=

>>> to use a flag field in the verification method. Another would be to=20=

>>> use the IPv4-mapped address format (::ffff:1.2.3.4). Both would work=20=

>>> AFAIK.
>>>
>> sure
>> but i guess that my point is that if already identify that we may=20
>> need additional flags about the address information, it would be a=20
>> good option to redefine the verification method to a generic flag=20
>> octet, in order to support future flexibility (this is somehow=20
>> independent of the option selected for the v4 address, is more a=20
>> general observation)
>
> Yes, but if we want to support different types of locators (IPv6,=20
> IPv4, and perhaps IPv4+UDP port for NAT traversal), then maybe we=20
> should have an explicit locator type? (and length?)
>

well, i guess that we don't need more than 128 bits, so probably if the=20=

locator field is 128 bits, this would be enough.
w.r.t. to the explicit locator type, perhaps the option would be to use=20=

3 bits of the flag field to express this...
> ...

>> wouldn't the Sent locator option and the received locator option be=20=

>> useful for dealing with NATs?
>> I mean using these, a host can find out if the addresses were=20
>> rewritten (whether by a router or a nat doesn't matter i guess), and=20=

>> discover its own addresses and eventually add them to the Ls
>
> Yes, that is potentially useful. Hadn't thought about that (I try to=20=

> avoid spending precious brain cycles on inventing yet another NAT=20
> traversal mechanism).

> But the semantics might be different.
> With a NAT rewrite it presumably means that some other address (the=20
> one in the source in the Sent locator option) is no longer useful.
> But with router rewriting, that address might still be useful but just=20=

> isn't recommended (e.g., due to TE) for this communication.
>

agree but i guess that it is easy to determine that in the NAT case (in=20=

the general case). I mean, if the other end sees a public address and=20
we are using a private address, then the private address is unreachable=20=

from the peer and must be substituted with the public one received in=20
the option.

If both v4 addresses received are public, then this is TE and both are=20=

ok.

(The problem is that now some very big sites are starting to use public=20=

addresses when they run out of private addresses=20
(http://www.arin.net/policy/proposals/2004_3.html) which would break=20
the simplicity of the approach. But even in this case, the local host=20
should be able to know which are the addresses reserved for private use=20=

and apply the same heuristic mentioned above

>> In addition, as you mention in the draft the failure detection=20
>> mechanism could be used to preserve the NAT state if detected.
>
> Detection and preservation are different things.
> To preserve one would have to determine how often the NAT needs to see=20=

> packets in order for it to not discard the state. This is hard. Or one=20=

> has to send a packet every N seconds all the time, which generates=20
> lots of extra load.
>

yes, but shim is supposed to be able to deal with failures and the NAT=20=

loosing its state can be seen as a failure. So, the shim will be able=20
to recover from this and even recreate the NAT state. Probably we need=20=

some additional logic to be smart enough to identify that what is going=20=

on is that the nat is loosing state and increase the keepalive=20
frequency.
I mean, in the case of the shim, the communication can be recovered=20
from a nat loosing state and the state in the nat can be easily=20
recovered.

>> Probably the piece that is missing is somehow of rendez vous server=20=

>> for initial contact for hosts behind nat
>
> Which is hard.
>
> One can observe some similarities between a NAT rendez-vous server,=20
> and a mobility home agent, in that they both could be a (set of) fixed=20=

> locators where one can send packets (and have them be forwarded on,=20
> and later route optimized away), with a mechanism for the host to keep=20=

> the rendez-vous server updated with the current locators of the host.
>
> But this is harder for NAT than for IP mobility; with full cone NATs=20=

> it could be a single IPv4 address plus port as the locator. But with=20=

> stricter NATs then each peer has to find a different IP+port to talk=20=

> via the NAT.
>

yes this is a completely new piece that is missing and i guess that in=20=

this case, it would make sense to try to import any solution designed=20
somewhere else to deal with this problem

> All good reasons for me to focus on
>  - completing the ID/loc split,
>  - TE feedback
>

i agree with the approach, just wanted to explore this a bit

> and let somebody else think about NAT traversal.
>
>
>> I guess that locator selection incorporates several elements:
>> - Rechability
>> - Preferences with respect the different types of locators (CoA vs.=20=

>> HoA, scope, private etc)
>> - TE/policing issues (including local preferences and remote=20
>> preferences (the preferences of the peer))
>> RFC3484 can be used to express the tree items somehow (with its=20
>> limitations) (note that it does not help to determine if an address=20=

>> is reachable or not, but it does take this information into account=20=

>> when performing dest address selection)
>> In addition, the locator selection is performed:
>> - when the communication is initiated (whether when a routable id is=20=

>> selected (just RFC3484) or when a non routable id is used and the=20
>> shim selects a correspondent locator)
>> - when a failure occurs,
>> - because of TE considerations, a host may choose to change the=20
>> locator
>> I would say that in any of those cases, it is importnat to take into=20=

>> account all the aforementioned 3 items, so i guess that RFC3484 may=20=

>> be a good candidate for locator selection
>
> s/good candidate for/provides some additional constraints on/
>
>> But not only availability must be taken into account for exploring=20
>> and rehoming i guess.
>> First of all, you probably want to take into account considerations=20=

>> like scope, HoA vs. CoA, private vs. public addresses when selecting=20=

>> which address to explore
>
> Sure. But the tradeoffs are quite different in a shim6 locator=20
> selection
> context that in address selection.
>
> For instance, for private vs. public in order to provide pseudonymity=20=

> the host has to keep multiple sets of <temporary ULID, set of=20
> temporary locators> and never combine ULID/locators across the sets,=20=

> since that would provide a linkage over time where the host can be=20
> tracked even though it changes its ULIDs and locators over time.
>
> For HoA/CoA the tradeoffs are different when used as address today=20
> (where using a CoA can easily cause communication to fail), and using=20=

> it as a locator with shim6.
>
> So RFC 3484 does help to list some things to consider, but doesn't=20
> provide useful answers for the issues with shim6.
>

i agree with the points made above

So what we need is a locator selection algorithm that is likely to take=20=

into account the same issues that RFC3484 does, but with different=20
trade-offs being made, right?

>
>>> One of the interesting things is the combination of the unroutable=20=

>>> ULID and getting router feedback during the context establishment=20
>>> exchange (which will be done before any ULP packets are exchanged in=20=

>>> that case).
>>> If we only allow rewriting on packets with the payload extension=20
>>> header we wouldn't have that capability of early locator selection=20=

>>> according to the policy implicit in the router's rewriting.
>>>
>> but how early is this compared to the case where no rewriting is=20
>> supported for control packets?
>> I mean, in the case of no rewriting for control packets, the first=20
>> payload packet would get rewritten and the capability would only get=20=

>> delayed for 2 packets i.e. the shim context establishment packets,=20
>> which i don't think is an issue...
>> but probably i am missing the point you are making...
>
> My point is that if we have unroutable ULIDs, then there isn't any=20
> extra cost I can see for allowing the rewrite of on the I1/R1 etc=20
> messages; those messages get a bit larger that's all.
>
> Without rewriting in I1/R1/etc we get rewriting for the first data=20
> packet resulting in an Update Request message to inform the sender of=20=

> a rewrite, potentially followed by an Update Request in the reverse=20
> direction to inform the peer that it can use that new locator as a=20
> destination.
>
>
> And if we are pursuing router rewriting, a simple test for "can this=20=

> packet be rewritten" might be important for performance. With=20
> rewriting on shim6 control packets, this test is
> 	if (ip6->ip6_nxthdr =3D=3D IPPPROTO_SHIM6)
>

i may agree with that.
It is just that i am concerned about the added complexity to the=20
protocol itself, which is already quite complex...

regards, marcelo

>
>    Erik
>

Delivery-date: Tue, 07 Mar 2006 02:51:48 +0000
Date: Mon, 06 Mar 2006 18:49:56 -0800
From: Erik Nordmark <[email-masked]>
To: "Jason Schiller ([email-masked])" <[email-masked]>
CC: shim6-wg <[email-masked]>
Subject: Re: shim6 @ NANOG   (forwarded note from John Payne)

Jason Schiller ([email-masked]) wrote:
> Is it the consensus of the shim6 working group that the full suite of TE
> capabilities should not be a requirement?  Or is this just the opinion of
> a few vocal people?

Jason,

I don't know what "full suite" means.

In a system with multiple prefixes per site the type of TE one can do, 
at least with current tools, is likely to be quite different than in a 
multihoming system with a single prefix per site.

For instance, if I understood GSE <draft-ietf-ipngwg-gseaddr-00.txt> it 
would also mean that something on the initiating host would have to 
select between multiple 16 byte addresses (RG+ESD).
But the support for router rewriting of the RG means that the 
communication will switch to using the RG that the routers desire.

Have you or somebody else though about whether this would be sufficient 
for TE? Or would there be a need to be able to influence the destination 
locator that is used for the first packet?

    Erik

Delivery-date: Tue, 07 Mar 2006 00:13:56 +0000
Date: Mon, 06 Mar 2006 16:11:25 -0800
From: Erik Nordmark <[email-masked]>
To: marcelo bagnulo braun <[email-masked]>
CC: shim6 <[email-masked]>
Subject: Re: [Fwd: I-D ACTION:draft-nordmark-shim6-esd-00.txt]

marcelo bagnulo braun wrote:

> I agree that we should care about robustness
> But my point is that if someone is doing shim and it is using non 
> routable ids, then he probably cares about robustness and so he should 
> know that he must properly populate the reverse DNS.

They might think they have populated it correctly, but if it "never"
gets used there is no easy way for them to tell.

I think some careful judgment is needed here, so ensure that it isn't 
only during failures in combination with other events (such as referrals 
or long-lived application sessions), that the ID->locators would be 
looked up in the DNS.

Taking a car analogy. Folks are reasonably confident that the breaks on 
the car works, because they use them when slowing down, and when 
stopping at intersections.
If the breaks where only used for emergencies (such as somebody jumping 
into the street just in front of you, or the car in front suddenly 
slamming its breaks), then it would be too late to discover you were out 
of break fluid.

>> We can encode the address type in different ways. One way would be to 
>> use a flag field in the verification method. Another would be to use 
>> the IPv4-mapped address format (::ffff:1.2.3.4). Both would work AFAIK.
>>
> 
> sure
> but i guess that my point is that if already identify that we may need 
> additional flags about the address information, it would be a good 
> option to redefine the verification method to a generic flag octet, in 
> order to support future flexibility (this is somehow independent of the 
> option selected for the v4 address, is more a general observation)

Yes, but if we want to support different types of locators (IPv6, IPv4, 
and perhaps IPv4+UDP port for NAT traversal), then maybe we should have 
an explicit locator type? (and length?)

>> I'm concerned about the slippery slope.
>>
> 
> Ok, let's explore this a bit more...

We're slipping, we're slipping ... ;-)

> wouldn't the Sent locator option and the received locator option be 
> useful for dealing with NATs?
> I mean using these, a host can find out if the addresses were rewritten 
> (whether by a router or a nat doesn't matter i guess), and discover its 
> own addresses and eventually add them to the Ls

Yes, that is potentially useful. Hadn't thought about that (I try to 
avoid spending precious brain cycles on inventing yet another NAT 
traversal mechanism).

But the semantics might be different.
With a NAT rewrite it presumably means that some other address (the one 
in the source in the Sent locator option) is no longer useful.
But with router rewriting, that address might still be useful but just 
isn't recommended (e.g., due to TE) for this communication.

> In addition, as you mention in the draft the failure detection mechanism 
> could be used to preserve the NAT state if detected.

Detection and preservation are different things.
To preserve one would have to determine how often the NAT needs to see 
packets in order for it to not discard the state. This is hard. Or one 
has to send a packet every N seconds all the time, which generates lots 
of extra load.

> Probably the piece that is missing is somehow of rendez vous server for 
> initial contact for hosts behind nat

Which is hard.

One can observe some similarities between a NAT rendez-vous server, and 
a mobility home agent, in that they both could be a (set of) fixed 
locators where one can send packets (and have them be forwarded on, and 
later route optimized away), with a mechanism for the host to keep the 
rendez-vous server updated with the current locators of the host.

But this is harder for NAT than for IP mobility; with full cone NATs it 
could be a single IPv4 address plus port as the locator. But with 
stricter NATs then each peer has to find a different IP+port to talk via 
the NAT.

All good reasons for me to focus on
  - completing the ID/loc split,
  - TE feedback

and let somebody else think about NAT traversal.

> I guess that locator selection incorporates several elements:
> - Rechability
> - Preferences with respect the different types of locators (CoA vs. HoA, 
> scope, private etc)
> - TE/policing issues (including local preferences and remote preferences 
> (the preferences of the peer))
> 
> RFC3484 can be used to express the tree items somehow (with its 
> limitations) (note that it does not help to determine if an address is 
> reachable or not, but it does take this information into account when 
> performing dest address selection)
> 
> In addition, the locator selection is performed:
> - when the communication is initiated (whether when a routable id is 
> selected (just RFC3484) or when a non routable id is used and the shim 
> selects a correspondent locator)
> - when a failure occurs,
> - because of TE considerations, a host may choose to change the locator
> 
> I would say that in any of those cases, it is importnat to take into 
> account all the aforementioned 3 items, so i guess that RFC3484 may be a 
> good candidate for locator selection

s/good candidate for/provides some additional constraints on/

> But not only availability must be taken into account for exploring and 
> rehoming i guess.
> First of all, you probably want to take into account considerations like 
> scope, HoA vs. CoA, private vs. public addresses when selecting which 
> address to explore

Sure. But the tradeoffs are quite different in a shim6 locator selection
context that in address selection.

For instance, for private vs. public in order to provide pseudonymity 
the host has to keep multiple sets of <temporary ULID, set of temporary 
locators> and never combine ULID/locators across the sets, since that 
would provide a linkage over time where the host can be tracked even 
though it changes its ULIDs and locators over time.

For HoA/CoA the tradeoffs are different when used as address today 
(where using a CoA can easily cause communication to fail), and using it 
as a locator with shim6.

So RFC 3484 does help to list some things to consider, but doesn't 
provide useful answers for the issues with shim6.

> Second, you also may want to take some policy/TE considerations into 
> account
> (e.g. suppose you have a site with 3 links, 2 of them with a flat rate 
> and the third one with per traffic rate. Probably you will want to use 
> always that flat rate first and only fall back to the per traffic rate 
> if those two ae down, independently of the number of bits that the 
> source address has in common)

Yes, we are agreeing that RFC 3484 isn't the complete answer ;-)

> but i guess we need to consider scope/HoA/private etc when selecting 
> destiantion addresses

Out of those we only know the scope (for the *destination*).

>> One of the interesting things is the combination of the unroutable 
>> ULID and getting router feedback during the context establishment 
>> exchange (which will be done before any ULP packets are exchanged in 
>> that case).
>> If we only allow rewriting on packets with the payload extension 
>> header we wouldn't have that capability of early locator selection 
>> according to the policy implicit in the router's rewriting.
>>
> 
> but how early is this compared to the case where no rewriting is 
> supported for control packets?
> I mean, in the case of no rewriting for control packets, the first 
> payload packet would get rewritten and the capability would only get 
> delayed for 2 packets i.e. the shim context establishment packets, which 
> i don't think is an issue...
> but probably i am missing the point you are making...

My point is that if we have unroutable ULIDs, then there isn't any extra 
cost I can see for allowing the rewrite of on the I1/R1 etc messages; 
those messages get a bit larger that's all.

Without rewriting in I1/R1/etc we get rewriting for the first data 
packet resulting in an Update Request message to inform the sender of a 
rewrite, potentially followed by an Update Request in the reverse 
direction to inform the peer that it can use that new locator as a 
destination.

And if we are pursuing router rewriting, a simple test for "can this 
packet be rewritten" might be important for performance. With rewriting 
on shim6 control packets, this test is
	if (ip6->ip6_nxthdr == IPPPROTO_SHIM6)

    Erik

Delivery-date: Mon, 06 Mar 2006 03:40:32 +0000
Date: Mon, 6 Mar 2006 03:37:36 +0000 (GMT)
From: Paul Jakma <[email-masked]>
To: Iljitsch van Beijnum <[email-masked]>
cc: Geoff Huston <[email-masked]>, [email-masked]
Subject: Re: shim6 @ NANOG   (forwarded note from John Payne)
Mail-Copies-To: [email-masked]

On Fri, 24 Feb 2006, Iljitsch van Beijnum wrote:

> This is doable if we support proxy shim6 in middleboxes.
>
> Doing shim6 in a middlebox for a legacy host may be challenging, but 
> worthwhile because it makes deployment so much easier.

FWIW, being one of the people who argued for this a while ago, on 
mature reflection I think this is unimportant and now agree with 
anyone who argued against me (particularly if from a "state in the 
middle is awkward" POV).

> Right. So how do we solve this? Second shim? Shim-before-syn?

Why not shim in parallel? The network locator in the packet which 
just arrived in is, 99.99% of the time, good for replying to at 
least.

regards,
-- 
Paul Jakma	[email-masked]	[email-masked]	Key ID: 64A2FF6A
Fortune:
Accept people for what they are -- completely unacceptable.

Delivery-date: Sun, 05 Mar 2006 19:32:03 +0000
Date: Mon, 06 Mar 2006 06:30:42 +1100
To: [email-masked]
From: Geoff Huston <[email-masked]>
Subject: Re: shim6 @ NANOG   (forwarded note from John Payne) (fwd)

 >Message-Id: <[email-masked]>
 >Cc: Igor Gashinsky <[email-masked]>, shim6-wg <[email-masked]>
 >From: John Payne <[email-masked]>
 >Subject: Re: shim6 @ NANOG   (forwarded note from John Payne) (fwd)
 >Date: Fri, 3 Mar 2006 13:01:18 -0500
 > To: Kurt Erik Lindqvist <[email-masked]>

On Mar 3, 2006, at 3:26 AM, Kurt Erik Lindqvist wrote:

 > This was something that was brought up in the famous L.A Nanog
 > session, but I actually fail to see in what way you think shim6 add
 > delay to the initial hit? You will look up the destination in DNS,
 > get multiple AAAA's back, perform RFC3483 and start communicating.
 > That is the same, with or without shim6. If it is a short lived
 > session and failure occurs, that is no worse than IPv4 which will
 > also have to timeout and retry.

The issue is first hit under failed conditions.

AIUI with the current plan, you'd get multiple AAAAs back vs one
multihomed A leading to a potentially long cycle of TCP timeouts
before getting a AAAA that actually worked.

Unless you want to try parallalising SYNs and racing them off...
which has other issues :)

Delivery-date: Sat, 04 Mar 2006 14:09:12 +0000
Cc: Igor Gashinsky <[email-masked]>, shim6-wg <[email-masked]>
From: Kurt Erik Lindqvist <[email-masked]>
Subject: Re: shim6 @ NANOG   (forwarded note from John Payne) (fwd)
Date: Sat, 4 Mar 2006 12:58:15 +0100
To: John Payne <[email-masked]>

On 3 mar 2006, at 19.01, John Payne wrote:

>
> On Mar 3, 2006, at 3:26 AM, Kurt Erik Lindqvist wrote:
>
>> This was something that was brought up in the famous L.A Nanog  
>> session, but I actually fail to see in what way you think shim6  
>> add delay to the initial hit? You will look up the destination in  
>> DNS, get multiple AAAA's back, perform RFC3483 and start  
>> communicating. That is the same, with or without shim6. If it is a  
>> short lived session and failure occurs, that is no worse than IPv4  
>> which will also have to timeout and retry.
>
> The issue is first hit under failed conditions.
>
> AIUI with the current plan, you'd get multiple AAAAs back vs one  
> multihomed A leading to a potentially long cycle of TCP timeouts  
> before getting a AAAA that actually worked.

Ok, so this is actually not a DNS timeout issue, and it isn't a TCP  
timeout issues really either. When shim6 "suspects" a path failure it  
will try locator pairs until one working is found. The time it takes  
to to pair wise trials is relative to the number of locators but  
should be fairly quick.

> Unless you want to try parallalising SYNs and racing them off...  
> which has other issues :)

There are no SYNs involved..

- kurtis -

Delivery-date: Fri, 03 Mar 2006 18:23:29 +0000
Date: Fri, 03 Mar 2006 10:21:45 -0800
From: Erik Nordmark <[email-masked]>
To: Kurt Erik Lindqvist <[email-masked]>
CC: [email-masked]
Subject: Re: [Fwd: Review comments on draft-ietf-shim6-proto-03.txt]

Kurt Erik Lindqvist wrote:

> As I said, I can't say I feel strongly one way or the other...Did Geoff 
> have any more thoughts?

No (other than that we should discuss it on the mailing list)

    Erik

Delivery-date: Fri, 03 Mar 2006 16:15:09 +0000
Cc: shim6-wg <[email-masked]>
From: Kurt Erik Lindqvist <[email-masked]>
Subject: Re: shim6 @ NANOG   (forwarded note from John Payne) (fwd)
Date: Fri, 3 Mar 2006 09:27:20 +0100
To: Mikael Abrahamsson <[email-masked]>

On 2 mar 2006, at 09.13, Mikael Abrahamsson wrote:

> On Wed, 1 Mar 2006, Iljitsch van Beijnum wrote:
>
>> This is not what I think of when I say "multihoming". I don't see  
>> an easy way to do this with shim6 (there may be a non-easy way,  
>> though) so maybe hosting companies that are big enough to do  
>> peering will need to continue to do this as in IPv4.
>
> Couldn't this be implemented into the resolving DNS, in that it  
> could include priority information into the DNS response sent to  
> the host that does the initial connect? Or it could be done via  
> reverse-DNS (in the case where the "called" party needs to know  
> which IP to send to when there are several to choose from)?

this is pretty much SRV records.

> Perhaps one state in the shim6 initiation could be to do reverse- 
> lookup and via some record there gain knowledge about which IP  
> should be prioritized? This means that routingpolicy needs to be  
> somehow implemented into the DNS server, which would require new  
> code there as well.

Then you need secure binding beetween forward and reverse DNS. Which  
just isn't there.

- kurtis -

Delivery-date: Fri, 03 Mar 2006 16:14:38 +0000
Cc: shim6-wg <[email-masked]>
From: Kurt Erik Lindqvist <[email-masked]>
Subject: Re: shim6 @ NANOG   (forwarded note from John Payne) (fwd)
Date: Fri, 3 Mar 2006 09:26:12 +0100
To: Igor Gashinsky <[email-masked]>

On 1 mar 2006, at 09.10, Igor Gashinsky wrote:

> 3) While TE has been discussed at length already, but it is something
> which is absolutely required for a content provider to deploy  
> shim6. There
> has been quite a bit of talk about what TE is used for, but it  
> seems that
> few people recognize it as a way of expressing "business/financial
> policies". For example, in the v4 world, the (multi-homed) end-user  
> maybe
> visible via both a *paid* Transit path (say UUNET), and a *free*  
> peering
> link (say Cogent), and I would wager that most content providers would
> choose the free link (even if performance on that link is (not hugely)
> worse). That capability all but disappears in the v6 world if the  
> Client
> ID was sourced from their UUnet ip address (since that's who they  
> chose
> to use for outbound traffic), and the (web) server does not know that
> that locator also corresponds to a Cogent IP (which they can reach for
> free). This change alone would add millions to the bw bills of said
> content providers, and well, reduce the likelyhood of adoption of the
> protocol by them. Now, if the shim6 init takes place in the 3way
> handshake process, then the servers "somewhat" know what all possible
> paths to reach that locator are, but then would need some sort of a
> policy server telling them who to talk to on what ip, and that's  
> something
> which will not simply scale for 100K+ machines.

In this case you are hoping for non-asymmetrical paths, which might  
or might not be the case. See Jason's example of the UUnet and Sprint  
customer.

> 4) As has also been discussed before, the initial connect time has  
> to be
> *very* low. Anything that takes longer then 4-5 seconds the end- 
> users have
> a funny way of clicking "stop" in their browser, deeming that "X is  
> down,
> let me try Y", which is usually not a very acceptable scenario :-) So,
> whatever methodology we use to do the initial set-up has to account  
> for
> that, and be able to get a connection that is actually starting to do
> something in under 2 seconds, along with figuring out which  
> sourceIP and
> destIP pairs actually can talk to each other.

This was something that was brought up in the famous L.A Nanog  
session, but I actually fail to see in what way you think shim6 add  
delay to the initial hit? You will look up the destination in DNS,  
get multiple AAAA's back, perform RFC3483 and start communicating.  
That is the same, with or without shim6. If it is a short lived  
session and failure occurs, that is no worse than IPv4 which will  
also have to timeout and retry.

- kurtis -

Delivery-date: Fri, 03 Mar 2006 16:14:08 +0000
Cc: [email-masked]
From: Kurt Erik Lindqvist <[email-masked]>
Subject: Re: [Fwd: Review comments on draft-ietf-shim6-proto-03.txt]
Date: Fri, 3 Mar 2006 07:57:44 +0100
To: Erik Nordmark <[email-masked]>

On 2 mar 2006, at 02.13, Erik Nordmark wrote:

> Kurt Erik Lindqvist wrote:
>>
>> I do make the observation that our charter says :
>> o Compatibility will remain for existing mobility mechanisms. It will
>> be possible to use Mobile IPv6 on a node that also supports Shim6.
>> However, any optimizations or advanced configurations are out of
>> scope for shim6.
>> So I am kind of inclined to agree with Geoff :-)
>> Although I can't say I feel strongly either...
>
> I know what the charter says and I don't have problem with that.
>
> But I don't think the charter text means that we can't make  
> *observations* in our documents - there are no optimizations or  
> advanced configurations specified in the text that is in the document.

As I said, I can't say I feel strongly one way or the other...Did  
Geoff have any more thoughts?

- kurtis -

Delivery-date: Fri, 03 Mar 2006 07:53:49 +0000
Cc: shim6 <[email-masked]>
From: marcelo bagnulo braun <[email-masked]>
Subject: Re: [Fwd: I-D ACTION:draft-nordmark-shim6-esd-00.txt]
Date: Fri, 3 Mar 2006 09:53:00 +0200
To: Erik Nordmark <[email-masked]>

El 02/03/2006, a las 22:42, Erik Nordmark escribi=F3:
>
>> In section 2.6  Design Alternatives it is stated
>>    If the identifiers are placed in the DNS using AAAA records, then=20=

>> the
>>    lookup for the AAAA record set (to find the identifier) might also
>>    return a list of locators.
>> another alternative that probably would result in similar behaviour=20=

>> would be that the identifier is included in the identifier record,=20
>> but that the same query that returns the ID record also returns the=20=

>> locator set associated to this identifier in the additional=20
>> information field
>> Such information would be delivered by the resolver to the shim=20
>> process which would store it for when the packet addressed to the=20
>> identifier arrives
>
> Yes, apart from the point below ;-)
>
> Putting the ids in AAAA records, assuming it can be made to work, have=20=

> a performance benefit. Without it, the host has to
>  - look for an ID RRset
>  - if none found look for AAAA RRset
>  - if none found, look for A RRset
>
> This is extra overhead when the target host doesn't have any ID RRs.
>
> If we could put the identifiers in the AAAA RRset in such a way that=20=

> legacy IPv6 hosts wouldn't try to communicate with them (due to the=20
> RFC 3484 ordering putting them last in the list), then even for target=20=

> hosts that don't have IDs there is no extra DNS lookup.
>

Yes this makes sense to me

>>   Such a list can potentially be useful to
>>    avoid the ip6.arpa lookup to find the locators.  But relying on=20
>> this
>>    means that the reverse lookup from the identifier will only be =
used
>>    in uncommon cases such as:
>>    o  The shim6 context state having been garbage collected too =
early,
>>       and the upper-layer protocol sends down a packet with a
>>       destination ULID which is a non-routable identifier.
>>    o  Application callbacks, referrals, and long-lived application
>>       handles [27] that are IP addresses.
>>    For this reason it makes sense to be more consistent and always=20
>> rely
>>    on the reverse lookup when the context is established.
>> I fail to understand this point. I mean avoiding extra DNS lookups is=20=

>> good, since it reduces latency and load to the servers. why do we=20
>> want to impose this? just to make sure that the reverse information=20=

>> is in place i.e. that the reverse tree is properly populated? i think=20=

>> this is an expensive price to pay for this and i would rather prefer=20=

>> that those that have the reverse tree poorly populated simply have=20
>> problems with the referrals rather than imposing a penalty to all=20
>> shim communications
>
> FWIW it isn't only referrals and the like that would be a problem when=20=

> the ID->locator lookup isn't populated; also the case when the shim=20
> discards the context state would not work since there wouldn't be a=20
> way to rediscover some locators once the context state is gone.
>

agree

> My take is that we are doing shim6 and its extensions to gain better=20=

> robustness of the network as seen from the communicating applications.=20=

> Thus saying that we don't care about robustness by not ensuring that=20=

> the id->locators lookup mechanism is populated doesn't make much sense=20=

> to me.
>

I agree that we should care about robustness
But my point is that if someone is doing shim and it is using non=20
routable ids, then he probably cares about robustness and so he should=20=

know that he must properly populate the reverse DNS.
I mean i consider that the penalty of artificially imposing a reverse=20
lookup when establishing every shim context in order to make sure that=20=

the reverse entry exists, seems an overkill to me (especially because=20
we will be always paying the cost even once the reverse dns is properly=20=

populated for these id)

> There isn't a free lunch here - introducing a separate  128-bit=20
> identifier space provides significant benefits, but also comes with=20
> some cost.

Yes, but imho we should try to minimize that cost.
Having to do the ID->loc lookup when a referral, call back, lost=20
context event occurs, is the cost that we really need to pay
Having to do it for every shim context establishment is not strictly=20
necesary, and imho we should avoid that

>> About using v4 locators
>> I think that supporting v4 locators would be very valuable for the=20
>> protocol. I would suggest to move this to a separate document and=20
>> adopt it. (maybe it could be included in the base spec since it seems=20=

>> quite trivial to do)
>
> OK. Without any NAT support?
>

Yes, without NAT support for the base spec or initial v4 support.
Then we should move on and figure out how existent NAT traversal=20
techniques are suitable for SHIM6 and see if additional stuff is needed

>> The option proposed for this in the draft is:
>>    When CGA is used to prevent redirection attacks in shim6, there is=20=

>> no
>>    constraint on the locators that are used apart from host B must=20
>> know
>>    its own locators so it can pass it them to host A. In particular,=20=

>> we
>>    can use IPv4 addresses as locators; this doesn't require anything
>>    more than defining how an IPv4 address is carried in the 128-bit
>>    fields in the Locator List option.
>> I wonder if it wouldn't be better to change the verification method=20=

>> field to a generic Flag field and use some bits there to specify the=20=

>> address family, what do you think?
>
> We can encode the address type in different ways. One way would be to=20=

> use a flag field in the verification method. Another would be to use=20=

> the IPv4-mapped address format (::ffff:1.2.3.4). Both would work=20
> AFAIK.
>

sure
but i guess that my point is that if already identify that we may need=20=

additional flags about the address information, it would be a good=20
option to redefine the verification method to a generic flag octet, in=20=

order to support future flexibility (this is somehow independent of the=20=

option selected for the v4 address, is more a general observation)

>> w.r.t. NAT i guess that an additional consideration is how does the=20=

>> host detects its own addresses in order to include them in its own=20
>> locator set and communicate them to the peer. Of course, shim should=20=

>> be able to detect private addresses and not include them in the=20
>> Ls(local)
>
> This is non-trivial. It also would need to know which peers are on the=20=

> near and far side of the NAT, since it needs to use different locators=20=

> in the two cases.
>

right

>> IMHO a reasonable trade-off would be now to specify how IPv4=20
>> addresses could be used as locators when no nat is involved and leave=20=

>> the nat support for further study
>
> I'm concerned about the slippery slope.
>

Ok, let's explore this a bit more...
wouldn't the Sent locator option and the received locator option be=20
useful for dealing with NATs?
I mean using these, a host can find out if the addresses were rewritten=20=

(whether by a router or a nat doesn't matter i guess), and discover its=20=

own addresses and eventually add them to the Ls
In addition, as you mention in the draft the failure detection=20
mechanism could be used to preserve the NAT state if detected
Probably the piece that is missing is somehow of rendez vous server for=20=

initial contact for hosts behind nat
>

...

>> In the draft it is assumed that a single identifier will be available=20=

>> for an endpoint. I don't see the need to have more than one, but i=20
>> guess it would make sense to consider the case, but we can probably=20=

>> deffer this till later. However, it seems reasonable to have more=20
>> than one locator. The question would be at this point why not use=20
>> RFC3484 (or RFC3484bis) to select among the multiple locator pair=20
>> combinations?
>
> I guess there are two aspects of locator selection and the draft=20
> focuses on the TE aspects (selecting between different global locators=20=

> with different prefixes based on the TE wishes of the destination);=20
> selecting  locators with different scope, home vs. CoA, private vs.=20
> public is the other part.
>
> I suspect that one would want some coupling between the ULID selection=20=

> and the locator selection. For instance, a host that wishes the=20
> privacy benefits of temporary locators presumably also would use a=20
> temporary ULID.
>

right
I guess that locator selection incorporates several elements:
- Rechability
- Preferences with respect the different types of locators (CoA vs.=20
HoA, scope, private etc)
- TE/policing issues (including local preferences and remote=20
preferences (the preferences of the peer))

RFC3484 can be used to express the tree items somehow (with its=20
limitations) (note that it does not help to determine if an address is=20=

reachable or not, but it does take this information into account when=20
performing dest address selection)

In addition, the locator selection is performed:
- when the communication is initiated (whether when a routable id is=20
selected (just RFC3484) or when a non routable id is used and the shim=20=

selects a correspondent locator)
- when a failure occurs,
- because of TE considerations, a host may choose to change the locator

I would say that in any of those cases, it is importnat to take into=20
account all the aforementioned 3 items, so i guess that RFC3484 may be=20=

a good candidate for locator selection

>> Moreover, i would wonder if it didn't make sense to use RFC3484 to=20
>> select locators after a failure i.e. to select the locators to=20
>> explore after a failure is detected using RFC3484 for that, or even=20=

>> if we have multiple address pairs that are working as locators, to=20
>> use RFC3484 to select which one among them to use to rehome the=20
>> communication.
>
> What part of RFC 3484 do you see as useful for this?
> As an example, one way to explore other locators after a failure is to=20=

> try one that is maximally different from the failed locator (i.e.=20
> differs in the leftmost bit) since such a locator might be a more=20
> independent path.  The point with this example is that RFC 3484 takes=20=

> no such consideration into account.

But not only availability must be taken into account for exploring and=20=

rehoming i guess.
First of all, you probably want to take into account considerations=20
like scope, HoA vs. CoA, private vs. public addresses when selecting=20
which address to explore
Second, you also may want to take some policy/TE considerations into=20
account
(e.g. suppose you have a site with 3 links, 2 of them with a flat rate=20=

and the third one with per traffic rate. Probably you will want to use=20=

always that flat rate first and only fall back to the per traffic rate=20=

if those two ae down, independently of the number of bits that the=20
source address has in common)

>
>> I guess that the information in SRV records is about the peer's=20
>> preferences about its own locators
>> The same thing w.r.t. the preference information received in the=20
>> preference option of the shim protocol
>> however, the local host may also have some policy w.r.t. which=20
>> locator prefer when there are multiple of them for a given=20
>> destiantion.
>> I guess that the local preferences will be expressed in the RFC3484=20=

>> table
>
> I was actually envisioning that the source locator selection would be=20=

> driven by the same type of information (priority and weight) as the=20
> remote end sends us (in the Locator Preference option) for the=20
> destination locators.

> Thus one way to do this is independent selection of source locator=20
> (based on the locally configured priority and weights) and destination=20=

> locator (based on the Locator Preference option).
>
> I don't see what the RFC 3484 table would add; we do need to consider=20=

> address scope etc which is something to carry forward from RFC 3484.
>

Not sure i understand the last sentence.

but i guess we need to consider scope/HoA/private etc when selecting=20
destiantion addresses

And i guess that a multihomed host may also have preferences about=20
which destiantion address to use for reaching a destiantion and may=20
want to override the preferences of the peer, especially when he is=20
sending traffic. I mean, each end has the possibility of selecting the=20=

locator pair that it want to use for sending. It may take into account=20=

the peer's preferences, but it will have the final choice in any case.=20=

does this makes sense?

So for both of these issues, RFC3484 could be useful

In any case, i guess your model is much direct and maybe we could adopt=20=

it and not precluding more complex setups in the future

> ....

>> I have two comments w.r.t. to the mechanisms presented:
>> First, i think that the idea to carry the Sent Locator Pair and=20
>> Received Locator pair options to discover and allow coordination=20
>> between routers and hosts is very clever. As i understand it the main=20=

>> focus is putted in the case where host A learns through this option=20=

>> new locators of its own.
>> However, it may be case that a host receives a payload packet with a=20=

>> new locator from its peer i.e. the source locator is not included in=20=

>> Ls(peer). In this case, it will accept the packet since the CT match,=20=

>> but it cannot use it to send packets until a CGA/HBA verification of=20=

>> this locator is presented by the peer. However, when it sends the new=20=

>> locator in the received locator option, the peer should sent an=20
>> Update request the includes the new locator signed with CGA. I am not=20=

>> sure if this is what is expressed in the last bullet of section=20
>> 3.4...
>
> That bullet just says to inform the peer that its locators were=20
> rewritten in a new way.
>
> But the draft mentions somewhere that the host can "adopt" a locator=20=

> that the routers using, and if it does this, it would presumably send=20=

> an Update Message to the peer saying that it is ok to send to this new=20=

> locator (and provide the CGA signature to prove it).
>
>> Second, i think that the presented mechanisms are potentially very=20
>> useful for rewriting payload packets, but i really don't think it=20
>> worth the effort to rewrite the source address of the shim control=20
>> message. I mean, they are a fairly reduced amount of packets, so i=20
>> guess that they don't really affect TE considerations. Supporting=20
>> this adds additional complexity to a protocol which is already fairly=20=

>> complex imho. I mean, i guess it can be done as you present in the=20
>> draft, but i don't think we win much with allowing rewriting of these=20=

>> few packet and we would be adding much complexity. I would keep it=20
>> just for data packets and not for signalling packets
>
> One of the interesting things is the combination of the unroutable=20
> ULID and getting router feedback during the context establishment=20
> exchange (which will be done before any ULP packets are exchanged in=20=

> that case).
> If we only allow rewriting on packets with the payload extension=20
> header we wouldn't have that capability of early locator selection=20
> according to the policy implicit in the router's rewriting.
>

but how early is this compared to the case where no rewriting is=20
supported for control packets?
I mean, in the case of no rewriting for control packets, the first=20
payload packet would get rewritten and the capability would only get=20
delayed for 2 packets i.e. the shim context establishment packets,=20
which i don't think is an issue...
but probably i am missing the point you are making...

Regards, marcelo

>    Erik
>

Delivery-date: Thu, 02 Mar 2006 22:05:04 +0000
Date: Thu, 02 Mar 2006 14:03:21 -0800
From: Erik Nordmark <[email-masked]>
To: marcelo bagnulo braun <[email-masked]>
CC: [email-masked], Iljitsch van Beijnum <[email-masked]>
Subject: Re: shim6 @ NANOG   (forwarded note from John Payne)

marcelo bagnulo braun wrote:
>> Wouldn't the hosts in X and Y have two IPv6 addresses?
>> Thus BGP itself at site X wouldn't know that B:Y and D:Y is the same 
>> host, hence BGP wouldn't compare them to find the best.
>>
> 
> yes, but i think could work anyway
> I mean, suppose that we are performing source address rewriting by the 
> multihomed site exit routers
> Now suppose that intra site routing is performed based solely in the 
> destination address (just as it is in the current paradigm)
> Assume that the multihomed site obtains BGP feed through both of its 
> providers.
> Now, based on the BGP information obtained, the intra site routing will 
> preffer one or other exit to reach a given destiantion.
> So, when a host within the multihomed site sends a packet to that 
> destination the exit route will be determined by this BGP information 
> hence the policy information pushed by the transit ISPs can affect the 
> intra site route selection
> Since the site exit routers can rewrite the source address, they will 
> select the appropriate source prefix

My response to Iljitsch was about his assertion about the current 
workings of shim6 - not taking into account any hypothetical extensions 
like router rewriting about locators.

But even with such a hypothetical extension, there would be nothing in 
BGP that would know that the B:Y and D:Y address prefixes are for the 
same site, hence nothing in BGP that would compare the metrics (AS path 
length or whatever) for the routes to those two separate prefixes.

    Erik

Delivery-date: Thu, 02 Mar 2006 20:44:59 +0000
Date: Thu, 02 Mar 2006 12:42:01 -0800
From: Erik Nordmark <[email-masked]>
To: marcelo bagnulo braun <[email-masked]>
CC: shim6 <[email-masked]>
Subject: Re: [Fwd: I-D ACTION:draft-nordmark-shim6-esd-00.txt]

marcelo bagnulo braun wrote:
> Hi Erik,
> 
> I think the draft is very useful and provides very good insight of these 
> interesting issues.
> Allow me to do some comments.

Thanks for your comments.

> all the above conditions seem to be fulfilled by ULAs, but they are not 
> mentioned anywhere (only in the references)
> did you have something different in mind or ULAs are a good candidate 
> for this?

Centrally allocated ULAs would be AFAIK.
But I though (and could be wrong) that only the random ULAs were moved 
forward.

> In sections 2.4 and 2.5 the handling of referrals is presented.
> 
> I guess that an additional problem that shows up in the case that ids 
> are not valid locators is the support of legacy hosts.
> I mean, since the ones that are passing ulid information are the 
> applications, it is possible that the ulid ends up in a legacy host that 
> does not implement the shim and that is not able to translate the id 
> into a locator, even if the directory service is available.
> So, even with this approach, the referral case still presents some 
> issues imho

Good point.

> In section 2.6  Design Alternatives it is stated
> 
>    If the identifiers are placed in the DNS using AAAA records, then the
>    lookup for the AAAA record set (to find the identifier) might also
>    return a list of locators.
> 
> another alternative that probably would result in similar behaviour 
> would be that the identifier is included in the identifier record, but 
> that the same query that returns the ID record also returns the locator 
> set associated to this identifier in the additional information field
> Such information would be delivered by the resolver to the shim process 
> which would store it for when the packet addressed to the identifier 
> arrives

Yes, apart from the point below ;-)

Putting the ids in AAAA records, assuming it can be made to work, have a 
performance benefit. Without it, the host has to
  - look for an ID RRset
  - if none found look for AAAA RRset
  - if none found, look for A RRset

This is extra overhead when the target host doesn't have any ID RRs.

If we could put the identifiers in the AAAA RRset in such a way that 
legacy IPv6 hosts wouldn't try to communicate with them (due to the RFC 
3484 ordering putting them last in the list), then even for target hosts 
that don't have IDs there is no extra DNS lookup.

>   Such a list can potentially be useful to
>    avoid the ip6.arpa lookup to find the locators.  But relying on this
>    means that the reverse lookup from the identifier will only be used
>    in uncommon cases such as:
> 
>    o  The shim6 context state having been garbage collected too early,
>       and the upper-layer protocol sends down a packet with a
>       destination ULID which is a non-routable identifier.
> 
>    o  Application callbacks, referrals, and long-lived application
>       handles [27] that are IP addresses.
> 
>    For this reason it makes sense to be more consistent and always rely
>    on the reverse lookup when the context is established.
> 
> I fail to understand this point. I mean avoiding extra DNS lookups is 
> good, since it reduces latency and load to the servers. why do we want 
> to impose this? just to make sure that the reverse information is in 
> place i.e. that the reverse tree is properly populated? i think this is 
> an expensive price to pay for this and i would rather prefer that those 
> that have the reverse tree poorly populated simply have problems with 
> the referrals rather than imposing a penalty to all shim communications

FWIW it isn't only referrals and the like that would be a problem when 
the ID->locator lookup isn't populated; also the case when the shim 
discards the context state would not work since there wouldn't be a way 
to rediscover some locators once the context state is gone.

My take is that we are doing shim6 and its extensions to gain better 
robustness of the network as seen from the communicating applications. 
Thus saying that we don't care about robustness by not ensuring that the 
id->locators lookup mechanism is populated doesn't make much sense to me.

There isn't a free lunch here - introducing a separate  128-bit 
identifier space provides significant benefits, but also comes with some 
cost. But this cost of doing the id->locator lookup only occurs when 
creating the shim6 context, thus it is NOT for every connection.

> About using v4 locators
> 
> I think that supporting v4 locators would be very valuable for the 
> protocol. I would suggest to move this to a separate document and adopt 
> it. (maybe it could be included in the base spec since it seems quite 
> trivial to do)

OK. Without any NAT support?

> The option proposed for this in the draft is:
> 
>    When CGA is used to prevent redirection attacks in shim6, there is no
>    constraint on the locators that are used apart from host B must know
>    its own locators so it can pass it them to host A. In particular, we
>    can use IPv4 addresses as locators; this doesn't require anything
>    more than defining how an IPv4 address is carried in the 128-bit
>    fields in the Locator List option.
> 
> I wonder if it wouldn't be better to change the verification method 
> field to a generic Flag field and use some bits there to specify the 
> address family, what do you think?

We can encode the address type in different ways. One way would be to 
use a flag field in the verification method. Another would be to use the 
IPv4-mapped address format (::ffff:1.2.3.4). Both would work AFAIK.

> w.r.t. NAT i guess that an additional consideration is how does the host 
> detects its own addresses in order to include them in its own locator 
> set and communicate them to the peer. Of course, shim should be able to 
> detect private addresses and not include them in the Ls(local)

This is non-trivial. It also would need to know which peers are on the 
near and far side of the NAT, since it needs to use different locators 
in the two cases.

> IMHO a reasonable trade-off would be now to specify how IPv4 addresses 
> could be used as locators when no nat is involved and leave the nat 
> support for further study

I'm concerned about the slippery slope.

> About TE and source address rewriting by exit routers
> 
> In section 1.2 it is stated that
> 
>    o  With ULIDs that are non-routable identifiers, there will most
>       likely be only one identifier for the destination as well as the
>       source.  Thus the role of RFC 3484 is largely removed.  But there
>       is an additional step of looking up the identifier to find the
>       locator, and at that point in time it makes sense to consider
>       traffic engineering for selecting the initial locator pair.
> 
> The point here, i guess is that RFC3484 is used today to select 
> addresses for initial contact. As those addresses are going to be used 
> both as ULIDs and locator, then RFC3484 is currently used to select both.
> In the case that we use a ULID that it is not a locator, then which 
> mechanism are we going to use to select each of them,

If the peer has only one ID and this host has only one ID, then the 
selection is trivial.
I don't know of a use case where each host would have multiple IDs.
(We do need to make sure that the source ULID is a non-routable address 
if and only if the destination ULID is a non-routable address.)

> In the draft it is assumed that a single identifier will be available 
> for an endpoint. I don't see the need to have more than one, but i guess 
> it would make sense to consider the case, but we can probably deffer 
> this till later. However, it seems reasonable to have more than one 
> locator. The question would be at this point why not use RFC3484 (or 
> RFC3484bis) to select among the multiple locator pair combinations?

I guess there are two aspects of locator selection and the draft focuses 
on the TE aspects (selecting between different global locators with 
different prefixes based on the TE wishes of the destination); selecting 
  locators with different scope, home vs. CoA, private vs. public is the 
other part.

I suspect that one would want some coupling between the ULID selection 
and the locator selection. For instance, a host that wishes the privacy 
benefits of temporary locators presumably also would use a temporary ULID.

> Moreover, i would wonder if it didn't make sense to use RFC3484 to 
> select locators after a failure i.e. to select the locators to explore 
> after a failure is detected using RFC3484 for that, or even if we have 
> multiple address pairs that are working as locators, to use RFC3484 to 
> select which one among them to use to rehome the communication.

What part of RFC 3484 do you see as useful for this?
As an example, one way to explore other locators after a failure is to 
try one that is maximally different from the failed locator (i.e. 
differs in the leftmost bit) since such a locator might be a more 
independent path.  The point with this example is that RFC 3484 takes no 
such consideration into account.

> I guess that the information in SRV records is about the peer's 
> preferences about its own locators
> The same thing w.r.t. the preference information received in the 
> preference option of the shim protocol
> however, the local host may also have some policy w.r.t. which locator 
> prefer when there are multiple of them for a given destiantion.
> I guess that the local preferences will be expressed in the RFC3484 table

I was actually envisioning that the source locator selection would be 
driven by the same type of information (priority and weight) as the 
remote end sends us (in the Locator Preference option) for the 
destination locators.

Thus one way to do this is independent selection of source locator 
(based on the locally configured priority and weights) and destination 
locator (based on the Locator Preference option).

I don't see what the RFC 3484 table would add; we do need to consider 
address scope etc which is something to carry forward from RFC 3484.

> later on in section 3.1  Recommending use of DNS SRV
> 
>    In shim6 as specified, the host rely on existing DNS mechanisms, such
>    as AAAA records or any other mechanism, to find a list of locators to
>    try.  When AAAA records are used, there is no mechanism for the
>    destination site to express any ranking for primary/fallback, or any
>    mechanism to spread load across the paths that are represented by the
>    locators, since the AAAA resource record set is treated as a set with
>    no implied order.
> 
> This is not strictly true, i believe
> 
> I mean, rfc3484 preserves the order of the received locator list if no 
> rules apply

The AAAA records for a resource record *set*, hence there is no order 
defined. People might wish that the DNS provided a *list* instead of a 
*set*, and in some cases one can pretend it is a list and get some 
benefits, but it is still defined to be an (unordered) set.

> I have two comments w.r.t. to the mechanisms presented:
> 
> First, i think that the idea to carry the Sent Locator Pair and Received 
> Locator pair options to discover and allow coordination between routers 
> and hosts is very clever. As i understand it the main focus is putted in 
> the case where host A learns through this option new locators of its own.
> However, it may be case that a host receives a payload packet with a new 
> locator from its peer i.e. the source locator is not included in 
> Ls(peer). In this case, it will accept the packet since the CT match, 
> but it cannot use it to send packets until a CGA/HBA verification of 
> this locator is presented by the peer. However, when it sends the new 
> locator in the received locator option, the peer should sent an Update 
> request the includes the new locator signed with CGA. I am not sure if 
> this is what is expressed in the last bullet of section 3.4...

That bullet just says to inform the peer that its locators were 
rewritten in a new way.

But the draft mentions somewhere that the host can "adopt" a locator 
that the routers using, and if it does this, it would presumably send an 
Update Message to the peer saying that it is ok to send to this new 
locator (and provide the CGA signature to prove it).

> Second, i think that the presented mechanisms are potentially very 
> useful for rewriting payload packets, but i really don't think it worth 
> the effort to rewrite the source address of the shim control message. I 
> mean, they are a fairly reduced amount of packets, so i guess that they 
> don't really affect TE considerations. Supporting this adds additional 
> complexity to a protocol which is already fairly complex imho. I mean, i 
> guess it can be done as you present in the draft, but i don't think we 
> win much with allowing rewriting of these few packet and we would be 
> adding much complexity. I would keep it just for data packets and not 
> for signalling packets

One of the interesting things is the combination of the unroutable ULID 
and getting router feedback during the context establishment exchange 
(which will be done before any ULP packets are exchanged in that case).
If we only allow rewriting on packets with the payload extension header 
we wouldn't have that capability of early locator selection according to 
the policy implicit in the router's rewriting.

    Erik

Delivery-date: Thu, 02 Mar 2006 15:20:23 +0000
Cc: shim6 <[email-masked]>
From: marcelo bagnulo braun <[email-masked]>
Subject: Re: [Fwd: I-D ACTION:draft-nordmark-shim6-esd-00.txt]
Date: Thu, 2 Mar 2006 17:19:42 +0200
To: Erik Nordmark <[email-masked]>

Hi Erik,

I think the draft is very useful and provides very good insight of=20
these interesting issues.
Allow me to do some comments.

About full id/locator split in shim6

In section 2.1 it is stated that

    Since we are likely to have applications communicate to both hosts
    which have an IP identifier and those which only have the IP
    locators, it is highly desirable to be able to have a syntactic =
means
    to tell an IP identifier apart from an IP locator.  A reasonable
    approach is to allocate a small subset of the IPv6 address space =
[23]
    to be non-routable IP identifiers, in a similar means to the KHI
    approach [24].

    The desired semantics of an IP identifier is to be a name that =
refers
    to an instance of the IP protocol.  Hence it should not be bound to =
a
    particular network interface.  Also, it is desirable for the
    identifier to be long term stable, for instance ensuring that the
    identifier survives renumbering.

    As we will discuss below, for application referrals to work using
    128-bit IP "addresses" as handles for another host, there has to be
    an efficient and scalable way to look up an identifier and find the
    locators.  An obvious way to get scalability is to do hierarchical
    allocation of the identifiers, since this allows for a scalable,
    hierarchical organization of a lookup system and also ensures that
    the identifiers are unique.

    While it certainly isn't the only possibility, in order to work out =
a
    complete picture, this document suggests such a hierarchical
    allocation, in such a way that at least 64 bits of the identifier is
    left to each site to allocate.  (With 64 bits we can use CGA to
    "prove" identifier ownership as a way to prevent redirection attacks
    from off-path attackers.)

all the above conditions seem to be fulfilled by ULAs, but they are not=20=

mentioned anywhere (only in the references)
did you have something different in mind or ULAs are a good candidate=20
for this?

In sections 2.4 and 2.5 the handling of referrals is presented.

I guess that an additional problem that shows up in the case that ids=20
are not valid locators is the support of legacy hosts.
I mean, since the ones that are passing ulid information are the=20
applications, it is possible that the ulid ends up in a legacy host=20
that does not implement the shim and that is not able to translate the=20=

id into a locator, even if the directory service is available.
So, even with this approach, the referral case still presents some=20
issues imho

In section 2.6  Design Alternatives it is stated

    If the identifiers are placed in the DNS using AAAA records, then =
the
    lookup for the AAAA record set (to find the identifier) might also
    return a list of locators.

another alternative that probably would result in similar behaviour=20
would be that the identifier is included in the identifier record, but=20=

that the same query that returns the ID record also returns the locator=20=

set associated to this identifier in the additional information field
Such information would be delivered by the resolver to the shim process=20=

which would store it for when the packet addressed to the identifier=20
arrives

   Such a list can potentially be useful to
    avoid the ip6.arpa lookup to find the locators.  But relying on this
    means that the reverse lookup from the identifier will only be used
    in uncommon cases such as:

    o  The shim6 context state having been garbage collected too early,
       and the upper-layer protocol sends down a packet with a
       destination ULID which is a non-routable identifier.

    o  Application callbacks, referrals, and long-lived application
       handles [27] that are IP addresses.

    For this reason it makes sense to be more consistent and always rely
    on the reverse lookup when the context is established.

I fail to understand this point. I mean avoiding extra DNS lookups is=20
good, since it reduces latency and load to the servers. why do we want=20=

to impose this? just to make sure that the reverse information is in=20
place i.e. that the reverse tree is properly populated? i think this is=20=

an expensive price to pay for this and i would rather prefer that those=20=

that have the reverse tree poorly populated simply have problems with=20
the referrals rather than imposing a penalty to all shim communications

About using v4 locators

I think that supporting v4 locators would be very valuable for the=20
protocol. I would suggest to move this to a separate document and adopt=20=

it. (maybe it could be included in the base spec since it seems quite=20
trivial to do)

The option proposed for this in the draft is:

    When CGA is used to prevent redirection attacks in shim6, there is =
no
    constraint on the locators that are used apart from host B must know
    its own locators so it can pass it them to host A. In particular, we
    can use IPv4 addresses as locators; this doesn't require anything
    more than defining how an IPv4 address is carried in the 128-bit
    fields in the Locator List option.

I wonder if it wouldn't be better to change the verification method=20
field to a generic Flag field and use some bits there to specify the=20
address family, what do you think?

w.r.t. NAT i guess that an additional consideration is how does the=20
host detects its own addresses in order to include them in its own=20
locator set and communicate them to the peer. Of course, shim should be=20=

able to detect private addresses and not include them in the Ls(local)

IMHO a reasonable trade-off would be now to specify how IPv4 addresses=20=

could be used as locators when no nat is involved and leave the nat=20
support for further study

About TE and source address rewriting by exit routers

In section 1.2 it is stated that

    o  With ULIDs that are non-routable identifiers, there will most
       likely be only one identifier for the destination as well as the
       source.  Thus the role of RFC 3484 is largely removed.  But there
       is an additional step of looking up the identifier to find the
       locator, and at that point in time it makes sense to consider
       traffic engineering for selecting the initial locator pair.

The point here, i guess is that RFC3484 is used today to select=20
addresses for initial contact. As those addresses are going to be used=20=

both as ULIDs and locator, then RFC3484 is currently used to select=20
both.
In the case that we use a ULID that it is not a locator, then which=20
mechanism are we going to use to select each of them,
In the draft it is assumed that a single identifier will be available=20
for an endpoint. I don't see the need to have more than one, but i=20
guess it would make sense to consider the case, but we can probably=20
deffer this till later. However, it seems reasonable to have more than=20=

one locator. The question would be at this point why not use RFC3484=20
(or RFC3484bis) to select among the multiple locator pair combinations?

In other words, i don't see that the role of RFC3484 removed, but=20
rather that RFC3484 is used to perform locator pair selection rather=20
than ULID pair selection... does this makes sense?

Moreover, i would wonder if it didn't make sense to use RFC3484 to=20
select locators after a failure i.e. to select the locators to explore=20=

after a failure is detected using RFC3484 for that, or even if we have=20=

multiple address pairs that are working as locators, to use RFC3484 to=20=

select which one among them to use to rehome the communication.

It would also make sense imho to explore the relation between RFC3484=20
and the locator preference option. I mean, the preference option is=20
used by the local host to inform the peer about its preferences when=20
the peer selects the locator to use. RFC3484 is about which of the=20
local locators will the local host use for communication. Probably,=20
there these two are somehow related i.e. in many cases probably the=20
local host would want that these two criteria are similar i guess

Essentially, the question is whether RFC3484 is a good candidate for=20
locator selection and how this relates to the preference information=20
available in SHIM.

in section 3.  Traffic Engineering Support

    The traffic engineering pieces that might be desirable and that are
    easy to implement in this model are outlined in this section.  If =
the
    ULID is a routable locator, then it makes sense to recommend that
    applications use DNS SRV records for the initial (non-shimmed)
    contact, and also provide at least a DHCPv6 option by which a site
    administrator can control what each host in the site uses in the
    shim6 Locator Preference option.

    In the case when the ULID is a non-routable identifier, a different
    set of mechanisms are desirable.  Instead of using DNS SRV records
    for the lookup of the domain name of the peer, we want similar
    control when looking up an identifier to find the set of locators.

I guess that the information in SRV records is about the peer's=20
preferences about its own locators
The same thing w.r.t. the preference information received in the=20
preference option of the shim protocol
however, the local host may also have some policy w.r.t. which locator=20=

prefer when there are multiple of them for a given destiantion.
I guess that the local preferences will be expressed in the RFC3484=20
table
I guess that an additional issue to consider would be the interaction=20
between remote policy information obtained through SRV/preference=20
option and the local policy information in rfc3484 policy table

later on in section 3.1  Recommending use of DNS SRV

    In shim6 as specified, the host rely on existing DNS mechanisms, =
such
    as AAAA records or any other mechanism, to find a list of locators =
to
    try.  When AAAA records are used, there is no mechanism for the
    destination site to express any ranking for primary/fallback, or any
    mechanism to spread load across the paths that are represented by =
the
    locators, since the AAAA resource record set is treated as a set =
with
    no implied order.

This is not strictly true, i believe

I mean, rfc3484 preserves the order of the received locator list if no=20=

rules apply
So, if for the local host all the addresses are equal, then the order=20
in which the locators are returned by the DNS is preserved, and the app=20=

is likely to pick the first one
So, changing the order in which DNS returns the locator list would=20
likely result in some form of load sharing AFAIK
Perhaps if the order is always the same, a behaviour similar to primary=20=

backup could be achieved.

I agree though that using SRV records would result in a more fine=20
grained well documented behaviour

In sections 3.4 and section 3.5 a mechanisms for supporting source=20
address rewriting by hosts is presented.

I have two comments w.r.t. to the mechanisms presented:

First, i think that the idea to carry the Sent Locator Pair and=20
Received Locator pair options to discover and allow coordination=20
between routers and hosts is very clever. As i understand it the main=20
focus is putted in the case where host A learns through this option new=20=

locators of its own.
However, it may be case that a host receives a payload packet with a=20
new locator from its peer i.e. the source locator is not included in=20
Ls(peer). In this case, it will accept the packet since the CT match,=20
but it cannot use it to send packets until a CGA/HBA verification of=20
this locator is presented by the peer. However, when it sends the new=20
locator in the received locator option, the peer should sent an Update=20=

request the includes the new locator signed with CGA. I am not sure if=20=

this is what is expressed in the last bullet of section 3.4...

Second, i think that the presented mechanisms are potentially very=20
useful for rewriting payload packets, but i really don't think it worth=20=

the effort to rewrite the source address of the shim control message. I=20=

mean, they are a fairly reduced amount of packets, so i guess that they=20=

don't really affect TE considerations. Supporting this adds additional=20=

complexity to a protocol which is already fairly complex imho. I mean,=20=

i guess it can be done as you present in the draft, but i don't think=20
we win much with allowing rewriting of these few packet and we would be=20=

adding much complexity. I would keep it just for data packets and not=20
for signalling packets

regards, marcelo

El 01/03/2006, a las 21:45, Erik Nordmark escribi=F3:

>
>
> -------- Original Message --------
> Subject: I-D ACTION:draft-nordmark-shim6-esd-00.txt
> Date: Tue, 28 Feb 2006 15:50:02 -0500
> From: [email-masked]
> Reply-To: [email-masked]
> To: [email-masked]
>
> A New Internet-Draft is available from the on-line Internet-Drafts=20
> directories.
>
>
> 	Title		: Extended Shim6 Design for ID/loc split and
>                           Traffic Engineering
> 	Author(s)	: E. Nordmark
> 	Filename	: draft-nordmark-shim6-esd-00.txt
> 	Pages		: 25
> 	Date		: 2006-2-28
> =09
>    The Shim6 protocol provides for locator agility while satisfying =
the
>    'first, do no harm' security requirements.  This document outlines
>    some extensions to Shim6 that in addition provides complete
>    separation between identifiers and locators, and allows routers to
>    rewrite the locators in the shim6 packets as a way to provide=20
> traffic
>    engineering information to the hosts.
>
>    The document also outlines a simple extension to allow shim6, with =
a
>    CGA upper-layer ID, to operate using IPv4 addresses as locators.
>
>    The purpose of this outline is to stimulate discussions.
>
> A URL for this Internet-Draft is:
> http://www.ietf.org/internet-drafts/draft-nordmark-shim6-esd-00.txt
>
> To remove yourself from the I-D Announcement list, send a message to
> [email-masked] with the word unsubscribe in the body of=20=

> the message.
> You can also visit https://www1.ietf.org/mailman/listinfo/I-D-announce
> to change your subscription settings.
>
>
> Internet-Drafts are also available by anonymous FTP. Login with the=20
> username
> "anonymous" and a password of your e-mail address. After logging in,
> type "cd internet-drafts" and then
> 	"get draft-nordmark-shim6-esd-00.txt".
>
> A list of Internet-Drafts directories can be found in
> http://www.ietf.org/shadow.html
> or ftp://ftp.ietf.org/ietf/1shadow-sites.txt
>
>
> Internet-Drafts can also be obtained by e-mail.
>
> Send a message to:
> 	[email-masked].
> In the body type:
> 	"FILE /internet-drafts/draft-nordmark-shim6-esd-00.txt".
> =09
> NOTE:	The mail server at ietf.org can return the document in
> 	MIME-encoded form by using the "mpack" utility.  To use this
> 	feature, insert the command "ENCODING mime" before the "FILE"
> 	command.  To decode the response(s), you will need "munpack" or
> 	a MIME-compliant mail reader.  Different MIME-compliant mail =
readers
> 	exhibit different behavior, especially when dealing with
> 	"multipart" MIME messages (i.e. documents which have been split
> 	up into multiple messages), so check your local documentation on
> 	how to manipulate these messages.
> 	=09
> 	=09
> Below is the data which will enable a MIME compliant mail reader
> implementation to automatically retrieve the ASCII version of the
> Internet-Draft.
>
> Content-Type: text/plain
> Content-ID: <[email-masked]>
>
>
> _______________________________________________
> I-D-Announce mailing list
> [email-masked]
> https://www1.ietf.org/mailman/listinfo/i-d-announce
>

Delivery-date: Thu, 02 Mar 2006 12:10:58 +0000
Cc: shim6 <[email-masked]>
From: marcelo bagnulo braun <[email-masked]>
Subject: Re: Defining and updating CT(peer)
Date: Thu, 2 Mar 2006 13:54:50 +0200
To: Shinta Sugimoto <[email-masked]>

El 02/03/2006, a las 13:07, Shinta Sugimoto escribi=F3:

> Hi Marcelo,
>
> Please find a few more comments below.
>
> On Thu, 2 Mar 2006 12:06:18 +0200
> marcelo bagnulo braun <[email-masked]> wrote:
>
> ...[snip]...
>
>>>> - Receiving I2 packets in I1-SENT state
>>>>
>>>> This scenario is the case of simultaneous context
>>>> establishment and the draft states that the context
>>>> state remains unchanged (waiting for the R2 message)
>>>> so no problem in this case and no change is required.
>>>
>>> I assume the above case is depicted in Figure 26 in the
>>> protocol document <draft-ietf-shim6-proto-03.txt>.
>>> However, I need some clarification here.  Let me expand a
>>> little bit:  According to the protocol draft, when a host
>>> detects concurrent context establishment (e.g. receiving I2/I2bis
>>> message in I1-SENT state), it replies with R2 message and wait
>>> until it receives R2 message from the peer for state transition
>>> from I1-SENT to ESTABLISHED.  But I wonder why the state
>>> transition should be deferred until receipt of R2 message.
>>> Does it help ruling out some sort of security threat ?
>>>
>>
>> no as far i can tell,
>> i think you can do it either way, either you update CT(peer) and move
>> to ESTABLISHED state at the reception of the I2 or at the reception =
of
>> R2
>> Do you see any reason why prefer either one of them?
>
> I'd prefer the first choice (not to wait until receipt of R2) because
> it simply requires less latency.  If R2 message from the peer is lost,
> another cycles of round-trip delay may be needed.
>

ok, i think this makes sense

regards, marcelo

>>> In addition, I think there is an important issue in the
>>> choice of CT when sending R2 message in reply to I2/I2bis
>>> message.  That is, the CT(responder/local) to be included
>>> in R2 message must be consistent with the CT(initiator/local)
>>> which had been included in the I1 message.  Otherwise, the
>>> end hosts may be confused which CT(peer) to use for establishing
>>> the context.
>>
>> right, but this would be derived from the fact that the I2 message is
>> received in I1-SENT state i.e. there is already state for this ULID
>> pair context so the CT(local) is already defined (which was included=20=

>> in
>> the I1 packet) hence the same one must be used for the R2 packet
>
> Ok, I see.  As far as the host maintains state information per ULID=20
> pair
> there should be no concern.
>
>>
>>>   Maybe some texts are needed in Section 7.11
>>> with regard to the treatment of CT value to be included in
>>> R2 message.
>>>
>>
>> I am not sure about this.
>> As i mention, the CT(peer) information is associated to the context
>> state itself.
>> But i don't have strong preferences for this
>
> No need for the texts I think.  Thanks for the clarification.
>
>
> Regards,
> Shinta
>

Delivery-date: Thu, 02 Mar 2006 11:08:18 +0000
Date: Thu, 02 Mar 2006 20:07:47 +0900
From: Shinta Sugimoto <[email-masked]>
To: marcelo bagnulo braun <[email-masked]>
Subject: Re: Defining and updating CT(peer)
Cc: shim6 <[email-masked]>

Hi Marcelo,

Please find a few more comments below.

On Thu, 2 Mar 2006 12:06:18 +0200
marcelo bagnulo braun <[email-masked]> wrote:

=2E..[snip]...

> >> - Receiving I2 packets in I1-SENT state
> >>
> >> This scenario is the case of simultaneous context
> >> establishment and the draft states that the context
> >> state remains unchanged (waiting for the R2 message)
> >> so no problem in this case and no change is required.
> >
> > I assume the above case is depicted in Figure 26 in the
> > protocol document <draft-ietf-shim6-proto-03.txt>.
> > However, I need some clarification here.  Let me expand a
> > little bit:  According to the protocol draft, when a host
> > detects concurrent context establishment (e.g. receiving I2/I2bis
> > message in I1-SENT state), it replies with R2 message and wait
> > until it receives R2 message from the peer for state transition
> > from I1-SENT to ESTABLISHED.  But I wonder why the state
> > transition should be deferred until receipt of R2 message.
> > Does it help ruling out some sort of security threat ?
> >
>=20
> no as far i can tell,
> i think you can do it either way, either you update CT(peer) and move=20
> to ESTABLISHED state at the reception of the I2 or at the reception of=20
> R2
> Do you see any reason why prefer either one of them?

I'd prefer the first choice (not to wait until receipt of R2) because
it simply requires less latency.  If R2 message from the peer is lost,
another cycles of round-trip delay may be needed.

> > In addition, I think there is an important issue in the
> > choice of CT when sending R2 message in reply to I2/I2bis
> > message.  That is, the CT(responder/local) to be included
> > in R2 message must be consistent with the CT(initiator/local)
> > which had been included in the I1 message.  Otherwise, the
> > end hosts may be confused which CT(peer) to use for establishing
> > the context.
>=20
> right, but this would be derived from the fact that the I2 message is=20
> received in I1-SENT state i.e. there is already state for this ULID=20
> pair context so the CT(local) is already defined (which was included in=
=20
> the I1 packet) hence the same one must be used for the R2 packet

Ok, I see.  As far as the host maintains state information per ULID pair
there should be no concern.

>=20
> >   Maybe some texts are needed in Section 7.11
> > with regard to the treatment of CT value to be included in
> > R2 message.
> >
>=20
> I am not sure about this.
> As i mention, the CT(peer) information is associated to the context=20
> state itself.
> But i don't have strong preferences for this

No need for the texts I think.  Thanks for the clarification.

Regards,
Shinta

Delivery-date: Thu, 02 Mar 2006 10:15:24 +0000
Cc: [email-masked], Iljitsch van Beijnum <[email-masked]>
From: marcelo bagnulo braun <[email-masked]>
Subject: Re: shim6 @ NANOG   (forwarded note from John Payne)
Date: Thu, 2 Mar 2006 12:15:11 +0200
To: Erik Nordmark <[email-masked]>

El 02/03/2006, a las 3:56, Erik Nordmark escribi=F3:

> Geoff Huston wrote:
>> Reposted on behalf of Iljitsch van Beijnum
>
>
>> Push back by transit ASes is harder. This is what I mean by that:
>>      A --- B
>>    /         \
>> X             Y
>>    \         /
>>      C --- D
>> C's link to D may be low capacity or expensive, so D would prefer it
>> if X would send traffic to Y over another route if possible. C can
>> make this happen in BGP by prepending its AS one or more times so X
>> will see the following AS paths:
>> A B Y
>> C C C D Y
>> All else being equal, X will choose the path over A to reach Y.
>> The simple answer here is that if the multihomed site receives a BGP
>> feed just like today (except that it's a read only feed) and thus
>> makes outgoing path selection decisions just like today, transit ASes
>> have exactly the same tools as they have today. But presumably, if
>> shim6 takes off many smaller sites that aren't comfortable with BGP
>> will multihome also, so this push back won't work as well anymore.
>> Creating a new way to accomplish this result is probably possible,
>> but not entirely non-trivial, and probably something we wouldn't want
>> to deliver on day one.
>
> Wouldn't the hosts in X and Y have two IPv6 addresses?
> Thus BGP itself at site X wouldn't know that B:Y and D:Y is the same=20=

> host, hence BGP wouldn't compare them to find the best.
>

yes, but i think could work anyway
I mean, suppose that we are performing source address rewriting by the=20=

multihomed site exit routers
Now suppose that intra site routing is performed based solely in the=20
destination address (just as it is in the current paradigm)
Assume that the multihomed site obtains BGP feed through both of its=20
providers.
Now, based on the BGP information obtained, the intra site routing will=20=

preffer one or other exit to reach a given destiantion.
So, when a host within the multihomed site sends a packet to that=20
destination the exit route will be determined by this BGP information=20
hence the policy information pushed by the transit ISPs can affect the=20=

intra site route selection
Since the site exit routers can rewrite the source address, they will=20
select the appropriate source prefix

makes sense?

regards, marcelo

>    Erik
>
>

Delivery-date: Thu, 02 Mar 2006 10:07:12 +0000
Cc: shim6 <[email-masked]>
From: marcelo bagnulo braun <[email-masked]>
Subject: Re: Defining and updating CT(peer)
Date: Thu, 2 Mar 2006 12:06:18 +0200
To: Shinta Sugimoto <[email-masked]>

Hi Shinta,

El 02/03/2006, a las 9:46, Shinta Sugimoto escribi=F3:

>>
>> -- Receiving I1 packets in I1-SENT state
>>
>> In this case, we are in the case of simultaneous shim6
>> context establishment. In this case the draft states that
>> CT(peer) is set to the value contained in the received I1
>> message. Again, this would open the door to the
>> considered DoS attack. Since the hosts will receive a R2
>> message later anyway, the proposed solution is to set
>> CT(peer) only upon the reception of the R2 message (which
>> must be validated with the nonce)
>>
>> Proposed change: not  setting CT(peer) upon the reception
>> of I1 in I1-SENT  state but do it upon the reception of R2
>
> I agree.
>
> I also think it's fine to keep the optimization in case of
> concurrent context exchange (replying to I1 with R2).
>

yes, this is kept as it was, the only change is that the=20
definition/update of the CT(peer) is performed when the R2 packet is=20
received

...
>>
>> - Receiving I2 packets in ESTABLISHED state
>>
>> No action is performed upon the reception on an I2
>> packet with a different CT(peer) in ESTABLISHED state
>> However, this needs to be changed in order to support
>> context recovery, since we are in the case where the
>> peer is trying to establish a context that the local
>> node has already available. So, upon the reception of
>> a I2 message for a context that is in ESTABLISHED
>> state, the node needs to verify the validator, and the
>> relation between the source locator and the source
>> identifier (i.e. HBA/CGA check) and if it succeeds,
>> then it must update CT(peer) and eventually the other
>> information related to the context e.g. the locator set
>>
>> Question: does this means that we update the existent
>> context with the new information or that we create a
>> new context with the new information and we let the
>> old context to age and expire?
>
> I think the former choice is fine. (see below)
>
>> Proposed change: upon the reception of an I2 message
>> in ESTABLISHED state, the validator and ULID-locator
>> context is verified and the context state information
>> (CT(peer) and Ls(peer)) is updated
>
> I think above is fine.  In such case (a responder receives
> I2 message from an initiator) in ESTABLISHED state, it
> implies that the context was lost (for some reason) at the
> peer (initiator).  So, it seems that holding the old
> context does not make much sense.

yes

>>
>> - Receiving I2 packets in I1-SENT state
>>
>> This scenario is the case of simultaneous context
>> establishment and the draft states that the context
>> state remains unchanged (waiting for the R2 message)
>> so no problem in this case and no change is required.
>
> I assume the above case is depicted in Figure 26 in the
> protocol document <draft-ietf-shim6-proto-03.txt>.
> However, I need some clarification here.  Let me expand a
> little bit:  According to the protocol draft, when a host
> detects concurrent context establishment (e.g. receiving I2/I2bis
> message in I1-SENT state), it replies with R2 message and wait
> until it receives R2 message from the peer for state transition
> from I1-SENT to ESTABLISHED.  But I wonder why the state
> transition should be deferred until receipt of R2 message.
> Does it help ruling out some sort of security threat ?
>

no as far i can tell,
i think you can do it either way, either you update CT(peer) and move=20
to ESTABLISHED state at the reception of the I2 or at the reception of=20=

R2

Do you see any reason why prefer either one of them?

> In addition, I think there is an important issue in the
> choice of CT when sending R2 message in reply to I2/I2bis
> message.  That is, the CT(responder/local) to be included
> in R2 message must be consistent with the CT(initiator/local)
> which had been included in the I1 message.  Otherwise, the
> end hosts may be confused which CT(peer) to use for establishing
> the context.

right, but this would be derived from the fact that the I2 message is=20
received in I1-SENT state i.e. there is already state for this ULID=20
pair context so the CT(local) is already defined (which was included in=20=

the I1 packet) hence the same one must be used for the R2 packet

>   Maybe some texts are needed in Section 7.11
> with regard to the treatment of CT value to be included in
> R2 message.
>

I am not sure about this.
As i mention, the CT(peer) information is associated to the context=20
state itself.
But i don't have strong preferences for this

thanks again for your in depth review

Delivery-date: Thu, 02 Mar 2006 09:06:19 +0000
Cc: shim6-wg <[email-masked]>
From: marcelo bagnulo braun <[email-masked]>
Subject: Re: shim6 @ NANOG   (forwarded note from John Payne) (fwd)
Date: Thu, 2 Mar 2006 11:05:42 +0200
To: Igor Gashinsky <[email-masked]>

Hi Igor,

It is very nice from you to give us feedback about this...

i will try to comment some of the issues that you mention below...

El 01/03/2006, a las 10:10, Igor Gashinsky escribi=F3:
>
>
> 1) Most connections to content providers (with the exceptions of
> long-lived streaming sessions, but those sessions are fairly "few" per
> server) are very short-lived http (think about 15 packets in each
> direction including the setup/teardown). Since, shim6 (as designed =20
> right
> now) does not initiate from the first packet(s), it might not take =20
> effect
> for these short-lived sessions, and therefore will not help in case of
> failure, so in effect, *does not work* at all for fast http =20
> transactions
>

First of all i think it is important to remember that the goal of the =20=

SHIM6 protocol is to _preserve_ established sessions through outages.

However, there are other tools that are being discussed, that would =20
allow to establish new communication _after_ an outage.

The rationale behind the different tools as i understand it would be =20
something like the following:
- If two hosts have long lived sessions (that could be a long tcp =20
session, or many short tcp sessions or a long udp exchange), then it is =20=

likely that it is important for them to preserve this session through =20=

outages. In addition, since the session is long lived, and as the =20
probability of having an outage affecting the communication raises with =20=

the lifetime of the communication, it seems reasonable to try to =20
protect the session. Moreover, as the session is long lived, the amount =20=

of packets will be large enough to reduce the effect of the overhead =20
introduced by the shim context establishment
- However, if two hosts have a short lived session, like a short TCP =20
connection, the the above conditions are not true. Basically, this =20
means that since the session is short, then the probability of an =20
outage affecting this session during its lifetime is reduced. Moreover, =20=

since the session has just been established, and an outage affects it, =20=

the assupmtion is that the host will be willing to retry to establish =20=

the session again. For this there are mechanisms being proposed in =20
order to allow the hosts to be able to establish new connections n the =20=

case that a failure is affecting one of the available addresses. In =20
other words, the rationale here is that since the session is short =20
lived, the host will prefer to take the risk of having to reestablish =20=

the session in the case of an outage rather than paying the shim6 =20
overhead in all its communications (when it is likely that no outage =20
will affect them). It should also be noted, that as you mention the =20
patience of the users is quite limited and they are likely to retry if =20=

the connection takes too long, which seems in line with the above case =20=

for retrying to establish the connection. In addition i would like to =20=

point out that because of the time that it may take to reconverge, a =20
BGP based solution for multihoming does not preserves established =20
communication through all the outages, especially when you have anxious =20=

users that are willing to hit the reload button.

So the effort for this case imho is putted in enabling the capacity or =20=

establishing new sessions after an outage rather than in preserving =20
established connections, do you think this makes any sense to you

> 1) In order to "fix" #1, shim6 has the potential to put a sizable =
(over
> 10%) state penalties on our servers (to service end-sites w/ shim6),
> something which is arguably the most painful thing for those servers,
> which can translate into millions of dollars of additional hardware, =20=

> and
> many more millions of dollars per year to power/cool that hardware.
>

Well, the good thing about mechanisms to establish new communications =20=

through outages is that they are located in the client only and have no =20=

effect in the server

> 3) While TE has been discussed at length already, but it is something
> which is absolutely required for a content provider to deploy shim6. =20=

> There
> has been quite a bit of talk about what TE is used for, but it seems =20=

> that
> few people recognize it as a way of expressing "business/financial
> policies". For example, in the v4 world, the (multi-homed) end-user =20=

> maybe
> visible via both a *paid* Transit path (say UUNET), and a *free* =20
> peering
> link (say Cogent), and I would wager that most content providers would
> choose the free link (even if performance on that link is (not hugely)
> worse). That capability all but disappears in the v6 world if the =20
> Client
> ID was sourced from their UUnet ip address (since that's who they =
chose
> to use for outbound traffic), and the (web) server does not know that
> that locator also corresponds to a Cogent IP (which they can reach for
> free).

I fail to understand the example the you are presenting here...

are you considering the case where both the client and the server are =20=

both multihomed to Cognet and UUnet?
something like

      UUnet
     /     \
    C       S
     \     /
      Cognet

I mean in this case, the selection of the server provider is determined =20=

by the server's address not by the client address, right?
The server can influence such decision using SRV records in the DNS, =20
but not sure yet if this is the case you are considering

>  This change alone would add millions to the bw bills of said
> content providers, and well, reduce the likelyhood of adoption of the
> protocol by them. Now, if the shim6 init takes place in the 3way
> handshake process, then the servers "somewhat" know what all possible
> paths to reach that locator are, but then would need some sort of a
> policy server telling them who to talk to on what ip, and that's =20
> something
> which will not simply scale for 100K+ machines.
>

I am not sure i understand the scaling problem here
Suppose that you are using a DHCP option for distributing the SHIM6 =20
preferences of the RFC3484 policy table, are you saying that DHCP does =20=

not scale for 100K+ machines? or is there something else other than =20
DHCP that

> 4) As has also been discussed before, the initial connect time has to =20=

> be
> *very* low. Anything that takes longer then 4-5 seconds the end-users =20=

> have
> a funny way of clicking "stop" in their browser, deeming that "X is =20=

> down,
> let me try Y", which is usually not a very acceptable scenario :-) So,
> whatever methodology we use to do the initial set-up has to account =
for
> that, and be able to get a connection that is actually starting to do
> something in under 2 seconds, along with figuring out which sourceIP =20=

> and
> destIP pairs actually can talk to each other.

As i mentioned above, we are working in other mechanisms than the shim6 =20=

protocol itself that can be used for establishing new communication =20
through outages.

you can find some work in this area in

ftp://ftp.rfc-editor.org/in-notes/internet-drafts/draft-bagnulo-ipv6-=20
rfc3484-update-00.txt

if you have comments, and especially improvements on the ideas of this =20=

draft or other ideas of how to tackkle this problem of initial contact, =20=

it would be really useful

>
> I hope this gives people some visibility as to what some content =20
> providers
> think about shim6, and why deploying it is, well, not something that
> people will scramble (or very possibly chose) to do, unless those are
> addresses. And, yes, everyone understands that it's all about making
> trade-offs, but if you make the wrong trade-offs, and not enough =
people
> deploy the protocol, it's simply not going to fly, and people will =20
> just go
> back to de-aggregating in v6 and let Moore's Law deal with the issue =20=

> (and
> anyone who thinks that people will prevent paying customers from
> deagregating has not seen how many hoops ISP's will jump through for =20=

> that
> extra revenue, or how fast customers will jump to other ISP's which =20=

> will
> allow them to do just that). I don't know if more work on shim6 is the
> answer, or GSE/8+8 is a better alterntive, but it sure looks like what =
=20
> we
> have in shim6 today (and it's current direction) isn't going to cut =
it.
>
> Just my $0.02
>

yes your feedback is very welcome

thanks, marcelo

> Thanks,
> -igor
>
>
>

Delivery-date: Thu, 02 Mar 2006 08:13:25 +0000
Date: Thu, 2 Mar 2006 09:13:09 +0100 (CET)
From: Mikael Abrahamsson <[email-masked]>
To: shim6-wg <[email-masked]>
Subject: Re: shim6 @ NANOG   (forwarded note from John Payne) (fwd)
Organization: People's Front Against WWW

On Wed, 1 Mar 2006, Iljitsch van Beijnum wrote:

> This is not what I think of when I say "multihoming". I don't see an easy way 
> to do this with shim6 (there may be a non-easy way, though) so maybe hosting 
> companies that are big enough to do peering will need to continue to do this 
> as in IPv4.

Couldn't this be implemented into the resolving DNS, in that it could 
include priority information into the DNS response sent to the host that 
does the initial connect? Or it could be done via reverse-DNS (in the case 
where the "called" party needs to know which IP to send to when there 
are several to choose from)?

Perhaps one state in the shim6 initiation could be to do reverse-lookup 
and via some record there gain knowledge about which IP should be 
prioritized? This means that routingpolicy needs to be somehow implemented 
into the DNS server, which would require new code there as well.

-- 
Mikael Abrahamsson    email: [email-masked]

Delivery-date: Thu, 02 Mar 2006 07:46:39 +0000
Date: Thu, 02 Mar 2006 16:46:16 +0900
From: Shinta Sugimoto <[email-masked]>
To: marcelo bagnulo braun <[email-masked]>
Subject: Re: Defining and updating CT(peer)
Cc: shim6 <[email-masked]>

Hello Marcelo,

Thank you for your thorough clarification and sorry that it took
so long for me to reply.  Please find my comments inline:

On Sun, 19 Feb 2006 19:27:33 +0100
marcelo bagnulo braun <[email-masked]> wrote:

> Hi,
> 
> One of the issues that Shinta brought up in his review is the potential 
> attack concerning updating CT(peer) with shim6 control packets.
> We have been discussing this point with Erik, and this is an attempt to 
> summarize my understanding of the problem so far.
> Any comments are welcome
> 
> Defining and updating CT(peer)
> 
> # Issue description
> 
> -- When a shim6 context is established (or in its way to be 
> established), the reception of shim6 control packets may result
> in defining/updating the CT(peer) associated to the shim6 context.
> This may pose some security concerns if this operation is not
> properly secured.
> 
> # Threat related to this operation:
> 
> -- The main threat identified is that this could allow an attacker 
> to change the CT(peer) of a context leading to an DoS
> attack, since subsequent packets with the new CT won't be
> recognized by the receiver
> 
> # Level of security targeted
> 
> -- The proposed level of security for this operation is that 
> CT(peer) is only updated when the shim control message that
> requests so contains a token that proves prior contact and that
> the source locator can be verified that it is securely bound
> to the source ULID of the context, does this seems a
> reasonable requirement?

Yes, I think so.

> 
> # Scope
> 
> -- Messages involved
> I1, I2, R2 and I2bis contain CT information so they must be
> considered in this analysis
> R1 does not contain CT information, so no need to consider it
> R1bis contain CT(packet) which does not result in updating
> CT(peer), so there is no need to consider it neither for this
> particular threat.
> 
> -- States involved
> This concern is clearly raised when the CT(peer) for the
> context is defined. So, the following states are
> affected:
> - ESTABLISHED
> - I2bis-SENT
> 
> In addition, when a state is being created, the reception
> of different messages result in the definition of CT(peer)
> for that context. In this case, the situation is somehow
> different, since CT(peer) is not defined yet, but the
> result of the attack is similar, since CT(peer) may end
> up with a fake value. This can occur in the following states:
> - I1-SENT
> - I2-SENT
> 
> # Summary of the proposed approach
> 
> The idea is not to update nor define CT(peer) upon the
> reception of messages that don't contain information of
> prior contact. So, the proposal is to defer the
> update/define of the CT(peer) until the reception of:
> - R2 messages received in I1-SENT, I2-SENT or I2bis-SENT
>    states. The R2 contains an valid Initiator nonce
>    that proves prior exchange and has a valid ULID-locator
>    relation proven through HBA/CGA information, or
> - I2 messages received in ESTABLISHED state. The I2
>    contains a valid validator and
>    a valid ULID-locator relation proven through HBA/CGA
>    information.

Ok.

> 
> In this approach, it is still possible for an attacker to
> assign an invalid CT(peer) for an established context, but
> in order to do that, the attacker needs to be able to
> perform a 4-way handshake from one of the locators securely
> bound (through HBA/CGA technique) to the used ULID

Right.

> 
> Note: next the complete case by case analysis is detailed,
> so, if you are not really interested i would suggest stop
> reading here...
> 
> 
> # Analysis of different packet processing
> 
> -- Receiving I1 packets in ESTABLISHED state
> 
> In case that a host receives a I1 packet for a context that
> is in the ESTABLISHED state, the draft assumes that the peer
> has lost context and triggers the context recovery mechanism,
> which consists in  updating the CT(peer) to the one contained
> in the I1 packet and then sending an R2 packets.
> I guess this is too weak.
> Probably we need to improve this, dropping the optimization
> and requiring the full 4-way exchange in this scenario. This
> means updating the CT(peer) only on I2 packets. This would
> have the additional cost that it would take 2 more packets
> to create the loss context in this case.
> 
> Suggested change: drop optimization for this case and go
> for R1, I2, R2 procedure

Yes, I agree.

> 
> -- Receiving I1 packets in I1-SENT state
> 
> In this case, we are in the case of simultaneous shim6
> context establishment. In this case the draft states that
> CT(peer) is set to the value contained in the received I1
> message. Again, this would open the door to the
> considered DoS attack. Since the hosts will receive a R2
> message later anyway, the proposed solution is to set
> CT(peer) only upon the reception of the R2 message (which
> must be validated with the nonce)
> 
> Proposed change: not  setting CT(peer) upon the reception
> of I1 in I1-SENT  state but do it upon the reception of R2

I agree.

I also think it's fine to keep the optimization in case of
concurrent context exchange (replying to I1 with R2).

> 
> -- Receiving I1 packets in I2-SENT state
> 
> This case is similar to the previous one. It affects the
> case of almost simultaneous context establishment. The
> draft states that CT(peer) is defined upon the reception
> of an I1 message in I2-SENT state. This present similar
> threats as the previous case, and the proposed solution
> is the same. Define CT(peer) upon the reception of R2.
> 
> Proposed change: not setting CT(peer) upon the reception
> of I1 in I2-SENT, but do it with R2

Ok.

> 
> -- Receiving I1 packets in I2bis-SENT state
> 
> Similar to the previous case, this scenario occurs when
> one end attempts to create a context that the other end
> is trying to recover. The draft states that CT(peer)
> must be updated upon the reception of an I1 message,
> but this open the possibility of the considered attack.
> The solution is again to update CT(peer) upon the
> reception of the R2 message.
> 
> Proposed change: not updating CT(peer) upon the reception
> of I1 in I2bis-SENT, but do it with R2

Ok.

> 
> - Receiving I2 packets in ESTABLISHED state
> 
> No action is performed upon the reception on an I2
> packet with a different CT(peer) in ESTABLISHED state
> However, this needs to be changed in order to support
> context recovery, since we are in the case where the
> peer is trying to establish a context that the local
> node has already available. So, upon the reception of
> a I2 message for a context that is in ESTABLISHED
> state, the node needs to verify the validator, and the
> relation between the source locator and the source
> identifier (i.e. HBA/CGA check) and if it succeeds,
> then it must update CT(peer) and eventually the other
> information related to the context e.g. the locator set
> 
> Question: does this means that we update the existent
> context with the new information or that we create a
> new context with the new information and we let the
> old context to age and expire?

I think the former choice is fine. (see below)

> Proposed change: upon the reception of an I2 message
> in ESTABLISHED state, the validator and ULID-locator
> context is verified and the context state information
> (CT(peer) and Ls(peer)) is updated

I think above is fine.  In such case (a responder receives
I2 message from an initiator) in ESTABLISHED state, it
implies that the context was lost (for some reason) at the
peer (initiator).  So, it seems that holding the old
context does not make much sense.
> 
> - Receiving I2 packets in I1-SENT state
> 
> This scenario is the case of simultaneous context
> establishment and the draft states that the context
> state remains unchanged (waiting for the R2 message)
> so no problem in this case and no change is required.

I assume the above case is depicted in Figure 26 in the
protocol document <draft-ietf-shim6-proto-03.txt>.
However, I need some clarification here.  Let me expand a
little bit:  According to the protocol draft, when a host
detects concurrent context establishment (e.g. receiving I2/I2bis
message in I1-SENT state), it replies with R2 message and wait
until it receives R2 message from the peer for state transition
from I1-SENT to ESTABLISHED.  But I wonder why the state
transition should be deferred until receipt of R2 message.
Does it help ruling out some sort of security threat ?
In addition, I think there is an important issue in the
choice of CT when sending R2 message in reply to I2/I2bis
message.  That is, the CT(responder/local) to be included
in R2 message must be consistent with the CT(initiator/local)
which had been included in the I1 message.  Otherwise, the
end hosts may be confused which CT(peer) to use for establishing
the context.  Maybe some texts are needed in Section 7.11
with regard to the treatment of CT value to be included in
R2 message.

> 
> - Receiving I2 packets in I2-SENT and I2bis-SENT state
> 
> I can't find a scenario for this case to make sense,
> but the draft states that no change is performed upon
> the reception of an I2 message in this state, so i
> guess no threats are involved.
> 
> (Note that even if we update the context information upon
> the reception of an I2 message, this wouldn't present a
> threat since I2 contains a validator to check prior contact
> and HBA/CGA information to check ULID-locator relation)

I agree.

> 
> - Receiving R2 packets in ESTABLISHED state
> 
> No action is performed upon the reception on an R2 packet
> with a different CT(peer) in ESTABLISHED state

This should be another mysterious case.  I agree that there is
no need to process the R2 packet.

> 
> - Receiving R2 packets in I1-SENT, I2-SENT and I2bis-SENT
> 
> The draft states that the host would need to update the
> CT(peer) upon the reception of an R2 packet in I1-SENT,
> I2-SENT and I2bis-SENT state. this does not presents a
> security issue because R2 packet must contain the
> Initiator Nonce which was included in previously sent
> messages

I agree.

> 
> - Receiving I2bis packets in ESTABLISHED state
> 
> No action is performed upon the reception on an I2bis
> packet with a different CT(peer). Moreover, this situation
> is unlikely to happen, since no R1bis message can be generated
> for an existent context, so no proper I2bis validator can be
> obtained, in order to generate a I2bis message for an
> established context

I agree.  In such case, I2bis should be discarded.

> 
> - Receiving I2bis packets in I1-SENT state state
> 
> We are in the simultaneous context establishment situation
> and the draft states that no context information is updated,
> since this is deferred upon the reception of the R2 message

Same as the case when receiving I2 packet in I1-SENT.

> 
> - Receiving I2bis packets in I2-SENT and I2bis-SENT state
> 
> I can't identify which scenario would result in this case,
> but in any case, the context information is not updated upon
> the reception of an I2bis message in this case, so no threat
> is involved

Neither can I imagine such case.  I agree that I2bis packet
should be discarded in this case.

Regards,
Shinta

Delivery-date: Thu, 02 Mar 2006 07:14:27 +0000
Cc: Erik Nordmark <[email-masked]>, shim6-wg <[email-masked]>
From: marcelo bagnulo braun <[email-masked]>
Subject: Re: TE & SHIM6 (was Re: comments on draft-ietf-shim6-proto-03
Date: Thu, 2 Mar 2006 09:13:12 +0200
To: Iljitsch van Beijnum <[email-masked]>

Hi Iljitsch,

El 01/03/2006, a las 17:12, Iljitsch van Beijnum escribi=F3:

> [This message will go into both source address rewriting and traffic=20=

> engineering.]
>
> On 27-feb-2006, at 20:48, Erik Nordmark wrote:
>
>>> There are two problems with allowing routers to rewrite source=20
>>> addresses:
>>> 1. The routers must know which packets are "legacy" and can't have=20=

>>> their source address changed vs which packets are controlled by=20
>>> shim6 or another mechanism that can handle rewritten source=20
>>> addresses.
>>> 2. In current shim6, only previously negotiated source addresses may=20=

>>> be used, which means the shim6-enabled hosts in a site and the=20
>>> rewriting routers must coordinate their efforts so correspondent=20
>>> hosts don't see unexpected source addresses.
>
>> FWIW draft-nordmark-shim6-esd-00.txt is on the way to the I-D=20
>> directory, and it has some ideas for how to address this.
>
> Right. Lots of good points in there, but unfortunately, I disagree=20
> with the mechanisms proposed. I really hope I'll never have to run=20
> DHCPv6 to configure my hosts, it's a big, fat, unelegant protocol.
>

otoh, the protocol is already defined and hosts already implement it=20
and it is nice to only have to run less protocols for configuring hosts=20=

in a site i guess

>>> The first issue is readily solvable by simply having shim6 hosts put=20=

>>> a magic value in the upper 64 bits of the source address that=20
>>> indicates "rewriting permitted".
>
>> Or next hdr =3D IPPROTO_SHIM6.
>
> I don't find this very suitable. If we're going to send many shimmed=20=

> packets, it's more important than ever that we omit the shim header=20
> whenever possible.

i may agree with this but we still don't know how complex would be the=20=

resulting protocol when context tags are carried without the shim=20
header... i recall that someone was going to write a draft about such a=20=

solution looked like and how complex it was in order to be able to=20
evaluate it properly....

> Apart from that, using the source address to signal that the source=20
> address may be changed is much cleaner. It also has the advantage that=20=

> we can now borrow some bits  to make the process easier. What we can=20=

> do is have shim6 capable hosts emit a "source rewriting information=20
> request". That would be a packet addressed to a shim6 correspondent=20
> that has the magic prefix in the source address that triggers source=20=

> address rewriting, and an additional bit combination that tells the=20
> router to send back a list of prefixes it will use to rewrite. The=20
> host can then make sure that the correspondent knows to expect packets=20=

> with these source addresses.
>

but this magic source prefix would not be routable right?
i mean, packets carrying this source prefix MUST be rewritten before=20
they reach the public internet, right?

if this is so, this is not so great because we need again some form of=20=

coordination between the hosts and the routers i.e. a host must knwo if=20=

there is a router that will rewrite the packets before they are routed=20=

through the public internet.

The nice thing about Erik's mechanism is that it does eliminates the=20
need for coordination, since if there is a router that can rewrite the=20=

source address, it may do so, but there is none, there is no problem=20
neither

> If this is an ordered list, the host can then use bits in the data=20
> packets with the rewrite prefix in the source address to tell the=20
> router which addresses it may insert. (Not sure what would happen=20
> though if the router wants to rewrite into Y but the host only allows=20=

> X and Z.)
>
> I've been thinking about something similar for traffic engineering=20
> ever since my message yesterday where I mentioned A6 records. The=20
> problem is that it's far from inconceivable that at some point, a=20
> disconnect forms between the info in the DNS and the actual state of=20=

> the network. The way I see it, we have four ways to convey TE related=20=

> info:
>
> 1. out of band end-to-end: this would be stuff in the DNS
> 2. out of band hop-by-hop: BGP is like this
> 3. in-band end-to-end: measured timing and packet loss information
> 4. in-band hop-by-hop: feedback from routers
>
> The problem is that 2. needs aggregation to scale. 3. and 4. need to=20=

> have contact with the correspondent already, so it's useless in some=20=

> cases, like in the case where we want one or more backup addresses=20
> that are only tried if the primary addresses don't work.

I am not sure i understand why this wouldn't work
I mean if many addresses are included in SRV records and different=20
priorities are included to different prefixes, then if the initiator=20
honnours the SRV information, it will use the primary unless it is=20
broken, right?

>  The only way to convey this is with 1. We can either reuse SRV=20
> records for individual services for this, which has the advantage that=20=

> it's already available today, but the disadvantage that this mechanism=20=

> isn't really used and it needs to be supported on an=20
> application-by-application basis. Alternatively, we can do some magic=20=

> in the resolver library to make this happen.
>

yes, this would make sense imho
> ...

> But an in-band hop-by-hop TE mechanism would allow exactly this. The=20=

> way it would work is that routers are configured to provide feedback=20=

> for packets with a shim header, if necessary. This feedback would be=20=

> in the form of entries that go into the address selection policy=20
> table. The site egress router would probably want to inform hosts=20
> about which source addresses go well with certain destination=20
> prefixes. All routers between the source and destination (including=20
> the site exit router) would signal back "this prefix (which would be=20=

> the prefix that the destination for the packets falls into) preference=20=

> value XXX". To avoid trouble, the preference value shouldn't be=20
> allowed to completely override locally configured info.
>

But how would be the trust model for this, i mean any signaling packet=20=

can configure the preferences of a host in a site, wouldn't this be a=20
problem?

regards, marcelo=

Delivery-date: Thu, 02 Mar 2006 01:57:51 +0000
Date: Wed, 01 Mar 2006 17:56:36 -0800
From: Erik Nordmark <[email-masked]>
To: Iljitsch van Beijnum <[email-masked]>
CC: [email-masked]
Subject: Re: shim6 @ NANOG   (forwarded note from John Payne)

Geoff Huston wrote:
> Reposted on behalf of Iljitsch van Beijnum

> Push back by transit ASes is harder. This is what I mean by that:
> 
>      A --- B
>    /         \
> X             Y
>    \         /
>      C --- D
> 
> C's link to D may be low capacity or expensive, so D would prefer it
> if X would send traffic to Y over another route if possible. C can
> make this happen in BGP by prepending its AS one or more times so X
> will see the following AS paths:
> 
> A B Y
> C C C D Y
> 
> All else being equal, X will choose the path over A to reach Y.
> 
> The simple answer here is that if the multihomed site receives a BGP
> feed just like today (except that it's a read only feed) and thus
> makes outgoing path selection decisions just like today, transit ASes
> have exactly the same tools as they have today. But presumably, if
> shim6 takes off many smaller sites that aren't comfortable with BGP
> will multihome also, so this push back won't work as well anymore.
> Creating a new way to accomplish this result is probably possible,
> but not entirely non-trivial, and probably something we wouldn't want
> to deliver on day one.

Wouldn't the hosts in X and Y have two IPv6 addresses?
Thus BGP itself at site X wouldn't know that B:Y and D:Y is the same 
host, hence BGP wouldn't compare them to find the best.

    Erik

Delivery-date: Thu, 02 Mar 2006 01:48:34 +0000
Date: Wed, 01 Mar 2006 17:47:15 -0800
From: Erik Nordmark <[email-masked]>
To: Iljitsch van Beijnum <[email-masked]>
CC: shim6-wg <[email-masked]>
Subject: Re: shim6 @ NANOG   (forwarded note from John Payne)

Iljitsch van Beijnum wrote:

> True enough, and I certainly hope that frameworks, libraries and the 
> like that implement a connect-by-name type call do this. However, this 
> isn't the only way to solve this. One very interesting feature that I 
> discovered this summer when finally upgrading to a more recent version 
> of FreeBSD was the address selection policy mechanism (see the 
> ip6addrctl command, Windows also has it as netsh interface ipv6 show/set 
> prefixpolicy). With this, it's possible to control address preferences 
> system-wide. I'm not sure how it's implemented exactly, but apparently 
> this policy is applied somewhere between the DNS resolver and the 
> application.

ip6addrctl is an implementation of the configurable policy table which 
is suggested in RFC 3484. Thus it provides a static table.

> One way to put this mechanism to good use is when a DNS lookup for an 
> AAAA record returns more than one address, is to go out and check which 
> address is alive and/or apply local, remote and in-the-middle traffic 
> engineering policies. Since the application hasn't committed to any 
> particular address choice yet, this should work very well. The only 
> downside is that all of this will take an extra roundtrip.

Implementations of the RFC 3484 policy table do not have a way to 
execute some extra code as part of doing the address selection, thus to 
do what you suggest one would have to hack around.
So the existence of ip6addrctl doesn't seem to help to do something like 
this.

I also question doing extra work each time a DNS lookup is performed; 
the lookup might only be done to put an address/name in a log file; 
checking reachability and TE policy for such a lookup would be wasteful.

> I imagine that we can send out some kind of quick reachability check / 
> information request to the other side, maybe even towards all the 
> addresses returned by the DNS. I have to read up on the R1/R2 stuff, but 
> I imagine this could be an R1, without necessarily completing the full 
> handshake at this time.

If an I1/R1 exchange was extended to return all the locators and their 
preferences in the R1 message (currently this isn't required), then such 
an exchange could be used to find out the peer's preferences.

But doing this for all peer IP addresses would be expensive since the 
number of packets could be large and the ULP communication might be 
quite short. So we would be trading off better TE capability against the 
benefits of deferred context establishment.

    Erik

> Since we're waiting for information to come back from the correspondent 
> anyway, we can use this time to query a policy server that's local to 
> the site or is located at one or more of the site's ISPs. If the servers 
> don't waste time, the answer for this should be back by the time the 
> correspondent answers anyway.
> 
> Another advantage of checking reachability at this stage is that we can 
> easily avoid ingress filtering: simply try the reachability check 
> towards the correspondent with the cartesian product of all local and 
> remote addresses at the same time. The probes with an ingress filtered 
> source address shouldn't even take up any non-local bandwidth if the 
> egress routers also perform this filtering.

Delivery-date: Thu, 02 Mar 2006 01:14:51 +0000
Date: Wed, 01 Mar 2006 17:13:14 -0800
From: Erik Nordmark <[email-masked]>
To: Kurt Erik Lindqvist <[email-masked]>
CC: [email-masked]
Subject: Re: [Fwd: Review comments on draft-ietf-shim6-proto-03.txt]

Kurt Erik Lindqvist wrote:
>
> I do make the observation that our charter says :
> 
> o Compatibility will remain for existing mobility mechanisms. It will
> be possible to use Mobile IPv6 on a node that also supports Shim6.
> However, any optimizations or advanced configurations are out of
> scope for shim6.
> 
> So I am kind of inclined to agree with Geoff :-)
> 
> Although I can't say I feel strongly either...

I know what the charter says and I don't have problem with that.

But I don't think the charter text means that we can't make 
*observations* in our documents - there are no optimizations or advanced 
configurations specified in the text that is in the document.

    Erik

Delivery-date: Wed, 01 Mar 2006 19:47:46 +0000
Date: Wed, 01 Mar 2006 11:45:58 -0800
From: Erik Nordmark <[email-masked]>
To: shim6 <[email-masked]>
Subject: [Fwd: I-D ACTION:draft-nordmark-shim6-esd-00.txt]

This is a multi-part message in MIME format.
--------------070406060909010001050603
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit

-------- Original Message --------
Subject: I-D ACTION:draft-nordmark-shim6-esd-00.txt
Date: Tue, 28 Feb 2006 15:50:02 -0500
From: [email-masked]
Reply-To: [email-masked]
To: [email-masked]

A New Internet-Draft is available from the on-line Internet-Drafts 
directories.

	Title		: Extended Shim6 Design for ID/loc split and
                           Traffic Engineering
	Author(s)	: E. Nordmark
	Filename	: draft-nordmark-shim6-esd-00.txt
	Pages		: 25
	Date		: 2006-2-28
	
    The Shim6 protocol provides for locator agility while satisfying the
    'first, do no harm' security requirements.  This document outlines
    some extensions to Shim6 that in addition provides complete
    separation between identifiers and locators, and allows routers to
    rewrite the locators in the shim6 packets as a way to provide traffic
    engineering information to the hosts.

    The document also outlines a simple extension to allow shim6, with a
    CGA upper-layer ID, to operate using IPv4 addresses as locators.

    The purpose of this outline is to stimulate discussions.

A URL for this Internet-Draft is:
http://www.ietf.org/internet-drafts/draft-nordmark-shim6-esd-00.txt

To remove yourself from the I-D Announcement list, send a message to
[email-masked] with the word unsubscribe in the body of 
the message.
You can also visit https://www1.ietf.org/mailman/listinfo/I-D-announce
to change your subscription settings.

Internet-Drafts are also available by anonymous FTP. Login with the username
"anonymous" and a password of your e-mail address. After logging in,
type "cd internet-drafts" and then
	"get draft-nordmark-shim6-esd-00.txt".

A list of Internet-Drafts directories can be found in
http://www.ietf.org/shadow.html
or ftp://ftp.ietf.org/ietf/1shadow-sites.txt

Internet-Drafts can also be obtained by e-mail.

Send a message to:
	[email-masked].
In the body type:
	"FILE /internet-drafts/draft-nordmark-shim6-esd-00.txt".
	
NOTE:	The mail server at ietf.org can return the document in
	MIME-encoded form by using the "mpack" utility.  To use this
	feature, insert the command "ENCODING mime" before the "FILE"
	command.  To decode the response(s), you will need "munpack" or
	a MIME-compliant mail reader.  Different MIME-compliant mail readers
	exhibit different behavior, especially when dealing with
	"multipart" MIME messages (i.e. documents which have been split
	up into multiple messages), so check your local documentation on
	how to manipulate these messages.

Below is the data which will enable a MIME compliant mail reader
implementation to automatically retrieve the ASCII version of the
Internet-Draft.

--------------070406060909010001050603
Content-Type: Message/External-body;
 name="draft-nordmark-shim6-esd-00.txt"
Content-Transfer-Encoding: 7bit
Content-Disposition: inline;
 filename="draft-nordmark-shim6-esd-00.txt"

Content-Type: text/plain
Content-ID: <[email-masked]>

--------------070406060909010001050603
Content-Type: text/plain;
 name="file:///tmp/nsmail.asc"
Content-Transfer-Encoding: 7bit
Content-Disposition: inline;
 filename="file:///tmp/nsmail.asc"

_______________________________________________
I-D-Announce mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/i-d-announce

--------------070406060909010001050603--

Delivery-date: Wed, 01 Mar 2006 15:46:53 +0000
Cc: Erik Nordmark <[email-masked]>, shim6-wg <[email-masked]>
From: Iljitsch van Beijnum <[email-masked]>
Subject: Re: TE & SHIM6 (was Re: comments on draft-ietf-shim6-proto-03
Date: Wed, 1 Mar 2006 16:46:38 +0100
To: Iljitsch van Beijnum <[email-masked]>

On 1-mrt-2006, at 16:12, Iljitsch van Beijnum wrote:

> But an in-band hop-by-hop TE mechanism would allow exactly this.  
> The way it would work is that routers are configured to provide  
> feedback for packets with a shim header, if necessary. This  
> feedback would be in the form of entries that go into the address  
> selection policy table.

In theory, routers could send this feedback for all packets.  
Obviously many hosts aren't going to be multihomed but they could be  
talking to multihomed correspondents so they may have to make TE  
decisions anyway. But limiting this to proto=shim6 packets is  
friendlier on the router's rate limiters and less wasteful. It would  
be even better if we could do this for packets that we know are part  
of a shim-enabled association. Would it be good enough to simply do  
this for all shim packets except I1 (which could be trying to  
initiate shim6 towards a non-shim6 enabled correspondent) or do we  
really want to have a bit in there that tells the router we are open  
to receiving TE info or not?

Delivery-date: Wed, 01 Mar 2006 15:37:57 +0000
Cc: shim6-wg <[email-masked]>
From: Iljitsch van Beijnum <[email-masked]>
Subject: Re: shim6 @ NANOG   (forwarded note from John Payne) (fwd)
Date: Wed, 1 Mar 2006 16:37:31 +0100
To: Igor Gashinsky <[email-masked]>

Just a few quick points...

On 1-mrt-2006, at 9:10, Igor Gashinsky wrote:

> :: > Right. So there is precedent for storing state for 30000  
> instances of
> :: > "something". Servers are getting a lot faster and memory is  
> getting
> :: > cheaper so adding a modest amount of extra state for longer lived
> :: > associations shouldn't be problematic.

> :: > (Visit a run of the mill content provider and see how many 100  
> byte
> :: > GIFs they send you over HTTP connections that have 700 - 1200  
> byte
> :: > overhead and of course all have high performance extensions  
> turned on
> :: > for extra bandwidth wastage.)

> Wow.. I realize I'm coming a little late to this party, but have  
> you asked
> anybody who works for a large-ish content provider about their views
> on what you are suggesting?

Not in this particular area, so far.

Note that shim6 is still a moving target. One way to optimize things  
for hosting providers would be to do an incomplete shim6 negotiation  
or remove the state on the server side as soon as it's created. As  
long as the client knows all the server's addresses and any failures  
are visible at the client (which would be the case for normal TCP  
traffic) the client can renegotiate the state when there is a failure  
that needs to be repaired. This way, shim6 doesn't have much impact  
on servers.

> Case and point: we all know that http/1.1 + keep-alive is
> more "efficient" from a bw perspective, yet there are some large  
> web shops
> which force all transactions into http/1.0 +no-keep-alive in order  
> to use
> "accept-filter" and therefore save on application state.

That's unfortunate, because reusing sessions makes the web experience  
much faster for people who sit far away from the server.

> sessions are fairly "few" per
> server) are very short-lived http (think about 15 packets in each
> direction including the setup/teardown). Since, shim6 (as designed  
> right
> now) does not initiate from the first packet(s), it might not take  
> effect
> for these short-lived sessions, and therefore will not help in case of
> failure, so in effect, *does not work* at all for fast http  
> transactions

The idea is that if you have a single short lived session, the chance  
that something bad happens during that second the session is around,  
is so small that it's not worth it to negotiate shim6. But if you  
have many short sessions, they add up and at some point shim6 kicks  
in. The current suggestion is to do this after 50 packets between the  
hosts involved, regardless whether this is 16 sessions or 1. But this  
is something that can and should be tuned by operators.

> For example, in the v4 world, the (multi-homed) end-user maybe
> visible via both a *paid* Transit path (say UUNET), and a *free*  
> peering
> link (say Cogent), and I would wager that most content providers would
> choose the free link

This is not what I think of when I say "multihoming". I don't see an  
easy way to do this with shim6 (there may be a non-easy way, though)  
so maybe hosting companies that are big enough to do peering will  
need to continue to do this as in IPv4.

> So,
> whatever methodology we use to do the initial set-up has to account  
> for
> that, and be able to get a connection that is actually starting to do
> something in under 2 seconds,

If anything, 2 seconds is on the long side.

> Just my $0.02

They are appreciated, thanks. (IETF budgets the way they are...)

Delivery-date: Wed, 01 Mar 2006 15:13:25 +0000
Cc: shim6-wg <[email-masked]>
From: Iljitsch van Beijnum <[email-masked]>
Subject: Re: TE & SHIM6 (was Re: comments on draft-ietf-shim6-proto-03
Date: Wed, 1 Mar 2006 16:12:42 +0100
To: Erik Nordmark <[email-masked]>

[This message will go into both source address rewriting and traffic  
engineering.]

On 27-feb-2006, at 20:48, Erik Nordmark wrote:

>> There are two problems with allowing routers to rewrite source  
>> addresses:
>> 1. The routers must know which packets are "legacy" and can't have  
>> their source address changed vs which packets are controlled by  
>> shim6 or another mechanism that can handle rewritten source  
>> addresses.
>> 2. In current shim6, only previously negotiated source addresses  
>> may be used, which means the shim6-enabled hosts in a site and the  
>> rewriting routers must coordinate their efforts so correspondent  
>> hosts don't see unexpected source addresses.

> FWIW draft-nordmark-shim6-esd-00.txt is on the way to the I-D  
> directory, and it has some ideas for how to address this.

Right. Lots of good points in there, but unfortunately, I disagree  
with the mechanisms proposed. I really hope I'll never have to run  
DHCPv6 to configure my hosts, it's a big, fat, unelegant protocol.

>> The first issue is readily solvable by simply having shim6 hosts  
>> put a magic value in the upper 64 bits of the source address that  
>> indicates "rewriting permitted".

> Or next hdr = IPPROTO_SHIM6.

I don't find this very suitable. If we're going to send many shimmed  
packets, it's more important than ever that we omit the shim header  
whenever possible. Apart from that, using the source address to  
signal that the source address may be changed is much cleaner. It  
also has the advantage that we can now borrow some bits  to make the  
process easier. What we can do is have shim6 capable hosts emit a  
"source rewriting information request". That would be a packet  
addressed to a shim6 correspondent that has the magic prefix in the  
source address that triggers source address rewriting, and an  
additional bit combination that tells the router to send back a list  
of prefixes it will use to rewrite. The host can then make sure that  
the correspondent knows to expect packets with these source addresses.

If this is an ordered list, the host can then use bits in the data  
packets with the rewrite prefix in the source address to tell the  
router which addresses it may insert. (Not sure what would happen  
though if the router wants to rewrite into Y but the host only allows  
X and Z.)

I've been thinking about something similar for traffic engineering  
ever since my message yesterday where I mentioned A6 records. The  
problem is that it's far from inconceivable that at some point, a  
disconnect forms between the info in the DNS and the actual state of  
the network. The way I see it, we have four ways to convey TE related  
info:

1. out of band end-to-end: this would be stuff in the DNS
2. out of band hop-by-hop: BGP is like this
3. in-band end-to-end: measured timing and packet loss information
4. in-band hop-by-hop: feedback from routers

The problem is that 2. needs aggregation to scale. 3. and 4. need to  
have contact with the correspondent already, so it's useless in some  
cases, like in the case where we want one or more backup addresses  
that are only tried if the primary addresses don't work. The only way  
to convey this is with 1. We can either reuse SRV records for  
individual services for this, which has the advantage that it's  
already available today, but the disadvantage that this mechanism  
isn't really used and it needs to be supported on an application-by- 
application basis. Alternatively, we can do some magic in the  
resolver library to make this happen.

But this doesn't really make it possible to react to traffic  
engineering events in anything close to real time, if at all (DNS may  
not be accessible by people who need to do TE.) The thing is, BGP  
isn't all that great for this either: with current multihoming, you  
can't engineer traffic such that link 1 gets the first 10 megabits,  
then everything between 10 and 15 goes to link 2 and if there's more  
than 15 Mbit it's balanced over the two links in a 2:1 ratio.  
(Believe me, this doesn't stop people from asking.)

But an in-band hop-by-hop TE mechanism would allow exactly this. The  
way it would work is that routers are configured to provide feedback  
for packets with a shim header, if necessary. This feedback would be  
in the form of entries that go into the address selection policy  
table. The site egress router would probably want to inform hosts  
about which source addresses go well with certain destination  
prefixes. All routers between the source and destination (including  
the site exit router) would signal back "this prefix (which would be  
the prefix that the destination for the packets falls into)  
preference value XXX". To avoid trouble, the preference value  
shouldn't be allowed to completely override locally configured info.

Ignoring non-shim6 traffic for a moment, this would allow any router  
in the path to push back traffic when the conditions warrant it. A  
router could be configured to start lowering the preference value  
when traffic hits a certain threshold and shim6 traffic would  
automatically be rerouted if possible. Obviously there's still the  
potential for conflicting preferences.

A less fortunate side effect could be that a lot of regular traffic  
would be shimmed when the initially chosed destination address isn't  
optimal, which is only discovered when shim state is created after  
the session reaches a certain number of packets exchanged. So this  
would work even better if shim packets are exchanged before the  
session starts, like you describe in your draft. Interestingly,  
suppressing the shim header makes shimming less problematic but with  
the shim header suppressed the traffic engineering doesn't work  
halfway through a long-lived exchange. This can be fixed by  
periodically sending a packet with a shim header, though. Shimming  
for TE reasons could also be problematic when one side garbage  
collects shim state too aggressively.

If we go down this road it may be useful to have one or more bits in  
the shim context tag to communicate with routers, so we'd probably  
want to make the context tag a bit smaller than 47 bits.

Last but not least: it's probably useful to use SRV records (if we're  
going to use those anyway) to tell hosts that:

- they shouldn't initiate shim6 (because the other end wants to  
control when this happens or shim6 isn't supported)
- they should defer shim6 negotiation as per local policy
- they should do shim6 negotiation before starting any sessions

The latter would probably be desireable for sites that want to  
optimize for TE or want to balance incoming sessions over different  
hosts.

Delivery-date: Wed, 01 Mar 2006 11:38:33 +0000
Cc: shim6-wg <[email-masked]>
From: marcelo bagnulo braun <[email-masked]>
Subject: Re: about R1bis
Date: Wed, 1 Mar 2006 12:38:18 +0100
To: Iljitsch van Beijnum <[email-masked]>

El 24/02/2006, a las 12:30, Iljitsch van Beijnum escribi=F3:

> On 8-nov-2005, at 19:20, marcelo bagnulo braun wrote:
>
>> One of the issues that are still open is how to detect context loss=20=

>> and recover it.
>
> [...]
>
>> At this point, this would work as follows:
>> - Regular IPv6 communication is established between A and B
>> - Shim context is set up
>> - Peer A losses the context
>> - a failure occurs
>
> I this case, A doesn't have any shim state so it can't do anything=20
> except time out. B on the other hand will notice a lack of incoming=20
> packets (unless the failure is unidirectional B -> A, that would be=20
> bad here) so after a while a reachability exploration will be started.
>
> It's been a while, so I'm not entirely sure if the way this is written=20=

> down now implies that a reachability exploration automatically gives A=20=

> all of B's addresses so that it's possible for control packets to=20
> start flowing again from A to B, but if it isn't, obviously we can add=20=

> this.
>
> At this point A will need to receive some context and security info=20
> from B and we're back in business. (If B uses a different address B'=20=

> towards A then B has to prove that B' is the same host as B, but=20
> that's what we have HBA for.)
>
>

I think this what is done in the current draft using R1bis and I2bis.

regards, marcelo

Delivery-date: Wed, 01 Mar 2006 08:11:18 +0000
Date: Wed, 1 Mar 2006 03:10:14 -0500 (EST)
From: Igor Gashinsky <[email-masked]>
To: shim6-wg <[email-masked]>
Subject: Re: shim6 @ NANOG   (forwarded note from John Payne) (fwd)

:: On Fri, 24 Feb 2006 Iljitsch van Beijnum writes:
:: > >   * Keeping shim6 state on the end host doesn't scale for content
:: > >     providers.  A single server may have 30,000 concurrent TCP
:: > > sessions
:: >
:: > Right. So there is precedent for storing state for 30000 instances of
:: > "something". Servers are getting a lot faster and memory is getting
:: > cheaper so adding a modest amount of extra state for longer lived
:: > associations shouldn't be problematic.
:: >
:: > (Visit a run of the mill content provider and see how many 100 byte
:: > GIFs they send you over HTTP connections that have 700 - 1200 byte
:: > overhead and of course all have high performance extensions turned on
:: > for extra bandwidth wastage.)

Wow.. I realize I'm coming a little late to this party, but have you asked 
anybody who works for a large-ish content provider about their views 
on what you are suggesting? (disclaimer: I do work for a fairly large 
content provider, but my views in this email are my own, and do not 
necessarily officially represent my employer's views in this email)

First of all, yes, content providers at times do waste quite a bit of 
bandwidth with (what sometimes may appear as unnecesery (and sometimes 
really is)) overhead. However, very often that is done not because people 
don't understand what they are doing, but in an effort to reduce overhead 
somewhere else. Case and point: we all know that http/1.1 + keep-alive is 
more "efficient" from a bw perspective, yet there are some large web shops 
which force all transactions into http/1.0 +no-keep-alive in order to use 
"accept-filter" and therefore save on application state. That state may be 
deemed as more "costly" to content providers then bandwidth. Now, since 
they are obviously going to great pains to save on this state, what makes 
you think that adding shim6 state into the picture "shouldn't be 
problematic."?

Also, while we are on it, let me offer you a view on some issues with 
shim6 from a content provider standpoint (some may be a duplication of 
what Jason has wrote previously, but I want to document it for sake of 
completeness):

1) Most connections to content providers (with the exceptions of 
long-lived streaming sessions, but those sessions are fairly "few" per 
server) are very short-lived http (think about 15 packets in each 
direction including the setup/teardown). Since, shim6 (as designed right 
now) does not initiate from the first packet(s), it might not take effect 
for these short-lived sessions, and therefore will not help in case of 
failure, so in effect, *does not work* at all for fast http transactions 

1) In order to "fix" #1, shim6 has the potential to put a sizable (over 
10%) state penalties on our servers (to service end-sites w/ shim6), 
something which is arguably the most painful thing for those servers, 
which can translate into millions of dollars of additional hardware, and 
many more millions of dollars per year to power/cool that hardware.

3) While TE has been discussed at length already, but it is something 
which is absolutely required for a content provider to deploy shim6. There 
has been quite a bit of talk about what TE is used for, but it seems that 
few people recognize it as a way of expressing "business/financial 
policies". For example, in the v4 world, the (multi-homed) end-user maybe 
visible via both a *paid* Transit path (say UUNET), and a *free* peering 
link (say Cogent), and I would wager that most content providers would 
choose the free link (even if performance on that link is (not hugely) 
worse). That capability all but disappears in the v6 world if the Client 
ID was sourced from their UUnet ip address (since that's who they chose 
to use for outbound traffic), and the (web) server does not know that 
that locator also corresponds to a Cogent IP (which they can reach for 
free). This change alone would add millions to the bw bills of said 
content providers, and well, reduce the likelyhood of adoption of the
protocol by them. Now, if the shim6 init takes place in the 3way 
handshake process, then the servers "somewhat" know what all possible 
paths to reach that locator are, but then would need some sort of a 
policy server telling them who to talk to on what ip, and that's something 
which will not simply scale for 100K+ machines.

4) As has also been discussed before, the initial connect time has to be 
*very* low. Anything that takes longer then 4-5 seconds the end-users have 
a funny way of clicking "stop" in their browser, deeming that "X is down, 
let me try Y", which is usually not a very acceptable scenario :-) So, 
whatever methodology we use to do the initial set-up has to account for 
that, and be able to get a connection that is actually starting to do 
something in under 2 seconds, along with figuring out which sourceIP and 
destIP pairs actually can talk to each other. 

I hope this gives people some visibility as to what some content providers 
think about shim6, and why deploying it is, well, not something that 
people will scramble (or very possibly chose) to do, unless those are 
addresses. And, yes, everyone understands that it's all about making 
trade-offs, but if you make the wrong trade-offs, and not enough people 
deploy the protocol, it's simply not going to fly, and people will just go 
back to de-aggregating in v6 and let Moore's Law deal with the issue (and 
anyone who thinks that people will prevent paying customers from 
deagregating has not seen how many hoops ISP's will jump through for that 
extra revenue, or how fast customers will jump to other ISP's which will 
allow them to do just that). I don't know if more work on shim6 is the 
answer, or GSE/8+8 is a better alterntive, but it sure looks like what we 
have in shim6 today (and it's current direction) isn't going to cut it.

Just my $0.02 

Thanks,
-igor

Delivery-date: Wed, 01 Mar 2006 01:12:40 +0000
Date: Wed, 01 Mar 2006 12:11:35 +1100
To: [email-masked]
From: Geoff Huston <[email-masked]>
Subject: SHIM6 WG agenda
Cc: Kurt Erik Lindqvist <[email-masked]>

Hi,

Here's what we've assembled for the SHIM6 WG meeting at IETF 65

thanks,

   Geoff & Kurtis

======================

Site Multihoming by IPv6 Intermediation WG (SHIM6)

Monday, March 20, 2006
1740-1840 Afternoon Session III
1850-1950 Afternoon Session IV
===============================

CHAIRS: Geoff Huston <[email-masked]>
        Kurtis Lindqvist <[email-masked]>

AGENDA

1. Agenda Bashing, Victimization of scribe
                                    [5 minutes]

    The agenda is broken into 2 sections. Agenda
    items 2 - 6 address topics related to the 
    current set of drafts and necessary steps
    to progress these documents through the WG.
    Agenda items 7 - 9 are a followup to the
    current consideration of (loosely phrased)
    "TE" considerations for possible extensions to 
    Shim6 framework, and the related aspect of
    re-chartering this WG to study this topic.

2. Review of Document Status and WG Roadmap
   Chairs

      The question to the working group is
      whether to adopt
      draft-huitema-shim6-ingress-filtering-00
      as a WG document

                                    [5 minutes]

3. Review of Shim6 Protocol Specification
   draft-ietf-shim6-proto-03.txt
   Erik Nordmark
   
     The question to the working group following
     a review of the document updates is whether
     the WG is ready for a WG Last Call on this
     document.
                                    [15 minutes]
                                    
4. Review of Shim6 Failure Detection Specification
   draft-ietf-shim6-failure-detection-03.txt
   Jari Arkko
   Illitsch vav Beijnum

     The question to the working group following
     a review of the document updates is whether
     the WG is ready for a WG Last Call on this
     document.
                                    [10 minutes]
       
5. Privacy Analysis for the SHIM6 protocol
   draft-bagnulo-shim6-privacy-00
   Marcelo Bagnulo
   
     Do the privacy issues identified in the
     draft need to be addressed? i.e. do we 
     need to define some extensions to address 
     the privacy concerns identified in the draft?
                                    [15 minutes]

6. Default Address Selection in RFC3484
   draft-arifumi-ipv6-policy-dist-00.txt
   draft-fujisaki-dhc-addr-select-opt-01.txt
   Arifumi Matsumoto

     This draft raises some issues about default
     address selection of RFC 3484 and proposes some 
     solutions towards them. (This is an informational
     briefing to the WG.)
                                    [10 minutes]

7. Report on IAB BOFs on multi-homing
   Dave Meyer
                                    [10 minutes]
                                    
8. Extended Shim6 Design for ID/loc split and 
   Traffic Engineering
   draft-nordmark-shim6-esd-00a.txt
   Erik Nordmark
                                    [35 minutes]
                                    
9. WG Discussion on Re-Chartering
   Chairs
   
                                    [15 minutes]

Recommended Reading:
  a) active wg documents to be discussed at the meeting
      draft-ietf-shim6-proto-03
      draft-ietf-shim6-failure-detection-03
  b) other documents whose status will be considered
      draft-huitema-shim6-ingress-filtering-00.txt
      draft-bagnulo-shim6-privacy-00
      draft-arifumi-ipv6-policy-dist-00.txt
      draft-fujisaki-dhc-addr-select-opt-01.txt
      draft-nordmark-shim6-esd-00a.txt
  c) other shim6 documents that may be referred to
      draft-ietf-shim6-arch-00.txt
      draft-iietf-shim6-applicability-00.txt
      draft-ietf-shim6-applicability-00.txt