Subject: RE: Last call - RSVP problems

From: Jonathan Lang <[email-masked]>
Date: Fri, 25 May 2001 14:13:40 -0700

Jennifer,
  Please see inline comments.

Thanks,
Jonathan

> -----Original Message-----
> From: Jennifer Yates [mailto:[email-masked]]
> Sent: Friday, May 25, 2001 11:06 AM
> To: [email-masked]; [email-masked]
> Subject: Last call - RSVP problems
> 
> 
> A number of potentially major protocol problems have been identified
> within the OIF with the current signaling protocols (focusing 
> on RSVP).
> These problems arise in transport networks because the data plane is
> separate from the signaling plane.
> 
> I don't believe these issues have been addressed on the mailing list -
> if they have, please explain where.
> 
> ****
> Connection (LSP) teardown:
> There is a problem with connection deletion in the current RSVP draft.
> If a PathTear is sent in the forward direction (with say a
> unidirectional connection), the first node (cross-connect) will delete
> the connection and then forward the PathTear to the next cross-connect
> and so on.
> 
> Since loss of light will propagate faster than the PathTears, all
> downstream cross-connects will detect the loss of light and 
> alarm on it, possibly triggering restoration.  This is clearly 
> unacceptable behavior.

If LMP fault management is used, the "multiple alarming" behavior will be
avoided.

> 
> The right solution is that the deletion message in the forward direction
> should inform all of the nodes that the connection is being deleted, and
> the actual deletion should propagate in the *reverse* direction.
> Therefore, additional protocol extensions are needed to get the correct
> behavior on deletion.
> 
> This issue has been already been discussed within OIF documents e.g.,
> OIF2001.200 (RSVP Extensions in support of deletion confirmation and
> maintenance during failure), and in the ITU e.g., "Signal flows during
> exception cases". This issue does not appear to be reflected in the
> GMPLS documents presented for last call.

Please see the LMP document for a description of the fault management
procedure.

> 
> ****
> Signaling channel failure:
> If any element along the RSVP signaling path of an LSP fails, the RSVP
> soft state timer in nodes downstream of the failure will timeout and the
> nodes will delete the LSP, even if the forwarding plane is still
> working! When GMPLS is used to control a transport network, this
> scenario will occur, since the reliability of control planes is likely
> quite a bit lower than the cross-connects themselves.  In transport
> networks, the data plane reliability expectations are very 
> high, and any factor that increases the already steep challenge of meeting
those
> expectations is not acceptable.
> 
> One of the cardinal principles of transport network architecture is that
> control plane failure must *not* cause the data plane to fail.  This
> point has been raised and acknowledged at the OIF, but so far does not
> seem to have been addressed in any of the GMPLS drafts.

  According to rfc2205, the state's lifetime L must satisfy: L>=
(K+0.5)*1.5*R where K is an integer and R is the refresh timer.  My
suggestion to the OIF back in January in Tampa was to use L (or effectively
K*R) to get the operation you are looking for.  I believe this will satisfy
your requirement and does not require any changes to the signaling protocol.

> Jen

---

From: Jonathan Lang <[email-masked]>
Date: Fri, 25 May 2001 14:29:51 -0700

Fong,

<snip>
> Same as (2), any proposals that remove the refresh mechanism 
> is going to be difficult to prove that all cases are covered.
> 
> Instead, we (will) recommend the following in OIF UNI document:
>    
>    Use RSVP Hello to detect control channel failure.
Why wouldn't you use the LMP Hello to detect control channel failure?  This
is exactly what it is designed for.  From draft-ietf-mpls-lsp-tunnel-08.txt,

  "This (RSVP Hello) mechanism is intended to be used when notification of
link layer
   failures is not available and unnumbered links are not used, or when
   the failure detection mechanisms provided by the link layer are not
   sufficient for timely node failure detection."
 
>    If a control channel failure is detected, LSPs states 
>    are maintained as if a node continues to receive 
>    RSVP refresh message from the failed control channel.
>    The recommended Hello timer will be in second range,
>    instead of ms range specified in RSVP-TE draft.  
> 
>    If a control channel failed permanently, manual intervention 
>    may be required. This is to be studied.
> 
> p.s The text is currently being drafted as we type.

---

From: Dimitrios Pendarakis <[email-masked]>
Date: Fri, 25 May 2001 18:16:12 -0400

Hi Jonathan,

One point to consider is that the signaling control 
channel might be realized over a different link layer 
than LMP. For example, LMP might be running in-fiber
over SONET/SDH overhead bytes, while RSVP is running 
over an out-of-band network such a shared Ethernet.
RSVP Hello allows independent detection of signaling 
channel failure, so it's a useful option to keep.

Thanks,
Dimitris

Dimitris Pendarakis
Tellium, Inc.

> From: Jonathan Lang [mailto:[email-masked]]
> Sent: Friday, May 25, 2001 5:30 PM
> To: 'Fong Liaw'; 'suresh Katukam'; [email-masked]
> Cc: 'Jennifer Yates'; [email-masked]; [email-masked]
> Subject: RE: Last call - RSVP problems
> 
> 
> Fong,
> 
> 
> <snip>

---

From: Fong Liaw <[email-masked]>
Date: Fri, 25 May 2001 15:25:31 -0700

See inline

>  From: David Charlap [mailto:[email-masked]]
>  Sent: Friday, May 25, 2001 2:49 PM
>  To: [email-masked]
>  Subject: Re: Last call - RSVP problems
>  
>  
>  Fong Liaw wrote:
>  > 
>  > "Infinite timer" is actually very problematice with RSVP.
>  > We have specifically recommend NOT to use this, after attempting
>  > to do it.
>  
>  Eliminating refreshes over a link can cause a lot of 
>  problems in RSVP,
>  unless two things are done:
>  
>  1: Hellos must be active on the link.  Without refreshes, the only
>     way to detect failures is either Hello or loss of carrier on the
>     fiber.  And loss of carrier won't happen in the case of a software
>     failure.  Without Hellos, the only way to detect a 
>  software failure
>     is missed refresh messages.

This is being recommended, plus using LMP hello etc, per Jonathan's
comment.

>  
>  2: The reliable message delivery (MESSAGE_ID) mechanism of the RSVP
>     Refresh Reduction RFC (Section 4 of RFC 2961) is in place on the
>     link.  Without this, you can't be sure that a message sent was
>     actually received.  Straight RSVP doesn't have a problem 
>  with this,
>     because a missed packet will be repeated via the refresh 
>  mechanism.
>     But if the refresh interval is very large, there must be 
>  some other
>     means to make sure a message is delivered.

This is also been recommended. The MESSAGE_ID ACK draft gives up
the retransmission after a preconfigured retry. We know we would
have very high probably that messages will be acked, but can we be
100% sure about the states in the case we receive no ackwnologyment 
to these message and we have extended the timer to infinite ?

>  
>  When both of these are in place, then there shouldn't be a problem
>  increasing the refresh interval to a very large amount, or infinite.
>  
>  If you don't tear down state when a failure is detected, what's the
>  point of having a failure detection mechanism at all?

This is to address the requirement from transport networks
where a failure of control channel can not affect the
data plane state.  We wouldn't be having this discussion
if that is not a requirement :-)

-Fong

>  -- David

---

From: Jonathan Lang <[email-masked]>
Date: Fri, 25 May 2001 15:55:31 -0700

Dimitris,
  The RSVP Hello Extension is intended to detect node failures and not
necessarily link (control channel) failures.  In particular, the tunnel
draft says, "It should be noted that node failure detection is not the same
as a link failure detection mechanism, particularly in the case of multiple
parallel unnumbered links."  What happens if you want redundant control
channels for control-channel resiliency?  I believe you would still want a
mechanism to detect control channel (link-layer) failures.

  In LMP, you can have multiple active control channels between a pair of
nodes.  You can configure control channels over shared Ethernet and over
SONET/SDH overhead, etc.  You send keep-alive Hellos over each control
channel.  If you so desire, you can send all signaling messages over the
Ethernet control channels.  You can also choose to send all remaining LMP
messages over the SONET/SDH-based control channels.  Keeping RSVP Hellos as
"optional" would be fine.  Making them mandatory doesn't seem like the right
solution.

Thanks,
Jonathan

P.S.  Should we move this discussion on the OIF list??

> From: Dimitrios Pendarakis [mailto:[email-masked]]
> Sent: Friday, May 25, 2001 3:16 PM
> To: 'Jonathan Lang'; 'Fong Liaw'; 'suresh Katukam'; [email-masked]

---

From: "Irfan Lateef" <[email-masked]>
Date: Fri, 25 May 2001 19:34:26 -0400

Dimitrios,

You have brought up a good point.
Is there any ongoing effort to standardize
the signalling mechanism on the optical side.
Namely in-band SONET-DCC OH bytes vs out-of-band
Ehternet vs in-band IP-over-SONET data plane etc.

Irfan Lateef

>From: Dimitrios Pendarakis <[email-masked]>
>To: "'Jonathan Lang'" <[email-masked]>, "'Fong Liaw'" 
><[email-masked]>,   "'suresh Katukam'" <[email-masked]>, 
>[email-masked]
>CC: "'Jennifer Yates'" <[email-masked]>, [email-masked],   
>[email-masked]
>Date: Fri, 25 May 2001 18:16:12 -0400
>
>Hi Jonathan,
>

_________________________________________________________________
Get your FREE download of MSN Explorer at http://explorer.msn.com

---

From: Dimitrios Pendarakis <[email-masked]>
Date: Fri, 25 May 2001 20:42:29 -0400

Hi Jonathan,

What needs to be detected here is the inability to send and 
receive signaling messages to and from your RSVP peer. In this 
respect, using RSVP Hello is a more accurate mechanism for 
detecting failures since it is part of the RSVP protocol and 
independent of the link layer used to realize the control 
channel. As you point out, if you prefer redundant control 
channels for resiliency, and want to rely on LMP only, you have
to correlate all LMP detected failures in order to declare a 
control channel failure. With RSVP Hello you would rely on a 
single set of event(s) to detect the failure.
To go even further, what if all your control channels are working
fine, but your RSVP daemon has crashed, for whatever reason.
LMP hellos will keep going across but RSVP messages will not...

All this points to the (certainly non-trivial) problem of failure 
detection and correlation at different layers. I am not trying to
say that RSVP Hello is the perfect or only mechanism we should rely 
on, but it certainly has a place in addressing the problem.

Thanks,
Dimitris

PS. Hopefully we are not too far apart so we might not have
to keep the discussion going for too long :-)
In any case I believe it has some relevance here as well.

> Sent: Friday, May 25, 2001 6:56 PM
> To: 'Dimitrios Pendarakis'; Jonathan Lang; 'Fong Liaw'; 'suresh
> Katukam'; [email-masked]

---

From: Jonathan Lang <[email-masked]>
Date: Tue, 29 May 2001 00:34:29 -0700

Dimitrios,
  If you're using the RSVP Hello to detect the failure of the RSVP module
only, then this is fine.  If you're using the RSVP Hello to detect (or
rather, infer) control channel failures, I have a problem.  For starters, I
don't think it works for parallel (redundant) control channels; if control
channel redundancy is *not* important for SPs, then it's not as much of an
issue, however, I think you're overloading the semantics of the RSVP Hello.

  See additional comments inline.

Thanks,
Jonathan

> control channel failure.
This is not correct.  LMP allows you to detect individual control channel
failures, even when there are multiple (redundant) control channels.  The
RSVP Hello cannot do this.

> With RSVP Hello you would rely on a 
To be clear, the failure of the RSVP module.

> LMP hellos will keep going across but RSVP messages will not...
This is where the RSVP Hello has its place.

> on, but it certainly has a place in addressing the problem.
agreed.  see above comment.

---

From: Jonathan Lang <[email-masked]>
Date: Wed, 30 May 2001 11:08:09 -0700

Dimitrios,
  The luxury of being in meetings all morning is that I had the chance to
read your email as well as the responses from David, Lyndon, and Eric before
I respond.  Since I agree with their response, I will try to be brief.
Using RSVP Hellos to detect RSVP module failures is in my mind the correct
usage for them.  Control channel failures should be detected by mechanisms
designed to detect them (e.g., LMP Hellos).   One scenario that carriers are
interested in supporting is the notion of having parallel redundant control
channels for signaling purposes.  We will need to support other mechanisms
for detecting control channel failures in this case.

Thanks,
Jonathan

> Sent: Wednesday, May 30, 2001 7:24 AM
> To: 'Jonathan Lang'; 'Fong Liaw'; 'suresh Katukam'
> Jonathan,
> Could you please clarify why you think RSVP Hello doesn't work for 
> parallel (redundant) control channels? Control channel redundancy 
> is obviously important; however I don't think it relates to the 
> usefulness of the RSVP hello mechanism, at least in the context of 
> this discussion.
> 
> The whole issue is one of layering: LMP hello allows you to detect 
> individual link (or control channel) failures; RSVP Hello allows
> you to detect that you 've lost contact with your signaling peer
> for whatever reason (failure of all redundant control channels, 
> failure of the RSVP daemon, etc.). Since they operate at different
> levels they are to some extend complementary; once RSVP Hello tells
> you that your peer is not reachable, you could use LMP Hello's to 
> try to pinpoint where the problem occurred. 
> 
> For clarification, the "control channel failure" mentioned in
> Fong's mail that started this discussion refers to a failure of the
> signaling channel, i.e., inability to communicate with your signaling
> peer, not failure of specific (individual) control channels. As such, 
> I still believe RSVP Hello is a more accurate detection mechanism.
> 
> Please see some additional comments inline.
> 
> Thanks,
> Dimitris
> 
> 
> Sure, as I pointed above, RSVP Hello is not intended to detect 
> individual link failures. At the risk of sounding repetitive, the
> point here is that these are different mechanisms that serve 
> different purpose. For example, if you have 3 (redundant) IP control 
> channels and rely on LMP Hello only, in order to infer failure of the 
> signaling channel with your peer, you would have to correlate LMP 
> failures in all three channels. 
> 
> Or simultaneous failure of all IP control channels...
> 
> > 
> > > > > > Why wouldn't you use the LMP Hello to detect 
> control channel 
> > > > > > failure?  This

---

From: Dimitrios Pendarakis <[email-masked]>
Date: Thu, 31 May 2001 09:55:14 -0400

David,

I agree with Lyndon's comment. If you read through the rest of
my email, you 'll see that the discussion is not about the 
ability of RSVP Hello to detect link failures:

DP> LMP hello allows you to detect 
DP> individual link (or control channel) failures; RSVP Hello allows
DP> you to detect that you 've lost contact with your signaling peer
DP> for whatever reason (failure of all redundant control channels, 
DP> failure of the RSVP daemon, etc.).

Thanks,
Dimitrios

> From: Lyndon Ong [mailto:[email-masked]]
> Sent: Wednesday, May 30, 2001 12:05 PM
> To: [email-masked]; David Charlap
> Hmm, I'm confused here - I did not think the discussion was 
> about RSVP Hello's function, it's understood to be detecting
> node failure rather than link or control channel failure.
> 
> I thought the question was whether RSVP Hello was necessary if
> you have an alternative way of detecting control channel
> failure, in which case you can tell when there are no control
> channels left standing.  Detection of control channel failure
> could be through LMP Hello or if you have some alternative 
> means, like monitoring control channel error rates.
> 
> Cheers,
> 
> Lyndon
> --
> 
> On Wed, 30 May 2001 10:55:39   David Charlap wrote:
> >Dimitrios Pendarakis wrote:
> >> 
> >> parallel (redundant) control channels?
> >
> >RSVP Hello messages are sent between adjacent IP addresses.  If there
> >are multiple unnumbered links between two routers, only one set of
> >Hellos are generated for all the links.  If the multiple 
> links are all
> >numbered, then Hello messages are sent for each of the links.
> >
> >As a result of this, RSVP Hello can only detect link failure when all
> >links between two routers are numbered.  If there are 
> unnumbered links
> >between two routers, Hellos will not be interrupted until all of the
> >links fail - meaning it will only detect node failure.
> >
> >-- David
> >
> 
> 
> Join 18 million Eudora users by signing up for a free Eudora 
> Web-Mail account at http://www.eudoramail.com
>

---

From: [email-masked]
Date: Thu, 31 May 2001 15:45:20 +0100

Some observations on this discussion:

The user-plane and the control-plane need individual OAM mechanisms that are
appropriate to the layer networking scenario considered.

At one extreme we have a purely cnls mode.....so here the control-plane is
simply composed of a routing facet (ie no signalling) and, moreover, both
the control-plane and user-plane share a common transport component, ie the
IP packet.  Here we need OAM only for the user-plane and the routing
protocol.

At the other extreme we have a co mode with separation of
control/user-planes onto their own network transport constructs (ie the
so-called 'control/bearer separation'.  This is a bit more complex because
the user-plane trails require their own OAM (usually embedded within the
path overhead of the trail) and the control-plane needs OAM for:
-	its own 'user-plane'...since this is logically disjoint network from
the 'user-plane' being offered to (external) customers;
-	the signalling protocol;
-	the routing protocol;
Now, since the routing and/or functionally is no longer common across these
facets it should be obvious that each will have its own specific failure
modes that will need addressing by OAM designed for that facet (clearly we
can extend this to any other protocols that are used in either control or
user planes).

In between these extrems there are others;  like the ones created by MPLS
LSP trails which (i) are more cnls-like for LDP created LSPs and (ii) more
co-like for explictly signalled LSPs using RSVP/CR-LDP.

So I am not sure that the discussion I have been viewing is really helpful.
RSVP hellos are maintaning the integrity of RSVP sessions, LMP hellos are
maintaining the integrity of 'LMP sessions'......and, also checking the
'user-plane' of the control-channel (the user-plane of the trails used to
carry external customer traffic are checked by other mechanisms in LMP).

neil

> Sent: 31 May 2001 14:55
> To: [email-masked]'; David Charlap
> Cc: [email-masked]