Subject: [Isis-wg] IS-IS TE Question

From: "ben abarbanel" <[email-masked]>
Date: Thu, 14 Sep 2000 10:52:59 -0400

I was reading the latest draft "draft-ietf-isis-traffic-02.txt" and a question came to mind regarding 
unreserved bandwidth. In section 5.4 SUB-TLV 11: Unreserved bandwidth states "for stability reasons, rapid changes in the values in this sub-TLV should not cause rapid generation of LSPs". 
 
I was wondering what if  2 ingress (head end RSVP) LSR routers are provisioning LSPs (each wanting 100 Mbytes) through the same core convergent LSR interface and each one of the routers at the time they privision their LSP think there is 100 Mbyte of bandwidth free to complete the job. In reality there is a total of 100 MBYTE for the whole interface, the first RSVP reservation succeeds while the second fails. 
 
The second LSP Ingress router would ask TE Source routing for another next bandwidth constraint best path to setup the LSP.  How efficient is that if many LSPs coverge in the core of the network this way and there is no precise way to say what is available at the time RSVP is trigered on?

Also, since the IS-IS and its TE source routing engine take snap shots of the bandwidth resources in the network it has a tunnel view of what other IS-IS(LSR) routers are doing to this information at that moment in time, and are making inaccurate determination of what is reserved and what is free. The IS-IS route convergent time could be much slower than the RSVP signalling time.

The following is my assumption, please clarify if I am wrong, thanks

if a series of LSP requests are hitting a given IS-IS router (TE source routing engine) at the same processing window/cycle, it makes assignments based on a frozen snap shot of the bandwidth in the network. Assignments are made without consideration for their RSVP signalling success/failure and eventually the signalling components would fail their reservation requests causing turbulance with LSP attempts.

Assuming most of what i said is true, do we need another more precise mechanism to solve this?  

Regards,
Ben 

Ben Abarbanel,  Software Engineering,  

Phone:  703 456 2982
FAX:     703 456 2952

IPOptical, Inc.
11480 Sunset Hills Road
Suite #200E
Reston, VA 20190

---

From: Sudheer Dharnikota <[email-masked]>
Date: Thu, 14 Sep 2000 08:05:12 -0700

Bena:

	This is a famous problem with the receiver-oriented
reservation mechanims. Where on the downstream side you do
the CAC and on the upstream side you do the actual reservaion.
I am not sure if we have a straight-forward answer for this.
I would also assume that the geneic consensus is pray god
that it will not happen (which is the generic approach taken
till now) or use different criteria for the second LSP after
it failed - as proposed in
draft-dharanikota-interarea-mpls-te-ext-00.txt

Cheers,

sudheer

> ben abarbanel wrote:
> unreserved bandwidth. In section 5.4 SUB-TLV 11: Unreserved bandwidth
> states "for stability reasons, rapid changes in the values in this
> sub-TLV should not cause rapid generation of LSPs".
> 
> I was wondering what if  2 ingress (head end RSVP) LSR routers are
> provisioning LSPs (each wanting 100 Mbytes) through the same core
> convergent LSR interface and each one of the routers at the time they
> privision their LSP think there is 100 Mbyte of bandwidth free to
> complete the job. In reality there is a total of 100 MBYTE for the
> whole interface, the first RSVP reservation succeeds while the second
> fails.
> 
> The second LSP Ingress router would ask TE Source routing for another
> next bandwidth constraint best path to setup the LSP.  How efficient
> is that if many LSPs coverge in the core of the network this way and
> there is no precise way to say what is available at the time RSVP is
> trigered on?
> 
> Also, since the IS-IS and its TE source routing engine take snap shots
> of the bandwidth resources in the network it has a tunnel view of what
> other IS-IS(LSR) routers are doing to this information at that moment
> in time, and are making inaccurate determination of what is reserved
> and what is free. The IS-IS route convergent time could be much slower
> than the RSVP signalling time.
> 
> if a series of LSP requests are hitting a given IS-IS router (TE
> source routing engine) at the same processing window/cycle, it makes
> assignments based on a frozen snap shot of the bandwidth in the
> network. Assignments are made without consideration for their RSVP
> signalling success/failure and eventually the signalling components
> would fail their reservation requests causing turbulance with LSP
> attempts.
> 
> 

_______________________________________________
Isis-wg mailing list  -  [email-masked]
http://external.juniper.net/mailman/listinfo/isis-wg

---

From: "Darek Skalecki" <[email-masked]>
Date: Thu, 14 Sep 2000 12:25:29 -0400

ben abarbanel wrote:

> I was reading the latest draft "draft-ietf-isis-traffic-02.txt" and a
> question came to mind regardingunreserved bandwidth. In section 5.4
> SUB-TLV 11: Unreserved bandwidth states "for stability reasons, rapid
> changes in the values in this sub-TLV should not cause rapid
> generation of LSPs".

> than the RSVP signalling time.

For CR-LDP a feedback mechanism was proposed to learn at signaling
intervals/speed whether or not there was b/w at interfaces. A snapshot
of that b/w is fed into TE database based on which source routes are
computed. With feedback, if there is insufficient b/w at an interface to
accomodate an LSP, a notification message with a feedback TLV is carried
towards the source. The feedback information is then input into
TE database at source and a new route computed. That route should now
not contain the interface where there was insufficient b/w. The feedback
mechanism is described in:
http://www.ietf.org/internet-drafts/draft-ietf-mpls-te-feed-01.txt

> Assuming most of what i said is true, do we need another more precise
> mechanism to solve this? Regards,

The feedback provides you ability to try, fail, learn and try again.
From our experience, a proprietary signaling protocol with feedback
worked just fine, i.e. convergence was fast (one or two tries/failures
and a satisfactory route was found).

> Ben  Ben Abarbanel,  Software Engineering, Phone:  703 456 2982
> FAX:     703 456 2952 IPOptical, Inc.
> 11480 Sunset Hills Road
> Suite #200E
> Reston, VA 20190

--
Darek Skalecki
Nortel
(613) 765-2252

   MPLS Working Group                                  Peter Ashwood-Smith
   Internet Draft                                      Bilel Jamoussi
   Expiration Date: December 2000                      Don Fedyk
                                                       Darek Skalecki
                                                       Nortel Networks

                                                       July 2000

         Improving Topology Data Base Accuracy with LSP Feedback

                     draft-ietf-mpls-te-feed-01.txt

Status of this Memo

   This document is an Internet-Draft and is in full conformance with
   all provisions of Section 10 of RFC2026.

   Internet-Drafts are working documents of the Internet Engineering
   Task Force (IETF), its areas, and its working groups. Note that
   other groups may also distribute working documents as Internet-
   Drafts.

   Internet-Drafts are draft documents valid for a maximum of six
   months and may be updated, replaced, or obsoleted by other documents
   at any time. It is inappropriate to use Internet-Drafts as reference
   material or to cite them other than as "work in progress."

   The list of current Internet-Drafts can be accessed at
   http://www.ietf.org/ietf/1id-abstracts.txt

   The list of Internet-Draft Shadow Directories can be accessed at
   http://www.ietf.org/shadow.html.

Abstract

   One key component of traffic engineering is a concept known as
   constraint based routing. In constraint based routing a topology
   database is maintained on all participating nodes. This database
   contains a complete list of all the links in the network that
   participate in traffic engineering and for each of these links a set
   of constraints which those links can meet. Bandwidth, for example,
   is one essential constraint. Since the bandwidth available changes
   as new LSPs are established and terminated the topology database
   will develop inconsistencies with respect to the real network. It is
   not possible to increase the flooding rates arbitrarily to keep the
   database discrepancies from growing. We propose a new mechanism
   whereby a source node can learn about the successes or failures of
   its path selections by receiving feedback from the paths it is
   attempting. This fed-back information can be incorporated into
   subsequent route computations, which greatly improves the accuracy

Ashwood-Smith, et. al.                                        [Page 1]
Internet Draft      LSP Feedback with CR-LDP    July, 2000

   of the overall routing solution by significantly reducing the
   database discrepancies.

1. Introduction

   Because the network is a distributed system, it is necessary to have
   a mechanism to advertise information about links to all nodes in the
   network [IS-IS], [OSPF].  A node can then build a topology map of
   the network.  This information is required to be as up-to-date as
   possible for accurate traffic engineered paths.  Information about
   link or node failures must be rapidly propagated through the network
   so that recovery can be initiated. Other information about links
   that may be useful for reasons of quality of service include
   parameters such as available bandwidth, and delay. The information
   in this topology database is often out of date with respect to the
   real network. Available bandwidth is the most critical of these
   attributes and it can drift substantially with respect to reality
   due to the low frequency of link state updates that can be sustained
   in a very large topology. We refer to the deviation in the topology
   database available bandwidth as being optimistic if the database
   shows more available bandwidth than there really is, or pessimistic
   if the topology database shows less bandwidth than there really is.
   This distinction is important because we shall propose an efficient
   algorithm to deal with optimistic databases without resorting to
   shorter flooding intervals.

   One of the major problems for a constraint based routing system is
   dealing with changing constraints. Obviously, since bandwidth is one
   of the essential constraints, dealing with the rapid changes in
   reserved bandwidth poses some interesting challenges. In smaller
   networks, one can resort to higher frequency flooding but this
   obviously does not scale.

   The basic proposal is to add to the signaling protocol the ability
   to piggyback actual link bandwidth availability information at every
   link that the signaling traverses. This is done as part of the
   reverse messaging on success or failure (mapping, release, withdraw
   or notification). What this means is that every time signaling
   messages flow backwards toward a source to tell it of the success,
   failure or termination of a request, that message contains a
   detailed slice of bandwidth availability information for the exact
   path that the message has followed. This slice of reservation
   information, which is very up to date, is received by the source
   node and attached to the source node's topology database prior to
   making any further source route computations. The result is that the
   source node's topology database will tend to stay synchronized with
   the slices of the network through which it is establishing paths.
   This is nothing more than learning from successes and failures and
   represents an intelligent alternative to either waiting for floods
   or introducing non-determinism (guessing) into the source
   algorithms. It is important to note that the fed-back data is never

Ashwood-Smith, et. al.       July  2000        [Page 2]
Internet Draft      LSP Feedback with CR-LDP    July, 2000

   re-flooded. It simply overrides flooded information for the purpose
   of route computation until a superceding flood or fed-back value
   arrives. As such, it is not actually inserted into a topology
   database, most likely it simply is linked to that database as an
   override used only by source route computations.

   Operating a constraint based routing system without such feedback is
   inefficient at best since a source node will continue to give out
   incorrect route over and over again until it gets an IGP update.
   This could be minutes away and as a result the worst case blocking
   time for a new route is the minimum repeatable flooding interval
   (often several minutes in big networks). Alternatives to feedback
   mechanisms involve adding some non-determinism (randomness) to the
   routing algorithm in the hopes that it will stumble onto a path that
   works. These sorts of approaches are seen in ATM dynamic routing
   systems, which do not have these forms of feedback.

   In order to get a good understanding of how the feedback works,
   imagine a network with precisely one path (with sufficient
   unreserved bandwidth) available from the source to the destination.
   Further, imagine that the topology database at the source is
   significantly out of date with respect to the real network in that
   the source topology database sees sufficient bandwidth available on
   many different routes to the destination. We call this being
   optimistic with respect to the network since the source thinks that
   more bandwidth is available than there really is.

   When such an optimistic source selects its first path it will likely
   contain links that do not in reality have sufficient unreserved
   bandwidth. Therefore, the path is only established up to the link
   that does not have sufficient bandwidth. A notification message is
   formatted that contains the actual unreserved bandwidth for this
   blocking link which flows back toward the source, collapsing the
   partially created path as it goes. In addition, at every link that
   this notification traverses, the current unreserved bandwidth
   information for each corresponding link is appended to the vector of
   unreserved bandwidth along the path. In this manner, an accurate
   view of the slice through the network we are traversing is
   constructed. Eventually this message arrives back at the source
   node, where the vector is taken and used to temporarily override the
   topology database for route computations. This node has just learned
   from its mistake and is now slightly less optimistic with respect to
   the real network conditions.

   Path selection can be attempted again but this time the node will
   not make the same mistake it made the previous time. The link in
   question, at which rejection occurred the first time, will not even
   be eligible this time around, so a source route computation is
   guaranteed to produce a different path (or none). The same procedure
   may be repeated as many times as is necessary, each time learning
   from its mistakes, until eventually no paths remain in the source
   topology to the destination, or we actually find a path that works.

Ashwood-Smith, et. al.       July  2000        [Page 3]
Internet Draft      LSP Feedback with CR-LDP    July, 2000

   This tendency to converge either to a solution or determine that
   there is no solution is an important property of a routing system
   (it actually behaves a lot like a depth first search). This property
   is not present with flooding mechanisms alone since the source node
   must randomly hunt, or continually make the same mistakes, or abort
   until the next flood arrives.

   In addition to feeding back bandwidth on failure, we also recommend
   feedback on success. This has important consequences on our ability
   to spread load or to spill over to new links as existing links fill.
   It is true that spilling over to new links does not require feedback
   on success since we could simply wait for a feedback on failure, but
   we can achieve better load spreading earlier.

   Finally, when a path is torn down the release/withdraw messages also
   contain bandwidth information that can be fed back to override the
   source topology database. This is very important during failure
   scenarios where the links we need to use to reroute the path share
   common sub-segments with the failed path. Without the feedback, the
   common sub-segments may not indicate sufficient available bandwidth
   until we get a flood that may mean many seconds without a
   connection. With feedback at least we will be up to date with
   respect to available bandwidth up to the point of failure in the
   path. Also since failure involves many paths tearing down and re-
   establishing this is the time that it is most critical to have an
   accurate view.

   When preemption is being employed it is also extremely important
   that the topology database inconsistencies be small. If not, high
   setup priority LSPs may unnecessarily preempt lower holding priority
   LSPs to obtain bandwidth that, had they had a more up to date view
   of unreserved bandwidth, they would have been able to find
   elsewhere. Since preempted LSPs may in turn preempt other LSPs in a
   domino like effect, the results of such database inconsistencies can
   have wide reaching ripple like impacts. These feedback mechanisms
   help reduce these occurrences significantly.

   There are a number of network conditions where feedback shows its
   value. One can think of a constraint-based network as being in one
   of three conditions. The first is called ramp-up, this is when the
   rate of arriving reservations exceeds the rate of departing
   reservations. The second is called steady-state, this is when the
   rate of arriving reservations is about the same as the rate of
   departing reservations. Finally, the ramp-down condition is that
   which has a greater rate of departing reservations than arriving
   reservations.

   These three network conditions show distinctly different types of
   error in the topology databases. In particular an optimistic view of
   available bandwidth by a source node is characteristic of the ramp-
   up condition of a network. A pessimistic view of available bandwidth
   by a source node is characteristic of the ramp-down condition of a

Ashwood-Smith, et. al.       July  2000        [Page 4]
Internet Draft      LSP Feedback with CR-LDP    July, 2000

   network. If one plots the average error in the topology databases
   with respect to the real network for the three different network
   conditions, one will see the error slowly go positive during ramp
   up, slowly go negative during ramp down, and drift slowly around 0
   for the steady condition. The effect of flooding on this plot is to
   periodically snap the error back to 0 at flooding intervals. The
   effect of the feedback algorithm is to bring an optimistic error
   back to zero without having to wait for the flood interval. On
   average then, the feedback algorithm tends to halve the absolute
   error, keeping it mostly negative or pessimistic. This makes sense
   since a routing system will never give paths to links that it thinks
   do not have resources and as a result its pessimistic view of the
   world stays that way until it gets a flood.  This relieves the IGP
   updates of the most urgent requirement of flooding when bandwidth is
   consumed. Availability of new bandwidth occurs when paths are
   released or new links become available.  New links are accompanied
   by floods. Significant releases of bandwidth can be broadcast at
   relatively low frequencies in the order of several minutes with
   little operational impact.

   Extensive operational experience with this feedback protocol in
   proprietary Nortel Networks (pre-standard CR-LDP) products has shown
   it to work very well for networks up to 1000 nodes with significant
   flooding intervals damped to several minutes. Without this protocol,
   these networks would block setups for up to several minutes. With
   this protocol, the blocking in most cases is reduced to a small
   number of retry attempts which is usually sub-second depending
   mostly on the propagation delays in the network.

   These feedback algorithms have been particularly beneficial in cases
   of failure recovery during which the network is in a sudden
   condition of ramp-up. Since a large number of reservations must be
   remade, it is highly likely that we will exceed the limits of
   certain key links in the network. Without feedback, the rerouting
   must block until a flood arrives telling us of the situation at
   those key links at which time rerouting can continue. With feedback,
   the rerouting simply continues until a feedback indicates that a
   link is full. In addition since reservation-balancing algorithms are
   also often used, feedback allows the balancing algorithms to make
   better distribution decisions based on immediate feedback.

   We have also explored through simulation and implementation a
   variety of mechanisms to deal with the pessimistic error in the
   database. One simple proposal is to use selective forgetting. In
   this algorithm, a reserved bandwidth value slowly drops back to zero
   over a relatively short time interval. The theory being that you
   shift the network back to an optimistic state (by forgetting your
   pessimism) where the feedback algorithm will again correctly
   operate. These algorithms have not shown any great advantage and are
   actually non-optimal when the error is purely optimistic.

Ashwood-Smith, et. al.       July  2000        [Page 5]
Internet Draft      LSP Feedback with CR-LDP    July, 2000

   Other algorithmic permutations we have explored include such
   variations as:

   Feeding-back to all intermediate nodes, information learned from
   control messages upstream of that intermediate node.

   Feeding back in both directions so that both the source and
   destination node's databases stay synchronized.

   Allowing a request to continue to its destination despite there
   being insufficient bandwidth at some intermediate hop. Then,
   rejecting the request with a full bandwidth vector slice all the way
   to the destination instead of just to the point of rejection.

   Our simulations have not show significant benefits relative to the
   simpler algorithm proposed here. However, it is an interesting
   research topic to explore and quantify the different feedback
   algorithms and their impacts on blocking times so we do not want to
   discourage the interested reader from exploring these concepts more
   fully.

2. Adding feedback TLVs to CR-LDP

   Two new TLVs are optionally added to the CR-LDP mapping,
   notification, and withdraw messages. There may be an arbitrary
   number of these TLV in any order or position in the message. It is
   recommended that they be placed such that they can be read and
   applied to override the topology database by scanning the message
   forwards and walking the topology database from the point where the
   last link feedback TLV left off.

   Each TLV consists of the 8 unreserved bandwidth values for each
   holding priority 0 through 7 as IEEE floating point numbers (the
   units are unidirectional bytes per second). Following this are the
   IP addresses of the two ends of the interface. Two TLVs are
   possible, one for IPV4 and one for IPV6 addressing of the link.

   Note: the feedback TLVs may also optionally be included in query or
   query-reply messages in response to bandwidth update queries from an
   LER. Details of this mechanism are provided in [QUERY].

2.1 Bandwidth directionality considerations

   The order of the two addresses in the feedback TLV implies the
   direction in which the bandwidth is available. For example if the
   first address is A and the second address is B the bandwidth is
   unreserved in the A to B direction.

   It is possible for an implementation to provide both the A to B
   direction and the B to A direction as part of the same feedback
   message. This is done by simply including a TLV with A,B as the
   addresses of the link and a different TLV with B,A as the addresses

Ashwood-Smith, et. al.       July  2000        [Page 6]
Internet Draft      LSP Feedback with CR-LDP    July, 2000

   of the link. Should CR-LDP evolve to be able to support bi-
   directional traffic flow and reservations it is expected that bi-
   directional feedback would also be implemented via this mechanism.

Ashwood-Smith, et. al.       July  2000        [Page 7]
Internet Draft      LSP Feedback with CR-LDP    July, 2000

3. IPV4 specified link feedback TLV

   0                   1                   2                   3
   0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |U|F|          0x830            |      Length                   |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |   BANDWIDTH UNRESERVED AT HOLDING PRIORITY 0 (IEEE float)     |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
      . . . . . . . .
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |   BANDWIDTH UNRESERVED AT HOLDING PRIORITY 7 (IEEE float)     |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                  IPV4 address of interface (near end)         |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                  IPV4 address of interface (far end)          |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

4. IPV6 specified link feedback TLV

   0                   1                   2                   3
   0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |U|F|          0x831            |      Length                   |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |   BANDWIDTH UNRESERVED AT HOLDING PRIORITY 0 (IEEE float)     |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
      . . . . . . . .
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |   BANDWIDTH UNRESERVED AT HOLDING PRIORITY 7 (IEEE float)     |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                  IPV6 address of interface (near end)         |
   |                           _____                               |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                  IPV6 address of interface (far end)          |
   |                           _____                               |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

5. Detailed Procedures

   On receipt of a withdraw, notification, query-reply, or mapping
   message pertaining to a request made by CR-LDP (as opposed to LDP),
   a feedback TLV of the appropriate format for the interface over
   which the message was received is inserted into the message before
   forwarding it back to the source of the request. The 8 bandwidth
   values are filled in with the outgoing bandwidth available on this
   interface for each of the 8 holding priorities in bytes per second.
   Finally the interface's address and far end address are placed in
   the TLV.

Ashwood-Smith, et. al.       July  2000        [Page 8]
Internet Draft      LSP Feedback with CR-LDP    July, 2000

   On receipt of a CR-LDP request message which cannot be satisfied. A
   notification message is formatted normally. The 8-bandwidth values
   are filled in with the outgoing bandwidth available on this
   interface for each of the 8 holding priorities in bytes per second.
   Finally, the interface's address and far end address are placed in
   the TLV.

   On receipt of a CR-LDP request message which has been satisfied and
   which results in a mapping being generated. No feedback TLV is added
   since the previous node will insert the proper TLV when it receives
   the reverse flowing mapping.

   When an LDP session goes down either because of a link failure,
   TCP/IP timeout, keepalive timeout, adjacency timeout etc. Other LDP
   sessions in the module must generate either notification, withdraw
   or release messages for LSPs that traversed the LDP in question. In
   the case that the LSP was created by CR-LDP and that a withdraw or
   notification is about to be generated, LDP will insert a feedback
   TLV for the interface which just went down that contains 0's for all
   the bandwidth values and attach to it the proper interface
   addresses.

   When the LDP session that originated a CR-LDP label request receives
   a mapping that contains feedback TLV's it is recommended that these
   bandwidth values supersede the corresponding values in the node's
   topology database for source route computations. Doing so permits
   this node to immediately synchronize its topology with respect to
   the real bandwidth reservations along the path that was just
   established.

   When the LDP session that originated a CR-LDP label request receives
   a notification that contains feedback TLV's it is recommended that
   these bandwidth values supersede the corresponding values in the
   node's topology database for source route computations. Doing so
   permits this node to immediately synchronize its topology with
   respect to the real bandwidth reservations along the path that just
   failed to establish. The source node may then re-compute a path
   knowing that the computation will take into account the failure if
   it was caused by the topology database being in error with respect
   to the real network state.

   6. IGP considerations

   Implementations MUST NOT permit bandwidth information learned by
   this feedback mechanism to be re-flooded via IS-IS, OSPF or any
   other IGP. The bandwidth information learned via these feedback
   mechanisms is to be used ONLY for source route computations on the
   nodes that are directly on the path that fed back the bandwidth.
   Normally only the source node of the LSP, or perhaps intermediate
   gateway nodes will use this information. It is however permitted for
   intermediate nodes that are forwarding this feedback information to
   store it for their own local source route computations.

Ashwood-Smith, et. al.       July  2000        [Page 9]
Internet Draft      LSP Feedback with CR-LDP    July, 2000

   There is a possibility of a race condition between the bandwidth
   information that is received via feedback and that which is received
   via a normal IGP flood. While there may be a discrepancy between the
   two, both are within a few 100 milliseconds of being correct.
   Solutions to allow us to determine which information is most up to
   date (say by adding a sequence number) do not add any significant
   benefit. Constraint based, source routed systems will always have
   errors in the local topology database with respect to the real
   network. We can reduce these errors through reduced flooding
   intervals, path following feedback and selective flooding but we
   cannot realistically reduce the errors below the second or so range.
   As a result propagation delay order race conditions are noise with
   respect to the average expected errors. An implementation SHOULD
   therefore consider the most recently received update (IGP or
   feedback) as being the most up to date.

7. Future considerations

   Constraint based routing systems such as CR-LDP will in the future
   offer other forms of constraint than simply reserved bandwidth.
   Actual utilization levels, current congestion levels, number of
   discrete channels/wavelengths available etc. are all possible
   constraints that change rapidly and which must be taken into
   consideration when computing a route. It is expected that this
   mechanism will be used to feedback these and other new forms of link
   constraining data.

8. RSVP consideration

   Nothing precludes the use of such feedback mechanisms with a similar
   TLV structure in the RSVP Resv and other reverse flowing messages
   although repeatedly applying a fed-back should be avoided since it
   increases the race condition window with flooded LSPs. This could be
   accomplished by a simple rule that only permits fed-back information
   on the original RESV, not on subsequent refreshes.

9. Intellectual Property Consideration

   The IETF has been notified of intellectual property rights claimed
   in regard to some or all of the specification contained in this
   document.  For more information consult the online list of claimed
   rights.

10. Security Considerations

   This document raises no new security considerations for CR-LDP, RSVP
   or MPLS in general.

11. Acknowledgments

Ashwood-Smith, et. al.       July  2000        [Page 10]
Internet Draft      LSP Feedback with CR-LDP    July, 2000

   The authors would like to thank Keith Dysart for his guidance, and
   Jerzy Miernik for helping implement these concepts and bringing them
   to life.

12. References

   [CR-LDP] Constraint-Based LSP Setup using LDP, draft-ietf-mpls-cr-
   ldp-04.txt

   [LDP] LDP Specification, draft-ietf-mpls-ldp-05.txt

   [IS-IS] Extensions to IS-IS for traffic engineering, draft-ietf-
   isis-traffic-01.txt

   [QUERY] MPLS LDP Query Message Description, draft-paraschiv-mpls-
   lsp-query-00.txt.

13. Author's Addresses

   Peter Ashwood-Smith               Bilel Jamoussi
   Nortel Networks Corp.             Nortel Networks Corp.
   P.O. Box 3511 Station C,          600 Technology Park Drive
   Ottawa, ON K1Y 4H7                Billerica, MA 01821
   Canada                            USA
   Phone: +1 613-763-4534            phone: +1 978-288-4506
   [email-masked]         [email-masked]

   Darek Skalecki                    Don Fedyk
   Nortel Networks Corp.             Nortel Networks Corp.
   P.O. Box 3511 Station C,          600 Technology Park Drive
   Ottawa, On K1Y 4H7                Billerica, MA 01821
   Canada                            USA
   Phone: +1 613-765-2252            Phone: +1 978-228-3041
   [email-masked]         [email-masked]

Full Copyright Statement

   Copyright (C) The Internet Society (date). All Rights Reserved. This
   document and translations of it may be copied and furnished to
   others, and derivative works that comment on or otherwise explain it
   or assist in its implementation may be prepared, copied, published
   and distributed, in whole or in part, without restriction of any
   kind, provided that the above copyright notice and this paragraph
   are included on all such copies and derivative works. However, this
   document itself may not be modified in any way, such as by removing
   the copyright notice or references to the Internet Society or other
   Internet organizations, except as needed for the purpose of
   developing Internet standards in which case the procedures for
   copyrights defined in the Internet Standards process must be
   followed, or as required to translate it into languages other than
   English.

Ashwood-Smith, et. al.       July  2000        [Page 11]
Internet Draft      LSP Feedback with CR-LDP    July, 2000

   The limited permissions granted above are perpetual and will not be
   revoked by the Internet Society or its successors or assigns.

Ashwood-Smith, et. al.       July  2000        [Page 12]

---

From: Matthew Kontoff <[email-masked]>
Date: Thu, 14 Sep 2000 16:08:44 -0400

Darek Skalecki wrote:
> 
> ben abarbanel wrote:
> 
> > I was reading the latest draft "draft-ietf-isis-traffic-02.txt" and a question came to mind regardingunreserved
> > bandwidth. In section 5.4 SUB-TLV 11: Unreserved bandwidth states "for stability reasons, rapid changes in the
> > values in this sub-TLV should not cause rapid generation of LSPs".

You still need to regenerate and flood LSPs when an interface state (i.e. bandwidth)
changes. How you do this is up to you. For example, you could reflood LSPs at an interval
that could be tied to the amount of bandwidth being used. The more frequently LSPs 
are reflooded the closer an approximation of actual unreserved bandwidth in the 
network you will have.

> 
> > I was wondering what if  2 ingress (head end RSVP) LSR routers are provisioning LSPs (each wanting 100 Mbytes)
> > through the same core convergent LSR interface and each one of the routers at the time they privision their LSP
> > think there is 100 Mbyte of bandwidth free to complete the job. In reality there is a total of 100 MBYTE for the
> > whole interface, the first RSVP reservation succeeds while the second fails.
> 
> > The second LSP Ingress router would ask TE Source routing for another next bandwidth constraint best path to setup
> > the LSP.  How efficient is that if many LSPs coverge in the core of the network this way and there is no precise way
> > to say what is available at the time RSVP is trigered on?
> 
> > Also, since the IS-IS and its TE source routing engine take snap shots of the bandwidth resources in the network it
> > has a tunnel view of what other IS-IS(LSR) routers are doing to this information at that moment in time, and are
> > making inaccurate determination of what is reserved and what is free. The IS-IS route convergent time could be much
> > slower than the RSVP signalling time.
> 
> For CR-LDP a feedback mechanism was proposed to learn at signaling intervals/speed whether or not there was b/w at
> interfaces. A snapshot of that b/w is fed into TE database based on which source routes are computed. With feedback,
> if there is insufficient b/w at an interface to accomodate an LSP, a notification message with a feedback TLV is
> carried towards the source. The feedback information is then input into TE database at source and a new route
> computed. That route should now not contain the interface where there was insufficient b/w. The feedback mechanism is
> described in:

Why would you need a feedback mechanism if you regenerate and flood IS-IS LSPs with updated TE TLVs
that contain up to the moment (second, millisecond, whatever) bandwidth info?

> > if a series of LSP requests are hitting a given IS-IS router (TE source routing engine) at the same processing
> > window/cycle, it makes assignments based on a frozen snap shot of the bandwidth in the network. Assignments are made
> > without consideration for their RSVP signalling success/failure and eventually the signalling components would fail
> > their reservation requests causing turbulance with LSP attempts.
> 
> > Assuming most of what i said is true, do we need another more precise mechanism to solve this? Regards,
> 
> The feedback provides you ability to try, fail, learn and try again. From our experience, a proprietary signaling
> protocol with feedback worked just fine, i.e. convergence was fast (one or two tries/failures and a satisfactory route
> was found).

I agree with the 'try fail and learn again' approach but I don't think you need
a separate feedback mechanism other than recent IS-IS LSPs from other LSRs.

Matt Kontoff

---

From: "Darek Skalecki" <[email-masked]>
Date: Thu, 14 Sep 2000 16:29:06 -0400

> that contain up to the moment (second, millisecond, whatever) bandwidth info?

If your flooding interval is very short then there is no need for feedback but you don't want to flood too frequently or
else only control traffic is running in the network and the network doesn't scale too well. We employed feedback in a
system where regular floods were every 30 minutes, feedback updates were at path establishment rates, i.e. feedback
information was piggy-backed on top of appropriate signaling messages.

--
Darek Skalecki
Nortel
(613) 765-2252

---

From: "ben abarbanel" <[email-masked]>
Date: Thu, 14 Sep 2000 16:44:25 -0400

Matthew and Darek:
  I kind of like the feedback mechanism defined in
"draft-ietf-mpls-te-feed-01.txt". I think what is missing and maybe Bilel or
Darek can add it to their spec is that a timestamp be added to both the
flooded LSA/LSP TE TLV to identify when the bandwidth sample was taken. Also
the CR/LDP or RSVP-TE new TLV have a time stamp identifying when the
reservation denial and bandwidth sample was taken from the rejecting LSR. As
the message is returned to the ingress LSR its bandwidth will be overwritten
into the linkstate database, since its the most up to date information.

I can see a race condition where IGP converging link state information might
have older data then Reservation Release/Withdrawl messages containing the
same (feedback) data. Thus its possible for the ingress LSR IGP to
momentarily over-write newer bandwidth with bad/older bandwidth into the
ingress LSR linkstate database. Ingress LSR IGP should check the timestamp
in the database with its current LSA/LSP packet and only update the
bandwidth value if it has the newer timestamp in the LSA/LSP packet.

regards,
Ben

Phone:  703 456 2982
FAX:      703 456 2952

IPOptical, Inc.
11480 Sunset Hills Road
Suite #200E
Reston, VA 20190

----- Original Message -----
From: "Matthew Kontoff" <[email-masked]>
To: "Darek Skalecki" <[email-masked]>
Cc: "ben abarbanel" <[email-masked]>; "isis-wg"
<[email-masked]>
Sent: Thursday, September 14, 2000 4:08 PM
Subject: Re: [Isis-wg] IS-IS TE Question

> > > I was reading the latest draft "draft-ietf-isis-traffic-02.txt" and a
question came to mind regardingunreserved
> > > I was wondering what if  2 ingress (head end RSVP) LSR routers are
provisioning LSPs (each wanting 100 Mbytes)
> > > through the same core convergent LSR interface and each one of the
routers at the time they privision their LSP
> > > think there is 100 Mbyte of bandwidth free to complete the job. In
reality there is a total of 100 MBYTE for the
fails.
> > > The second LSP Ingress router would ask TE Source routing for another
next bandwidth constraint best path to setup
> > > the LSP.  How efficient is that if many LSPs coverge in the core of
the network this way and there is no precise way
> > > Also, since the IS-IS and its TE source routing engine take snap shots
of the bandwidth resources in the network it
> > > has a tunnel view of what other IS-IS(LSR) routers are doing to this
information at that moment in time, and are
> > > making inaccurate determination of what is reserved and what is free.
The IS-IS route convergent time could be much
> > For CR-LDP a feedback mechanism was proposed to learn at signaling
intervals/speed whether or not there was b/w at
> > interfaces. A snapshot of that b/w is fed into TE database based on
which source routes are computed. With feedback,
> > if there is insufficient b/w at an interface to accomodate an LSP, a
notification message with a feedback TLV is
> > computed. That route should now not contain the interface where there
was insufficient b/w. The feedback mechanism is
> > > if a series of LSP requests are hitting a given IS-IS router (TE
source routing engine) at the same processing
> > > window/cycle, it makes assignments based on a frozen snap shot of the
bandwidth in the network. Assignments are made
> > > without consideration for their RSVP signalling success/failure and
eventually the signalling components would fail
> > The feedback provides you ability to try, fail, learn and try again.
From our experience, a proprietary signaling
> > protocol with feedback worked just fine, i.e. convergence was fast (one
or two tries/failures and a satisfactory route

---

From: "ben abarbanel" <[email-masked]>
Date: Thu, 14 Sep 2000 14:02:03 -0400

Darek:
  Thanks for the kind response. I was thinking about your answer. What happens if you have multiple LSP attempts with multiple retries does the originating LSR take into account potential attempts that have not initiated yet, but could collide in the core LSR interface and thus forces other potential LSP to go to different paths (although less optimal), or does it simply assume that all colliding LSPs have plenty of bandwidth even before they are actually assigned or signalled through the domain?

Second question, how do you keep the feedback information carried back in CR/LDP from being overwritten by the IGP flooding of the same bandwidth data?   Could this propose a synchronization issue or is the assumption that the IGP data will be as good as the feedback data?

Regards,
Ben

Phone:  703 456 2982
FAX:      703 456 2952

IPOptical, Inc.
11480 Sunset Hills Road
Suite #200E
Reston, VA 20190

  ----- Original Message ----- 
  From: Darek Skalecki 
  To: ben abarbanel 
  Cc: isis-wg 
  Sent: Thursday, September 14, 2000 12:25 PM
  Subject: Re: [Isis-wg] IS-IS TE Question

  ben abarbanel wrote: 
    I was reading the latest draft "draft-ietf-isis-traffic-02.txt" and a question came to mind regardingunreserved bandwidth. In section 5.4 SUB-TLV 11: Unreserved bandwidth states "for stability reasons, rapid changes in the values in this sub-TLV should not cause rapid generation of LSPs".
    I was wondering what if  2 ingress (head end RSVP) LSR routers are provisioning LSPs (each wanting 100 Mbytes) through the same core convergent LSR interface and each one of the routers at the time they privision their LSP think there is 100 Mbyte of bandwidth free to complete the job. In reality there is a total of 100 MBYTE for the whole interface, the first RSVP reservation succeeds while the second fails.
    The second LSP Ingress router would ask TE Source routing for another next bandwidth constraint best path to setup the LSP.  How efficient is that if many LSPs coverge in the core of the network this way and there is no precise way to say what is available at the time RSVP is trigered on?
    Also, since the IS-IS and its TE source routing engine take snap shots of the bandwidth resources in the network it has a tunnel view of what other IS-IS(LSR) routers are doing to this information at that moment in time, and are making inaccurate determination of what is reserved and what is free. The IS-IS route convergent time could be much slower than the RSVP signalling time.
  For CR-LDP a feedback mechanism was proposed to learn at signaling intervals/speed whether or not there was b/w at interfaces. A snapshot of that b/w is fed into TE database based on which source routes are computed. With feedback, if there is insufficient b/w at an interface to accomodate an LSP, a notification message with a feedback TLV is carried towards the source. The feedback information is then input into TE database at source and a new route computed. That route should now not contain the interface where there was insufficient b/w. The feedback mechanism is described in: 
  http://www.ietf.org/internet-drafts/draft-ietf-mpls-te-feed-01.txt 
    The following is my assumption, please clarify if I am wrong, thanks
    if a series of LSP requests are hitting a given IS-IS router (TE source routing engine) at the same processing window/cycle, it makes assignments based on a frozen snap shot of the bandwidth in the network. Assignments are made without consideration for their RSVP signalling success/failure and eventually the signalling components would fail their reservation requests causing turbulance with LSP attempts.
    Assuming most of what i said is true, do we need another more precise mechanism to solve this? Regards,
  The feedback provides you ability to try, fail, learn and try again. From our experience, a proprietary signaling protocol with feedback worked just fine, i.e. convergence was fast (one or two tries/failures and a satisfactory route was found). 

------------------------------------------------------------------------------

                                                         July 2000

  Status of this Memo

  Abstract

  1. Introduction

     0                   1                   2                   3

     0                   1                   2                   3

  5. Detailed Procedures

     6. IGP considerations

  7. Future considerations

  8. RSVP consideration

  11. Acknowledgments

  12. References

  13. Author's Addresses

     Darek Skalecki                    Don Fedyk

  Full Copyright Statement

---

From: Sudheer Dharnikota <[email-masked]>
Date: Thu, 14 Sep 2000 15:12:24 -0700

Hi Guys:

According to our draft.. our approach to this problem is:

	1. use a primary criteria to find a path 
	2. if it fails due to any reasons (unavailability of the
	   resources is being one of them) then use the crankback
           from the previous ABR by removing the links that are
	   failed. Note that we donot have to consider the
	   convergence problem in this case.
	3. If there are no other paths with the same characteistic
	   then use the secondary criteria to identify the
	   best path.

No links attached to the convergence and playing with the 
timers. Although i agree that changes in the ABW need to 
be propagated sooner.

Cheers,

sudheer

Darek Skalecki wrote:
> 
> Matthew Kontoff wrote:
> 
> > Darek Skalecki wrote:
> > >
> > > ben abarbanel wrote:
> > >
> > > > I was reading the latest draft "draft-ietf-isis-traffic-02.txt"
> > and a question came to mind regardingunreserved
> > > > bandwidth. In section 5.4 SUB-TLV 11: Unreserved bandwidth
> > states "for stability reasons, rapid changes in the
> > > > I was wondering what if  2 ingress (head end RSVP) LSR routers
> > are provisioning LSPs (each wanting 100 Mbytes)
> > > > through the same core convergent LSR interface and each one of
> > the routers at the time they privision their LSP
> > > > think there is 100 Mbyte of bandwidth free to complete the job.
> > In reality there is a total of 100 MBYTE for the
> > > > The second LSP Ingress router would ask TE Source routing for
> > another next bandwidth constraint best path to setup
> > > > the LSP.  How efficient is that if many LSPs coverge in the core
> > of the network this way and there is no precise way
> > > > Also, since the IS-IS and its TE source routing engine take snap
> > shots of the bandwidth resources in the network it
> > > > has a tunnel view of what other IS-IS(LSR) routers are doing to
> > this information at that moment in time, and are
> > > > making inaccurate determination of what is reserved and what is
> > free. The IS-IS route convergent time could be much
> > > interfaces. A snapshot of that b/w is fed into TE database based
> > on which source routes are computed. With feedback,
> > > if there is insufficient b/w at an interface to accomodate an LSP,
> > a notification message with a feedback TLV is
> > > carried towards the source. The feedback information is then input
> > into TE database at source and a new route
> > > computed. That route should now not contain the interface where
> > there was insufficient b/w. The feedback mechanism is
> > > described in:
> If your flooding interval is very short then there is no need for
> feedback but you don't want to flood too frequently or else only
> control traffic is running in the network and the network doesn't
> scale too well. We employed feedback in a system where regular floods
> were every 30 minutes, feedback updates were at path establishment
> rates, i.e. feedback information was piggy-backed on top of
> appropriate signaling messages.
> 
> >
> >
> > > > window/cycle, it makes assignments based on a frozen snap shot
> > of the bandwidth in the network. Assignments are made
> > > > without consideration for their RSVP signalling success/failure
> > and eventually the signalling components would fail
> > > > Assuming most of what i said is true, do we need another more
> > precise mechanism to solve this? Regards,
> > >
> > > The feedback provides you ability to try, fail, learn and try
> > again. >From our experience, a proprietary signaling
> > > protocol with feedback worked just fine, i.e. convergence was fast
> > (one or two tries/failures and a satisfactory route
> > > was found).
> >

---

From: "ben abarbanel" <[email-masked]>
Date: Thu, 14 Sep 2000 20:18:30 -0400

Hi Sudheer:
  Your draft/idea sounds like a good alternative. But if I understand
darek's spec. They are trying to adapt the topology to bandwidth changes as
quickly as possible so that the source routing engines can give the best
possible answer.  I agree most solutions still have that level of
inaccuracy, cause the data gathering mechanism (link state flooding) is much
slower than the execution (signalling) mechanism.

Regards,
Ben

Phone:  703 456 2982
FAX:      703 456 2952

IPOptical, Inc.
11480 Sunset Hills Road
Suite #200E
Reston, VA 20190

----- Original Message -----
From: "Sudheer Dharnikota" <[email-masked]>
To: "Darek Skalecki" <[email-masked]>
Cc: "Matthew Kontoff" <[email-masked]>; "ben abarbanel"
<[email-masked]>; "isis-wg" <[email-masked]>
Sent: Thursday, September 14, 2000 6:12 PM
Subject: Re: [Isis-wg] IS-IS TE Question

---

From: "Darek Skalecki" <[email-masked]>
Date: Fri, 15 Sep 2000 08:49:14 -0400

> Ben Abarbanel,  Software Engineering,

Timestamping is a fine idea but in our proprietary system we simply wrote any
learned  information into a database,
whether the information was from feedback or flooding. You are right that
more-up-to-date feedback information
may be then overwritten by older flooding information but in that case we may
simply re-learn newer information via feedback when needed so we never
implemented timestamping.

Darek

--
Darek Skalecki
Nortel
(613) 765-2252