Subject: [Idr] Requested number of paths enhancement for add-paths

From: Robert Raszuk <[email-masked]>
Date: Wed, 20 May 2009 09:17:44 -0700

<Sorry for duplicate emails>

Hi,

I am proposing the below enhancement to add-paths document:

Thx,
R.

5. ADD-PATH Capability

    The ADD-PATH Capability is a new BGP capability [RFC3392].  The
    Capability Code for this capability is specified in the IANA
    Considerations section of this document. The Capability Length field
    of this capability is variable. The Capability Value field consists
    of one or more of the following tuples:

              +------------------------------------------------+
              | Address Family Identifier (2 octets)           |
              +------------------------------------------------+
              | Subsequent Address Family Identifier (1 octet) |
              +------------------------------------------------+
              | Send/Receive (1 octet)                         |
              +------------------------------------------------+
              | Requested Number of Paths (1 octet)            |
              +------------------------------------------------+

    The meaning and use of the fields are as follows:

        Address Family Identifier (AFI):

           This field is the same as the one used in [RFC4760].

        Subsequent Address Family Identifier (SAFI):

           This field is the same as the one used in [RFC4760].

        Send/Receive:

           This field indicates whether the sender is (a) willing to
           receive multiple paths from its peer (value 1), (b) would
           like to send multiple paths to its peer (value 2), or (c)
           both (value 3) for the <AFI, SAFI>.
		
        Requested number of Paths:
		
           This field indicates maximum number of paths bgp speaker
           is willing to receive from it's peer for given AFI/SAFI.
           If Send/Receive field has the value of 2 the value of this
           field should be 0.
		
***		

Justification to go to 6. Operation section:
		  		
   The ability to signal how many paths bgp speaker has
   requested to be received has a significant operational
   advantage. As different routers have different roles in
   the network they need different number of paths to
   realize their application.
		
   As example ASBR willing to provide backup as well as load
   balancing may need more bgp paths then a P router just
   interested in backup application.

   Configuring all of those specific per peer information on
   for example a route reflector for hundreds of peers may
   result in configuration overhead and provisioning errors
   which can be easily avoided by the above extension.

   From the bgp speaker perspective such number can be considered
   just as a hint. An implementation may decide to overwrite it
   and to not comply with it.

---

From: Cayle Spandon <[email-masked]>
Date: Wed, 20 May 2009 14:54:06 -0400

I agree that adding the enhancement is useful.

>  and to not comply with it.

However, the above paragraph seems a little bit too permissive to me.

How about something a bit more strict:

"A sender may choose to send fewer paths than the receiver is willing
to accept and fewer paths than are available to the destination. A
sender must never send more paths than the receiver is willing to
accept. For example, if there are 5 paths to a destination and the
peer has indicated that it is willing to receive 4 paths, the sender
may choose to send only 2 paths but it is not allowed to send all 5
paths."

-- Cayle

---

From: Robert Raszuk <[email-masked]>
Date: Wed, 20 May 2009 12:03:30 -0700

Hi Cayle,

Many thx for the proposal of the alternative text. I would never said 
that better !

Cheers,
R.

---

From: Enke Chen <[email-masked]>
Date: Wed, 20 May 2009 12:29:20 -0700

Hi, Robert:

The idea was proposed and debated before.  The issue with the idea is 
that the receiver does not really know the number of paths that would be 
required for a particular application, and limiting the number of paths 
the sender can send may compromise a solution.  For example, one 
solution for the route oscillation is to advertise all the group-best 
paths.  The solution would be compromised if only a subset of the 
group-best paths were advertised.

Given what you said in the email:
------------------------------------------
 From the bgp speaker perspective such number can be considered
  just as a hint. An implementation may decide to overwrite it
  and to not comply with it.
------------------------------------------

I think that we are better off not to have it.

Thanks.   -- Enke

Robert Raszuk wrote:
> _______________________________________________
> Idr mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/idr

---

From: Robert Raszuk <[email-masked]>
Date: Wed, 20 May 2009 12:42:30 -0700

Hi Enke,

I think perhaps it requires second round of discussion.

I can really see the benefit where you have a lot of paths for a given 
SAFI and P routers need only two while edge may need more.

To distinguish by configuration 100s of IBGP clients of RR which would 
carry potentially 10s of paths if not more is a very much labor 
intensive and error prone configuration task. My intention is to 
eliminate such risk and easy the operator's life.

In fact I would improve what I originally suggested which may address 
the case of application not capable of knowing in advance what number of 
paths to receive. (Personally I have heard from SPs that they would be 
quite conservative with distributing more then best path and if not per 
peer hard limit the number of paths on the sender side on a per SAFI 
basis. That would effectively have even worse effect for your example 
then my suggestion :).

So the new text would say for section 5:

           This field indicates maximum number of paths bgp speaker
           is willing to receive from it's peer for given AFI/SAFI.
           If Send/Receive field has the value of 2 the value of this
           field should be 0. When Send/Receive field has the value of 1
           or 3 and Requested number of paths is set to 0 receiver is
           declaring willingness to accept all paths.

With this in mind would you think we are better off ?

Cheers,
R.

---

From: Robert Raszuk <[email-masked]>
Date: Wed, 20 May 2009 12:50:51 -0700

Hi Enke,

> The issue with the idea is that the receiver does not really know the number
 > of paths that would be required for a particular application

I can easily think of a list of applications where the receiver does 
know how many paths he would need.

If provider is willing to enable following applications he would 
configure corresponding value of the new Requested Number of Paths octet:

* BGP fast connectivity restoration -> value of 2

* Better load balancing -> value of 2-4

* Oscillations -> value of 0

I think it would be great to hear from service providers if they want to 
  have control over how many paths they may be getting from RR 
(including  to allow all) or if they would open up all their edge 
routers to receive all RR would be willing to send ?

Cheers,
R.

---

From: "Bhatia, Manav (Manav)" <[email-masked]>
Date: Thu, 21 May 2009 04:38:55 +0530

Hi Robert,

On what basis would the sender decide the subset of routes it needs to advertise? If you're doing ECMP then wouldn't you need to advertise *all* the information that you have?

Cheers, Manav

> -----Original Message-----
> From: [email-masked] [mailto:[email-masked]] On 
> Behalf Of Robert Raszuk
> Sent: Wednesday, May 20, 2009 9.48 PM
> To: idr
> Subject: [Idr] Requested number of paths enhancement for add-paths
>

---

From: Robert Raszuk <[email-masked]>
Date: Wed, 20 May 2009 16:21:47 -0700

Hi Bhatia,

 > On what basis would the sender decide the subset of routes it needs
 > to advertise?

On the basis of implementing ordered path list or on the basis of 
building group-best sets.

Are you suggesting that add-paths draft should always send all paths it 
has for a given net ? If so let's make sure that such point of view is 
reflected in the specification.

 > If you're doing ECMP then wouldn't you need to
 > advertise *all* the information that you have?

 From the point of view of large networks I doubt seriously that even 
for ECMP (or unequal cost multi path for tunneled core) you need as many 
paths as number of your peering points to other transit ISPs. That 
number can be clearly in tens today.

Cheers,
R.

> Hi Robert,
> 
> On what basis would the sender decide the subset of routes it needs
> to advertise? If you're doing ECMP then wouldn't you need to
> advertise *all* the information that you have?
> 
> Cheers, Manav
> 
>> -----Original Message----- From: [email-masked]
>> [mailto:[email-masked]] On Behalf Of Robert Raszuk Sent:
>> Wednesday, May 20, 2009 9.48 PM To: idr Subject: [Idr] Requested
>> number of paths enhancement for add-paths
>> 
>> Thx, R.
>> Considerations section of this document. The Capability Length
>> field of this capability is variable. The Capability Value field
>> consists of one or more of the following tuples:
>> 
>> 
>> +------------------------------------------------+ | Address Family
>> Identifier (2 octets)           | 
>> +------------------------------------------------+ | Subsequent
>> Address Family Identifier (1 octet) | 
>> +------------------------------------------------+ | Send/Receive
>> (1 octet)                         | 
>> +------------------------------------------------+ | Requested
>> Number of Paths (1 octet)            | 
>> This field indicates whether the sender is (a) willing to receive
>> multiple paths from its peer (value 1), (b) would like to send
>> multiple paths to its peer (value 2), or (c) both (value 3) for the
>> <AFI, SAFI>.  Requested number of Paths:  This field indicates
>> maximum number of paths bgp speaker is willing to receive from it's
>> peer for given AFI/SAFI. If Send/Receive field has the value of 2
>> the value of this field should be 0.  ***
>> 
>> Justification to go to 6. Operation section:  The ability to signal
>> how many paths bgp speaker has requested to be received has a
>> significant operational advantage. As different routers have
>> different roles in the network they need different number of paths
>> to realize their application.  As example ASBR willing to provide
>> backup as well as load balancing may need more bgp paths then a P
>> router just interested in backup application.
>> 
>> Configuring all of those specific per peer information on for
>> example a route reflector for hundreds of peers may result in
>> configuration overhead and provisioning errors which can be easily
>> avoided by the above extension.
>> 
>> From the bgp speaker perspective such number can be considered just
>> as a hint. An implementation may decide to overwrite it and to not
>> comply with it.
>> 
>> 
>> 
>> _______________________________________________ Idr mailing list 
>> [email-masked] https://www.ietf.org/mailman/listinfo/idr
>> 
>

---

From: "Bhatia, Manav (Manav)" <[email-masked]>
Date: Thu, 21 May 2009 06:29:47 +0530

Hi Robert,

I am not suggesting anything; I am merely trying to understand the issue.

Assume I have 3 paths to reach X :

Path1 : as_path {A1, B, C}
Path2 : as_path {A2, B, C}
Path2 : as_path {A3, B, C} 

An AS is load balancing X across these three paths. I cant understand how an AS can advertise its reachability using only a subset of these routes. Wouldn't all the downstream peers need to know all the AS details for their loop prevention algorithm to work?

BGP peers in IXes would also btw need to know all the routes.

Cheers, Manav

> From: Robert Raszuk [mailto:[email-masked]] 
> Sent: Thursday, May 21, 2009 4.52 AM
> To: Bhatia, Manav (Manav)
> Cc: idr
> Subject: Re: [Idr] Requested number of paths enhancement for add-paths
> 
> Hi Bhatia,
>

---

From: Robert Raszuk <[email-masked]>
Date: Thu, 21 May 2009 01:25:12 -0700

Hi,

My first observation is that we are talking about different things ...

I was focusing on intra-domain route distribution from route reflector 
to it's clients and allowing 100s of clients to inform route reflector 
on their application needs wrt numer of paths they would like to 
receive. That would be on the IBGP sessions only.

For clarity we could add to my proposed text that the requested number 
of paths applies to IBGP sessions. For EBGP for the time being it should 
be set to 0 (all paths) till an add-paths EBGP application rfc 
recommends otherwise.

Your point is between ASes over EBGP sessions. I think this is totally 
different application for which there is no even application draft using 
add-path. Sending inactive paths may be fatal. Sending subset of them 
may be dangerous as well if peer's inbound policy can drop some of them. 
  How to mark them which are active/best and which are not is not 
defined today. I think this application requires it's own discussion. 
Some scenarios have been already very well described and presented by 
Brian Dickson in the past. And that could be very good starting point 
for specification of this application.

Cheers,
R.

---

From: Jie Dong <[email-masked]>
Date: Thu, 21 May 2009 17:57:09 +0800

Hi Robert,

I agree with you that different specifications should be made for EBGP and
IBGP sessions, and here are some suggestions: 

For IBGP session: 
In order to make all IBGP speakers in one AS obtain consistent routing
information, after considering all of the requested number from IBGP peers,
the advertiser should determine the number of routes sent to each IBGP peer,
and in common case it should send the same number of routes to all its
peers. 

For EBGP session: 
The advertiser should honor the requested number #n from its peer, and send
its best #n routes to its peers. It may send different number of routes to
different EBGP peers.

Best regards,
Jie

> > An AS is load balancing X across these three paths. I cant understand
how
> an AS can advertise its reachability using only a subset of these routes.
> Wouldn't all the downstream peers need to know all the AS details for
their
> loop prevention algorithm to work?
> >

---

From: "Bhatia, Manav (Manav)" <[email-masked]>
Date: Thu, 21 May 2009 17:23:44 +0530

Hi,

> receive. That would be on the IBGP sessions only.

Are you suggesting that each client could potentially give a different number to the RR based on the application its running? This sounds a trifle odd to me. How would a client know that it can solve all cases of route oscillation if it receives "n" routes from the RR? This also means that the view inside the AS is inconsistent with some routers having more information than the others.

When the RR advertises the subset of routes, does it indicate the ones that have been installed in the FIB? What if it installs 3 routes and a peer requests for just 2? 

And I agree that EBGP is a different ball game altogether and should not be mixed with IBGP.

Cheers, Manav

> > An AS is load balancing X across these three paths. I cant 
> understand how an AS can advertise its reachability using 
> only a subset of these routes. Wouldn't all the downstream 
> peers need to know all the AS details for their loop 
> prevention algorithm to work?
> >

---

From: Robert Raszuk <[email-masked]>
Date: Thu, 21 May 2009 08:31:50 -0700

Hi,

So let's focus on the IBGP case ...

> Are you suggesting that each client could potentially give a
> different number to the RR based on the application its running? 

Yes.

> This sounds a trifle odd to me. 

Why ? Imagine network running some form of internal tunneling for 
example MPLS. Some of it's ASBRs/PEs are new and upgraded are running 
mission critical applications and would like to offer the service of 
fast bgp connectivity restoration by pre-programing their forwarding 
with additional bgp backup exit point.

What is wrong with this that some PEs will have two paths to the remote 
ASBRs and some will not ? Remember there are LSPs (LDP or RSVP-TE) or IP 
tunnels (exemple: GRE) to avoid any intra-domain looping issues.

 > How would a client know that it can solve
> all cases of route oscillation if it receives "n" routes from the RR?

For that application if service provider is interested in solving he may 
ask for all external paths.

> This also means that the view inside the AS is inconsistent with some
> routers having more information than the others.

How otherwise you would propose to realistically deploy this technology 
? If you enable it on one or few routers those will get as many paths as 
RR will give them while other clients of such RR not supporting 
add-paths will still get only the best path.

I have seem many proposal in the IDR which were rejected just due 
difficulty to provide gradual deployment. Would that be the next example 
of it ?

> When the RR advertises the subset of routes, does it indicate the
> ones that have been installed in the FIB? What if it installs 3
> routes and a peer requests for just 2?

In any case I think this new functionality does require a very good 
planning and consideration on the type of network this would be used in.

It is very clear that an implementation must provide the limiting tools 
not to send all paths. It could be as suggested via client to RR 
information then as commented by Jie RR would find the min common 
denominator or it should be configured manually on the RR on a per SAFI 
basis.

Thinking more on this perhaps such configuration on the RR on a per SAFI 
basis is the equally good choice of the solution reg the decision on how 
many paths to send to the clients.

Cheers,
R.