Subject: RE: MPLS/BGP routing question

From: GUESDON Herve FTRD/DAC/ISS <[email-masked]>
Date: Mon, 2 Oct 2000 08:04:28 +0200

Hi

If a network event breaks the MPLS LSP between two PE in a BGP/MPLS VPN
design, then I agree that the rerouting is an IGP issue. But there is also a
MP-BGP-MPLS synchronisation issue if this rerouting leads to a change in the
MP-BGP next hop. The switch to the LSP corresponding to the new MP-BGP net
hop has to be transparent to the customer VPN traffic.

Regards

hervé

****************************************************************
Hervé Guesdon                                    DAC/CPN/RRI
Research and Development Engineer
IP Routing and VPN lab
France Télécom - R&D
38-40 rue du Général-Leclerc  
92794 Issy Moulineaux Cedex 9  France
phone : +33.1.45.29.43.74  fax : +33.1.45.29.54.11
****************************************************************

"L'avenir c'est du passé en préparation."

>-----Message d'origine-----
>De : S.Matsushima [mailto:[email-masked]]
>Envoyé : samedi 30 septembre 2000 03:24
>À : [email-masked]; Satoru Matsushima
>Cc : [email-masked]
>Objet : Re: MPLS/BGP routing question 
>
>
>on 00.9.30 0:32 AM, Eric Rosen at [email-masked] wrote:
>
>> 
>> Matsushima> When a  LSP of PE to  PE broken, BGP  has no way 
>of  LSP broken.
>> Matsushima> Then, BGP keep  up of VPN routes and VPN  
>traffic going to black
>> Matsushima> hole, until LSP available.
>> 
>> Matsushima> As a result,  VPN customer can not back up  
>their traffic to any
>> Matsushima> link.
>> 
>> Matsushima> I think this is one of most seriously problem of 
>BGP/MPLS VPN.
>> 
>> The  situation you  are worried  about is  where there  is  
>IGP connectivity
>> between the edges,  but for some reason labeled packets  
>cannot make it from
>> one edge to another.
>
>Yes, exactly.
>
>> I guess we don't really see this as a realistic failure
>> scenario.  Sure, buggy software could cause this, but 
>there's a million ways
>> in which buggy software could cause undetected packet loss.
>> 
>
>I think that LSP failure was caused by not only buggy software but also
>oparation failure.
>For example, i) erase a interface as LDP ID ;-< , ii) routes 
>summarization
>failure on ospf area,...
>
>IMO, BGP which on PE should has some way to know of LSP failure.
>This is for customer.
>
>--
>Satoru Matsushima
>

---

From: "Metz, E.T." <[email-masked]>
Date: Mon, 02 Oct 2000 16:24:18 +0100

When considering a large-scale RFC2547-style VPN situation (many VPNs, many
VPN routes) I would not use the actual core-routers as RRs. Instead a
core-router type-of-box (or multiple), or something similar with stable BGP
and sufficient memory, that sits outside the forwarding path seems a better
idea to me. In particular with regard to core stability.

cheers,
	Eduard

> -----Original Message-----
> From: Curtis Villamizar [mailto:[email-masked]]
> Sent: vrijdag 29 september 2000 21:27
> To: [email-masked]
> Cc: [email-masked]; 'Chris Flores'; 'Javier Antich'; 'Michel Redondo
> Ferrero'; [email-masked]
> Subject: Re: MPLS/BGP routing question
> 
> 
> 
> In message 
> <[email-masked]>, "Brijesh 
> Kumar" writes:
> > Eric,
> > 
> > I don't think I said that Route Reflectors need to be in 
> the path, or
> > this function cannot be implemented on an Edge router itself. But
> > given the performance difference in access routers and core routers,
> > and the role of a route reflector in applying policies on behalf of
> > clients, it is pretty obvious that a core router is more 
> suitable for
> > route reflection function. And we both agree that a router 
> designated
> > as Route Reflector need to run BGP.
> > 
> > Cheers,
> > 
> > --brijesh
> 
> 
> A route reflector need not be a router but it does have to run BGP.
> Though common practice is to use the core routers since currently they
> must IBGP anyway and generally have the memory and CPU power and a
> stable BGP implementation to do it well.
> 
> Curtis
>

---

From: Jim Guichard <[email-masked]>
Date: Mon, 02 Oct 2000 15:43:01 +0100

it seems reasonable to suppose that if the BGP next-hop changes then the
MP-BGP update has been originated by a different PE router. This being the
case, then one can assume that the attached VPN site will have advertised
the route across two separate paths so that two separate PE routers can
originate the route. In this scenario, we should pick the best path based
on BGP decision process and if this best path goes away, we would revert to
the other path, hence tranparency to the VPN traffic. Not sure how else the
BGP next-hop would change .. regards, Jim

>hop has to be transparent to the customer VPN traffic.

Jim Guichard CCIE #2069
Network Design Consultant EMEA
Global Solutions Engineering 

+44 (0)181 756 8806
Mobile: +44 802 809763

---

From: GUESDON Herve FTRD/DAC/ISS <[email-masked]>
Date: Mon, 2 Oct 2000 17:47:04 +0200

Hi Jim

You are right, I forgot to mention that the MP-BGP next hop can change due
to the fact that the CPE is multihomed. That change must be transparent for
the customer traffic. Transparency of the provider network like opaque
packet transport should be a requirements for designing IP VPN.

Regards

herve

>De : Jim Guichard [mailto:[email-masked]]
>Envoyé : lundi 2 octobre 2000 16:43
>À : GUESDON Herve FTRD/DAC/ISS; 'S.Matsushima'; [email-masked]
>Cc : [email-masked]; [email-masked]'
>Objet : RE: MPLS/BGP routing question 
>
>

---

From: Jim Boyle <[email-masked]>
Date: Thu, 5 Oct 2000 05:52:19 -0600 (MDT)

On Fri, 29 Sep 2000, Bala Rajagopalan wrote:

> Hello,
> 
> Curtis Villamizar wrote:
> 
> > In message <[email-masked]>, Michel Redondo Ferrero writes:
> > >
> >
> > For example, optical switches will definitely NOT run IBGP with the
> > Internet routers.  They run an IGP and MPLS (plus LMP) but not BGP.
> > (In any reasonably sane network).
> 
> Why not E-BGP between OXCs and routers?

what if the routers are in the same AS, is this some new use of
confederates?

Jim

p.s. to answer the *original* question, from a protocol perspective, as
long as there are LSPs edge to edge (no need to be tunnels), one doesn't
have to run BGP on interior nodes.  In reality, there are other issues
that might keep one running it, though.

---

From: Jieyun Jessica Yu <[email-masked]>
Date: Mon, 9 Oct 2000 07:42:51 -0700

Robert Raszuk said:

>2. Satoru Matsushima said:

>> When a LSP of PE to PE broken, BGP has no way of LSP broken.
>> I think this is one of most seriously problem of BGP/MPLS VPN.

>Not true when we are talking about prefix based LSPs. If you are using
>prefix based LSPs they will disappear only if ip route to bgp next hop
>will disappear what in turn will cause bgp route withdrawn (invalid bgp
>next hop) at a periodic scanner. If there is no route - you will not be
>able to route anyway - no miracles here.

I think the black-hole senario Satoru described can happen in prefix based
LSPs. The BGP next hop is learned via IGP so if for some reason the IGP is
up but the mpls tunnel fail to established, the blackhole would happen. In
this case, the RR will still pass the route to the PE via IBGP because it
has no idea te LSP is broken and the PE will still install routes from the
RR since the bgp next hop learned via IGP is there.

The question is how likely this will happen in operation. For that, Eric and
Satoru had an exchange which I will include below.

This is sort of similar to the situation of Route Server (RS) serving at the
NAPs where FDDI or gigaE were used. NAP is a facility where ISPs routers
connect directly via a common media such as FDDI in elary days or others and
exchange routing information via BGP. To improve scalability, instead of
having all ISP routers BGP peer with each other (full mesh peering), a RS is
used to pass the information so each ISP router just needs to peer with the
RS. The problem encountered was that somehow the layer2 connection lost
between a pair of routers while the RS won't know (if can since it is not in
the data pass) it and still passing routes between them. As a result, it
causes traffic blackholing. When I was involved in RS operation, it happened
more than what anticipated. So it is hard to say definitely this is not
going to (or goig to) happen in the MPLS VPN situation until we have more
operational experiences.

                                                           --Jessica

The exchange between Eric R. and Satoru: 

on 00.9.30 0:32 AM, Eric Rosen at [email-masked] wrote:

> 
> Matsushima> When a  LSP of PE to  PE broken, BGP  has no way of  LSP
broken.
> Matsushima> Then, BGP keep  up of VPN routes and VPN  traffic going to
black
> Matsushima> As a result,  VPN customer can not back up  their traffic to
any
> Matsushima> link.
> 
> Matsushima> I think this is one of most seriously problem of BGP/MPLS VPN.
> 
> The  situation you  are worried  about is  where there  is  IGP
connectivity
> between the edges,  but for some reason labeled packets  cannot make it
from

Yes, exactly.

> scenario.  Sure, buggy software could cause this, but there's a million
ways

I think that LSP failure was caused by not only buggy software but also
oparation failure.
For example, i) erase a interface as LDP ID ;-< , ii) routes summarization
failure on ospf area,...

--
Satoru Matsushima