Subject: [RADIR] ROAP was discussed on the IAB TechChat Today

From: Russ Housley <[email-masked]>
Date: Wed, 23 May 2007 12:52:38 -0400

The same slides that were used in the INT Area Session in Prague were 
used, so there you already have access to the presentation 
material.  However, there were a few nuggets that came up in the 
discussion that I wanted to pass on for consideration in the problem 
statement document.  I'm not the expert by any stretch, so I am just 
the messenger.  Any follow-up will come during the expanding circle 
review that we discussed yesterday.

The number number of unique routes or prefixes is not the only aspect 
of the problem.  As denseness of interconnection grows, number of 
paths expands.  A typical edge router may have 500K-800K paths with 
250K unique prefixes.  A typical core router may have 3M-6M 
paths.  Thus, selecting from the possible paths takes computation and 
memory usage, adding to latency and convergence time.

Most ISPs use route reflection, which implicitly acts as an 
aggregation function per RRs only announce their best paths, but it 
has two negative aspects: things like persistent route oscillation 
may occur, and more fundamentally, multiple route reflectors are 
added per cluster and this results in an amplification of routes as 
each RR reflects all it's best paths.  The result is a 5-15 times 
increase of number of paths in iBGP cores versus edge devices.  So a 
site advertising a more-specific on the far end of the Internet, just 
one, might mean 50 unique paths in an operator's network.

Russ

_______________________________________________
RADIR mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/radir

---

From: Thomas Narten <[email-masked]>
Date: Thu, 24 May 2007 14:59:50 -0400

Thanks Russ.

> memory usage, adding to latency and convergence time.

Agree with this point and at one point we talked about it or had some
wording. I think. But it may have gotten lost.

What this suggests to me is that we shouldn't have a major heading
(for the problem) be "routing update rate", but "routing update
convergence pressures" (or "factors" or something). If the number of
prefixes remains constant, the resources needed to maintain tables
could still go up, just from an increase in the richness of
interconnections.

> one, might mean 50 unique paths in an operator's network.

OK. Some of our BGP experts will have to educate me on this point in
more detail. :-)

Thomas

---

From: Jason Schiller <[email-masked]>
Date: Thu, 24 May 2007 18:04:47 -0400 (EDT)

> OK. Some of our BGP experts will have to educate me on this point in
In some topologies yes.  

Consider an AS1 who runs eBGP with AS 2 in a single location.

If AS1 has a flat iBGP full mesh, then AS1 will have one path for AS2.

Now assume that AS1 is divided into four regions, and each region is
serviced by a single cluster with a pair of route reflectors that
are doing client reflection.  Let us numer the clusters (and their
respective regions) 1 through 4.  

If AS2 runs eBGP to a single edge device in AS1's region 1, the that edge
device will learn a single eBGP path.  That path will be advertised to
both route reflectors that serve region 1.  

All of the route reflects will form the highest level of BGP aggregation,
and will thus run iBGP with each other.  This means both of the route
reflectors will advertise one best path each, to all of the other route
refelectors via iBGP.  

The route refelctors in cluster 1 will have one path via iBGP from the
edge device that is their client.  They will each advertise one best path
to all other route reflectors servers via iBGP.  The route reflector
servers in cluster 1 will ignor the iBGP paths as theit own cluster id
"1" is in the cluster list.

All route reflector server not in cluster 1 will learn two path, one for
each of the route reflectors in cluster 1.

Now each route reflector will reflect one best path to all of its
clients.  Since all of the clients have two route reflector servers, they
will all lear one path for both of their route reflector servers.

Confederate BGP is similar, but has different path charastics as in some
cases an iBGP path may trump an eiBGP path and lead to path supression. 

As for route oscillation, yes that can occur if you have hierarchy.  In
other words if you have more than two clusters or two sub-confederations,
route oscillation can occur.

With out getting too detailed, what happens is you have three routes where
route A is better than route B, and route B is better than route C, and
route A is better than route C.

This can happen if for example you have the following:

#   route       AS path  MED   IGP cost  
A   1.2.3.0/24     2  1    0         40
B   1.2.3.0/24     2  1   10         20   
C   1.2.3.0/24     3  1    0         30

A<B path A has a lower MED

B<C path B has a different neighbor AS, don't compair MED.  path B has a
lower IGP cost than path C

C<A path C has a different neighbor AS, don't compair MED.  path C has a
lower IGP cost than path A

If two of these routes are advertised to a single route reflector cluster,
or a single sub-confed, then only one best path will be chosen and
forwarded onward.  This can cause oscillation due to the fact that the far
end router in another cluster or another sub-AS does not have all three
paths to evulate.

For more details see below...  (I think I have this right)

Region 1 has paths A and B.  Region 3 has path C.  Region 2 lies between
region 1 and region 3.

First region 1 learns path B.  
Region 1 chooses path B as the best.
Region 1 advertises path B to region 2.

Region 2 advertises path B to region 3.

Region 3 knows path B.
Region 3 learns path C.  
Region 3 eiBGP router or route reflector in the iBGP reflector mesh
evaluates the best path.  
Region 3 router compairs path B and path C.  
Path B is best path due to lower IGP distance.  
Path C is not forwarded to region 2.  

====

Region 1 knows path B.
Region 1 now learns path A.
Region 1 evaluates paths B and A.
Path A is better due to lower MED.
Path B is retracted from region 2.
Path A is announced to region 2.

Region 2 knows path B.
Path B is retraced from region 2
Path A is annouced to region 2.
Path A is best
Path B is retracted from region 3.
Region 2 advertises path A to region 3.

Region 3 knows paths B and C.
Path B is retracted.
Path A is learned
Region 3 evaluates paths A and C.
Path C is best due to lower IGP cost.
Region 3 advertises path C to region 2

Region 2 knows path A
Region 2 learns path C
Region 2 evaluates paths A and C
Path C is best due to lower IGP cost
Region 2 advertises path C to region 1

Region 1 knows paths A and B
Region 1 learns path C
Region 1 evaluates paths A, B and C.
Path A is better than B.
Path C is better than A
Best path is B.
Path A is no longer best and get retracted
Path B gets advertised to region 2

=====

Region 2 knows paths A and C
Path A is retracted.
Path B is learned
Region 2 evaluates path B and C
Path B is better due to lower IGP cost.
Path B is advertised to region 3
Path C is no longer best, and is retracted from region 1

Region 1 knows paths A, B and C
Path C is retracted
Region 1 evaluates path A and B
Path A is better due to lower MED
Path B is retracted from region 2

Region 2 knows paths B and C
Path B is retracted
Path C is best
Path C is advertised to region 1

rinse and repeate.