Subject: Re: RFC 3065, confeds..
From: Alex Zinin <[email-masked]>
To: "John G. Scudder" <[email-masked]>
Cc: Ben Black <[email-masked]>, John Engelhart <[email-masked]>, Jeffrey Haas <[email-masked]>, <[email-masked]>
Date: Fri, 29 Jun 2001 11:16:52 -0700

John,

  Thanks for historical info.
  I don't feel it is necessary, but if more people believe
  it makes sense to explicitly mention this, I won't object.

-- 
Alex Zinin

Friday, June 29, 2001, 10:57:02 AM, John G. Scudder wrote:

> At 10:48 AM -0700 6/29/01, Alex Zinin wrote:
>>  > It wouldn't.
>>
>>IMO, we should avoid getting into the business of specifying how
>>things do not work, unless there's an ambiguity. Among other things,
>>the set of these "solutions" :) is infinite.

> In general I agree.  In this particular instance there is some case 
> to be made for mentioning you can't nest confeds since an earlier 
> iteration of the RFC alluded to nesting them.  Since in fact you 
> cannot nest them, that was removed in the latest revision.  (I think 
> the mention of nesting was a consequence of incompletely cribbing 
> confeds from IDRP.  The ISO 10747 flavor *can* be nested, overlapped, 
> etc.)

> It's not a big deal IMO, and by itself certainly doesn't justify 
> revving the RFC.

> --John

Date: Fri, 29 Jun 2001 13:57:02 -0400
To: Alex Zinin <[email-masked]>
From: "John G. Scudder" <[email-masked]>
Subject: Re: RFC 3065, confeds..
Cc: Ben Black <[email-masked]>, John Engelhart <[email-masked]>, Jeffrey Haas <[email-masked]>, [email-masked]
Sender: [email-masked]

At 10:48 AM -0700 6/29/01, Alex Zinin wrote:
>  > It wouldn't.
>
>IMO, we should avoid getting into the business of specifying how
>things do not work, unless there's an ambiguity. Among other things,
>the set of these "solutions" :) is infinite.

In general I agree.  In this particular instance there is some case 
to be made for mentioning you can't nest confeds since an earlier 
iteration of the RFC alluded to nesting them.  Since in fact you 
cannot nest them, that was removed in the latest revision.  (I think 
the mention of nesting was a consequence of incompletely cribbing 
confeds from IDRP.  The ISO 10747 flavor *can* be nested, overlapped, 
etc.)

It's not a big deal IMO, and by itself certainly doesn't justify 
revving the RFC.

--John

Date: Fri, 29 Jun 2001 10:48:38 -0700
From: Alex Zinin <[email-masked]>
Reply-To: Alex Zinin <[email-masked]>
Organization: Nexsi Systems
To: Ben Black <[email-masked]>
Cc: John Engelhart <[email-masked]>, Jeffrey Haas <[email-masked]>, [email-masked]
Subject: Re: RFC 3065, confeds..
Sender: [email-masked]

Ben,

> It wouldn't.

IMO, we should avoid getting into the business of specifying how
things do not work, unless there's an ambiguity. Among other things,
the set of these "solutions" :) is infinite.

Alex.

>> > Might it also be useful to explicitly state that nesting confederations
>> > is not a valid configuration?
>> 
>> Hmmm... Never thought about this. How would such a beast work?

Date: Fri, 29 Jun 2001 09:33:33 -0700
From: Ben Black <[email-masked]>
To: Alex Zinin <[email-masked]>
Cc: John Engelhart <[email-masked]>, Jeffrey Haas <[email-masked]>, [email-masked]
Subject: Re: RFC 3065, confeds..
Sender: [email-masked]

--6TrnltStXW4iwmi0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable

It wouldn't.

Ben

On Fri, Jun 29, 2001 at 12:09:55AM -0700, Alex Zinin wrote:
>=20
> > Might it also be useful to explicitly state that nesting confederations
> > is not a valid configuration?
>=20
> Hmmm... Never thought about this. How would such a beast work?
>=20
> Alex.
>=20

--6TrnltStXW4iwmi0
Content-Type: application/pgp-signature
Content-Disposition: inline

--6TrnltStXW4iwmi0--

Date: Fri, 29 Jun 2001 00:09:55 -0700
From: Alex Zinin <[email-masked]>
Reply-To: Alex Zinin <[email-masked]>
Organization: Nexsi Systems
To: Ben Black <[email-masked]>
Cc: John Engelhart <[email-masked]>, Jeffrey Haas <[email-masked]>, [email-masked]
Subject: Re: RFC 3065, confeds..
Sender: [email-masked]

> Might it also be useful to explicitly state that nesting confederations
> is not a valid configuration?

Hmmm... Never thought about this. How would such a beast work?

Alex.

To: [email-masked]
From: Danny McPherson <[email-masked]>
Reply-To: [email-masked]
Subject: Re: RFC 3065, confeds.. 
Date: Thu, 28 Jun 2001 18:54:27 -0600
Sender: [email-masked]

I'll give it some work within the next few weeks, as 
folks seem to be pretty concerned.  There are some 
other clarifications that need to be made, so I'll 
get to fix those as well.

I'd encourage you to have your vendors fix their 
code if [you|they] think it's broken.  Of course, 
this is what usually happens, regardless of what 
any document says.

-danny

Date: Thu, 28 Jun 2001 18:44:01 -0700
From: Ben Black <[email-masked]>
To: Alex Zinin <[email-masked]>
Cc: John Engelhart <[email-masked]>, Jeffrey Haas <[email-masked]>, [email-masked]
Subject: Re: RFC 3065, confeds..
Sender: [email-masked]

--Fba/0zbH8Xs+Fj9o
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable

Might it also be useful to explicitly state that nesting confederations
is not a valid configuration?

Ben

On Thu, Jun 28, 2001 at 06:44:46PM -0700, Alex Zinin wrote:
>=20
> Ben,
>=20
>  Seems to be indeed better to me.
>=20
>  "the any" should be "any"
>=20
> Alex.
>=20
> > I believe this could more effectively be communicated if you instead ma=
ke
> > a minor change to step 1 in (c):
>=20
> >       1) if the first path segment of the AS_PATH is of type
> >          AS_CONFED_SEQUENCE, that segment and any immediately following
> >          segments of the type AS_CONFED_SET or AS_CONFED_SEQUENCE are
> >          removed from the AS_PATH attribute, leaving the sanitized
> >          AS_PATH attribute to be operated on by steps 2, or 3.
>=20
> > Change this statement to:
>=20
> >       1) if the any path segments of the AS_PATH are of type AS_CONFED_=
SET
> >          or AS_CONFED_SEQUENCE, those segments are removed from the AS_=
PATH
> >          attribute, leaving the sanitized AS_PATH attribute to be opera=
ted
> >          on by steps 2 or 3.
>=20
> > This removes the conflict between (c) and (c.1) that is present with the
> > change you proposed above.
>=20
>=20
> > Ben
>=20

--Fba/0zbH8Xs+Fj9o
Content-Type: application/pgp-signature
Content-Disposition: inline

--Fba/0zbH8Xs+Fj9o--

Date: Thu, 28 Jun 2001 18:44:46 -0700
From: Alex Zinin <[email-masked]>
Reply-To: Alex Zinin <[email-masked]>
Organization: Nexsi Systems
To: Ben Black <[email-masked]>
Cc: John Engelhart <[email-masked]>, Jeffrey Haas <[email-masked]>, [email-masked]
Subject: Re: RFC 3065, confeds..
Sender: [email-masked]

Ben,

 Seems to be indeed better to me.

 "the any" should be "any"

Alex.

> I believe this could more effectively be communicated if you instead make
> a minor change to step 1 in (c):

>       1) if the first path segment of the AS_PATH is of type
>          AS_CONFED_SEQUENCE, that segment and any immediately following
>          segments of the type AS_CONFED_SET or AS_CONFED_SEQUENCE are
>          removed from the AS_PATH attribute, leaving the sanitized
>          AS_PATH attribute to be operated on by steps 2, or 3.

> Change this statement to:

>       1) if the any path segments of the AS_PATH are of type AS_CONFED_SET
>          or AS_CONFED_SEQUENCE, those segments are removed from the AS_PATH
>          attribute, leaving the sanitized AS_PATH attribute to be operated
>          on by steps 2 or 3.

> This removes the conflict between (c) and (c.1) that is present with the
> change you proposed above.

> Ben

Date: Thu, 28 Jun 2001 17:59:17 -0700
From: Ben Black <[email-masked]>
To: Alex Zinin <[email-masked]>
Cc: John Engelhart <[email-masked]>, Jeffrey Haas <[email-masked]>, [email-masked]
Subject: Re: RFC 3065, confeds..
Sender: [email-masked]

--wac7ysb48OaltWcw
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable

On Thu, Jun 28, 2001 at 02:36:09AM -0700, Alex Zinin wrote:
>=20
> 6.1. AS_PATH Modification Rules
>=20
> [...]
>=20
>    c) When a given BGP speaker advertises the route to a BGP speaker
>       located in a neighboring autonomous system that is not a member of
>       the current autonomous system confederation, the advertising
>       speaker shall update the AS_PATH attribute as follows:
>=20
> /* Add:
>                                               ...as follows. Note that
> the BGP speaker must ensure that no AS_CONDEF_SEQUENCE or AS_CONFED_SET
> path segments are left in the AS_PATH attribute when it is announced outs=
ide
> of the confederation.
>=20
> */
>=20

I believe this could more effectively be communicated if you instead make
a minor change to step 1 in (c):

      1) if the first path segment of the AS_PATH is of type
         AS_CONFED_SEQUENCE, that segment and any immediately following
         segments of the type AS_CONFED_SET or AS_CONFED_SEQUENCE are
         removed from the AS_PATH attribute, leaving the sanitized
         AS_PATH attribute to be operated on by steps 2, or 3.

Change this statement to:

      1) if the any path segments of the AS_PATH are of type AS_CONFED_SET
         or AS_CONFED_SEQUENCE, those segments are removed from the AS_PATH
         attribute, leaving the sanitized AS_PATH attribute to be operated
         on by steps 2 or 3.

This removes the conflict between (c) and (c.1) that is present with the
change you proposed above.

Ben

--wac7ysb48OaltWcw
Content-Type: application/pgp-signature
Content-Disposition: inline

--wac7ysb48OaltWcw--

Date: Thu, 28 Jun 2001 02:36:09 -0700
From: Alex Zinin <[email-masked]>
Reply-To: Alex Zinin <[email-masked]>
To: John Engelhart <[email-masked]>
Cc: Jeffrey Haas <[email-masked]>, [email-masked]
Subject: Re: RFC 3065, confeds..
Sender: [email-masked]

John,

> Allright, take 2 on this whole fiasco.  This RFC needs work, ASAP.  It has
> now caused two major outages in the internet.

I agree, to make sure people don't mess future implementations, this
should be documented and the text should be changed to be more explicit
about points a), b), and c) you mentioned. I agree with Jeff on d) however.

Below is what I think can be added. It is redundant on purpose,
so the WG can decide what to leave/add/take out.

Regards,

Alex.

5. AS_CONFED Segment Type Extension

[...]
   This document reserves two additional segment types:

      3       AS_CONFED_SEQUENCE: ordered set of Member AS Numbers
              in the local confederation that the UPDATE message has
              traversed

      4       AS_CONFED_SET: unordered set of Member AS Numbers in
              the local confederation that the UPDATE message has
              traversed

/* Add:

The scope of distribution of the new segment types is limited
to the borders of a specific confederation, i.e., no BGP update message
sent over an external BGP session with a peer outside of the confederation
should include an AS_CONFED_SEQUENCE or an AS_CONFED_SET segment in
the AS_PATH attribute.

*/

6. Operation

[...]
   A BGP speaker receiving an AS_PATH attribute containing an autonomous
   system matching its own confederation shall treat the path in the
   same fashion as if it had received a path containing its own AS
   number.
//         ^+See Section 6.2 for more details.

6.1. AS_PATH Modification Rules

[...]

   c) When a given BGP speaker advertises the route to a BGP speaker
      located in a neighboring autonomous system that is not a member of
      the current autonomous system confederation, the advertising
      speaker shall update the AS_PATH attribute as follows:

/* Add:
                                              ...as follows. Note that
the BGP speaker must ensure that no AS_CONDEF_SEQUENCE or AS_CONFED_SET
path segments are left in the AS_PATH attribute when it is announced outside
of the confederation.

*/

/* Add:

6.2 Processing Incoming Updates

  When processing an incoming update that contains an AS_CONFED_SEQUENCE
  or an AS_CONDEF_SET segment in the AS_PATH attribute, the BGP speaker
  should perform the following check.

  If the update is received from an external BGP neighbor that does
  not belong to the same AS confederation, or the update is received
  from an internal BGP neighbor, but the speaker itself is not configured
  to perform AS confederation functionality, the update packet
  should be ignored and an error should be logged locally.   

*/

8. Compatability Considerations

[...]
/* Add:

   To avoid potential disruption in the autonomous systems that do not
   support extensions described in this document, implementations must 
   ensure that no confederation-specific information is announced to or
   received from the external BGP peers outside of the confederation
   (see Sections 6.1 and 6.2).

*/

Date: Thu, 28 Jun 2001 01:21:50 -0700 (PDT)
From: John Engelhart <[email-masked]>
To: Jeffrey Haas <[email-masked]>
Cc: <[email-masked]>
Subject: Re: RFC 3065, confeds..
Sender: [email-masked]

On Wed, 27 Jun 2001, Jeffrey Haas wrote:

> On Tue, Jun 26, 2001 at 07:38:43PM -0700, John Engelhart wrote:
>
> > c) Implementations should only accept AS_CONFED_* info if confederations
> > are enabled, else drop them as malformed as_paths (from edc)
>
> This is the currently specified behavior:
>
> :    Any BGP speaker not supporting these extensions will generate a
> :    notification message specifying an "UPDATE Message Error" and a sub-
> :    code of "Malformed AS_PATH".

Allright, take 2 on this whole fiasco.  This RFC needs work, ASAP.  It has
now caused two major outages in the internet.

Specifically, the above quoted section from the RFC is being interpreted
by Cisco and Juniper to mean that even if confederations aren't enabled
and configured, it's still valid to accept and pass them on to other IBGP
peers.

This is having devastating effects to non-3065 speakers in many providers
networks.

Date: Wed, 27 Jun 2001 11:22:32 -0700 (PDT)
From: John Engelhart <[email-masked]>
To: Jeffrey Haas <[email-masked]>
Cc: <[email-masked]>
Subject: Re: RFC 3065, confeds..
Sender: [email-masked]

On Wed, 27 Jun 2001, Jeffrey Haas wrote:

> On Tue, Jun 26, 2001 at 07:38:43PM -0700, John Engelhart wrote:
> > We, Exodus, don't run with confederations.  Some of our speakers are non
> > RFC3065 capable.  But, as point #1 states, this isn't a problem.  Unless a
> > vendor has bugs and starts leaking confed info into the BGP routing
> > tables.
>
> If the vendor has bugs, the bugs should be FIXED.  If the issue is
> you're getting leakage due to bugs, how can you trust the peering
> session at all?

Yep.  The vendor is fixing the bugs.  But, IMHO, the spec needs some
tweaking as well.

> > a) Explicit language to not send AS_PATH AS_CONFED_* to EBGP peers
>
> The only way you SHOULD get this is if the router you are peering
> with has explicitly configured your AS as a confederation member:
>
> :    c) When a given BGP speaker advertises the route to a BGP speaker
> :       located in a neighboring autonomous system that is not a member of
> :       the current autonomous system confederation, the advertising
> :       speaker shall update the AS_PATH attribute as follows:
>
> And several sane rules follow.

Yes, it's generally agreed that the spec says that you shouldn't leak
confed info across AS boundires.  I'm saying update the language to make
it specific and explicit.

> > c) Implementations should only accept AS_CONFED_* info if confederations
> > are enabled, else drop them as malformed as_paths (from edc)
>
> This is the currently specified behavior:
>
> :    Any BGP speaker not supporting these extensions will generate a
> :    notification message specifying an "UPDATE Message Error" and a sub-
> :    code of "Malformed AS_PATH".

Here I disagree with you.  Even if it's just an implementation tweak, I
still think language should be added to the RFC 3065 for this.  Why?
We're here to learn from our mistakes.  Big mistakes have been made.
Let's document them and pass them on to future writers of BGP
implemenations.

I think everyone agrees that the spirit of the above was to disallow
CONFED info if confederations weren't enabled.  Unfortunatly, one very
popular router vendor has a different interpertation of things.  This is
why I want it clarified.

> > d) Confederations should be a negotiated capability, as it fundamentally
> > alters RFC 1771 with dire results for non 3065 speakers.  With no warning.
> > A rude suprise for operational networks if someone just turns Confeds on
> > without warning,
>
> The only way this SHOULD happen is your peer is misconfigured.

This isn't an academic problem.  This very scenerio whizzed through the
internet last weekend and took down huge swaths in its wake.

The spec is ambiguous on a few points.  Let's clarify them with the
painful knowledge we've learned from running a network.

> > or it accidentally leaks across because non conformance
> > with A or B.
>
> In which case, can you trust the router at all?

This crosses my mind daily, unfortunatly.  :(

> > AS_PATH is a Mandatory attribute.  Anything altering
> > mandatory attributes in the core spec should be negotiated, because you're
> > no longer core spec.
>
> IMO, capability advertisement (note - not negotiation) doesn't solve
> any of your core issues, which seems to be buggy vendor code.  If
> you don't advertise the confederation capability, and the vendor
> doesn't sanitize the AS_PATH, we still have the same issue.

We've got buggy router code.  Actually, probably most everyone out there
does right now, as of today.

But let's not leave things open for interpertation.  Let's make it
explicit.  Ambiguity in the spec is bad, bad, bad.

If everyone had followed rule A, nothing would have happened.
Unfortunatly, a bug let it leak across.  And then, they didn't implement
rule B.  And they didn't implment rule C, because they support RFC 3065
confederations, which means it's perfectly legal data for them to get.
And everyone implements rule A, so I'm not getting this from an EBGP
peer.  It then happily passed all of this along to all the other
routers like a virus.  And killed non-RFC 3065 speakers.  Again.  And
again.  And again.

> The only big bonus that capability advertisement would buy us is
> having the capability payload verify that the peers are in the
> same confederation and dropping the peering session otherwise.

Point D is up for debate.  It's mostly religous on my part, I hate things
that are getting modified on you without your knowledge.  It's one thing
if you negotiate for the modifications, then everyone has the same common
language.  But just being able to switch extra bits in and out without
forwarning.... not good.

Had rules A through C been implemented, this would have been a total non
issue.  Since they can possibly help contain future outbreaks of similar
confed bugs, I recommend the RFC be updated.  Then, when someone else
decides to write or add BGP confederation support, they'll go "Hmmm, makes
sense" and avoid our mistakes.

> --
> Jeff Haas
> NextHop Technologies
>

From: "Ken Benstead" <[email-masked]>
To: <[email-masked]>
Subject: Testing Again
Date: Wed, 27 Jun 2001 13:40:48 -0400
Importance: Normal
Sender: [email-masked]

Please ignore.

Date: Wed, 27 Jun 2001 10:53:43 -0400
From: Jeffrey Haas <[email-masked]>
To: [email-masked]
Cc: John Engelhart <[email-masked]>
Subject: Re: RFC 3065, confeds..
Sender: [email-masked]

On Tue, Jun 26, 2001 at 07:38:43PM -0700, John Engelhart wrote:
> We, Exodus, don't run with confederations.  Some of our speakers are non
> RFC3065 capable.  But, as point #1 states, this isn't a problem.  Unless a
> vendor has bugs and starts leaking confed info into the BGP routing
> tables.

If the vendor has bugs, the bugs should be FIXED.  If the issue is
you're getting leakage due to bugs, how can you trust the peering
session at all?

> a) Explicit language to not send AS_PATH AS_CONFED_* to EBGP peers

The only way you SHOULD get this is if the router you are peering
with has explicitly configured your AS as a confederation member:

:    c) When a given BGP speaker advertises the route to a BGP speaker
:       located in a neighboring autonomous system that is not a member of
:       the current autonomous system confederation, the advertising
:       speaker shall update the AS_PATH attribute as follows:

And several sane rules follow.

> c) Implementations should only accept AS_CONFED_* info if confederations
> are enabled, else drop them as malformed as_paths (from edc)

This is the currently specified behavior:

:    Any BGP speaker not supporting these extensions will generate a
:    notification message specifying an "UPDATE Message Error" and a sub-
:    code of "Malformed AS_PATH".

> d) Confederations should be a negotiated capability, as it fundamentally
> alters RFC 1771 with dire results for non 3065 speakers.  With no warning.
> A rude suprise for operational networks if someone just turns Confeds on
> without warning,

The only way this SHOULD happen is your peer is misconfigured.  

> or it accidentally leaks across because non conformance
> with A or B.

In which case, can you trust the router at all?

> AS_PATH is a Mandatory attribute.  Anything altering
> mandatory attributes in the core spec should be negotiated, because you're
> no longer core spec.

IMO, capability advertisement (note - not negotiation) doesn't solve
any of your core issues, which seems to be buggy vendor code.  If
you don't advertise the confederation capability, and the vendor
doesn't sanitize the AS_PATH, we still have the same issue.

The only big bonus that capability advertisement would buy us is
having the capability payload verify that the peers are in the
same confederation and dropping the peering session otherwise.

-- 
Jeff Haas 
NextHop Technologies

From: "Ken Benstead" <[email-masked]>
To: <[email-masked]>
Subject: Test
Date: Wed, 27 Jun 2001 09:48:55 -0400
Importance: Normal
Sender: [email-masked]

Please ignore

Subject: question of rfc2858
To: [email-masked]
From: [email-masked]
Date: Wed, 27 Jun 2001 10:46:39 +0800
Sender: [email-masked]

As rfc2858 says,in a update message,the MP_REACH_NLRI attribute carry
only one single instance of <AFI,SAFI,Next Hop Information,...>.

I want to know,if i have many NLRIs to advertise by MP_REACH_NLRI,should
I build same number of update messages as that of NLRIs?And is it
efficient?

thanks for your help.

Date: Tue, 26 Jun 2001 19:38:43 -0700 (PDT)
From: John Engelhart <[email-masked]>
To: <[email-masked]>
Subject: RFC 3065, confeds..
Sender: [email-masked]

We've had some debate on this internally and I'd like to bounce them off
the list.

It is my opinion, and a few others, that RC 3065 should be modified to be
a Capability Negotiated item.

Why?  Because it screws your network wholesale if it's not.  RFC 3065
fundamentally alters RFC 1771.  The problem with this is it gives the far
side no forewarning that it's about to send data that it may not be
prepared for.

This may be fine if this is taken into account.  Anyone who's going be
using confederations had best make sure that all speakers can talk
confederations.  Usually not a problem.  It's the "usually" part that
screws you, and screwed us, big time.

We, Exodus, don't run with confederations.  Some of our speakers are non
RFC3065 capable.  But, as point #1 states, this isn't a problem.  Unless a
vendor has bugs and starts leaking confed info into the BGP routing
tables.  What's the only recourse for non speakers?  Shut down the peering
session.  In a big network, this can be lots of fun.  Keep in mind, this
is no longer theoretical, it's happened.  It's painful.

So, IMHO, and open for discussion, I think the following changes should be
made:

a) Explicit language to not send AS_PATH AS_CONFED_* to EBGP peers

b) Explicit language to not accept AS_PATH AS_CONFED_* from EBGP peers

c) Implementations should only accept AS_CONFED_* info if confederations
are enabled, else drop them as malformed as_paths (from edc)

d) Confederations should be a negotiated capability, as it fundamentally
alters RFC 1771 with dire results for non 3065 speakers.  With no warning.
A rude suprise for operational networks if someone just turns Confeds on
without warning, or it accidentally leaks across because non conformance
with A or B.  AS_PATH is a Mandatory attribute.  Anything altering
mandatory attributes in the core spec should be negotiated, because you're
no longer core spec.

I'm really adamant about point D.  If you open up a BGP connection with a
peer, and nothing gets negotiated, you're speaking RFC 1771.  If one guy
decides to start sending extra information, but the other doesn't support
it, who's at fault?  In the end, it's the people using the network because
the only recourse available is to close the peering session.  And lemme
tell you, this thing propagates like a VIRUS once let loose in the
network.

To: [email-masked]
From: Danny McPherson <[email-masked]>
Reply-To: [email-masked]
Subject: Re: RFC 3065, Confederation...
Date: Tue, 26 Jun 2001 19:11:17 -0600
Sender: [email-masked]

> My email doesn't appear to be showing up on idr.  I'm mailing you because
> I think it's important.

Yep, the idr-post(?) list...

As for your points, they'll be clarified/corrected if/when 
another update is made.

Thanks!

-danny

Date: Tue, 26 Jun 2001 15:09:34 -0500
From: [email-masked]
To: Yakov Rekhter <[email-masked]>
Cc: [email-masked]
Subject: as path attr, as segment t(3|4) handling
Sender: [email-masked]

3065 specifies a specific order for as path segments in it's
ebgp announcement cleanup operations (sec.6.1,c,1).  This is valid
if you assume that all vendors implement spec correctly;  this
is obviously not going to be the case.  

It would be better if the the entire as-path attr. was examined for
cleanup, rather than assuming correct ordering.

Also, it would might be good to specify that processing for 
as segment type 3,4 is /off/ when confed. is not configured on
a receiving device, and that, as a result, a notification should 
be generated as in 3065, sec.8. 

	-ed

	--> Yakov Rekhter <[email-masked]> [010626 14:18]:

> fyi
> ------- Forwarded Message
> 
> Date:    Tue, 26 Jun 2001 13:03:54 -0700
> From:    John Engelhart <[email-masked]>
> To:      <[email-masked]>, <[email-masked]>
> cc:      <[email-masked]>,
> 	 "Ed 'I turned out just television' Crabbe" <[email-masked]>
> Subject: RFC 3065, Confederation clarification (fwd)
> 
> Hi guys,
> 
> My email doesn't appear to be showing up on idr.  I'm mailing you because
> I think it's important.
> 
> Furthermore, we've since found one additional point of contention.  RFC
> 3065, section 6.1.b.1 and 6.1.b.2 specifiy the position in which AS_CONFED
> path info must appear in an AS_PATH announcement.  IMHO, position is
> irrelevent, as long as it's only one of each type (seq & set).
> 
> Do you have to subscribe to another list in order to post to IDR?
> 
> - ---------- Forwarded message ----------
> Date: Mon, 25 Jun 2001 16:17:38 -0700 (PDT)
> From: John Engelhart <[email-masked]>
> To: [email-masked]
> Subject: RFC 3065, Confederation clarification
> 
> Correct me if I'm wrong, but shouldn't the behaviour of confederation
> logic work something like this:
> 
> a) Do not send AS_PATH AS_CONFED_* to EBGP peers
> 
> b) Do not accept AS_PATH AS_CONFED_* from EBGP peers
> 
> While the RFC is clear about item A, it is not clear about item B.  If a
> vendors implementation doesn't support confederations, the result of
> sending CONFED info across EBGP links is clear- tear down the session with
> a NOTIFY of malformed as-path.
> 
> However, if the vendors implementation DOES support confederations, the
> result is less clear.  Expected, intuitive behaviour is either the
> sanitization of the AS_PATH, or more preferebly, a NOTIFY with malformed
> as-path sent to the remote peer.
> 
> If a vendors BGP implementation doesnt do this, and you don't run
> confederations, you end up leaking confederation information into your AS
> and destorying BGP with non-confederation speaking BGP peers.  This is not
> good.
> 
> I bring this up because it's already happened.  I'm tossing it out with
> the suggestion that the RFC be clarified with regard to receiving CONFED
> AS_PATHs from an EBGP peer.
> 
> 
> 
> 
> ------- End of Forwarded Message

-- 
Edward Crabbe
=============
Sr. Network Architect  
Backbone Group
Exodus Communications
-------------
v0x: 408 346 1544

To: [email-masked]
Subject: reposting e-mail
Date: Tue, 26 Jun 2001 14:15:33 -0700
From: Yakov Rekhter <[email-masked]>
Sender: [email-masked]

fyi
------- Forwarded Message

Date:    Tue, 26 Jun 2001 13:03:54 -0700
From:    John Engelhart <[email-masked]>
To:      <[email-masked]>, <[email-masked]>
cc:      <[email-masked]>,
	 "Ed 'I turned out just television' Crabbe" <[email-masked]>
Subject: RFC 3065, Confederation clarification (fwd)

Hi guys,

My email doesn't appear to be showing up on idr.  I'm mailing you because
I think it's important.

Furthermore, we've since found one additional point of contention.  RFC
3065, section 6.1.b.1 and 6.1.b.2 specifiy the position in which AS_CONFED
path info must appear in an AS_PATH announcement.  IMHO, position is
irrelevent, as long as it's only one of each type (seq & set).

Do you have to subscribe to another list in order to post to IDR?

- ---------- Forwarded message ----------
Date: Mon, 25 Jun 2001 16:17:38 -0700 (PDT)
From: John Engelhart <[email-masked]>
To: [email-masked]
Subject: RFC 3065, Confederation clarification

Correct me if I'm wrong, but shouldn't the behaviour of confederation
logic work something like this:

a) Do not send AS_PATH AS_CONFED_* to EBGP peers

b) Do not accept AS_PATH AS_CONFED_* from EBGP peers

While the RFC is clear about item A, it is not clear about item B.  If a
vendors implementation doesn't support confederations, the result of
sending CONFED info across EBGP links is clear- tear down the session with
a NOTIFY of malformed as-path.

However, if the vendors implementation DOES support confederations, the
result is less clear.  Expected, intuitive behaviour is either the
sanitization of the AS_PATH, or more preferebly, a NOTIFY with malformed
as-path sent to the remote peer.

If a vendors BGP implementation doesnt do this, and you don't run
confederations, you end up leaking confederation information into your AS
and destorying BGP with non-confederation speaking BGP peers.  This is not
good.

I bring this up because it's already happened.  I'm tossing it out with
the suggestion that the RFC be clarified with regard to receiving CONFED
AS_PATHs from an EBGP peer.

------- End of Forwarded Message

Date: Tue, 26 Jun 2001 12:52:02 -0700
From: Alex Zinin <[email-masked]>
Reply-To: Alex Zinin <[email-masked]>
Organization: Nexsi Systems
To: Alvaro Retana <[email-masked]>
Cc: Ben Black <[email-masked]>, Enke Chen <[email-masked]>, Danny McPherson <[email-masked]>, <[email-masked]>
Subject: Re: route reflection language change suggestion
Sender: [email-masked]

Alvaro,

  Something like the text below is what I meant. :)
  We definitely don't want to make the spec a deployment
  guidelines doc, but a small note is good.

  Thanks,
-- 
Alex Zinin

Tuesday, June 26, 2001, 11:37:42 AM, Alvaro Retana wrote:

> On Tue, 26 Jun 2001, Alex Zinin wrote:

> Alex:

->>   Maybe we should also add some text about hierarchical
->>   route reflection.

> Like what? ;-)

> IMHO, hierarchical RRs are "already covered" as the clusters are
> independent from each other.  In other words, a client that is also a RR
> (for a different set of clients) is just a "special case " in which the
> non-client(s) also happen to be RRs. ;-)

> On the other hand, Section 9 reads:

> 9. Configuration and Deployment Considerations

>    The BGP protocol provides no way for a Client to identify itself
>    dynamically as a Client of an RR.  The simplest way to achieve this
>    is by manual configuration.

> The only thing I can think about adding is something like:

> A commonly deployed configuration is the use of hierarchical route
> reflectors.  Specifically, a hierarchical route reflector configuration is
> one in which a client in a cluster is also configured as a route reflector
> (for a different cluster).  Care should be taken so that the CLUSTER_ID
> assigned to the route reflector and the client (acting as a route
> reflector for a different set of clients) is not the same.

> My 2c.

> Alvaro.

Date: Tue, 26 Jun 2001 14:37:42 -0400 (EDT)
From: Alvaro Retana <[email-masked]>
To: Alex Zinin <[email-masked]>
Cc: Ben Black <[email-masked]>, Enke Chen <[email-masked]>, Danny McPherson <[email-masked]>, <[email-masked]>
Subject: Re: route reflection language change suggestion
Sender: [email-masked]

On Tue, 26 Jun 2001, Alex Zinin wrote:

Alex:

->   Maybe we should also add some text about hierarchical
->   route reflection.

Like what? ;-)

IMHO, hierarchical RRs are "already covered" as the clusters are
independent from each other.  In other words, a client that is also a RR
(for a different set of clients) is just a "special case " in which the
non-client(s) also happen to be RRs. ;-)

On the other hand, Section 9 reads:

9. Configuration and Deployment Considerations

   The BGP protocol provides no way for a Client to identify itself
   dynamically as a Client of an RR.  The simplest way to achieve this
   is by manual configuration.

The only thing I can think about adding is something like:

A commonly deployed configuration is the use of hierarchical route
reflectors.  Specifically, a hierarchical route reflector configuration is
one in which a client in a cluster is also configured as a route reflector
(for a different cluster).  Care should be taken so that the CLUSTER_ID
assigned to the route reflector and the client (acting as a route
reflector for a different set of clients) is not the same.

My 2c.

Alvaro.

Date: Tue, 26 Jun 2001 10:54:16 -0700
From: Alex Zinin <[email-masked]>
Reply-To: Alex Zinin <[email-masked]>
Organization: Nexsi Systems
To: Ben Black <[email-masked]>
Cc: Alvaro Retana <[email-masked]>, Enke Chen <[email-masked]>, Danny McPherson <[email-masked]>, <[email-masked]>
Subject: Re: route reflection language change suggestion
Sender: [email-masked]

Ben, Alvaro

  Maybe we should also add some text about hierarchical
  route reflection.

-- 
Alex Zinin

Friday, June 15, 2001, 8:40:14 PM, Ben Black wrote:

> I like your wording solution.

> Ben

> On Fri, Jun 15, 2001 at 11:25:50PM -0400, Alvaro Retana wrote:
>> On Fri, 15 Jun 2001, Ben Black wrote:
>> 
>> Ben:
>> 
>> -> If routes are reflected between clients (meaning they aren't meshed
>> -> with each other), then the reflectors serving those clients should
>> -> have unique cluster IDs.
>> 
>> Recommending different cluster-ids in all cases where the clients are not
>> meshed is not the solution.  As Enke pointed out, it will result in
>> additional routes.  The problem you're pointing to is real, but in most
>> topologies will only occur in a failure scenario.
>> 
>> -> If routes are not reflected between clients (meaning they are meshed
>> -> with each other), then the reflectors serving those clients may have
>> -> the same cluster ID.
>> 
>> On the same token, what happens if the connectivity between two clients is
>> lost?  We'll run into the same problem because the RRs have the same
>> cluster-id... :-(
>> 
>> -> Further comments?
>> 
>> How about this?
>> 
>> 6. Redundant RRs
>> 
>>    While a set of clients may be served by a single RR, this
>>    configuration represents a single point of failure.  To mitigate the
>>    risk, the same set of clients may be served by one or more additional
>>    RRs.  Configuration of all the RRs in the cluster with the same 4-
>>    byte CLUSTER_ID allows them to discard routes from other RRs in the
>>    same cluster.  However, it can lead to reachability problems between
>>    clients if conectivity is lost between one of the RRs and at least
>>    one of the clients.
>> 
>>    The CLUSTER_ID assignment in multiple RR scenarios should be
>>    considered carefully in light of the deployment considerations
>>    mentioned in Section 9.
>> 
>> 
>> I think it addresses your concern about clarifying the fact that using the
>> same cluster-id may lead to blackholes...and it also points to Section 9
>> for more criteria about configuration (where it talks about the importance
>> of the topology).  It also doesn't set absolute constraints on any one
>> configuration, but it leaves it up to the people configuring the network.
>> :-)
>> 
>> Alvaro.

From: "proto man" <[email-masked]>
To: [email-masked]
Subject: invalid origin code
Date: Wed, 20 Jun 2001 23:06:37 -0000
Sender: [email-masked]

Folks,
  I just noticed a behaviour in a major vendor box accepting and
  propagating an invalid origin value (4)
  From the latest draft, i understand an invalid origin value should be
  handled disruptively.
  "If the ORIGIN attribute has an undefined value, then the Error
   Subcode is set to Invalid Origin Attribute"
  Is this just a vendor's bug or is there work going on to make BGP
  sessions more stable.
Thanks,
Priya
( I had similar issue with confederation value (1966) but the RFC has been 
updated to reflect a vendo'rs behaviour )
_________________________________________________________________
Get your FREE download of MSN Explorer at http://explorer.msn.com

Date: Wed, 20 Jun 2001 11:35:12 -0400
From: Jeffrey Haas <[email-masked]>
To: [email-masked]
Subject: Re: question on cluster-list
Sender: [email-masked]

>From Section 7 of said RFC:

   When a RR reflects a route, it must prepend the local CLUSTER_ID to
   the CLUSTER_LIST.  If the CLUSTER_LIST is empty, it must create a new
   one. 

On Tue, Jun 19, 2001 at 04:06:36PM -0400, Tsillas, Jim wrote:
> RFC 2796 says that a route received from a client-peer
> should be sent to all client and non-client peers. When
> sending these routes to non-client peers, is it still
> required that the RR prepend/add the cluster-id to the
> cluster-list attribute?
> 
> thanks,
> -Jim.
> 

-- 
Jeff Haas 
NextHop Technologies

From: "Tsillas, Jim" <[email-masked]>
To: "[email-masked]'" <[email-masked]>
Subject: question on cluster-list
Date: Tue, 19 Jun 2001 16:06:36 -0400
Sender: [email-masked]

RFC 2796 says that a route received from a client-peer
should be sent to all client and non-client peers. When
sending these routes to non-client peers, is it still
required that the RR prepend/add the cluster-id to the
cluster-list attribute?

thanks,
-Jim.

Date: Mon, 18 Jun 2001 12:30:19 -0400
From: Jeffrey Haas <[email-masked]>
To: [email-masked]
Subject: Re: Previous work on TTL on bgp routes?
Sender: [email-masked]

On Mon, Jun 18, 2001 at 12:02:22PM -0400, Martin, Christian wrote:
> How about an extension to
> NO_EXPORT, perhaps using a new extended community that says "Announce this
> NLRI only to this(these) ASNs - otherwise, NO_EXPORT?
> 
> Apologies if this has been considered before...

IDRP has the dist_list_incl and dist_list_excl.

IDRP is ISO document 10747.  We keep adding stuff from IDRP into BGP,
probably to the amusement of several people who said we should have
gone to IDRP in the first place. :-)

> chris

-- 
Jeff Haas 
NextHop Technologies

Date: Mon, 18 Jun 2001 10:37:40 -0400
From: Jeffrey Haas <[email-masked]>
To: [email-masked]
Subject: Re: Previous work on TTL on bgp routes?
Sender: [email-masked]

On Wed, Jun 13, 2001 at 01:00:00PM -0700, Yakov Rekhter wrote:
> For previous work on TTL on BGP routes see IDRP, and specifically
> text on RD_HOP_COUNT attribute.

After taking a look at the relevant section of IDRP, this isn't
quite what I had intended.

The version in IDRP essentially gives you the diameter of RD's
that routes have passed through.  What I had envisioned was
having the route originator set a TTL value which is decremented
as it passes through an RD (AS) and is not propagated when the value
is zero.

I have received some comments offline asking if an ID is in
the works for something like this.  Honestly, I was casually asking
to see if anyone else had thoughts on this.  Please consider:

o This is a hammer.  It is a very clumsy one.
o Depending on the "topology of the Internet", it may be an ineffective
  hammer and wont actually solve any problems.  It may create more
  problems than we want.
o This hammer would work best when some amount of "hierarchy" in
  the Internet is present.  With the "topology of the Internet"
  that I think we're seeing, this may be the wrong hammer.
o This hammer really wants the IDRP DIST_LIST_INCL as an override
  to the proposed TTL to allow selective bleedover.  This is
  yet more complexity.

Several parties have observed that the "topology of the Internet"
is flattening out.  A more effective hammer may involve classifying
the AS's that constitute the "core" and using a global community
that says "these more specifics need not cross the core".

Given the wrangling over who is a tier-1 ISP on other mailing lists,
I find it unlikely that we can get this classification to happen
easily without some impartial third-party.

> Yakov.

-- 
Jeff Haas 
NextHop Technologies

Date: Fri, 15 Jun 2001 20:40:14 -0700
From: Ben Black <[email-masked]>
To: Alvaro Retana <[email-masked]>
Cc: Enke Chen <[email-masked]>, Danny McPherson <[email-masked]>, [email-masked]
Subject: Re: route reflection language change suggestion
Sender: [email-masked]

I like your wording solution.

Ben

On Fri, Jun 15, 2001 at 11:25:50PM -0400, Alvaro Retana wrote:
> On Fri, 15 Jun 2001, Ben Black wrote:
> 
> Ben:
> 
> -> If routes are reflected between clients (meaning they aren't meshed
> -> with each other), then the reflectors serving those clients should
> -> have unique cluster IDs.
> 
> Recommending different cluster-ids in all cases where the clients are not
> meshed is not the solution.  As Enke pointed out, it will result in
> additional routes.  The problem you're pointing to is real, but in most
> topologies will only occur in a failure scenario.
> 
> -> If routes are not reflected between clients (meaning they are meshed
> -> with each other), then the reflectors serving those clients may have
> -> the same cluster ID.
> 
> On the same token, what happens if the connectivity between two clients is
> lost?  We'll run into the same problem because the RRs have the same
> cluster-id... :-(
> 
> -> Further comments?
> 
> How about this?
> 
> 6. Redundant RRs
> 
>    While a set of clients may be served by a single RR, this
>    configuration represents a single point of failure.  To mitigate the
>    risk, the same set of clients may be served by one or more additional
>    RRs.  Configuration of all the RRs in the cluster with the same 4-
>    byte CLUSTER_ID allows them to discard routes from other RRs in the
>    same cluster.  However, it can lead to reachability problems between
>    clients if conectivity is lost between one of the RRs and at least
>    one of the clients.
> 
>    The CLUSTER_ID assignment in multiple RR scenarios should be
>    considered carefully in light of the deployment considerations
>    mentioned in Section 9.
> 
> 
> I think it addresses your concern about clarifying the fact that using the
> same cluster-id may lead to blackholes...and it also points to Section 9
> for more criteria about configuration (where it talks about the importance
> of the topology).  It also doesn't set absolute constraints on any one
> configuration, but it leaves it up to the people configuring the network.
> :-)
> 
> Alvaro.

Date: Fri, 15 Jun 2001 23:25:50 -0400 (EDT)
From: Alvaro Retana <[email-masked]>
To: Ben Black <[email-masked]>
Cc: Enke Chen <[email-masked]>, Danny McPherson <[email-masked]>, <[email-masked]>
Subject: Re: route reflection language change suggestion
Sender: [email-masked]

On Fri, 15 Jun 2001, Ben Black wrote:

Ben:

-> If routes are reflected between clients (meaning they aren't meshed
-> with each other), then the reflectors serving those clients should
-> have unique cluster IDs.

Recommending different cluster-ids in all cases where the clients are not
meshed is not the solution.  As Enke pointed out, it will result in
additional routes.  The problem you're pointing to is real, but in most
topologies will only occur in a failure scenario.

-> If routes are not reflected between clients (meaning they are meshed
-> with each other), then the reflectors serving those clients may have
-> the same cluster ID.

On the same token, what happens if the connectivity between two clients is
lost?  We'll run into the same problem because the RRs have the same
cluster-id... :-(

-> Further comments?

How about this?

6. Redundant RRs

   While a set of clients may be served by a single RR, this
   configuration represents a single point of failure.  To mitigate the
   risk, the same set of clients may be served by one or more additional
   RRs.  Configuration of all the RRs in the cluster with the same 4-
   byte CLUSTER_ID allows them to discard routes from other RRs in the
   same cluster.  However, it can lead to reachability problems between
   clients if conectivity is lost between one of the RRs and at least
   one of the clients.

   The CLUSTER_ID assignment in multiple RR scenarios should be
   considered carefully in light of the deployment considerations
   mentioned in Section 9.

I think it addresses your concern about clarifying the fact that using the
same cluster-id may lead to blackholes...and it also points to Section 9
for more criteria about configuration (where it talks about the importance
of the topology).  It also doesn't set absolute constraints on any one
configuration, but it leaves it up to the people configuring the network.
:-)

Alvaro.

Date: Fri, 15 Jun 2001 12:22:03 -0700
From: Ben Black <[email-masked]>
To: Alvaro Retana <[email-masked]>
Cc: Enke Chen <[email-masked]>, Danny McPherson <[email-masked]>, [email-masked]
Subject: Re: route reflection language change suggestion
Sender: [email-masked]

How about the following guidelines:

If routes are reflected between clients (meaning they aren't meshed
with each other), then the reflectors serving those clients should
have unique cluster IDs.

If routes are not reflected between clients (meaning they are meshed
with each other), then the reflectors serving those clients may have
the same cluster ID.

Further comments?

Ben

Date: Fri, 15 Jun 2001 15:09:12 -0400 (EDT)
From: Alvaro Retana <[email-masked]>
To: Enke Chen <[email-masked]>
Cc: Ben Black <[email-masked]>, Danny McPherson <[email-masked]>, <[email-masked]>
Subject: Re: route reflection language change suggestion 
Sender: [email-masked]

On Fri, 15 Jun 2001, Enke Chen wrote:

-> Here we are talking about ibgp peers which should be using the
-> loopbacks for peering. So the peering can not be lost unless there
-> is no longer any connectivity between the peering routers, in which
-> case configuring different cluser-id would not have helped.

In fact, it **would** help. :-)

Going back to Ben's diagram...assume different cluster-ids...

[A]----[B]
 | \  / |
 |  \/  |
 |  /\  |
 | /  \ |
[1]    [2]

If we assume that the loss in connectivity *only* happens between [1] and
[B] *and* between [2] and [A], then the routes from [2] will in fact reach
all the other routers in the picture. :-)

Of course we can both come up with multiple topologies in which problems
may or may not happen, depending on the failure mode. ;-)

-> It is certainly the case that the RRs in a cluster do not need to be
-> configured with the same cluser-id. Storing the extra paths would
-> lead to higher memory consumption, though.

Completely agreed. :-)  I also agree that in most cases iBGP peers should
peer using the loopback interface.

The rfc reads:

9. Configuration and Deployment Considerations
...
   To prevent routing loops and maintain consistent routing view, it is
   essential that the network topology be carefully considered in
   designing a route reflection topology. In general, the route
   reflection topology should congruent with the network topology when
   there exist multiple paths for a prefix.

If I remember correctly, there are "harmful" topologies (IMHO, mostly
resulting from misconfigurations) that can be achieved if the physical
topology is not congruent when assigning RRs.

IMHO, Ben's point is valid as to potential problems.  But the RFC is
*clear* in spelling out the possible pitfalls one may run into when not
following the physical topology.  In most networks (specially ones built
w/the physical topology in mind and using loopback peering), the failure
scenarios must be pretty radical (as Danny pointed out) for a blackhole or
a loop to occur...it all depends on the failure mode *and* the steady
state configuration. =:-)

For whatever's worth, I think that the rfc already points out possible
problems, so no extra text in needed.  Taking the above paragraph into
account, the correct decision should be made by the network implementors.

My 2c.

Alvaro.

Date: Fri, 15 Jun 2001 12:14:03 -0700
From: Ben Black <[email-masked]>
To: Alvaro Retana <[email-masked]>
Cc: Enke Chen <[email-masked]>, Danny McPherson <[email-masked]>, [email-masked]
Subject: Re: route reflection language change suggestion
Sender: [email-masked]

On Fri, Jun 15, 2001 at 03:09:12PM -0400, Alvaro Retana wrote:
> 
> For whatever's worth, I think that the rfc already points out possible
> problems, so no extra text in needed.  Taking the above paragraph into
> account, the correct decision should be made by the network implementors.
> 

The RFC also reads:

6.  Redundant RRs

   Usually a cluster of clients will have a single RR. In that case, the
   cluster will be identified by the ROUTER_ID of the RR. However, this
   represents a single point of failure so to make it possible to have
   multiple RRs in the same cluster, all RRs in the same cluster can be
   configured with a 4-byte CLUSTER_ID so that an RR can discard routes
   from other RRs in the same cluster.

This is the source of my concern, not a vague reference to the possibility
of bad things happening with the "wrong" topology (of which there are no
examples in the RFC).

Ben

Date: Fri, 15 Jun 2001 11:55:24 -0700
From: Ben Black <[email-masked]>
To: "Martin, Christian" <[email-masked]>
Cc: Danny McPherson <[email-masked]>, [email-masked]
Subject: Re: route reflection language change suggestion
Sender: [email-masked]

Christian,

As Rex has kindly reminded me, you need not lose 2 peering sessions, as
in my extreme example, you need only lose 1, at which point one of the
reflectors will start rejecting routes (for example, B would start
rejecting routes from 2 reflected by A, which is fine until the peering
from 2 to B fails).  I do not find the insistence on the use of a
loopback address for peering compelling as the specification does not
require it.

Router B need not be in the forwarding path (nor was I saying that in
my example) for this to be a problem.

To directly address your assertion of a routing loop, I think you have
forgotten the behavior of the ORIGINATOR_ID.  A route which came back
through to router [1] would be rejected since the ORIGINATOR_ID and
the ROUTER_ID would be the same.  There is no loop here.  Also note that
your example is not a routing loop, but an oscillation, as it fails to
actually converge.

Ben

On Fri, Jun 15, 2001 at 06:32:21AM -0400, Martin, Christian wrote:
> 
> > If the problem exists in the configuration the RFC suggests, but not
> > in the configuration I am suggesting, why not make the change?  Is
> > there a drawback I do not see for unique CLUSTER_IDs?
> 
> Ben,
> 
> The reason I have been able to come up with in this regard is that this
> would allow the reflected route that originated from [1] to come back to
> [1], which violates split horizon - thus creating a _routing_ loop.  Given
> the following example, and using your diagram for reference:
> 
> An external route R is announced to [1].  [1] announces it to [A}.  [A], by
> rule reflects it to [B] after adding the originator id of [1] and [A]'s
> cluster id.  [B] receives the route and prepends its cluster id to R, then
> reflectes it back to [1].  Now, if [1] has a policy to set a local
> preference of 100 to routes from [B], and 90 for external routes, [1] shall
> install a route to R via [B].  At this point, the next-hop changes for R via
> [B], and since the best path is now internal, [1] wthdraws the anouncement
> from [A], which withdraws from [B], which withdraws from [1], which
> reannounces to [A] with external nh, and on and on around we go!
> 
>  > > 
> > > > [A]----[B]
> > > >  | \  / |
> > > >  |  \/  |
> > > >  |  /\  |
> > > >  | /  \ |
> > > > [1]    [2]
> > > > 
> > > > Routers A and B are both reflectors with cluster ID 100.  
> > Routers 1 and    
> > > > 2 are reflector clients of both A and B.  If router 1 
> > loses its peering     
> > > > session with router B and router 2 loses its peering 
> > session with router 
> > > > A, then router A will reject the routes to router 2 via 
> > router B, and
> > > > likewise for B and routes via A to 1.
> 
> This is really no different in a full mesh topology.  If any two iBGP
> speakers lose connectivity to each other, they no longer can see routes from
> each other.
> 
> I should point out that what you propose works to provide redundancy, so
> long as reflected routes are never prefered over external or local
> originated routes on a client.  I believe some NSPs are doing this now for
> redundancy purposes, but it can break things.  Perhaps it can be reworded to
> state this a a caveat to your proposal.
> 
> regards,
> chris
> 
> 
> > > 
> > > So realistically, what's the probability of:
> > > 
> > >  o router 1 losing it's peering session to router B
> > >  o AND router 2 losing it's peering session to router A
> > >  o AND routers A & B STILL having connectivity 
> > >  o AND routers 1 & 2 still having connectivity
> > >  o AND the BGP NEXT_HOP via router 2 being accessible from 
> > >    router 1 (and/or the BGP NEXT_HOP via router 1 being accessible 
> > >    from router 2)? 
> > > 
> > > > I don't feel this is a desirable trait.
> > > 
> > > Then perhaps not employing client-to-client reflection would 
> > > alleviate your concern?
> > > 
> > > -danny
> > 

To: Ben Black <[email-masked]>
Cc: Danny McPherson <[email-masked]>, [email-masked]
Subject: Re: route reflection language change suggestion 
Date: Fri, 15 Jun 2001 09:58:00 -0700
From: Enke Chen <[email-masked]>
Sender: [email-masked]

Ben,
Here we are talking about ibgp peers which should be using the loopbacks
for peering. So the peering can not be lost unless there is no longer
any connectivity between the peering routers, in which case configuring
different cluser-id would not have helped.

It is certainly the case that the RRs in a cluster do not need to be
configured with the same cluser-id. Storing the extra paths would lead
to higher memory consumption, though.

-- Enke

> Received: (qmail 28217 invoked by uid 1000); 15 Jun 2001 08:55:19 -0000
> Date: Fri, 15 Jun 2001 01:55:19 -0700
> From: Ben Black <[email-masked]>
> To: Danny McPherson <[email-masked]>
> Cc: [email-masked]
> Subject: Re: route reflection language change suggestion
> Message-ID: <[email-masked]>
> References: <[email-masked]>
> Mime-Version: 1.0
> Content-Type: text/plain; charset=us-ascii
> Content-Disposition: inline
> User-Agent: Mutt/1.2.5i
> In-Reply-To: <[email-masked]>; from [email-masked] on Fri, Jun 15, 2001 at 01:48:42AM -0600
> Sender: [email-masked]
> Precedence: bulk
> 
> Danny,
> 
> Consider the following:
> 
> [A]----[B]
>  | \  / |
>  |  \/  |
>  |  /\  |
>  | /  \ |
> [1]    [2]
> 
> Routers A and B are both reflectors with cluster ID 100.  Routers 1 and    
> 2 are reflector clients of both A and B.  If router 1 loses its peering     
> session with router B and router 2 loses its peering session with router 
> A, then router A will reject the routes to router 2 via router B, and
> likewise for B and routes via A to 1.
> 
> I don't feel this is a desirable trait.
> 
> 
> Ben
> 
> On Fri, Jun 15, 2001 at 01:48:42AM -0600, Danny McPherson wrote:
> > 
> > Ben,
> > Can you describe an example of when this occurs?
> > 
> > Thanks!
> > 
> > -danny
> > 
> > > RFC2796 makes several references to configuring multiple reflectors
> > > with the same cluster ID.  After discussing the issue with several
> > > people, I believe this to be incorrect.  Placing multiple reflectors
> > > in the same cluster can render sets of clients mutually unreachable.
> > > 
> > > I would suggest that section 6 be reworded along the following lines:
> > > 
> > >    While a set of clients may be served by a single RR, this represents
> > >    a single point of failure.  To mitigate this risk, the same set of
> > >    clients may be served by one or more additional RRs, each configured
> > >    with its own, unique, 4-byte CLUSTER_ID, generally the ROUTER_ID of
> > >    each RR.  Configuration of multiple RRs with the same CLUSTER_ID
> > >    is discouraged as it can lead to reachability problems between 
> > >    clients of those RRs.
> > > 
> > > 
> > > 
> > > Ben
> > 

Date: Fri, 15 Jun 2001 02:35:53 -0700
From: Ben Black <[email-masked]>
To: Danny McPherson <[email-masked]>
Cc: [email-masked]
Subject: Re: route reflection language change suggestion
Sender: [email-masked]

If the problem exists in the configuration the RFC suggests, but not
in the configuration I am suggesting, why not make the change?  Is
there a drawback I do not see for unique CLUSTER_IDs?

Ben

On Fri, Jun 15, 2001 at 02:27:20AM -0600, Danny McPherson wrote:
> 
> > Consider the following:
> > 
> > [A]----[B]
> >  | \  / |
> >  |  \/  |
> >  |  /\  |
> >  | /  \ |
> > [1]    [2]
> > 
> > Routers A and B are both reflectors with cluster ID 100.  Routers 1 and    
> > 2 are reflector clients of both A and B.  If router 1 loses its peering     
> > session with router B and router 2 loses its peering session with router 
> > A, then router A will reject the routes to router 2 via router B, and
> > likewise for B and routes via A to 1.
> 
> So realistically, what's the probability of:
> 
>  o router 1 losing it's peering session to router B
>  o AND router 2 losing it's peering session to router A
>  o AND routers A & B STILL having connectivity 
>  o AND routers 1 & 2 still having connectivity
>  o AND the BGP NEXT_HOP via router 2 being accessible from 
>    router 1 (and/or the BGP NEXT_HOP via router 1 being accessible 
>    from router 2)? 
> 
> > I don't feel this is a desirable trait.
> 
> Then perhaps not employing client-to-client reflection would 
> alleviate your concern?
> 
> -danny

To: [email-masked]
From: Danny McPherson <[email-masked]>
Reply-To: [email-masked]
Subject: Re: route reflection language change suggestion 
Date: Fri, 15 Jun 2001 02:27:20 -0600
Sender: [email-masked]

> Consider the following:
> 
> [A]----[B]
>  | \  / |
>  |  \/  |
>  |  /\  |
>  | /  \ |
> [1]    [2]
> 
> Routers A and B are both reflectors with cluster ID 100.  Routers 1 and    
> 2 are reflector clients of both A and B.  If router 1 loses its peering     
> session with router B and router 2 loses its peering session with router 
> A, then router A will reject the routes to router 2 via router B, and
> likewise for B and routes via A to 1.

So realistically, what's the probability of:

 o router 1 losing it's peering session to router B
 o AND router 2 losing it's peering session to router A
 o AND routers A & B STILL having connectivity 
 o AND routers 1 & 2 still having connectivity
 o AND the BGP NEXT_HOP via router 2 being accessible from 
   router 1 (and/or the BGP NEXT_HOP via router 1 being accessible 
   from router 2)? 

> I don't feel this is a desirable trait.

Then perhaps not employing client-to-client reflection would 
alleviate your concern?

-danny

Date: Fri, 15 Jun 2001 01:55:19 -0700
From: Ben Black <[email-masked]>
To: Danny McPherson <[email-masked]>
Cc: [email-masked]
Subject: Re: route reflection language change suggestion
Sender: [email-masked]

Danny,

Consider the following:

[A]----[B]
 | \  / |
 |  \/  |
 |  /\  |
 | /  \ |
[1]    [2]

Routers A and B are both reflectors with cluster ID 100.  Routers 1 and    
2 are reflector clients of both A and B.  If router 1 loses its peering     
session with router B and router 2 loses its peering session with router 
A, then router A will reject the routes to router 2 via router B, and
likewise for B and routes via A to 1.

I don't feel this is a desirable trait.

Ben

On Fri, Jun 15, 2001 at 01:48:42AM -0600, Danny McPherson wrote:
> 
> Ben,
> Can you describe an example of when this occurs?
> 
> Thanks!
> 
> -danny
> 
> > RFC2796 makes several references to configuring multiple reflectors
> > with the same cluster ID.  After discussing the issue with several
> > people, I believe this to be incorrect.  Placing multiple reflectors
> > in the same cluster can render sets of clients mutually unreachable.
> > 
> > I would suggest that section 6 be reworded along the following lines:
> > 
> >    While a set of clients may be served by a single RR, this represents
> >    a single point of failure.  To mitigate this risk, the same set of
> >    clients may be served by one or more additional RRs, each configured
> >    with its own, unique, 4-byte CLUSTER_ID, generally the ROUTER_ID of
> >    each RR.  Configuration of multiple RRs with the same CLUSTER_ID
> >    is discouraged as it can lead to reachability problems between 
> >    clients of those RRs.
> > 
> > 
> > 
> > Ben
> 

To: [email-masked]
From: Danny McPherson <[email-masked]>
Reply-To: [email-masked]
Subject: Re: route reflection language change suggestion 
Date: Fri, 15 Jun 2001 01:48:42 -0600
Sender: [email-masked]

Ben,
Can you describe an example of when this occurs?

Thanks!

-danny

> RFC2796 makes several references to configuring multiple reflectors
> with the same cluster ID.  After discussing the issue with several
> people, I believe this to be incorrect.  Placing multiple reflectors
> in the same cluster can render sets of clients mutually unreachable.
> 
> I would suggest that section 6 be reworded along the following lines:
> 
>    While a set of clients may be served by a single RR, this represents
>    a single point of failure.  To mitigate this risk, the same set of
>    clients may be served by one or more additional RRs, each configured
>    with its own, unique, 4-byte CLUSTER_ID, generally the ROUTER_ID of
>    each RR.  Configuration of multiple RRs with the same CLUSTER_ID
>    is discouraged as it can lead to reachability problems between 
>    clients of those RRs.
> 
> 
> 
> Ben

Date: Fri, 15 Jun 2001 01:05:40 -0700
From: Ben Black <[email-masked]>
To: [email-masked]
Subject: route reflection language change suggestion
Sender: [email-masked]

RFC2796 makes several references to configuring multiple reflectors
with the same cluster ID.  After discussing the issue with several
people, I believe this to be incorrect.  Placing multiple reflectors
in the same cluster can render sets of clients mutually unreachable.

I would suggest that section 6 be reworded along the following lines:

   While a set of clients may be served by a single RR, this represents
   a single point of failure.  To mitigate this risk, the same set of
   clients may be served by one or more additional RRs, each configured
   with its own, unique, 4-byte CLUSTER_ID, generally the ROUTER_ID of
   each RR.  Configuration of multiple RRs with the same CLUSTER_ID
   is discouraged as it can lead to reachability problems between 
   clients of those RRs.

Ben

From: "proto man" <[email-masked]>
To: [email-masked]
Cc: [email-masked]
Subject: Re: Update Message Notification error subcodes
Date: Fri, 15 Jun 2001 02:03:41 -0000
Sender: [email-masked]

Hi,
Thanks,
I am guessing the hole is not filled up for backwards compatibility
of error subcodes...
Can we change the listing to include
   7 - unused value

-Priya

>From: Tony Li <[email-masked]>
>To: "proto man" <[email-masked]>
>CC: [email-masked]
>Subject: Update Message Notification error subcodes
>Date: Thu, 14 Jun 2001 18:41:52 -0700 (PDT)
>
>
>  | I meant to add, is it because sending notification when receiving 
>looped
>  | paths is disruptive..?
>
>
>Exactly.  One thing that we discovered was that it was far less disruptive
>to simply log things or to ignore and count them than to generate a
>NOTIFICATION.
>
>Breaking BGP connections when you don't have to is Harmful to The Internet.
>
>Tony

_________________________________________________________________
Get your FREE download of MSN Explorer at http://explorer.msn.com

From: Tony Li <[email-masked]>
Date: Thu, 14 Jun 2001 18:41:52 -0700 (PDT)
To: "proto man" <[email-masked]>
Cc: [email-masked]
Subject: Update Message Notification error subcodes
Sender: [email-masked]

 | I meant to add, is it because sending notification when receiving looped
 | paths is disruptive..?

Exactly.  One thing that we discovered was that it was far less disruptive
to simply log things or to ignore and count them than to generate a
NOTIFICATION.

Breaking BGP connections when you don't have to is Harmful to The Internet.

Tony

From: "proto man" <[email-masked]>
To: [email-masked]
Subject: Update Message Notification error subcodes
Date: Fri, 15 Jun 2001 01:38:21 -0000
Sender: [email-masked]

oops.
I meant to add, is it because sending notification when receiving looped
paths is disruptive..?
Thanks,
-Priya Moniker
_________________________________________________________________
Get your FREE download of MSN Explorer at http://explorer.msn.com

From: "proto man" <[email-masked]>
To: [email-masked]
Subject: Update Message Notification error subcodes
Date: Fri, 15 Jun 2001 01:34:48 -0000
Sender: [email-masked]

Hi,
  The Update Message error sub codes as per draft-ietf-idr-bgp4-12.txt
  are listed as :
  UPDATE Message Error subcodes:

                               1 - Malformed Attribute List.
                               2 - Unrecognized Well-known Attribute.
                               3 - Missing Well-known Attribute.
                               4 - Attribute Flags Error.
                               5 - Attribute Length Error.
                               6 - Invalid ORIGIN Attribute
                               8 - Invalid NEXT_HOP Attribute.
                               9 - Optional Attribute Error.
                              10 - Invalid Network Field.
                              11 - Malformed AS_PATH.

Value 7 was used in RFC 1771 for routing-loop.
Why is it no longer listed...

Thanks,
-Priya Moniker
_________________________________________________________________
Get your FREE download of MSN Explorer at http://explorer.msn.com

From: "Hire, Ejay" <[email-masked]>
To: "[email-masked]'" <[email-masked]>
Subject: RE: Previous work on TTL on BGP routes?
Date: Thu, 14 Jun 2001 08:38:21 -0400
Sender: [email-masked]

I was thinking about it this morning, and their is a problem with this.  
In the scenario with an as path of 1-2-3, Where 2 advertises the route and a
aggregate sufficient enough to trigger the route drop but ignores the MPAAL
(as in the case of a router without this feature enabled).  3 will receive
both routes but discard the more specific route.  In the event that 2
withdraws the summary route, 2 will not re-advertise the more specific route
causing 3 to lose reachability information to the more specific subnet.

Rewording it to prevent this becomes:

The "Maximum-Parallel-Advertised-Aggregate-Length" (MPAAL) is a optional
transitive path attribute that says  "If this route is being advertised to a
peer in addition to an aggregate route encompassing it with a prefix of less
than the MPAAL, Remove the more specific route from the Adj-RIB-Out.  

If the aggregate route is withdrawn from the Adj-RIB-Out, the more specific
route should be reinserted into the Adj-RIB-Out.

In this Context, Prefix is referring to the subnet mask length expressed in
bits.

-----Original Message-----
From: Ben Black [mailto:[email-masked]]
Cc: [email-masked]'
Subject: Re: Previous work on TTL on BGP routes?
This was suggested some time ago by Curtis, but I've not seen a
full draft on it.
Ben
On Wed, Jun 13, 2001 at 03:59:27PM -0400, Hire, Ejay wrote:
> What if the instead of a TTL value, there was a
> "Maximum-Parallel-Advertised-Aggregate-Length" (MPAAL).  A value that says
> "If this route is received on an interface in addition to an aggregate
route
> encompassing it with a prefix (Subnet mask length expressed in bits) of
less
> than the MPAAL, Discard the more specific route.
> 
> If it was a optional transitive attribute, it would be backwards
compatible
> with current BGP implementations because it would be passed on and ignored
> by systems that did not understand the function.
> 
> Ejay Hire
> 804-591-4564
> 
> Is this a new idea?  Is it worthy of an RFC?

Date: Wed, 13 Jun 2001 16:08:45 -0700
From: Ben Black <[email-masked]>
To: "Hire, Ejay" <[email-masked]>
Cc: "[email-masked]'" <[email-masked]>
Subject: Re: Previous work on TTL on BGP routes?
Sender: [email-masked]

This was suggested some time ago by Curtis, but I've not seen a
full draft on it.

Ben

On Wed, Jun 13, 2001 at 03:59:27PM -0400, Hire, Ejay wrote:
> What if the instead of a TTL value, there was a
> "Maximum-Parallel-Advertised-Aggregate-Length" (MPAAL).  A value that says
> "If this route is received on an interface in addition to an aggregate route
> encompassing it with a prefix (Subnet mask length expressed in bits) of less
> than the MPAAL, Discard the more specific route.
> 
> If it was a optional transitive attribute, it would be backwards compatible
> with current BGP implementations because it would be passed on and ignored
> by systems that did not understand the function.
> 
> Ejay Hire
> 804-591-4564
> 
> Is this a new idea?  Is it worthy of an RFC?
> 
> -----Original Message-----
> From: Jeffrey Haas [mailto:[email-masked]]
> Sent: Wednesday, June 13, 2001 3:42 PM
> To: [email-masked]
> Subject: Re: Previous work on TTL on bgp routes?
> 
> 
> On Wed, Jun 13, 2001 at 03:13:27PM -0400, Hire, Ejay wrote:
> > This feature could cause sporadic and difficult to troubleshoot
> reachability
> > issues.
> > Assume to following scenario.  
> > You are multihomed to two providers that are X Hops apart.
> > You are advertising non-aggregate-able space to one or both providers. 
> > If your BGP TTL > X then you should retain reachability to both providers
> in
> > the event of a single link failure.
> > 
> > In the event of a link failure on your part, and a routing failure on
> their
> > part causing X+n hops to separate the two providers, your BGP TTL may <
> X+n
> > and you will not be reachable by both providers.
> > 
> > Is their a break in my logic here?
> 
> The logic is sound.
> 
> However, this feature - if it existed - should only be used on
> routes that are covered by a less specific route.
> In the event of a topology shift as suggest above, your inter-domain
> traffic engineering would suffer, but you'd maintain reachability.
> 
> An implementation should only propagate the TTL attribute iff
> a less specific route is also being propagated.  If the less
> specific is to be withdrawn, the more specific must be re-advertised
> without the TTL attribute.
> 
> Additionally, only the originator of a route should set the TTL.
> 
> -- 
> Jeff Haas 
> NextHop Technologies

From: "Hire, Ejay" <[email-masked]>
To: "[email-masked]'" <[email-masked]>
Subject: RE: Previous work on TTL on BGP routes?
Date: Wed, 13 Jun 2001 15:59:27 -0400
Sender: [email-masked]

What if the instead of a TTL value, there was a
"Maximum-Parallel-Advertised-Aggregate-Length" (MPAAL).  A value that says
"If this route is received on an interface in addition to an aggregate route
encompassing it with a prefix (Subnet mask length expressed in bits) of less
than the MPAAL, Discard the more specific route.

If it was a optional transitive attribute, it would be backwards compatible
with current BGP implementations because it would be passed on and ignored
by systems that did not understand the function.

Ejay Hire
804-591-4564

Is this a new idea?  Is it worthy of an RFC?

-----Original Message-----
From: Jeffrey Haas [mailto:[email-masked]]
Sent: Wednesday, June 13, 2001 3:42 PM
To: [email-masked]
Subject: Re: Previous work on TTL on bgp routes?

On Wed, Jun 13, 2001 at 03:13:27PM -0400, Hire, Ejay wrote:
> This feature could cause sporadic and difficult to troubleshoot
reachability
> issues.
> Assume to following scenario.  
> You are multihomed to two providers that are X Hops apart.
> You are advertising non-aggregate-able space to one or both providers. 
> If your BGP TTL > X then you should retain reachability to both providers
in
> the event of a single link failure.
> 
> In the event of a link failure on your part, and a routing failure on
their
> part causing X+n hops to separate the two providers, your BGP TTL may <
X+n
> and you will not be reachable by both providers.
> 
> Is their a break in my logic here?

The logic is sound.

However, this feature - if it existed - should only be used on
routes that are covered by a less specific route.
In the event of a topology shift as suggest above, your inter-domain
traffic engineering would suffer, but you'd maintain reachability.

An implementation should only propagate the TTL attribute iff
a less specific route is also being propagated.  If the less
specific is to be withdrawn, the more specific must be re-advertised
without the TTL attribute.

Additionally, only the originator of a route should set the TTL.

-- 
Jeff Haas 
NextHop Technologies

To: [email-masked]
Subject: Re: Previous work on TTL on bgp routes? 
Date: Wed, 13 Jun 2001 13:00:00 -0700
From: Yakov Rekhter <[email-masked]>
Sender: [email-masked]

Folks,

For previous work on TTL on BGP routes see IDRP, and specifically
text on RD_HOP_COUNT attribute.

Yakov.

Date: Wed, 13 Jun 2001 15:42:03 -0400
From: Jeffrey Haas <[email-masked]>
To: [email-masked]
Subject: Re: Previous work on TTL on bgp routes?
Sender: [email-masked]

On Wed, Jun 13, 2001 at 03:13:27PM -0400, Hire, Ejay wrote:
> This feature could cause sporadic and difficult to troubleshoot reachability
> issues.
> Assume to following scenario.  
> You are multihomed to two providers that are X Hops apart.
> You are advertising non-aggregate-able space to one or both providers. 
> If your BGP TTL > X then you should retain reachability to both providers in
> the event of a single link failure.
> 
> In the event of a link failure on your part, and a routing failure on their
> part causing X+n hops to separate the two providers, your BGP TTL may < X+n
> and you will not be reachable by both providers.
> 
> Is their a break in my logic here?

The logic is sound.

However, this feature - if it existed - should only be used on
routes that are covered by a less specific route.
In the event of a topology shift as suggest above, your inter-domain
traffic engineering would suffer, but you'd maintain reachability.

An implementation should only propagate the TTL attribute iff
a less specific route is also being propagated.  If the less
specific is to be withdrawn, the more specific must be re-advertised
without the TTL attribute.

Additionally, only the originator of a route should set the TTL.

-- 
Jeff Haas 
NextHop Technologies

From: "Hire, Ejay" <[email-masked]>
To: [email-masked]
Subject: RE: Previous work on TTL on bgp routes?
Date: Wed, 13 Jun 2001 15:13:27 -0400
Sender: [email-masked]

This feature could cause sporadic and difficult to troubleshoot reachability
issues.
Assume to following scenario.  
You are multihomed to two providers that are X Hops apart.
You are advertising non-aggregate-able space to one or both providers. 
If your BGP TTL > X then you should retain reachability to both providers in
the event of a single link failure.

In the event of a link failure on your part, and a routing failure on their
part causing X+n hops to separate the two providers, your BGP TTL may < X+n
and you will not be reachable by both providers.

Is their a break in my logic here?

-----Original Message-----
From: Alex Zinin [mailto:[email-masked]]
Sent: Wednesday, June 13, 2001 2:57 PM
To: Brijesh Kumar
Cc: Jeffrey Haas; [email-masked]
Subject: Re: Previous work on TTL on bgp routes?

Brijesh,

> On the basis of what knowledge will you recommend to choose this TTL
value?

Same as today---based on the limited knowledge of your upstream
topology.

> And more so, what exactly will you gain by this? 

See below.

> To me restricting routing information on the basis of TTL defeats the
basic
> principle that information about routes must be propogated to every router
> that needs to know this information (let them filter on the basis of their
> needs). Expressing policies in terms of TTL is meaningless since you may
> have no idea how this information is applicable beyond your direct
> neighbours.

This is not about policies in terms of TTL.
A TTL would provide an mechanism to automatically remove specifics and
leave an aggregate in the multihoming scenarios without having an arbitrary
AS configure route aggregation. The benefit is---decreased size of
the routing table.

> If you need to control the distribution of routing information, then use
the
> method similar to IDRP which has attributes to control the distribution of
> routing information by expressely listing the ADs to which this route
> information is to be distributed or excluded from.

A good thing about the TTL approach is that the originator does not
need to know the (constantly changing) AS topology to control the radius
of the route distribution region.
There always are multiple methods to do the same thing with their
pros and cons.

Thanks,

Alex.

Date: Wed, 13 Jun 2001 11:57:15 -0700
From: Alex Zinin <[email-masked]>
Reply-To: Alex Zinin <[email-masked]>
Organization: Nexsi Systems
To: Brijesh Kumar <[email-masked]>
Cc: Jeffrey Haas <[email-masked]>, [email-masked]
Subject: Re: Previous work on TTL on bgp routes?
Sender: [email-masked]

Brijesh,

> On the basis of what knowledge will you recommend to choose this TTL value?

Same as today---based on the limited knowledge of your upstream
topology.

> And more so, what exactly will you gain by this? 

See below.

> To me restricting routing information on the basis of TTL defeats the basic
> principle that information about routes must be propogated to every router
> that needs to know this information (let them filter on the basis of their
> needs). Expressing policies in terms of TTL is meaningless since you may
> have no idea how this information is applicable beyond your direct
> neighbours.

This is not about policies in terms of TTL.
A TTL would provide an mechanism to automatically remove specifics and
leave an aggregate in the multihoming scenarios without having an arbitrary
AS configure route aggregation. The benefit is---decreased size of
the routing table.

> If you need to control the distribution of routing information, then use the
> method similar to IDRP which has attributes to control the distribution of
> routing information by expressely listing the ADs to which this route
> information is to be distributed or excluded from.

A good thing about the TTL approach is that the originator does not
need to know the (constantly changing) AS topology to control the radius
of the route distribution region.
There always are multiple methods to do the same thing with their
pros and cons.

Thanks,

Alex.

Date: Tue, 12 Jun 2001 01:17:29 -0700
From: Alex Zinin <[email-masked]>
Reply-To: Alex Zinin <[email-masked]>
Organization: Nexsi Systems
To: Jeffrey Haas <[email-masked]>
Cc: [email-masked]
Subject: Re: Previous work on TTL on bgp routes?
Sender: [email-masked]

Jeffrey,

  The idea has been floating around for a while, but
  I do not think it was documented. I believe it has merit,
  especially considering the way route aggregation is
  implemented and deployed now.

-- 
Alex Zinin

Monday, June 11, 2001, 2:03:00 PM, Jeffrey Haas wrote:

> Has there been any previous work on TTL in announced bgp routes?
> I'm using TTL in the IP sense - it goes through N hops,
> in this case AS's, and is propagated no further.

Date: Mon, 11 Jun 2001 16:50:43 -0700
From: Alex Zinin <[email-masked]>
Reply-To: Alex Zinin <[email-masked]>
Organization: Nexsi Systems
To: [email-masked]
Subject: Re: Proposed text for 9.1.2
Sender: [email-masked]

Folks,

 Below is the text incorporating the latest comments,
 and including some minor changes I thought were useful.

Alex.

> 3.  Summary of Operation
> 
>    Two systems form a transport protocol connection between one another.
>    They exchange messages to open and confirm the connection parameters.
>    The initial data flow is complete BGP routing information. Incremental
>    updates are sent as the routing information changes. BGP does not
>    require periodic refresh of all BGP routes. Therefore, a BGP speaker
>    must retain the current version of the entire BGP routing tables of
>    all of its peers for the duration of the connection.  If the 
>    implementation decides to not store the routes that have been received
>    from a peer, but have been filtered out according to configured local
>    policy, the BGP Route Refresh option [10] may be used to request the
>    full set of routes from a peer without resetting the BGP session when
>    the local policy configuration changes...
> 
> [...]
> 
> 3.1 Routes: Advertisement and Storage
> 
> [...]
>       - Routes are stored in the Routing Information Bases (RIBs):
>       namely, the Adj-RIBs-In, the Loc-RIB, and the Adj-RIBs-Out. Routes
>       that will be advertised to other BGP speakers must be present in
>       the Adj-RIB-Out; routes that will be used by the local BGP speaker
>       must be present in the Loc-RIB, and the next hop for each of these
>       routes must be present in the local BGP speaker's Routing Table;
>       and routes that are received from other BGP speakers are present
>       in the Adj-RIBs-In.
> 
> [...]
> 
> 3.2 Routing Information Bases
> 
> [...]
> 
>    In summary, the Adj-RIBs-In contain unprocessed routing information
>    that has been advertised to the local BGP speaker by its peers; the
>    Loc-RIB contains the routes that have been selected by the local BGP
>    speaker's Decision Process; and the Adj-RIBs-Out organize the routes
>    for advertisement to specific peers by means of the local speaker's
>    UPDATE messages.
> 
>    Routing information that the router uses to forward packets
>    (or to construct the forwarding table that is used for packet
>    forwarding) is maintained in the Routing Table. The Routing Table
>    accumulates routes to directly connected networks, static routes,
>    routes learned from the IGP protocols, and routes learned from BGP.
>    Whether or not a specific BGP route should be installed in the Routing
>    Table, and whether a BGP route should override a route to the same
>    destination installed by another source is a local policy decision,
>    not specified in this document. Besides actual packet forwarding,
>    the Routing Table is used for resolution of the next-hop addresses
>    specified in BGP updates (see Section 9.1.2).
> 
> [...]
> 
> 6.  BGP Error Handling.
> 
> [...]
> 
>    The phrase "the BGP connection is closed" means that the transport
>    protocol connection has been closed, the associated Adj-RIB-In has
>    been cleared, and that all resources for that BGP connection have
>    been deallocated. Entries in the Loc-RIB associated with the remote
>    peer are marked as invalid...   
> 
> [...]
> 
> 6.1 Multiple Networks Per Message
> 
> 
>    The BGP protocol allows for multiple address prefixes with the same
>    AS path and next-hop gateway to be specified in one message. Making
>    use of this capability is highly recommended. With one address prefix
>    per message there is a substantial increase in overhead in the
>    receiver. Not only does the system overhead increase due to the
>    reception of multiple messages, but the overhead of scanning the
>    Loc-RIB for updates to BGP peers and other routing protocols
>    (and sending the associated messages) is incurred multiple times as
>    well. One method of building messages containing many address
>    prefixes per AS path and gateway from the Loc-RIB that is not
>    organized per AS path is to build many messages as the Loc-RIB
>    is scanned. As each address prefix is processed, a message for the
>    associated AS path and gateway is allocated, if it does not exist,
>    and the new address prefix is added to it.  If such a message exists,
>    the new address prefix is just appended to it. If the message lacks
>    the space to hold the new address prefix, it is transmitted, a new
>    message is allocated, and the new address prefix is inserted into the
>    new message. When the entire Loc-RIB has been scanned, all
>    allocated messages are sent and their resources released.  Maximum
>    compression is achieved when all  the destinations covered by the
>    address prefixes share a gateway and common path attributes, making
>    it possible to send many address prefixes in one 4096-byte message.
>    When peering with a BGP implementation that does not compress
>    multiple address prefixes into one message, it may be necessary to
>    take steps to reduce the overhead from the flood of data received
>    when a peer is acquired or a significant network topology change
>    occurs. One method of doing this is to limit the rate of updates.
>    This will eliminate the redundant scanning of the Loc-RIB to
> 
> [...]
> 
> 
> 9.1.2 Phase 2: Route Selection
> 
> [...]
> 
>    If the NEXT_HOP attribute of a BGP route is not resolvable or it would
>    become unresolvable if the route was installed in the routing table,
>    the BGP route should be excluded from the Phase 2 decision function
>    (see Section 9.1.2.1).
> 
> [...]
> 
>    For each set of destinations for which a feasible route exists in the
>    Adj-RIBs-In, the local BGP speaker shall identify the route that has:
> 
>       a) the highest degree of preference of any route to the same set
>       of destinations, or
> 
>       b) is the only route to that destination, or
> 
>       c) is selected as a result of the Phase 2 tie breaking rules
>       specified in 9.1.2.1.
> 
> //                 ^^^^^^^->"9.1.2.2"
> 
> 
> [...]
> 
>    The local speaker SHALL then install that route in the Loc-RIB,
>    replacing any route to the same destination that is currently being
>    held in the Loc-RIB. If the new BGP route is installed in the Routing
>    Table (as a result of the local policy decision), care must be taken
>    to ensure that invalid BGP routes to the same destination are removed
>    from the Routing Table. Whether or not the new route replaces an already
>    existing non-BGP route in the routing table depends on the policy
>    configured on the BGP speaker.
> 
>    The local speaker MUST determine the immediate next hop to the address
>    depicted by the NEXT_HOP attribute of the selected route by performing
>    a best matching route lookup in the Routing Table and selecting one
>    of the possible paths (if multiple best paths to the same prefix are
>    available). If the route to the address depicted by the NEXT_HOP
>    attribute changes such that the immediate next hop or the IGP cost to
>    the NEXT_HOP (if the NEXT_HOP is resolved through an IGP route) changes,
>    route selection should be recalculated as specified above.
> 
>    Notice that even though BGP routes do not have to be installed in
>    the Routing Table with the immediate next hop(s), implementations
>    must take care that before any packets are forwarded along a BGP
>    route, it's associated NEXT_HOP address is resolved to the immediate
>    (directly connected) next-hop address and this address (or multiple
>    addresses) is finally used for actual packet forwarding.
> 
>    Unfeasible routes SHALL be removed from the Loc-RIB and the routing
>    table. However, corresponding unfeasible routes MUST be kept in
>    the Adj-RIBs-In.
> 
> 9.1.2.1 Route Resolvability Condition
> 
>    As indicated in Section 9.1.2, BGP routers should exclude unresolvable
>    routes from the Phase 2 decision. This ensures that only valid routes
>    are installed in Loc-RIB and the Routing Table.
> 
>    The route resolvability condition is defined as follows.
> 
>       1.  A route Rte1, referencing only the intermediate network address,
>           is considered resolvable if the Routing Table contains at least
>           one resolvable route Rte2 that matches Rte1's intermediate network
>           address and is not recursively resolved (directly or indirectly)
>           through Rte1. If multiple matching routes are available, only
>           the longest matching route should be considered.
> 
>       2.  Routes referencing interfaces (with or without intermediate
>           addresses) are considered resolvable if the state of the referenced
>           interface is up and IP processing is enabled on this interface.
> 
>    BGP routes do not refer to interfaces, but can be resolved through the
>    routes in the Routing Table that can be of both types. IGP routes and
>    routes to directly connected networks are expected to specify the
>    outbound interface.
> 
>    Note that a BGP route is considered unresolvable not only in situations
>    where the router's Routing Table contains no route matching the BGP
>    route's NEXT_HOP. Mutually recursive routes (routes resolving each other
>    or themselves), also fail the resolvability check.
> 
>    It is also important that implementations do not consider feasible
>    routes that would become unresolvable if they were installed in
>    the Routing Table even if their NEXT_HOPs are resolvable using the
>    current contents of the Routing Table. This check ensures that a BGP
>    speaker does not install in the Routing Table routes that will be
>    removed and not used by the speaker. Therefore, in addition to local
>    Routing Table stability, this check also improves behavior of
>    the protocol in the network.
> 
>    Whenever a BGP speaker identifies a route that fails the resolvability
>    check because of mutual recursion, an error message should be logged.
> 
> 
> 9.1.2.1 Breaking Ties (Phase 2)
> 
> ^^^^^^^-->9.1.2.2 ...
> 
> [...]
> 
>       b) Remove from consideration any routes with less-preferred
>       interior cost.  The interior cost of a route is determined by
>       calculating the metric to the next hop for the route using the
>       Routing Table.  If the next hop for a route is reachable, but no
>       cost can be determined, then this step should be should be skipped
>       (equivalently, consider all routes to have equal costs).
> 
> [...]
> 
> 9.1.3   Phase 3: Route Dissemination
> 
> [...]
> 
>    When the updating of the Adj-RIBs-Out and the Routing Table is complete,
>    the local BGP speaker shall run the external update process of 9.2.2.
> 
> 
> 9.2.2 External Updates
> 
>    The external update process is concerned with the distribution of
>    routing information to external peers.  As part of Phase 3 route
>    selection process, the BGP speaker has updated its Adj-RIBs-Out and
>    its Routing Table. 
> 
> [...]
> 
> 
> References
>
> [...]
> 
>    [10] Chen, E., "Route Refresh Capability for BGP-4", RFC 2918,
>    September 2000.

Date: Mon, 11 Jun 2001 17:03:00 -0400
From: Jeffrey Haas <[email-masked]>
To: [email-masked]
Subject: Previous work on TTL on bgp routes?
Sender: [email-masked]

Has there been any previous work on TTL in announced bgp routes?
I'm using TTL in the IP sense - it goes through N hops,
in this case AS's, and is propagated no further.

-- 
Jeff Haas 
NextHop Technologies

From: Yosi Yarchi <[email-masked]>
To: "'Jeffrey Haas'" <[email-masked]>
Cc: [email-masked]
Subject: RE: reposting an e-mail
Date: Mon, 11 Jun 2001 17:57:22 +0300
Sender: [email-masked]

Jeffrey

I'm talking about not to implement the Adj-RIBs-Out at all, not by pointers,
nor by any other methods. On naive implementation, when BGP decides new
path, it have to update the Adj-RIBs-Out and, if Adj-RIBs-Out actually
changed (depending on output filters) - to advertise route to the peer. I'm
talking about shortening the process. When BGP decides new path, it passes
(1) the old path through the output filter, then (2) the new path through
output filter. The delta is sent to peer. 

My question regards not only compliancy to the standard, but also
implementation and performance aspects. I'm afraid that 
(1) I didn't pay attention to SW activities required by current RFC's or
drafts that assumes real existence of Adj-RIBs-Out (=absent of Adj-RIBs-Out
will lead to compensation by over processing, complicated code, etc.)
(2) Implementation complexity will be too high. For example, what happens
when the operator tries to replace route map for specific peer? You have to
maintain temporarily two maps, the old and the new, to re-calculate the
Adj-RIBs-Out for both, to send the deltas to your peer, then cancel the old
one and remain with the new one.

With best regards
Yosi yarchi

--------------------------------------------------
Yosi Yarchi 		http://www.kerenix.com     
Phone: +972-3-9050783	Mobile: 972-58-512783
Fax:  +972-3-9347720	
mailto:[email-masked] 
70 Pinsker St., Petach-Tikva 49221 ISRAEL
--------------------------------------------------

> -----Original Message-----
> From: Jeffrey Haas [mailto:[email-masked]]
> Sent: Friday, June 08, 2001 6:08 PM
> To: [email-masked]
> Cc: [email-masked]
> Subject: Re: reposting an e-mail
> 
> 
> Yosi,
> 
> This is answered in section 3.2 of RFC 1771:
> 
> :    Although the conceptual model distinguishes between Adj-RIBs-In,
> :    Loc-RIB, and Adj-RIBs-Out, this neither implies nor 
> requires that an
> :    implementation must maintain three separate copies of the routing
> :    information. The choice of implementation (for example, 
> 3 copies of
> :    the information vs 1 copy with pointers) is not 
> constrained by the
> :    protocol.
> 
> 
> -- 
> Jeff Haas 
> NextHop Technologies
> 

Date: Sun, 10 Jun 2001 20:05:17 -0500
To: [email-masked]
From: Bill Nickless <[email-masked]>
Subject: Updated draft-nickless-safi-mcast-app-00.txt
Sender: [email-masked]

The rest of this note is an updated version of 
draft-nickless-safi-mcast-app-00.txt which includes changes based on David 
Meyer's comments (Thanks, Dave!)

Under separate cover I have sent this draft to [email-masked] for 
posting.

    Internet Draft                                           B. Nickless
    Document: draft-nickless-safi-mcast-app-00.txt      Argonne National
                                                              Laboratory
    Expires: December 2001                                     June 2001

        Application of Multiprotocol BGP-4 to IPv4 Multicast Routing

Status of this Memo

    This document is an Internet-Draft and is in full conformance
    with all provisions of Section 10 of RFC2026.

    Internet-Drafts are working documents of the Internet Engineering
    Task Force (IETF), its areas, and its working groups.  Note that
    other groups may also distribute working documents as Internet-
    Drafts.

    Internet-Drafts are draft documents valid for a maximum of six
    months and may be updated, replaced, or obsoleted by other documents
    at any time.  It is inappropriate to use Internet-Drafts as
    reference material or to cite them other than as "work in progress."

    The list of current Internet-Drafts can be accessed at
         http://www.ietf.org/ietf/1id-abstracts.txt

    The list of Internet-Draft Shadow Directories can be accessed at
         http://www.ietf.org/shadow.html.

Abstract

    BGP-4 Multiprotocol Extensions [BGP-MP] defines the values for the
    Subsequent Address Family Identifier field carried in the
    MP_REACH_NLRI and MP_UNREACH_NLRI attributes.  This document defines
    how compliant systems should make use of these values for the
    purpose of routing IP multicast traffic.

Table of Contents

    Status of this Memo................................................1
    Abstract...........................................................1
    Conventions used in this document..................................2
    Introduction and Terminology.......................................2
    Application of Multiprotocol BGP-4 SAFI Field......................2
    Security Considerations............................................3
    Acknowledgements...................................................3
    References.........................................................4
    Author's Address...................................................4

    Nickless     Informational - Expires November 2001               1
                   Application of Multiprotocol BGP-4        April 2001
                       To IPv4 Multicast Routing

Overview

    Inter-Autonomous System IPv4 multicast routing is supported by the
    BGPv4 Multiprotocol Extensions.  The topology defined by BGPv4 for
    multicast routing is used for two purposes: guiding the creation and
    direction of source-based multicast distribution trees, and
    assisting MSDP to flood knowledge of active Anycast sources.

Conventions used in this document

    The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
    "SHOULD", "SHOULD NOT", "RECOMMENDED",  "MAY", and "OPTIONAL" in
    this document are to be interpreted as described in RFC-2119
    [RFC2119].

Introduction and Terminology

    IPv4 multicast [MCAST] is an internetwork service that allows IPv4
    datagrams sent from a source to be delivered to more than one
    interested receiver.  That is, a given source sends a packet the
    network with a destination address 224/4 CIDR [CIDR] range.  The
    network transports this packet to all receivers (replicated where
    necessary) that have registered their interest in receiving these
    packets.

Application of Multiprotocol BGP-4 SAFI Field

    The topology of inter-domain IPv4 multicast forwarding is determined
    by BGPv4 [BGPV4] policy, as is IPv4 unicast forwarding.  BGP
    provides reachability information.  Reachability information for
    IPv4 Unicast and IPv4 Multicast prefixes can be advertised
    separately.  (See [MBGP] for details and the definition of Network
    Layer Reachability Information (NLRI) and Subsequent Address Family
    Information (SAFI).) The practical definition of reachability is
    different for IPv4 unicast (NLRI=Unicast, SAFI=1) and IPv4 multicast
    (NLRI=Multicast, SAFI=2).

    In the case of BGP multicast advertisements (NLRI=Multicast,
    SAFI=2), reachability is defined to mean two things:

    First, IPv4 datagrams can be requested (or accepted) from multicast
    sources within the advertised prefix range.  This causes the
    creation of source-rooted multicast distribution trees.

    Such requests MAY be made to the advertised NEXT_HOP by means of the
    PIM Sparse Mode [PIM-SM] protocol.   Any other mutually agreed upon

    Nickless     Informational - Expires December 2001               2
                   Application of Multiprotocol BGP-4        April 2001
                       To IPv4 Multicast Routing

    protocol that supports source-specific traffic requests MAY be used
    to support this requirement.

    Second, the MSDP [MSDP] speaker with the NEXT_HOP address SHOULD
    provide MSDP Source Active messages with the Source Active RP-
    Address field matching the advertised prefix range.  Any other
    mutually agreed upon protocol that supports the spreading of
    knowledge of active Anycast sources MAY be used to support this
    requirement in lieu of, or in addition to, MSDP.

    These two definitions of BGP NLRI=Multicast flow from the original
    use of the Distance Vector Multicast Routing Protocol [DVMRP] for
    IPv4 multicast routing.  (The BGP NLRI=Multicast topology replaces
    the topology functions of DVMRP.)  DVMRP is a "dense" routing
    protocol, which means traffic is flooded outwards from the sources
    to all possible receivers.  In this situation, an IPv4 multicast
    router has to decide which incoming interface may accept IPv4
    datagrams from a given source (to avoid forwarding loops), as well
    as which incoming interface is appropriate for learning about a
    given multicast source.  The current "sparse" inter-domain
    forwarding model (requiring source-specific requests for traffic)
    requires both interpretations of BGP NLRI=Multicast became necessary
    for interoperability with the DVMRP-based model.

    Note that while MSDP is not strictly necessary for Autonomous
    Systems that only support Source Specific Multicast [SSM], MSDP
    depends on the latter interpretation of BGP NLRI=Multicast to avoid
    MSDP SA forwarding loops.  There is a real danger of causing MSDP SA
    forwarding "black holes" unless Autonomous Systems with BGP
    NLRI=Multicast capable peerings also exchange MSDP Source Active
    messages.

    MBGP also supports combined multicast and unicast advertisements
    (SAFI=3).  We define these advertisements to include the multicast
    meanings above in addition to reachability for unicast forwarding.
    This is an optimization for the case where the unicast forwarding
    and multicast acceptance/request/source-discovery topologies are
    congruent.

Security Considerations

    The extensions defined in this document allow BGP speakers to make
    use of reachability information about IPv4 multicast sources.  As
    such, no new security issues are raised beyond those that already
    exist in BGPv4, PIM-SM, and MSDP.

Acknowledgements

    The author would like to thank David Meyer for his useful comments
    and suggestions, which have been incorporated in this Draft.

    Nickless     Informational - Expires December 2001               3
                   Application of Multiprotocol BGP-4        April 2001
                       To IPv4 Multicast Routing

    This work was supported by the Mathematical, Information, and
    Computational Sciences Division subprogram of the Office of Advanced
    Scientific Computing Research, U.S. Department of Energy, under
    Contract W-31-109-Eng-38.

References

    [RFC2119] RFC 2119: Key Words for use in RFCs to Indicate
       Requirement Levels.  S. Bradner.  March 1997.

    [MCAST] RFC 1112: Host extensions for IP multicasting. S.E. Deering.
       Aug-01-1989.

    [CIDR] RFC 1519: Classless Inter-Domain Routing (CIDR): an Address
       Assignment and Aggregation Strategy. V. Fuller, T. Li, J. Yu, K.
       Varadhan. September 1993.

    [BGPV4] RFC 1771: A Border Gateway Protocol 4 (BGP-4).  Y. Rekhter,
       T. Li.  March 1995.

    [MBGP] RFC 2858: Multiprotocol Extensions for BGP-4.  T. Bates, Y.
       Rekhter, R. Chandra, D. Katz.  June 2000.

    [PIM-SM] RFC 2117: Protocol Independent Multicast-Sparse Mode (PIM-
       SM): Protocol Specification.  D. Estrin, D. Farinacci, A. Helmy,
       D. Thaler, S. Deering, M. Handley, V. Jacobson, C. Liu, P.
       Sharma, L. Wei.  June 1997.

    [MSDP] draft-ietf-msdp-spec-09.txt: Multicast Source Discovery
       Protocol (MSDP).  D. Meyer (Editor).  March 2001.

    [DVMRP] RFC 1075: Distance Vector Multicast Routing Protocol.  D.
       Waitzman, C. Partridge, S.E. Deering.  November 1988.

Author's Address

    Bill Nickless
    Argonne National Laboratory
    9700 South Cass Avenue #221     Phone:  +1 630 252 7390
    Argonne, IL 60439               Email:  [email-masked]

    Nickless     Informational - Expires December 2001               4

===
Bill Nickless    http://www.mcs.anl.gov/people/nickless      +1 630 252 7390
PGP:0E 0F 16 80 C5 B1 69 52 E1 44 1A A5 0E 1B 74 F7     [email-masked]

Date: Fri, 08 Jun 2001 14:30:56 -0400
From: [email-masked]
To: [email-masked]
Subject: Valid IP host address
Sender: [email-masked]

The BGP spec requires several fields (BGP identifier in OPEN, next-hop
in UPDATE, and aggregator address in UPDATE) to be a "valid IP (host)
address".

Could we add some more precise language to the spec to define more
precisely what constitutes a valid host address. For example:
a) Is a class D or class E address a "valid IP host address"?
b) Is a loopback address (e.g. 127.0.0.1) a "valid IP host address"?
c) Is network zero (e.g. 0.0.0.1) a "valid IP host address"?

Also, what should a BGP implementation do when it receives an aggregator
address which is not a valid IP address? Send a notification? 

__________________________________________________________________
Get your own FREE, personal Netscape Webmail account today at http://webmail.netscape.com/

Date: Fri, 8 Jun 2001 11:08:01 -0400
From: Jeffrey Haas <[email-masked]>
To: [email-masked]
Cc: [email-masked]
Subject: Re: reposting an e-mail
Sender: [email-masked]

Yosi,

This is answered in section 3.2 of RFC 1771:

:    Although the conceptual model distinguishes between Adj-RIBs-In,
:    Loc-RIB, and Adj-RIBs-Out, this neither implies nor requires that an
:    implementation must maintain three separate copies of the routing
:    information. The choice of implementation (for example, 3 copies of
:    the information vs 1 copy with pointers) is not constrained by the
:    protocol.

-- 
Jeff Haas 
NextHop Technologies

From: "Tsillas, Jim" <[email-masked]>
To: "'Yakov Rekhter'" <[email-masked]>, [email-masked]
Cc: [email-masked]
Subject: RE: reposting an e-mail
Date: Fri, 8 Jun 2001 10:59:13 -0400 
Sender: [email-masked]

yes

-----Original Message-----
From: Yakov Rekhter [mailto:[email-masked]]
Sent: Friday, June 08, 2001 11:00 AM
To: [email-masked]
Cc: [email-masked]
Subject: reposting an e-mail

Folks,

I've been asked to repost the following.

Yakov.
------- Forwarded Message
From: [email-masked]
To: Idr (E-mail); [email-masked]'
Sent: 6/6/2001 11:40 AM
Subject: BGP4 implementation of Adj-RIBs-Out

Hi

I have question regarding possible BGP4 implementation:

Anlyzing BGP, I found that:

(1) Implementor is required to implement Adj-RIBs-Out for each peer, to
hold
advertised routes (RFC1771)

(2) The Adj-RIBs-Out data base is not really involved in update/withdraw
operations, because when BGP decides to prefer another path in it's
Loc-RIB,
it should update Adj-RIBs-Out in accordance with route-maps etc., and
send
update to all peers. The route-map filter is applied anyway, and the
update
is sent anyway. So, the Adj-RIBs-Out does not really participating in
the
update/withdraw process

(3) The Adj-RIBs-Out data base participates in special operations. For
example, when peer request for refresh, or when the operator request to
show
the advertised routes of specific peer. Both operations are not
performance
critical, because they are not frequent and because they could be
executed
in lower priority than other BGP activities, thus not degrading the
forwarding performance.

(4) The Adj-RIBs-Out consumes lot of memory, because it is per peer.

My question is: is it possible to implement BGP4 without real
Adj-RIBs-Out,
and instead maintain virtual Adj-RIBs-Out, where each entry is generated
upon demand from Loc-RIB and the relevant route-maps (something like
/proc
file system in linux)? 

With best regards
Yosi yarchi

- --------------------------------------------------
Yosi Yarchi 		http://www.kerenix.com     
Phone: +972-3-9050783	Mobile: 972-58-512783
Fax:  +972-3-9347720	
mailto:[email-masked] 
70 Pinsker St., Petach-Tikva 49221 ISRAEL
- --------------------------------------------------

------- End of Forwarded Message

To: [email-masked]
Cc: [email-masked]
Subject: reposting an e-mail
Date: Fri, 08 Jun 2001 08:00:27 -0700
From: Yakov Rekhter <[email-masked]>
Sender: [email-masked]

Folks,

I've been asked to repost the following.

Yakov.
------- Forwarded Message
From: [email-masked]
To: Idr (E-mail); [email-masked]'
Sent: 6/6/2001 11:40 AM
Subject: BGP4 implementation of Adj-RIBs-Out

Hi

I have question regarding possible BGP4 implementation:

Anlyzing BGP, I found that:

(1) Implementor is required to implement Adj-RIBs-Out for each peer, to
hold
advertised routes (RFC1771)

(2) The Adj-RIBs-Out data base is not really involved in update/withdraw
operations, because when BGP decides to prefer another path in it's
Loc-RIB,
it should update Adj-RIBs-Out in accordance with route-maps etc., and
send
update to all peers. The route-map filter is applied anyway, and the
update
is sent anyway. So, the Adj-RIBs-Out does not really participating in
the
update/withdraw process

(3) The Adj-RIBs-Out data base participates in special operations. For
example, when peer request for refresh, or when the operator request to
show
the advertised routes of specific peer. Both operations are not
performance
critical, because they are not frequent and because they could be
executed
in lower priority than other BGP activities, thus not degrading the
forwarding performance.

(4) The Adj-RIBs-Out consumes lot of memory, because it is per peer.

My question is: is it possible to implement BGP4 without real
Adj-RIBs-Out,
and instead maintain virtual Adj-RIBs-Out, where each entry is generated
upon demand from Loc-RIB and the relevant route-maps (something like
/proc
file system in linux)? 

With best regards
Yosi yarchi

- --------------------------------------------------
Yosi Yarchi 		http://www.kerenix.com     
Phone: +972-3-9050783	Mobile: 972-58-512783
Fax:  +972-3-9347720	
mailto:[email-masked] 
70 Pinsker St., Petach-Tikva 49221 ISRAEL
- --------------------------------------------------

------- End of Forwarded Message

Date: Thu, 7 Jun 2001 21:35:11 -0400 (EDT)
From: Mark Mentovai <[email-masked]>
To: <[email-masked]>
Cc: IDR <[email-masked]>
Subject: Re: =?gb2312?b?tPC4tDpSRTo=?=
Sender: [email-masked]

[email-masked] wrote:
>maybe i have not expressed my question clearly.
>
>I just want to know while we do the connection collision detection,
>should we know the state of PEER speaker,and if we should know peer
>speaker's state,how can we know?

A speaker always knows the local state of its own connections.

When a speaker receives an OPEN message, it is guaranteed that the sending
neighbor is in the OpenSent or OpenConfirm state.

Since a speaker's own ID is always known, and a neighbor's ID is known once
an OPEN is received, implementation of the collision avoidance algorithm is
now trivial.

Mark

Date: Mon, 4 Jun 2001 12:45:09 -0700
From: Alex Zinin <[email-masked]>
Reply-To: Alex Zinin <[email-masked]>
Organization: Nexsi Systems
To: "Shekhar, Ravi" <[email-masked]>
Cc: [email-masked]
Subject: Re: Proposed text for 9.1.2
Sender: [email-masked]

Ravi,

  I'll change this part as Yakov suggested.
  I guess this should satisfy all parties.

-- 
Alex Zinin

Monday, June 04, 2001, 8:06:09 AM, Shekhar, Ravi wrote:

>> > have revised opinion here:
>> > It MUST be MUST
>> 
>> Yes, I think this is the better way.
>> It is also consistent with Section 3:
>> 
>> >                                                  BGP does 
>> not require
>> >    periodic refresh of the entire BGP routing table.  
>> Therefore, a BGP
>> >    speaker must retain the current version of the entire BGP routing
>> >    tables of all of its peers for the duration of the connection.
>> 

> Alex, that simply means this does not hold true anymore.. unless you 
> are insistent on not considering route-refresh, which I dont
> see any reason not to consider because route-refresh is already 
> an RFC..

> - Ravi Shekhar.

Date: Fri, 1 Jun 2001 17:31:37 -0400
From: Jeffrey Haas <[email-masked]>
To: [email-masked]
Subject: Re: NULLROUTE Community
Sender: [email-masked]

On Fri, Jun 01, 2001 at 01:51:36PM -0600, Danny McPherson wrote:
> > Often, it would be useful for one BGP speaker to inform another that it has
> > no interest in receiving traffic addressed to certain destinations.  Such a
> > capability could be used to effectively create destination-based access
> > lists on a BGP neighbor.

This topic was brought up on the NANOG list under the topic of
Subject: How does one make not playing nice with each other scale? (Was: net.terrorism)

> Isn't this the purpose of Enke's ORF stuff (i.e., draft-ietf-idr-route-filter-0
> 3.txt)?

This seems to be a poor fit for ORF.  Unless you mean "I don't want
routes with holes, so don't send me routes that have the community
that says there is a hole present".

> filtering policy, but the same mechanism could be 
> employed (though I'm not sure BGP is the right place 
> for it).

Well, you want to apply it for forwarding purposes, and passing
around reachability is what we're doing.

> The difference between the above and what you propose here is 
> that there'd be less scope of attribute advertisement.  Then again,
> it would still likely be a matter of policy within the local domain, 
> and could be broken if folks forget to configure things like BGP
> "send-community" in some implementations.  These, among other related
> reasons, are why folks weren't to keen on the AVOID_ME & PREFER_ME
> communities ID a while back either.

If we're going to pass around this stuff in bgp, it probably should
be in mpbgp since its a type of "non-reachability".  Perhaps a new
SAFI.

If we were to bother with this stuff, I'd want to see the following
behavior:

When a route is present that is a more specific and shares common
attributes as a less specific route,
And that route is tagged as a forwarding "hole",
And this hole isn't already covered by something that is reachable
and is more specific than the aggregate with the hole:
Find the next best route
Synthesize a route in your RIB/FIB for the hole that has the
next best route's properties.
Install *that* in your FIB, if policy says so.
NEVER PROPAGATE IT.

One of the arguments from the NANOG thread was that passing around
unreachability doesn't likely scale.  Storing it in the IRR, maybe,
but not passing it around in bgp.

> -danny

-- 
Jeff Haas 
NextHop Technologies

To: IDR <[email-masked]>
From: Danny McPherson <[email-masked]>
Reply-To: [email-masked]
Subject: Re: NULLROUTE Community 
Date: Fri, 01 Jun 2001 13:51:36 -0600
Sender: [email-masked]

> Often, it would be useful for one BGP speaker to inform another that it has
> no interest in receiving traffic addressed to certain destinations.  Such a
> capability could be used to effectively create destination-based access
> lists on a BGP neighbor.

Isn't this the purpose of Enke's ORF stuff (i.e., draft-ietf-idr-route-filter-0
3.txt)?  Given, it just
applies to routing information policy versus packet 
filtering policy, but the same mechanism could be 
employed (though I'm not sure BGP is the right place 
for it).
 
> There are many ways that this mechanism could be incorporated into BGP, but
> perhaps one of the cleanest and simplest is to use the existing COMMUNITIES
> attribute framework.  A NULLROUTE community could be defined and used to
> inform others that traffic sent to the networks given in the UPDATE
> message's NLRI is not desired and should be discarded immediately.

Besides ORFish (no pun intended :-) stuff, there are implementations
that provide capabilities more/less identical to this already (see 
Cisco's IOS Essentials WP wrt DOS/NH/CEF for a hint of what I'm 
referring to).

> Obviously, this can be configured on many existing routers, but I feel that
> everyone can benefit from a globally consistent system that does not require
> excess configuration.  As such, it is my opinion that optimum utility could
> be realized if the NULLROUTE community was defined as a well-known one.

The difference between the above and what you propose here is 
that there'd be less scope of attribute advertisement.  Then again,
it would still likely be a matter of policy within the local domain, 
and could be broken if folks forget to configure things like BGP
"send-community" in some implementations.  These, among other related
reasons, are why folks weren't to keen on the AVOID_ME & PREFER_ME
communities ID a while back either.

What Cisco's done is interesting and accomplishes this w/o touching 
the protocol. 

-danny

Date: Fri, 1 Jun 2001 15:56:36 -0400 (EDT)
From: Mark Mentovai <[email-masked]>
To: Ramana Gollamudi <[email-masked]>
Cc: IDR <[email-masked]>
Subject: RE: NULLROUTE Community
Sender: [email-masked]

Ramana Gollamudi wrote:
>Why would a BGP speaker advertise a route (with or without a NULL community)
>if it is not going to forward any traffic to that destination in the first
>place?

I'm sorry, this should have been more clear.  This is intended to be used
when the speaker is already advertising a less-specific route for the
destination, but does not wish to receive the more-specific traffic.  For
example, it provides an alternative to contacting an upstream provider to
build or modify an access list when a host or network is receiving unwanted
traffic (as in the case of a denial-of-service attack.)  It also has the
advantage of being able to propagate itself to other BGP speakers, so the
unwanted traffic traverses the fewest number of links and devices possible.

Mark

Date: Fri, 1 Jun 2001 14:59:18 -0400 (EDT)
From: Mark Mentovai <[email-masked]>
To: IDR <[email-masked]>
Subject: NULLROUTE Community
Sender: [email-masked]

Is there any interest in exploring a way to communicate null routes from
within BGP?

Often, it would be useful for one BGP speaker to inform another that it has
no interest in receiving traffic addressed to certain destinations.  Such a
capability could be used to effectively create destination-based access
lists on a BGP neighbor.

There are many ways that this mechanism could be incorporated into BGP, but
perhaps one of the cleanest and simplest is to use the existing COMMUNITIES
attribute framework.  A NULLROUTE community could be defined and used to
inform others that traffic sent to the networks given in the UPDATE
message's NLRI is not desired and should be discarded immediately.

Obviously, this can be configured on many existing routers, but I feel that
everyone can benefit from a globally consistent system that does not require
excess configuration.  As such, it is my opinion that optimum utility could
be realized if the NULLROUTE community was defined as a well-known one.

If there is interest, I will prepare an I-D.  I would appreciate the group's
comments and support.

Mark

To: Alex Zinin <[email-masked]>
Cc: [email-masked]
Subject: Re: Proposed text for 9.1.2 
Date: Fri, 01 Jun 2001 10:56:27 -0700
From: Yakov Rekhter <[email-masked]>
Sender: [email-masked]

Alex,

> Yes, I think this is the better way.
> It is also consistent with Section 3:
> 
> >                                                  BGP does not require
> >    periodic refresh of the entire BGP routing table.  Therefore, a BGP
> >    speaker must retain the current version of the entire BGP routing
> >    tables of all of its peers for the duration of the connection.
> 
> 
> OK, below is the result of an attempt to address the comments
> made on the list. I changed wording in Section 3 to not refer
> to the routing table, since the terms are not defined yet.
> I also introduced the term "Routing Table" in 3.2 and checked
> usage of "routing table" and "forwarding information table"
> to use the right term (either Loc-RIB or Routing Table).
> I also added some text in 9.1.2 to clarify usage of
> the intermediate next-hop, changed some wording, etc.
> The pseudo-code is removed from the subsection describing
> the route resolvability condition.
> 
> Comments are welcome.
> 
> ---
> Alex Zinin.
> 
> 
> 
> 3.  Summary of Operation
> 
>    Two systems form a transport protocol connection between one another.
>    They exchange messages to open and confirm the connection parameters.
>    The initial data flow is the entire BGP routing table.  Incremental
>    updates are sent as the routing tables change.  BGP does not require
>    periodic refresh of the entire BGP routing table.  Therefore, a BGP
>    speaker must retain the current version of the entire BGP routing
>    tables of all of its peers for the duration of the connection...
> 
> /* Change to:
>  
>    Two systems form a transport protocol connection between one another.
>    They exchange messages to open and confirm the connection parameters.
>    The initial data flow is complete BGP routing information. Incremental
>    updates are sent as the routing information changes. BGP does not
>    require periodic refresh of all BGP routes. Therefore, a BGP speaker
>    must retain the current version of the entire set of BGP routes of all
>    of its peers for the duration of the connection...
> */

If a BGP speaker and its peer use BGP Refresh (rfc2918), then the
last statement in the above is not correct. Therefore I would suggest
to change it as follows:

     Therefore, unless a BGP speaker and its peer use BGP Route
     Refresh [RFC2918], the speaker must retain the current version
     of the entire set of BGP routes received from the peer for the
     duration of the connection with the peer.

Yakov.

Date: Fri, 1 Jun 2001 11:47:05 -0400
From: Jeffrey Haas <[email-masked]>
To: ks s <[email-masked]>
Cc: [email-masked]
Subject: Re: bgp auth.
Sender: [email-masked]

On Thu, May 31, 2001 at 06:53:26PM -0000, ks s wrote:
> 1. are there any implementations, which support the auth.
>    as explained by rfc1771

None that I'm aware of.
Theoretically, one could use it for such a thing as a message
authentication digest for individual bgp messages.

> 2. whats the use of auth. in open packet, in lieu of rfc2385

If anyone knows of any use for this field, please let me know.
It'll be going into the next BGP mib.

-- 
Jeff Haas 
NextHop Technologies

Date: Thu, 31 May 2001 21:18:38 -0700
From: Alex Zinin <[email-masked]>
Reply-To: Alex Zinin <[email-masked]>
Organization: Nexsi Systems
To: [email-masked]
Cc: [email-masked]
Subject: Re: Proposed text for 9.1.2
Sender: [email-masked]

> have revised opinion here:
> It MUST be MUST

Yes, I think this is the better way.
It is also consistent with Section 3:

>                                                  BGP does not require
>    periodic refresh of the entire BGP routing table.  Therefore, a BGP
>    speaker must retain the current version of the entire BGP routing
>    tables of all of its peers for the duration of the connection.

OK, below is the result of an attempt to address the comments
made on the list. I changed wording in Section 3 to not refer
to the routing table, since the terms are not defined yet.
I also introduced the term "Routing Table" in 3.2 and checked
usage of "routing table" and "forwarding information table"
to use the right term (either Loc-RIB or Routing Table).
I also added some text in 9.1.2 to clarify usage of
the intermediate next-hop, changed some wording, etc.
The pseudo-code is removed from the subsection describing
the route resolvability condition.

Comments are welcome.

---
Alex Zinin.

3.  Summary of Operation

   Two systems form a transport protocol connection between one another.
   They exchange messages to open and confirm the connection parameters.
   The initial data flow is the entire BGP routing table.  Incremental
   updates are sent as the routing tables change.  BGP does not require
   periodic refresh of the entire BGP routing table.  Therefore, a BGP
   speaker must retain the current version of the entire BGP routing
   tables of all of its peers for the duration of the connection...

/* Change to:
 
   Two systems form a transport protocol connection between one another.
   They exchange messages to open and confirm the connection parameters.
   The initial data flow is complete BGP routing information. Incremental
   updates are sent as the routing information changes. BGP does not
   require periodic refresh of all BGP routes. Therefore, a BGP speaker
   must retain the current version of the entire set of BGP routes of all
   of its peers for the duration of the connection...
*/

[...]

3.1 Routes: Advertisement and Storage

[...]
      - Routes are stored in the Routing Information Bases (RIBs):
      namely, the Adj-RIBs-In, the Loc-RIB, and the Adj-RIBs-Out. Routes
      that will be advertised to other BGP speakers must be present in
      the Adj-RIB-Out; routes that will be used by the local BGP speaker
      must be present in the Loc-RIB, and the next hop for each of these
      routes must be present in the local BGP speaker's forwarding
      information base; and routes that are received from other BGP
      speakers are present in the Adj-RIBs-In.

/* Change to
                                   ...and the next hop for each of these
      routes must be present in the local BGP speaker's Routing Table...
*/

3.2 Routing Information Bases

[...]

   In summary, the Adj-RIBs-In contain unprocessed routing information
   that has been advertised to the local BGP speaker by its peers; the
   Loc-RIB contains the routes that have been selected by the local BGP
   speaker's Decision Process; and the Adj-RIBs-Out organize the routes
   for advertisement to specific peers by means of the local speaker's
   UPDATE messages.

/* Add:

   Routing information that the router uses to forward packets
   (or to construct the forwarding table that is used for packet
   forwarding) is maintained in the Routing Table. The Routing Table
   accumulates routes to directly connected networks, static routes,
   routes learnt from the IGP protocols, and routes learnt from BGP.
   Whether or not a specific BGP route should be installed in
   the routing table, and whether a BGP route should override a route
   to the same destination installed by another source is a local
   policy decision, not specified in this document. Besides actual
   packet forwarding, the Routing Table is used for resolution of
   the next-hop addresses specified in BGP updates (see Section 9.1.2).
*/

[...]

6.  BGP Error Handling.

[...]

   The phrase "the BGP connection is closed" means that the transport
   protocol connection has been closed and that all resources for that
   BGP connection have been deallocated.  Routing table entries
   associated with the remote peer are marked as invalid...

/* Change to:

   The phrase "the BGP connection is closed" means that the transport
   protocol connection has been closed, the associated Adj-RIB-In has
   been cleared, and that all resources for that BGP connection have
   been deallocated. Entries in the Loc-RIB associated with the remote
   peer are marked as invalid...   
*/

6.1 Multiple Networks Per Message

   The BGP protocol allows for multiple address prefixes with the same
   AS path and next-hop gateway to be specified in one message. Making
   use of this capability is highly recommended. With one address prefix
   per message there is a substantial increase in overhead in the
   receiver. Not only does the system overhead increase due to the
   reception of multiple messages, but the overhead of scanning the
   routing table for updates to BGP peers and other routing protocols
// ^^^^^^^^^^^^^^ change to "Loc-RIB"

   (and sending the associated messages) is incurred multiple times as
   well. One method of building messages containing many address
   prefixes per AS path and gateway from a routing table that is not
//                                         ^^^^^^^^^^^^^->"Loc-RIB"

   organized per AS path is to build many messages as the routing table
//                                             "Loc-RIB"<-^^^^^^^^^^^^^

   is scanned. As each address prefix is processed, a message for the
   associated AS path and gateway is allocated, if it does not exist,
   and the new address prefix is added to it.  If such a message exists,
   the new address prefix is just appended to it. If the message lacks
   the space to hold the new address prefix, it is transmitted, a new
   message is allocated, and the new address prefix is inserted into the
   new message. When the entire routing table has been scanned, all
//                              ^^^^^^^^^^^^^->"Loc-RIB"

   allocated messages are sent and their resources released.  Maximum
   compression is achieved when all  the destinations covered by the
   address prefixes share a gateway and common path attributes, making
   it possible to send many address prefixes in one 4096-byte message.
   When peering with a BGP implementation that does not compress
   multiple address prefixes into one message, it may be necessary to
   take steps to reduce the overhead from the flood of data received
   when a peer is acquired or a significant network topology change
   occurs. One method of doing this is to limit the rate of updates.
   This will eliminate the redundant scanning of the routing table to
//                                        "Loc-RIB"<-^^^^^^^^^^^^^

[...]

9.1.2 Phase 2: Route Selection

[...]

   If the NEXT_HOP attribute of a BGP route depicts an address to which
   the local BGP speaker doesn't have a route in its Loc-RIB, the BGP
   route should be excluded from the Phase 2 decision function.

/* Change to:

   If the NEXT_HOP attribute of a BGP route is not resolvable or it would
   become unresolvable if the route was installed in the routing table,
   the BGP route should be excluded from the Phase 2 decision function
   (see Section 9.1.2.1).
*/

[...]

   For each set of destinations for which a feasible route exists in the
   Adj-RIBs-In, the local BGP speaker shall identify the route that has:

      a) the highest degree of preference of any route to the same set
      of destinations, or

      b) is the only route to that destination, or

      c) is selected as a result of the Phase 2 tie breaking rules
      specified in 9.1.2.1.
//                 ^^^^^^^->"9.1.2.2"

[...]

   The local speaker SHALL then install that route in the Loc-RIB,
   replacing any route to the same destination that is currently being
   held in the Loc-RIB. The local speaker MUST determine the immediate
   next hop to the address depicted by the NEXT_HOP attribute of the
   selected route by performing a lookup in the IGP and selecting one of
   the possible paths in the IGP.  This immediate next hop MUST be used
   when installing the selected route in the Loc-RIB.  If the route to
   the address depicted by the NEXT_HOP attribute changes such that the
   immediate next hop changes, route selection should be recalculated as
   specified above.

/* Change to:

   The local speaker SHALL then install that route in the Loc-RIB,
   replacing any route to the same destination that is currently being
   held in the Loc-RIB. If the new BGP route is installed in the Routing
   Table (as a result of the local policy decision), care must be taken
   to ensure that invalid BGP routes to the same destination are removed
   from the Routing Table. Whether or not the new route replaces an already
   existing non-BGP route in the routing table depends on the policy
   configured on the BGP speaker.

   The local speaker MUST determine the immediate next hop to the address
   depicted by the NEXT_HOP attribute of the selected route by performing
   a best matching route lookup in the Routing Table and selecting one
   of the possible paths (if multiple best paths to the same prefix are
   available). If the route to the address depicted by the NEXT_HOP
   attribute changes such that the immediate next hop or the IGP cost to
   the NEXT_HOP (if the NEXT_HOP is resolved through an IGP route) changes,
   route selection should be recalculated as specified above.

   Notice that even though BGP routes do not have to be installed in
   the Routing Table with the immediate next hop(s), implementations
   must take care that before any packets are forwarded along a BGP
   route, it's associated NEXT_HOP address is resolved to the immediate
   (directly connected) next-hop address and this address (or multiple
   addresses) is finally used for actual packet forwarding.
*/

   Unfeasible routes shall be removed from the Loc-RIB, and
   corresponding unfeasible routes shall then be removed from the Adj-
   RIBs-In.

/* Change to:

   Unfeasible routes SHALL be removed from the Loc-RIB and the routing
   table. However, corresponding unfeasible routes MUST be kept in
   the Adj-RIBs-In.
*/

// Section on route resolvability:

9.1.2.1 Route Resolvability Condition

   As indicated in Section 9.1.2, BGP routers should exclude unresolvable
   routes from the Phase 2 decision. This ensures that only valid routes
   are installed in Loc-RIB and the Routing Table.

   The route resolvability condition is defined as follows.

      1.  A route Rte1, referencing only the intermediate network address,
          is considered resolvable if the Routing Table contains at least
          one resolvable route Rte2 that matches Rte1's intermediate network
          address and is not recursively resolved (directly or indirectly)
          through Rte1.

      2.  Routes referencing interfaces (with or without intermediate
          addresses) are considered resolvable if the state of the referenced
          interface is up and IP processing is enabled on this interface.

   BGP routes do not refer to interfaces, but can be resolved through the
   routes in the Routing Table that can be of both types. IGP routes and
   routes to directly connected networks are expected to specify the
   outbound interface.

   Note that a BGP route is considered unresolvable not only in situations
   where the router's Routing Table contains no route matching the BGP
   route's NEXT_HOP. Mutually recursive routes (routes resolving each other
   or themselves), also fail the resolvability check.

   It is also important that implementations do not consider feasible
   routes that would become unresolvable if they were installed in
   the Routing Table even if their NEXT_HOPs are resolvable using the
   current contents of the Routing Table. This check ensures that a BGP
   speaker does not install in the Routing Table routes that will be
   removed and not used by the speaker. Therefore, in addition to local
   Routing Table stability, this check also improves behavior of
   the protocol in the network.

   Whenever a BGP speaker identifies a route that fails the resolvability
   check because of mutual recursion, an error message should be logged.

9.1.2.1 Breaking Ties (Phase 2)

/* Change to:

9.1.2.2 ...
*/

9.1.3   Phase 3: Route Dissemination

[...]

   When the updating of the Adj-RIBs-Out and the Forwarding Information
   Base (FIB) is complete, the local BGP speaker shall run the external
   update process of 9.2.2.

/* Change to

   When the updating of the Adj-RIBs-Out and the Routing Table is complete,
   the local BGP speaker shall run the external update process of 9.2.2.
*/

9.2.2 External Updates

   The external update process is concerned with the distribution of
   routing information to external peers.  As part of Phase 3 route
   selection process, the BGP speaker has updated its Adj-RIBs-Out and
   its Forwarding Table. [...]
//     ^^^^^^^^^^^^^^^^->"Routing Table"