Subject: [Idr] Comparison between session-reset and treat-as-withdraw

From: Jie Dong <[email-masked]>
Date: Fri, 14 Sep 2012 05:21:22 +0000

Hi all,

Since the bgp-error-handling draft proposes to replace session-reset with treat-as-withdraw in some error cases, here I'd like to briefly compare these two mechanisms.

Basically, we could take session-reset as error-handling mechanism at session level, while treat-as-withdraw is error-handling at route level.

Impacts:
Session-reset impacts all the routes exchanged over the session, while treat-as-withdraw impacts only the routes carried in the malformed Update.

Recovery:
With session-reset, BGP speaker could recover the routes by re-establishing the session, possibly with some back-off mechanism; while currently treat-as-withdraw does not have corresponding recovery mechanism at route level.

Diagnosing:
With session-reset, the error codes/subcodes in Notification can be used for diagnosing; with treat-as-withdraw, the error could be locally logged, but Notification will not be sent. The mechanisms defined in operational-message draft may be used to provide diagnosing info.

Best regards,
Jie

---

From: Enke Chen <[email-masked]>
Date: Thu, 13 Sep 2012 23:59:18 -0700

Jie,

In terms of recovery, it has been shown that the session reset and 
reestablishment usually do not eliminate malformed updates.  Instead the 
session would often get into the cycle of reset and reestablishment 
until the culprit route is removed from the routing system by the 
operator manually.

-- Enke

On 9/13/12 10:21 PM, Jie Dong wrote:
>
> Hi all,
>
> Since the bgp-error-handling draft proposes to replace session-reset 
> with treat-as-withdraw in some error cases, here I'd like to briefly 
> compare these two mechanisms.
>
> Basically, we could take session-reset as error-handling mechanism at 
> session level, while treat-as-withdraw is error-handling at route level.
>
> Impacts:
>
> Session-reset impacts all the routes exchanged over the session, while 
> treat-as-withdraw impacts only the routes carried in the malformed Update.
>
> Recovery:
>
> With session-reset, BGP speaker could recover the routes by 
> re-establishing the session, possibly with some back-off mechanism; 
> while currently treat-as-withdraw does not have corresponding recovery 
> mechanism at route level.
>
> Diagnosing:
>
> With session-reset, the error codes/subcodes in Notification can be 
> used for diagnosing; with treat-as-withdraw, the error could be 
> locally logged, but Notification will not be sent. The mechanisms 
> defined in operational-message draft may be used to provide diagnosing 
> info.
>
> Best regards,
>
> Jie
>
>
>
> _______________________________________________
> Idr mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/idr

---

From: Jie Dong <[email-masked]>
Date: Fri, 14 Sep 2012 09:52:20 +0000

Hi Enke,

Agree that in some cases session reset may not solve the problem. But in many field deployments when error occurs the session will be reset and re-established, usually with some back-off mechanism.

If there is no session re-establishment after the session terminated, we may lose the chance of recovering the session automatically in some error cases. The back-off mechanism can help in preventing frequent session re-establishment.

Best regards,
Jie

From: Enke Chen [mailto:[email-masked]]
Sent: Friday, September 14, 2012 2:59 PM
To: Jie Dong
Cc: [email-masked]; Enke Chen
Subject: Re: [Idr] Comparison between session-reset and treat-as-withdraw

Jie,

In terms of recovery, it has been shown that the session reset and reestablishment usually do not eliminate malformed updates.  Instead the session would often get into the cycle of reset and reestablishment until the culprit route is removed from the routing system by the operator manually.

-- Enke

Impacts:

Recovery:

Diagnosing:

Best regards,
Jie

Idr mailing list

[email-masked]<mailto:[email-masked]>

---

From: Randy Bush <[email-masked]>
Date: Fri, 14 Sep 2012 20:15:33 +0900

> reestablishment usually do not eliminate malformed updates.  Instead
> the session would often get into the cycle of reset and
> reestablishment until the culprit route is removed from the routing
> system by the operator manually.

been there.  have coffee cup and tee shirt.  sadly sessions go bouncy
bouncy is not uncommon.

pathological extreme example, earlier this week i watched someone else
have to rboot their very large router to get a stuck route out.

randy

---

From: John Leslie <[email-masked]>
Date: Fri, 14 Sep 2012 11:00:14 -0400

Randy Bush <[email-masked]> wrote:
> [Enke Chen <[email-masked]> wrote:]
>  

   Yes... major pain, and very little information about the actual issue.

   But when the actual issue _is_ at the router getting the NOTIFICATION,
we are "routing around the problem".

   We now are facing cases where the actual issue is elsewhere.

Jie Dong <[email-masked]> wrote:
] 
] If there is no session re-establishment after the session terminated,
] we may lose the chance of recovering the session automatically in
] some error cases.

   By which I guess he means that treat-as-withdraw has no automatic
recovery path. And with that, I agree.

   We may succeed in "routing around the problem" when we use
treat-as-withdraw; but I think we may lack any way to restore the
original route when the actual issue is resolved.

   And that, IMHO, deserves some discussion...

--
John Leslie <[email-masked]>

---

From: Jie Dong <[email-masked]>
Date: Mon, 17 Sep 2012 07:29:05 +0000

Hi John,

> recovery path. And with that, I agree.

Yes, that's exactly what I mean.

>    And that, IMHO, deserves some discussion...

Agreed, after using treat-as-withdraw on some routes, there will be some inconsistency between the sending and receiving BGP speaker for some time, and some user traffic may be affected. Thus similar to session reset, maybe the receiving speaker could try to recover the particular routes with some back-off mechanism before manual intervention.

Best regards,
Jie

> -----Original Message-----
> From: [email-masked] [mailto:[email-masked]] On Behalf Of John
> Leslie
> Sent: Friday, September 14, 2012 11:00 PM
> To: Randy Bush
> Cc: idr wg

---

From: Robert Raszuk <[email-masked]>
Date: Mon, 17 Sep 2012 09:39:21 +0200

All,

Treat-as-withdraw was supposed to be used with bgp-operational-message. 
Using is as default before bgp-operational-message is available is IMHO 
a mistake.

Regards,
R.

---

From: Jie Dong <[email-masked]>
Date: Tue, 18 Sep 2012 06:49:39 +0000

Hi Robert,

Agree that bgp-operational-message could provide detailed diagnostic information which would be quite helpful for diagnosing of treat-as-withdraw events. And some mechanism for automatic route recovery can also be helpful after some important routes are treat-as-withdraw.

Regards,
Jie

> From: [email-masked] [mailto:[email-masked]] On Behalf Of Robert
> Raszuk
> Sent: Monday, September 17, 2012 3:37 PM
> To: Jie Dong
> Cc: John Leslie; Randy Bush; idr wg
> Treat-as-withdraw was supposed to be used with
> bgp-operational-message. Using is as default before
> bgp-operational-message is available is IMHO a mistake.
> 
> Regards,
> R.
> 
> On Mon, Sep 17, 2012 at 9:29 AM, Jie Dong <[email-masked]> wrote:
> > Hi John,
> >
> > Agreed, after using treat-as-withdraw on some routes, there will be some
> inconsistency between the sending and receiving BGP speaker for some time,
> and some user traffic may be affected. Thus similar to session reset, maybe the
> receiving speaker could try to recover the particular routes with some back-off
> mechanism before manual intervention.
> >
> >
> > Best regards,
> > Jie
> >

---

From: <[email-masked]>
Date: Tue, 18 Sep 2012 07:52:48 +0000

Hi Robert, all

>From: Robert Raszuk
>Sent: Monday, September 17, 2012 9:39 AM
>
>All,
>

I think I see what you mean and I agree that this could make sense from a theoretical point of view.
However, from a roadmap / network management POV, IMHO, bgp-error-handling is more important/urgent that bgp-operational-message.

Also, I would distinguish the iBGP and eBGP cases:

- for iBGP, the lack of bgp-operational-message is less of an issue as there are (should be) other existing means to know about the problem. Also IMO bgp-error-handling is crucial on iBGP as we can't afford to have a single error shutting down both redundant iBGP sessions. Moreover, possibly on 10s of PE at the same time. Especially in the absence of a "persistence" safety net.

- for eBGP, bgp-operational-message is indeed of importance, but whatever the order of feature delivery, deploying a local feature (bgp-error-handling) will always be easier and hence faster than a feature requiring interoperability. Especially between ASes. (cf GR vs NSR on eBGP PE-CE sessions).

Bottom line: IMO bgp-error-handling should not wait (and hence have required dependency) for bgp-operational-message.

Regards,
Bruno

>inconsistency between the sending and receiving BGP speaker for some time, and
>some user traffic may be affected. Thus similar to session reset, maybe the

_________________________________________________________________________________________________________________________

Ce message et ses pieces jointes peuvent contenir des informations confidentielles ou privilegiees et ne doivent donc
pas etre diffuses, exploites ou copies sans autorisation. Si vous avez recu ce message par erreur, veuillez le signaler
a l'expediteur et le detruire ainsi que les pieces jointes. Les messages electroniques etant susceptibles d'alteration,
France Telecom - Orange decline toute responsabilite si ce message a ete altere, deforme ou falsifie. Merci.

This message and its attachments may contain confidential or privileged information that may be protected by law;
they should not be distributed, used or copied without authorisation.
If you have received this email in error, please notify the sender and delete this message and its attachments.
As emails may be altered, France Telecom - Orange is not liable for messages that have been modified, changed or falsified.
Thank you.

---

From: "Susan Hares" <[email-masked]>
Date: Thu, 20 Sep 2012 18:58:18 -0400

Robert and Jie:

Could you both review exactly what information you feel is useful to put in
the bgp-operational-message for the treat-as-withdrawl case?

Sue 

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of Jie
Dong
Sent: Tuesday, September 18, 2012 2:50 AM
To: Robert Raszuk
Cc: idr wg; John Leslie
Subject: Re: [Idr] Comparison between session-reset and treat-as-withdraw

Hi Robert,

Agree that bgp-operational-message could provide detailed diagnostic
information which would be quite helpful for diagnosing of treat-as-withdraw
events. And some mechanism for automatic route recovery can also be helpful
after some important routes are treat-as-withdraw.

Regards,
Jie

> >>    By which I guess he means that treat-as-withdraw has no 
> >> automatic recovery path. And with that, I agree.
> >
> inconsistency between the sending and receiving BGP speaker for some 
> time, and some user traffic may be affected. Thus similar to session 
> reset, maybe the receiving speaker could try to recover the particular 
> routes with some back-off mechanism before manual intervention.
> >
> >
> > Best regards,
> > Jie
> >
> >> From: [email-masked] [mailto:[email-masked]] On Behalf 
> >> Of
> John
> >> Leslie
> >> >> In terms of recovery, it has been shown that the session reset 
> >> >> and reestablishment usually do not eliminate malformed updates.  
> >> >> Instead the session would often get into the cycle of reset and 
> >> >> reestablishment until the culprit route is removed from the 
> >> >> routing system by the operator manually.
> >> >
> >> > been there.  have coffee cup and tee shirt.  sadly sessions go 
> >> > bouncy bouncy is not uncommon.
> >>
> >>    But when the actual issue _is_ at the router getting the 
> >> NOTIFICATION, we are "routing around the problem".
> >>
> >> ] If there is no session re-establishment after the session 
> >> terminated, ] we may lose the chance of recovering the session 
> >> automatically in ] some error cases.
> >>
> >>    By which I guess he means that treat-as-withdraw has no 
> >> automatic recovery path. And with that, I agree.
> >>

---

From: Robert Raszuk <[email-masked]>
Date: Fri, 21 Sep 2012 06:58:09 +0200

Hi Sue,

As I am sure you remember that operational message in the current form 
of IDR WG document is a merge of advisory-message and diagnostic-message 
drafts proposed for adoption.

We have spent significant amount of time between Dave, Rob, Enke and 
myself to make sure it is sufficiently reach to be helpful in most BGP 
deployment cases today which does include error cases.

As you can see section "3.4.3.  DUMP TLVs" contains the essence of what 
needs to be reported to the peer at the event of "treat-as-withdraw" 
accident.

Ref:
http://tools.ietf.org/html/draft-ietf-idr-operational-message-00

In the light of the above we are very welcome to get more input from IDR 
WG if what we have currently in the bgp-operational-message is good 
enough or that it perhaps would benefit from addition of XYZ ... We will 
be happy to consider any input.

Other then that while Jie may his point of view I personally do not have 
additional comments to the draft.

Best regards,
R.

> the bgp-operational-message for the treat-as-withdraw case?
>
> Sue
>

---

From: Jie Dong <[email-masked]>
Date: Fri, 21 Sep 2012 06:05:16 +0000

Hi Sue and Robert, 

IMO bgp-operational-message can provide diagnostic information to the sending speaker of the malformed updates, and currently I do not have additional inputs to this draft. 

My concern is since we already provide bgp session level recovery mechanism for "session down" accident, accordingly do we also need recovery mechanism at route level for the new "route treat-as-withdraw" accident?

Best regards,
Jie

> From: Susan Hares [mailto:[email-masked]]
> Sent: Friday, September 21, 2012 6:58 AM
> To: Jie Dong; 'Robert Raszuk'
> Cc: 'idr wg'; 'John Leslie'
> Subject: RE: [Idr] Comparison between session-reset and treat-as-withdraw
> 
> Robert and Jie:
>

---

From: Jie Dong <[email-masked]>
Date: Fri, 21 Sep 2012 06:58:25 +0000

Hi Robert, 

I agree that the DUMP message should trigger the same error notification priority as session reset to the operators for diagnosing.

OTOH, Today's session reset also includes session re-establishment and re-advertisement of all the routes over the session, this is what I called "session level recovery". While we know that not all the errors can be solved with session re-establishment and route re-advertisement, usually such mechanism is deployed in networks, with some back-off mechanism. This is somewhat similar to restarting my cell phone when there is something wrong with it, before referring to the after-sale engineer.

Thus I would like to know if we need some equivalent for "treat-as-withdraw"

Regards,
Jie

> Sent: Friday, September 21, 2012 2:13 PM
> To: Jie Dong
> Cc: Susan Hares; idr wg; John Leslie
> Hi Jie,
> I understand your comment and concern. From the point of view of
> bgp-operational message point of view the DUMP message triggered by
> "treat-as-withdraw" should be treated with equal error level handling
> priority as today session reset.
> 
> The root cause for trigger of "treat-as-withdraw" IMHO can not be
> determined programatically today. So we do need operator's
> intervention on both ends to diagnose the real issue and at least try
> to fix it.
> 
> Rgs,
> R,
> 
> On Fri, Sep 21, 2012 at 8:05 AM, Jie Dong <[email-masked]> wrote:
> > Hi Sue and Robert,
> >
> > IMO bgp-operational-message can provide diagnostic information to the
> sending speaker of the malformed updates, and currently I do not have
> additional inputs to this draft.
> >
> > My concern is since we already provide bgp session level recovery mechanism
> for "session down" accident, accordingly do we also need recovery mechanism
> at route level for the new "route treat-as-withdraw" accident?
> >
> > Best regards,
> > Jie
> >