Subject: PWE3 question

From: Shahram Davari <[email-masked]>
Date: Tue, 7 May 2002 12:02:24 -0700

Hi,

Can we implement PWE3 with only one label (i.e., only VC label)?

-Shahram

---

From: "Andrew G. Malis" <[email-masked]>
Date: Tue, 07 May 2002 18:12:14 -0400

Shahram,

Well, it wouldn't scale very well since every switch along the path is 
involved in the signaling for each VC, and you can't do traffic 
engineering, so I would say that's pretty much a non-starter.

Cheers,
Andy

At 5/7/2002 12:02 PM -0700, Shahram Davari wrote:
>_______________________________________________
>pwe3 mailing list
>[email-masked]
>https://www1.ietf.org/mailman/listinfo/pwe3

---

From: Shahram Davari <[email-masked]>
Date: Wed, 8 May 2002 08:33:54 -0700

Andy,

Sure I understand the scaling issue. But ignoring the scalability,
I want to know if it can be done and if so how? 

It seems that you may not be able to use LDP any more, because VC FEC
is meaningless to the interior LSRs. But it seems that you could use
ER-LSPs and therefore do TE. Right? 

Yours,
-Shahram

> -----Original Message-----
> From: Andrew G. Malis [mailto:[email-masked]]
> Sent: Tuesday, May 07, 2002 6:12 PM
> To: Shahram Davari
> Cc: [email-masked]'
> Subject: Re: [PWE3] PWE3 question
> 
> 
> Shahram,
>

---

From: "Andrew G. Malis" <[email-masked]>
Date: Wed, 08 May 2002 11:51:06 -0400

Shahram,

Yeah, you would have to treat each VC as a separate TE tunnel and use 
either RSVP-TE or CR-LDP for signaling (suitably extended along Martini 
lines for VC type and ID).

Cheers,
Andy

-------

---

From: Sasha Vainshtein <[email-masked]>
Date: Wed, 8 May 2002 19:58:37 +0200

Andy and Shahram,

IMHO using only one label would, regardless
of the other issues, violate one of the requirements
 of the PWE3 Charter:

<quote>
PWE3 will not exert controls on the underlying PSN,
but only function at the endpoints of the pseudo wire, 
though the endpoints may be configured to set diffserv 
codepoints in the tunneling datagrams. 
<end quote>

Building an explicit TE tunnel for a single PW means to me 
exerting a control over the PSN (e.g., all the routers in the PSN
should recognize VC FECs etc.).

In other words, it probably can be done, but it would not be PWE3.

With best regards,
                                   Sasha Vainshtein
email:     [email-masked] <mailto:[email-masked]> 
tel:       +972-3-7659993 (office)
           +972-8-9254948 (res.)
           +972-58-674833 (cell.)

> From: Andrew G. Malis [mailto:[email-masked]]
> Sent: Wednesday, May 08, 2002 5:51 PM
> To: Shahram Davari
> Cc: 'Andrew G. Malis'; [email-masked]'
> Subject: RE: [PWE3] PWE3 question
> 
> 
> Shahram,
>

---

From: Shahram Davari <[email-masked]>
Date: Wed, 8 May 2002 12:05:01 -0700

Sasha,

I don't think in this case all the routers in the PSN
need to recognize the VC FEC.  You could set the U/F bits to 1
in VC FEC, therefore no need for the interior routers to
recognize the VC FEC.

-Shahram

> From: Sasha Vainshtein [mailto:[email-masked]]
> Sent: Wednesday, May 08, 2002 1:59 PM
> To: 'Andrew G. Malis'; Shahram Davari
> Cc: [email-masked]'; Danny McPherson (E-mail)

---

From: Sasha Vainshtein <[email-masked]>
Date: Thu, 9 May 2002 08:58:20 +0200

Shahram,

IMHO, if you use only one label for a PW, it is this label that
is switched through the path from ingress to egress. 
Aand if the routesr do not recognize the VC FEC, how should
they assign and switch labels associated with this FEC?

With best regards,
                                   Sasha Vainshtein

> From: Shahram Davari [mailto:[email-masked]]
> Sent: Wednesday, May 08, 2002 9:05 PM
> To: 'Sasha Vainshtein'; 'Andrew G. Malis'

---

From: Shahram Davari <[email-masked]>
Date: Thu, 9 May 2002 05:57:24 -0700

Sasha,

ER-LSPs work by DoD label assignment. And the ERO identifies the path.
VC FEC is not needed to be processed in the intermediate LSRs at all.

-Shahram

> Sent: Thursday, May 09, 2002 2:58 AM
> To: Shahram Davari; 'Andrew G. Malis'

---

From: Sasha Vainshtein <[email-masked]>
Date: Thu, 9 May 2002 18:05:52 +0200

Shahram,
I am confused.
If the intermediate routers recognize only ERO 
what would prevent them from advertizing upstream the same label for 
all the PWs along the same path? 
IMHO both PEs needs a unique label per PW (in one direction);
I do not understand how this can be achieved if the intermediate
routers do not recognize different VC FECs.

With best regards,
                                   Sasha Vainshtein

> Sent: Thursday, May 09, 2002 2:57 PM

---

From: Shahram Davari <[email-masked]>
Date: Thu, 9 May 2002 08:30:32 -0700

Sasha,

LSP ID is used to make sure that CR-LDP never advertises
the same label to two different requests.

And M-bit=0 (merge bit) could be used in RSVP-TE to make sure
that no merging occurs.

-Shahram

> Sent: Thursday, May 09, 2002 12:06 PM
> To: Shahram Davari; Sasha Vainshtein; 'Andrew G. Malis'

---

From: Shahram Davari <[email-masked]>
Date: Thu, 9 May 2002 13:59:02 -0700

Sasha,

> Sent: Thursday, May 09, 2002 5:45 PM
> I agree that it is possible to force the routers in
> the PSN to allocate a label per a VC FEC without
> understanding it.
> However I donot agree that this does not mean
> exerting control over the network - and this should
> not be done in PWE3.

In what respect do you see this exerting control over the
PSN network? In contrast I don't see any.

> Once more, I donot say that what you propose cannot be 
> done - I just claim that this is beyound the scope defined
> by the PWE3 chapter.

Since the PSN interior nodes need not know anything about the
PW, I think this actually falls within PWE3 charter.

-Shahram

> 
> --------------------------------------------------------------
> --------------
> --------
> With best regards,
>                           Sasha Vainshtein
> phone:  +972-3-7659993 (office)
>             +972-8-9254948 (home)
>             +972-58-674833 (cellular)
> 
> 
> > Sent: Thursday, May 09, 2002 5:31 PM
> > > > > > > > either RSVP-TE or CR-LDP for signaling 
> (suitably extended 
> > > > > > > > along Martini 
> > > > > > > > >is meaningless to the interior LSRs. But it 
> > seems that you 
> > > > > > > could use
> > > > > > > > > Well, it wouldn't scale very well since every 
> > switch along
> > > > > > > > > the path is
> > > > > > > > > >Can we implement PWE3 with only one label 
> (i.e., only 
> > > > > > VC label)?
> > > > > > > > > >
> > > > > > > > > >-Shahram
> > > > > > > > > >

---

From: Sasha Vainshtein <[email-masked]>
Date: Thu, 9 May 2002 23:45:12 +0200

Shahram,

----------------------------------------------------------------------------
--------
With best regards,
                          Sasha Vainshtein
email:   [email-masked] <mailto:[email-masked]> 
phone:  +972-3-7659993 (office)
            +972-8-9254948 (home)
            +972-58-674833 (cellular)

---

From: Vinay Bannai <[email-masked]>
Date: Thu, 9 May 2002 15:13:28 -0700

I am little confused by the discussion that has been going on so far
regarding using a single label vs stacked label (using MPLS encapsulation as
a example). Here is what I take from the PWE3 working group charter:

- The setting up of the pseudo wire end to end is a function of the end
points
- The existence of a tunnel LSP is a given and how it is setup does not fall
under the purview of PWE3

Maybe I am reading this wrong, isn't this what the charter implies?

Vinay

---

From: [email-masked]
Date: Thu, 9 May 2002 23:49:39 +0100

Interesting point Shahram.....I'd also like to know why LDP seems to be the
only signalling prtocol that seems to be sanctioned by some......for example
draft-martini-l2circuit-trans-mpls-09.txt says 'static' MAY be used, but if
signalled it MUST be via LDP.  I can see the VC_LSP is just 1-hop = single
link-connection of the client trail, so is it just 'because it might be
easiest/cheapest'....or are there any other reasons for this?  And
specifically, *why* is any other form of VC LSP signalling postively
disallowed?

regards, Neil
> Sent: 09 May 2002 13:57

---

From: [email-masked]
Date: Fri, 10 May 2002 01:03:15 +0100

Vinay,
Vinay Bannai wrote 09 May 2002 23:13
<snip>
> Maybe I am reading this wrong, isn't this what the charter implies?
NH=> If it does why is it so restrictive?  All Shahram has pointed out is
that 2 labels are not strictly needed if an ER LSP is used in a true
client/server sense.  My reading is that its sort-of been accepted as this
is how you do it without any explanation of such....Shahram just says this
is not a *forced* arch necessity, and if is a constraint it seems to be an
unconscious self-imposed one for some reason.  In a strict arch sense, the
(bottom) LSP trail is forming a single hop (= single link-connection) of the
technology trail being carried.  We do this all the time in telecoms
networks, eg if you want a ATM connection between A and B to carry protocol
X you don't force a double trail stack of VPC+VCC to be
implemented.....ditto in SDH for VC4+VC12 say, etc.  I suspect the
constraint is largely due to someone's orginal ideas of *one way* this could
be done using certain control-plane protocols, and everyone else has assumed
this has to be the right/only solution.

Maybe some of the orginators of this stuff could explain this restriction to
us so we can understand it better....it also relates to my earlier question
on why LDP is the only signalling protocol allowed for the lower LSP?  I may
of course be overlooking something, but currently I don't see it.

regards, Neil

---

From: "Andy Malis" <[email-masked]>
Date: Thu, 9 May 2002 17:10:21 -0700

Neil,
 
The point is that LDP used in extended discovery mode for this
application works perfectly and has great scaling properties.  So why
specify the necessary changes to anything else?  Interoperability is
enhanced by choosing one way to do things, especially when it just works
so well - this reduces implementation and testing costs for vendors, and
also reduces both capital and operational costs for service providers.
Capital costs are obvious - operational costs because there's less
interoperability testing the SPs have to do in their own labs, and fewer
protocols they have to learn how to provision, operate, and
troubleshoot.
 
Cheers,
Andy

	-----Original Message----- 
	From: [email-masked] 
	Sent: Thu 5/9/2002 18:49 
	To: [email-masked]; [email-masked]; Andy Malis 
	Cc: [email-masked] 
	Subject: RE: [PWE3] PWE3 question

	Interesting point Shahram.....I'd also like to know why LDP
seems to be the
	only signalling prtocol that seems to be sanctioned by
some......for example
	draft-martini-l2circuit-trans-mpls-09.txt says 'static' MAY be
used, but if
	signalled it MUST be via LDP.  I can see the VC_LSP is just
1-hop = single
	link-connection of the client trail, so is it just 'because it
might be
	easiest/cheapest'....or are there any other reasons for this?
And
	specifically, *why* is any other form of VC LSP signalling
postively
	disallowed?
	
	regards, Neil

---

From: Sasha Vainshtein <[email-masked]>
Date: Fri, 10 May 2002 08:56:57 +0200

Shahram and all.
Please see a brief comment inline.

> Sent: Thursday, May 09, 2002 10:59 PM

I would suggest a very simple criterion regarding exertion or
non-exertion of control over PSN: if setup/teradown of a PW
requires modification of the FIB in at least one PSN node that 
does not act as the PE for this PW, it means that control over 
this router has been exerted. 

This criterion does not depend on the protocol being used
(static,LDP, RSVP or else).

The single-label model requires allocation of a new label per PW
in each interior node on the path fromingress to egress, and hence
means exertion of control over the PSN unless the the pair of
PEs are adjacent (the "single hop" case mentioned by Neil).

Can we agree on that?

> > > > > > > > > Yeah, you would have to treat each VC as a 
> separate TE 
> > > > > > > > tunnel and use 
> > > > > > > > > >Sure I understand the scaling issue. But 
> ignoring the 
> > > > > > > scalability,
> > > > > > > > > >It seems that you may not be able to use LDP 
> any more, 
> > > > > > > > because VC FEC
> > > > > > > > > > involved in the signaling for each VC, and 
> you can't 
> > > > > > do traffic

---

From: "Gregory Wright"<[email-masked]>
Date: Fri, 10 May 2002 07:46:02 -0400

Andy, one could hardly call a flat fully meshed control plane as having
"great scaling properties". LDP works only in a shared trunk environment. If
a provider desires to offer both a shared trunk service and a "more rigid"
unstacked TE LSP based service why should we not allow it?

-----Original Message-----
From: Andy Malis [mailto:[email-masked]]
Sent: Thursday, May 09, 2002 8:10 PM
To: [email-masked]; [email-masked];
[email-masked]
Cc: [email-masked]

Neil,

Cheers,
Andy

	-----Original Message----- 
	From: [email-masked] 
	Sent: Thu 5/9/2002 18:49 
	To: [email-masked]; [email-masked]; Andy Malis 
	Cc: [email-masked] 
	Subject: RE: [PWE3] PWE3 question

	regards, Neil

---

From: Shahram Davari <[email-masked]>
Date: Fri, 10 May 2002 06:41:33 -0700

Hi,
 
Also one of the problems of LDP is that you are forced to use per-platform label space, which reduces scalability.
I don't think LDP should be made mandatory in PWE3.
 
-Shahram

-----Original Message-----
From: Gregory Wright [mailto:[email-masked]]
Sent: Friday, May 10, 2002 7:46 AM
To: 'Andy Malis'
Cc: [email-masked]
Subject: RE: [PWE3] PWE3 question

Andy, one could hardly call a flat fully meshed control plane as having "great scaling properties". LDP works only in a shared trunk environment. If a provider desires to offer both a shared trunk service and a "more rigid" unstacked TE LSP based service why should we not allow it?

-----Original Message----- 
From: Andy Malis [ mailto:[email-masked] <mailto:[email-masked]> ] 
Sent: Thursday, May 09, 2002 8:10 PM 
To: [email-masked]; [email-masked]; 
[email-masked] 
Cc: [email-masked] 

Neil, 

Cheers, 
Andy 

        -----Original Message----- 
        From: [email-masked] 
        Sent: Thu 5/9/2002 18:49 
        To: [email-masked]; [email-masked]; Andy Malis 
        Cc: [email-masked] 
        Subject: RE: [PWE3] PWE3 question 

        regards, Neil 

_______________________________________________ 
pwe3 mailing list 
[email-masked] 
https://www1.ietf.org/mailman/listinfo/pwe3 <https://www1.ietf.org/mailman/listinfo/pwe3>

---

From: Shahram Davari <[email-masked]>
Date: Fri, 10 May 2002 06:50:03 -0700

Vinay,

The exact wording of the charter is:

"PWE3 protocols and implementation focus on the edge-to-edge emulation and maintenance of service-specific pseudo-wires. Creation and placement of the tunnels is outside of the scope."

I don't see any thing here that prevents using one level tunnel. Do you see any?

-Shahram

> From: Vinay Bannai [mailto:[email-masked]]
> Sent: Thursday, May 09, 2002 6:13 PM
> To: Shahram Davari; 'Sasha Vainshtein'; 'Andrew G. Malis'
> Cc: [email-masked]'

---

From: Shahram Davari <[email-masked]>
Date: Fri, 10 May 2002 07:21:59 -0700

Hi Sasha,

Perhaps the charter is too restrictive. But that does not change the fact that we can build a PW with all the PWE3 encapsulations and tools which uses only one label.

I think such restriction must be removed. And I know customers that want a single label to implement PWE3.

-Shahram

> Sent: Friday, May 10, 2002 2:57 AM
> To: Shahram Davari
> Cc: [email-masked]'; Danny McPherson (E-mail); 'Andrew G. 
> Malis'; Neil.
> 2. Harrison (E-mail)
> > > > > > > > > exerting a control over the PSN (e.g., all 
> the routers 
> > > > > > in the PSN

---

From: Eric Rosen <[email-masked]>
Date: Fri, 10 May 2002 11:33:05 -0400

Our original assumption long  ago was that no one with a  clue would want to
use a  scheme in  which each pseudowire  lays down  state in the  P routers.
Otherwise it would be as bad (unscalable) as ATM. 

Given that  one wants instead to  tunnel the pseudowires, there  needs to be
some  applicable  signaling protocol  which  is  mandatory  to implement  in
devices  that set up  tunneled pseudowires.  This is  necessary in  order to
ensure multi-vendor interoperability. 

If  someone  wants to  standardize  a  method  for setting  up  non-tunneled
pseudowires, there is nothing to stop them, and no particular reason why LDP
should be involved.  

I imagine the networks which want to do non-tunneled pseudowires will be the
same  networks which  have deployed  CR-LDP, ITU's  standard  MPLS signaling
protocol, so you may wish to  look into that, perhaps within the auspices of
ITU. 

There is  already an  i-d proposing  to use RSVP-TE  to set  up non-tunneled
pseudowires, though  I did try to  discourage the authors on  the grounds of
the assumption in the first paragraph above.

---

From: Shahram Davari <[email-masked]>
Date: Fri, 10 May 2002 09:33:52 -0700

Eric, 

Thanks for your insightful reply. A few minor comments are in-line

-Shahram

> From: Eric Rosen [mailto:[email-masked]]
> Sent: Friday, May 10, 2002 11:33 AM
> To: Shahram Davari
> Cc: 'Sasha Vainshtein'; [email-masked]'; Danny McPherson (E-mail);
> 'Andrew G. Malis'; Neil. 2. Harrison (E-mail)

Sure, but what if the Provider network is small or supporting very few
PWE3 connections. 

> ensure multi-vendor interoperability.

But for TE purpose, those devices need to implement non-LDP
protocols. They might as well use them for setting up 
PWs too.
 
> protocol,

1) This method could be independent of the signaling protocol
2) Surprise: CR-LDP is IETF's standard too. 

> so you may wish to  look into that, perhaps within 
> the auspices of
> ITU.

I don't see any strong connection to ITU here.

---

From: "Thomas D. Nadeau" <[email-masked]>
Date: Fri, 10 May 2002 12:54:59 -0400

At 09:33 AM 5/10/2002 -0700, Shahram Davari wrote:
> > use a  scheme in  which each pseudowire  lays down  state in
> > the  P routers.  Otherwise it would be as bad (unscalable) as ATM.
>
>

         I agree with Eric's insightful assertion that making a solution 
scale realistically
to large numbers is paramount  What happens when the smaller networks
you refer to become large someday? Besides, given what I am hearing from
customer deployments and requirements, there is nothing small about either
their networks or the number of pseudo-wires they wish to install.

         --Tom

------------------------------------------------------------------------
Mathematics is the supreme nostalgia of our time.

---

From: Eric Rosen <[email-masked]>
Date: Fri, 10 May 2002 12:58:22 -0400

Shahram> Sure, but what if the  Provider network is small or supporting very
Shahram> few PWE3 connections. 

Certainly this was not the design  center, but the LDP method does work well
in  this case  as well,  and I  don't think  it introduces  costs  which are
onerous in  small scale  environments. So  I don't really  see the  need for
another standard that specifically targets small scale environments.

Shahram> But  for  TE  purpose,  those  devices need  to  implement  non-LDP
Shahram> protocols. They might as well use them for setting up PWs too. 

I don't  think we really  need a separate  standard targetted at the  set of
small scale MPLS networks which use RSVP-TE but not LDP. 

Shahram> Surprise: CR-LDP is IETF's standard too. 

Good thing IETF  has another standard that does the  same thing and actually
gets used!

---

From: Vinay Bannai <[email-masked]>
Date: Fri, 10 May 2002 10:11:10 -0700

Shahram,

I agree. The wording clearly does not tell what flavor of signaling, manual
intervention or otherwise is used to setup the tunnel. However, to me it
does appear that the process of setting up the PW is a edge to edge
functionality alone.

My read on the charter of the PWE3 efforts is the situation where the tunnel
is already provided to me and user is not in a position to know what
underlying mechanisms or networks support the tunnel. Having said that what
you are suggesting (single label) to me seems like the label switched path
setup that can be done using already proven technologies and does not need
any sanction or validation from PWE3 group. 

Thanks
Vinay

-----Original Message-----
From: Shahram Davari [mailto:[email-masked]] 
Sent: Friday, May 10, 2002 6:50 AM
To: 'Vinay Bannai'; 'Sasha Vainshtein'; 'Andrew G. Malis'
Cc: [email-masked]'
Subject: RE: [PWE3] PWE3 question

Vinay,

"PWE3 protocols and implementation focus on the edge-to-edge emulation and
maintenance of service-specific pseudo-wires. Creation and placement of the
tunnels is outside of the scope."

I don't see any thing here that prevents using one level tunnel. Do you see
any?

-Shahram

---

From: Shahram Davari <[email-masked]>
Date: Fri, 10 May 2002 10:30:03 -0700

Eric,

In non-tunneled case I can't use LDP without upgrading all interior nodes, 
because RFC 3036 says:

"  If it encounters a FEC Element type it cannot decode, it should stop
   decoding the FEC TLV, abort processing the message containing the
   TLV, and send an "Unknown FEC" Notification message to its LDP peer
   signaling an error."

-Shahram

> Sent: Friday, May 10, 2002 12:58 PM
> To: Shahram Davari

---

From: Eric Rosen <[email-masked]>
Date: Fri, 10 May 2002 13:41:18 -0400

Shahram> In non-tunneled case I can't use LDP

Sorry, I  didn't make  myself clear.  What  I meant  was that even  in small
scale environments, one might as well just use tunneling.

---

From: Sasha Vainshtein <[email-masked]>
Date: Sat, 11 May 2002 10:44:27 +0200

Shahram,
Thank you for a prompt response.
I read  it as your implicit  acceptance of the criteria that I have
tried to define :-). 

IMHO, if you find the charter too restrictive,
you should raise this issue explicitly with Danny and the ADs.

>From my point of view - for whatever it is worth- the charter 
limitations well ustified if you take the scalability considerations 
into account (the point that seems to be shared by Eric and others 
on this list). 

> Sent: Friday, May 10, 2002 4:22 PM
> To: 'Sasha Vainshtein'
> Malis'; Neil. 2. Harrison (E-mail)
> Perhaps the charter is too restrictive. But that does not 
> change the fact that we can build a PW with all the PWE3 
> encapsulations and tools which uses only one label.
> 
> I think such restriction must be removed. And I know 
> customers that want a single label to implement PWE3.
> 
> -Shahram
> 
>  
> 
> > > > > > > > > > In other words, it probably can be done, 
> but it would 
> > > > > > > not be PWE3.
> > > > > > > > > > 
> > > > > > > > > > With best regards,
> > > > > > > > > >                                    Sasha Vainshtein
> > > > > > > > > > email:     [email-masked] 
> > > > > > > > > > > > engineering, so I would say that's 
> pretty much a 
> > > > > > > non-starter.
> > > > > > > > > > > >
> > > > > > > > > > > > Cheers,
> > > > > > > > > > > > Andy
> > > > > > > > > > > >

---

From: [email-masked]
Date: Sun, 12 May 2002 23:04:56 +0100

Thanks Andy,

Well I am only half-convinced by your answer for the following reasons:
-	AFAI can tell, the top-layer (bottom label) LSP is a single-hop MPLS
trail (in all cases).....since architecturally what it is doing is proxying
for a link-connection in whatever client layer technology trail it is
carrying.  Hence, it is only using the extended/targetted LDP p2p
behaviour........which I am at least thankful for because the mp2p behaviour
of full-blown LDP does cause many problems IMO.
-	Given this, if the MPLS network has multiple nodes between IW
devices then one is forced to run some further server layer LSP underneath
this.  Which in itself is fine, as this is nothing more than a true
client/server approach which we will see recurse many times here before we
hit the duct layer.  But if we are to imbue this LSP with
enforeceable/measureable QoS/survivability attributes and fault-management
behaviour (for whatever the ultimate XoverMPLS technology client is being
carried), then this LSP needs to be ER p2p also IMO.
-	Hence, we have not avoided the need to provide an ER p2p LSP at
all....and architecturally you strictly only need a single proper
client/server relationship.

However, I do agree that a consistent approach is useful......so from this
perspective, and ignoring the strict arch relationships, then I can
understand why you may argue for keeping things as some have already decided
to do them.  All I want to show is that it is not a forced architecture.

regards, Neil

> Sent: 10 May 2002 01:10
> To: neil.2.harrison; Shahram_Davari; Sasha
> Cc: pwe3

---

From: [email-masked]
Date: Sun, 12 May 2002 23:05:24 +0100

> the  P routers.
NH=> This 'scalability' and 'what is bad' issue needs some qualification.
Sure, mp2p LSPs scale better than p2p LSPs but at what latent cost?
-	complex fault and performance management......probably relying on
customers as the defect detection device for within-MPLS failures, or the L1
networks for failures there which have decent fault management (courtesy of
those pagans in the ITU);
-	an inability to offer solid/measureable SLAs in VPN applications
(where XoverMPLS is a subset of these in principle) because QoS and
survivability get kludged together.  Connectivity is not the only criteria
to judge the merit of a given approach here.
-	operators have to throw BW (=money) at the VPN problem......cf with
FR where overbooking is common, but because FR is p2p you can still
prioritise properly on survivability.  So will a MPLS-based VPN network
consume (=cost) maybe >4x (?) that of a FR-based VPN network?  OK, its not
that simple I know....but senior managers like looking at utilisation
figures and margins, and these days post the bubble bursting in our industry
efficient network solutions are gaining in importance.

I also don't buy the N^2/2 reduction argument either.....if A wants to talk
to B 'something' has to be configured p2p to allow this.....so mp2p LSPs
simply push the problem up a layer, they don't get rid of it (and the BGP4
LSPs in rfc2457 do exactly this function).   The use of hierachical networks
is also an important factor wrt addressing scaling (though this is going in
the other direction towards the duct).  What operators need are better
OSS/NMS config tools to address the N^2/2 scaling problem properly.

The bottom-line is there is no free-lunch and merging is a blunt tool.....in
generic public networks it might be OK, but for some applications (like
VPNs) it fails to meet some basic service requirements IMO.  Scaling is not
the only criteria to judge bad/good.
> the assumption in the first paragraph above. 
NH=> I am failing to understand the issue here Eric.  Why is an ER LSP used
for application X any different to an ER LSP used for application Y?  IMO
the LSP should be agnostic to what it is carrying.  If you have a degenerate
p2p LDP 1-hop LSP you will still need a lower level (transport) LSP....so
you have to signal 'something' in the p2p LDP and also 'something' in the
lower LSP......which also has to be an ER p2p LSP IMO if you want to offer
proper QoS/management of such.  Further, because these LSPs are forming a
client/server relationship, the server LSP has to meet the 'service' demands
of its client LSP, ie the service requirement is recursively related.

The only argument I can see for discouraging people is to force everyone to
use the LDP approach which some have already adopted....but architecturally
dual LSP stacking seems an unecessary requirement, and I think that was all
that was originally being pointed out by Shahram.

regards, Neil

---

From: [email-masked]
Date: Mon, 13 May 2002 17:38:26 +0100

Thanks Robert,

Just 1 observation on your response:

<snipped>
> So to balance a bit what has been already said I would say that using
> single level of label or encapsulation may run great for any 
> application
> it carries over it until it is far from the limit of 
> scalability numbers
> of any routers in your network. When you get close you have three
> choices: 
> 
> * discontinue the service,
> * upgrade/replace all devices in the net,
> * use hierarchy (label stacking or other edge-edge 
> encapsulation) hiding
> P boxes from the # of instances of your PE-PE pipes.
NH=> I agree, esp wrt last bullet here....which is why in my initial
comments I said:
"The use of hierachical networks is also an important factor wrt addressing
scaling (though this is going in the other direction towards the duct).
This is something we have been doing for years with many
technologies.....this is simply G.805 defined client/server encapsulation of
layer networks.

However, the principle of the point made by Shahram is that a use of
targetted LDP is not forced....though some drafts say it is mandatory if
signalling is used here (ie not static).  So in principle I could use
ER/LSPs, and I could nest these if I wanted to get scaling (just like a road
network uses motorways between major sites, with minors roads dropped at at
various places).  However, it may be that targetted LDP in a 1-hop p2p mode
is simpler/cheaper (than an equivalent 1-hop ER/p2p replacement?), but I
feel I would for sure need a server ER/p2p LSP under this if I wanted solid
control of QoS/survivability of this 'trunk' LSP and, via inheritance, the
same for it's (XoverMPLS carrying) LSP clients.....I would certainly not
want these 'merged' with other traffic as its seems would follow for a
server layer use of general LDP with mp2p LSPs for reasons I have given
previously.

regards, Neil

---

From: Luca Martini <[email-masked]>
Date: Mon, 13 May 2002 15:33:35 -0600

Shahram Davari wrote:

> Hi,
> 
>  
> 
> Also one of the problems of LDP is that you are forced to use 
> per-platform label space, which reduces scalability.
> 

What we need to make mandatory is not modifying the underling network.
That is in the charter, Hence we argue the charter , or the case is closed.

"PWE3 protocols and implementation focus on the edge-to-edge emulation and 
maintenance of service-specific pseudo-wires. Creation and placement of the 
tunnels is outside of the scope."
When we wrote this , it was clearly argued by Scott , that we should not modify 
every network element in the path to achieve our goal. This , can of course be 
done , but you should take it to PPVPN.

A very good principle to follow is to avoid having customer specific state in 
the core network. Simple core , intelligent edge. the staking model used by 
IGP/BGP  or MPLS/IGP/BGP, where the IGP does not know about customer routes.

A certain vendor, implemented RSVP-TE to transport L2 frames for the only 
reason that , at the time , they did not have label stacking capabilities. Why 
would you, or anyone argue for this now ?

Luca

>     Andy, one could hardly call a flat fully meshed control plane as
>     having "great scaling properties". LDP works only in a shared trunk
>     environment. If a provider desires to offer both a shared trunk
>     service and a "more rigid" unstacked TE LSP based service why should
>     we not allow it?
> 
> 
>

---

From: Shahram Davari <[email-masked]>
Date: Tue, 14 May 2002 06:14:06 -0700

Luca,

> From: Luca Martini [mailto:[email-masked]]
> Sent: Monday, May 13, 2002 5:34 PM
> To: Shahram Davari
> Cc: 'Gregory Wright'; 'Andy Malis'; [email-masked]

This question is in-line with asking: why would anybody use PHP now?
Doing one label lookup at line rate is easier than 2.

-Shahram

> >     enhanced by choosing one way to do things, especially 
> when it just
> >     works
> >     so well - this reduces implementation and testing costs 
> for vendors,
> >     and
> >     interoperability testing the SPs have to do in their 
> own labs, and
> >     fewer

---

From: Shahram Davari <[email-masked]>
Date: Tue, 14 May 2002 07:12:36 -0700

BTW, even in case of LDP, why should DU be the only mode? why not permit DoD?

-Shahram

> From: Shahram Davari 
> Sent: Tuesday, May 14, 2002 9:14 AM
> To: 'Luca Martini'

---

From: "Rutemiller, John" <[email-masked]>
Date: Wed, 15 May 2002 06:12:24 -0400

> To: 'Shahram Davari'
[SNIP]
> PEs are adjacent (the "single hop" case mentioned by Neil).

Creation of a Pseudo wire requires a tunnel label. The creation
of this tunnel must be configured. The extablishment of the tunnel
will affect the FIB if the P routers. Further more, the tunnel must
be created with a forwarding class that is sufficient to carry the
desired pseudo wire traffic.

If traffic classes are to be separated (e.g, ATM CBR gets EF and
ATM UBR gets best effort), then multiple tunnels must be created
across the network.

Therefore, the PW has in some way exerted control over the P routers
even with the tunnel creation.

John

---

From: "Rutemiller, John" <[email-masked]>
Date: Wed, 15 May 2002 06:40:07 -0400

Protocols should not dictate network architecture.

The choice of whether an operator desires to use a separate
label inside the network for each psuedo-wire, or to use a
label stack to improve scaling inside the network is a
network design issue.

It is not a protocol design issue.

And this issue of ATM scalability is non-sense. Virtual Paths
provide a mechanism that helps in scalabililty. The two-levels
was a constraint if you wanted to provide a virutal path to
customers, and the VP space was a limit if trying to connect
lots of devices at the very edge of the network. But it was
fine for providing connection scaling within the network.

Although MPLS definitely has an advantage with multi-level stacks.

The ATM technology itself does not have any inherent scaling problems
with the number of connections that can exist within the network.
It is possible to have about 268 million connections (2**28) over a
single NNI link. The only limitation is what can be implemented in
ATM switches, and the performance of the control processors, at
a resonable cost. The number of connections supported by a single
ATM switch continues to climb.

I would hope that MPLS could follow the same trend. Unless you are
implying that there is something about MPLS that makes it harder to
support the same number of connections (LSPs) on a single device as
can be found on current ATM switches.

John

---

From: Sasha Vainshtein <[email-masked]>
Date: Wed, 15 May 2002 16:10:44 +0200

John,
Please see some comments/responses inline.
Hopefully they will clarify my position.
With best regards,
                                   Sasha Vainshtein
email:     [email-masked] <mailto:[email-masked]> 
tel:       +972-3-7659993 (office)
           +972-8-9254948 (res.)
           +972-58-674833 (cell.)

> From: Rutemiller, John [mailto:[email-masked]]
> Sent: Wednesday, May 15, 2002 12:12 PM
> To: [email-masked]'
> Creation of a Pseudo wire requires a tunnel label. 
>
Correct. You must associate every direction of a PW
over an MPLS network with an LSP leading from the 
ingress PE (for this direction) to egress PE.
>
>The creation of this tunnel must be configured. 
>
It can be created regardless of any specific PW that would use it.
And it could be created in many ways, including hop-by-hop forwarding
provided by the vanilla LDP.
Once created, such a tunnel can be used by - and, for scalability 
reasons, SHOULD be used - by more than one PW between the 
same pair of PEs.

And it could be created in many ways, including "vanilla" LDP
in DU mode.
>
>The establishment of the tunnel will affect the FIB if the P routers. 
>
Correct. However, as I said above, creation of the tunnel (to be  used
by multiple PWs) is not caused by creation of the PWs. Creation
of PWs using an already existing tunnel would not affect FIB in the
 P routers, only in PEs.
>
>Furthermore, the tunnel must be created with a forwarding class 
>that is sufficient to carry the desired pseudo wire traffic.
>
There are many ways to associate a certain forwarding behavior
with the EXP bits across an LSP. E.g., one can use 
Preconfigured EXP<-->PHB Mapping in LSPs that have been
created by LDP in DU mode.

Such an method cannot be used if reservation of resources
must be associated with an LSP. But even then nothing
prevents creation of a "planned" network of DiffServ-enabled 
LSPs with BW reservation beforehand regardless of any
specific PW, and then creation of PWs that use available
capaity on this "network".
> ATM UBR gets best effort), then multiple tunnels must be created
Not necessarily. A single E-LSP will may support up to 8 different
traffic classes. And this can be done by using preconfigured 
EXP<-->PHB mappings, too.
> even with the tunnel creation.
This would only happen if you apply the "One PW = One Tunnel" model
that originated this thread. And that is exactly why I said that, IMHO,  
this model is beyound the chartered PWE3 scope. The model that
separates creation of transport tunnels from creation of PWs is
within this scope.

---

From: "Rutemiller, John" <[email-masked]>
Date: Wed, 15 May 2002 05:55:19 -0400

---

From: "Rutemiller, John" <[email-masked]>
Date: Wed, 15 May 2002 06:31:23 -0400

The Layer is irrelevant.

The point Neil is trying to make is that VPN services may require
the ability to provide different guarantees or survivabilty on
a per VPN, or per member pair, basis.

When they get merged together, this becomes impossible.

And just because the traffic is IP doesn't relax these resrictions.

Using mp2p may be a great solution for some VPN applications.
But not all.

John

> Sent: Monday, May 13, 2002 1:10 PM
> To: [email-masked]
> Cc: [email-masked]
> Neil> I also don't buy the N^2/2 reduction argument 
> either.....if A wants to
> Neil> talk to B 'something' has to be configured p2p to allow this
> 
> I guess you mean "if A wants to  talk to B over a pseudowire 
> ...".  If layer
> 3 connectivity is all that  is required, then there's this 
> newfangled notion
> of connectionless  networking that eliminates  the p2p stuff. 
>   Nah, that'll
> never catch on ;-)
> 
> Assuming  the context  here is  pseudowires, then  for a  
> full mesh  among N
> endpoints  of course  one  needs N**2  PWs.   However, there  
> is no  O(N**2)
> scaling problem in the network because  (a) any given PW 
> endpoint only has N
> PWs, and (b)  in the core of the  network one only needs 
> state  for one mp2p
> LSP per endpoint.  So the endpoints scale O(N) and the core 
> scales O(N).
> 
> It is  true on  the other hand,  that the  PW provisioning 
> system  scales as
> O(N**2).  Not  much to do about that,  layer 2 just doesn't  
> scale nearly as
> well as layer  3; that's an inherent  problem of layer 2 
> which  can never be
> eliminated by NMS systems or even by VPLS-like schemes. 
> 
> Neil> mp2p LSPs simply push the problem up a layer, they 
> don't get rid of it
> Neil> (and the BGP4 LSPs in rfc2457 do exactly this function). 
> 
> The  BGP-distributed  labels of  RFC2547  do  not  create p2p 
>  anything.  In
> RFC2547, mp2p LSPs based on  BGP-distributed labels are 
> carried in mp2p LSPs
> based on LDP-distributed labels.  (That's why it's a layer 3 
> VPN service.)
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
>

---

From: Shahram Davari <[email-masked]>
Date: Wed, 15 May 2002 06:49:36 -0700

Sasha,

> From: Sasha Vainshtein [mailto:[email-masked]]
> Sent: Wednesday, May 15, 2002 10:11 AM
> To: 'Rutemiller, John'
> Cc: [email-masked]'

But if you don't have a PW between two PEs, you may not need any
tunnel between them. 

> of PWs using an already existing tunnel would not affect FIB in the

But what if a tunnel does not already exist? Then you need to create
a tunnel for the first PW between two PEs.

> capaity on this "network".

Sure, but the path that the tunnel should take depends on the
traffic parameters of the PWs inside that tunnel. In other words
the tunnel path computation needs to take into account the BW
requirement of the PWs, and if the BW of an existing tunnel is not
enough and can't be increased, then another tunnel needs to be created. 

> EXP<-->PHB mappings, too.

Sure, but you can't do TE with E-LSPs ! (See DS-aware TE drafts)

> This would only happen if you apply the "One PW = One Tunnel" model

No, even in case of N PW = 1 tunnel you still need to create the tunnel.
Even if a tunnel already exist between two PEs, it may not be a suitable
tunnel for carrying the PWs (e.g., may not have enough BW). 

> separates creation of transport tunnels from creation of PWs is

I think what John was trying to say is that, Even in case of tunneled
PWs, the P routers' configuration and FIB tables are not 100% PW independent.

Yours,
-Shahram

---

From: "Rutemiller, John" <[email-masked]>
Date: Wed, 15 May 2002 09:57:56 -0400

There are many ways to build a network.

The virtual "network" you describe is one way, in which case
the tunnel is not created explicitly for PW transport. But
others may choose to create a tunnel explicitly for PW transport. 
In the later case, the tunnel is closely tied to the PW. No, the
tunnel is not created in direct response to any PW stream. But it
is still created directly for the PW service.

Same goes with the choice of whether to use an E-LSP. Network
design decision.

Clearly, using a common LSP for all traffic between two nodes
and using E-LSPs has certain scaling and restoration advantages
(fewer restore faster for the entire network).

But, some carriers may want to seperate the fate of the different
traffic. For example, the carrier may want to restore PW services
before voice services because the PW users pay more for the bandwidth.

John

---

From: Sasha Vainshtein <[email-masked]>
Date: Wed, 15 May 2002 17:13:58 +0200

John,
Thank you for a prompt response.
Please see some more comments inline.
Hopefully they will be useful.

With best regards,
                                   Sasha Vainshtein

> Sent: Wednesday, May 15, 2002 3:58 PM
> To: [email-masked]'
No problem with that. But, IMHO, there is a difference between
creating a tunnel to b used by a (still unknown) group of PWs
and creating a tunnel per PW.
In the 1st case, there is no casual connection between creation
of any specific PW and the state of P routers; in the 2nd one,
there would be.
> tunnel is not created in direct response to any PW stream. 
>
IMHO this means that setup/teardown of PWs do not exert any
control over the state of P routers, so it's OK.
>
> But it is still created directly for the PW service.
>
My English may be faulty, but I would say that the tunnel is
created to be used by the (multiple and currently non-existing) PWs, 
not "for the (specific) PW service". If I install  new interface cards  in
my 
P routers and connect them over a new fiber with the intent to 
use new capacity thus created for (yet non-existent) PWs, 
these PWs still do not exert any control over my PSN!

---

From: Shahram Davari <[email-masked]>
Date: Wed, 15 May 2002 07:37:59 -0700

Sasha,

> Sent: Wednesday, May 15, 2002 11:14 AM
> To: 'Rutemiller, John'; [email-masked]'

OK, how about this: I have my PSN and don't have any PW yet, but I
expect to have N*PW (N is small) between to PEs. I could create 
N*LSP between the two PEs. Now After I receive a PW request, I choose
one of the LSPs that best fits that PW. 

If no suitable LSP is found, then I either will change the attributes 
of that LSP (by signaling) or create a new LSP. This last part is no
different in tunneled or non-tunneled solution, because even in case
of tunneled solution, if the tunnel is not suitable you need to take
exactly the same actions.

> control over the state of P routers, so it's OK.

What if all PWs are tear downed, do you still keep the tunnel? 
if so why?

> these PWs still do not exert any control over my PSN!

But PWs can't use those links either, unless you modify the
FIBs on those routers.

-Shahram

---

From: Vinay Bannai <[email-masked]>
Date: Wed, 15 May 2002 08:00:17 -0700

Shahram,

One of the points that you made at the bottom of this email  " but you can't
do TE with E-LSPs " actually can be addressed in some network architectures.
If you engineer the network with a some sort of centralized CAC mechanism
then this can be accomplished. However, this does restrict on the ways the
network can be architected. But, in reality, I would like to know how many
SPs really have the intelligence built into their networks so that the TE
happens on the fly among the network elements. 

Vinay

-----Original Message-----
From: Shahram Davari [mailto:[email-masked]] 
Sent: Wednesday, May 15, 2002 6:50 AM
To: 'Sasha Vainshtein'; 'Rutemiller, John'
Cc: [email-masked]'
Subject: RE: [PWE3] PWE3 question

Sasha,

I think what John was trying to say is that, Even in case of tunneled
PWs, the P routers' configuration and FIB tables are not 100% PW
independent.

Yours,
-Shahram

---

From: Sasha Vainshtein <[email-masked]>
Date: Wed, 15 May 2002 18:03:17 +0200

Shahram,
Please see some comments inline.
Hopefully they clarify my point.
With best regards,
                                   Sasha Vainshtein
email:     [email-masked] <mailto:[email-masked]> 
tel:       +972-3-7659993 (office)
           +972-8-9254948 (res.)
           +972-58-674833 (cell.)

> Sent: Wednesday, May 15, 2002 4:38 PM
> To: 'Sasha Vainshtein'; 'Rutemiller, John'; [email-masked]'
Probably I do - because I may use it for other traffic including other
PWs that have not been set up yet.
> But PWs can't use those links either, unless you modify the
The FIBs will be modified as the result of installation of interface
cards, regardless of the PWs.

---

From: Shahram Davari <[email-masked]>
Date: Wed, 15 May 2002 08:12:50 -0700

Sasha,

> Sent: Wednesday, May 15, 2002 12:03 PM
> To: Shahram Davari
> Cc: 'Rutemiller, John'; [email-masked]'

You could do the same with what I proposed above (N*LSPs).

> cards, regardless of the PWs.

Are you talking about LDP? But be realistic you need TE tunnels for PW,
otherwise you need massive over-provisioning. In TE case you need
to signal new LSPs !

-Shahram

---

From: Eric Rosen <[email-masked]>
Date: Wed, 15 May 2002 11:56:57 -0400

John> Protocols should not dictate network architecture.

John> The choice of whether an operator desires to use a separate
John> label inside the network for each psuedo-wire, or to use a
John> label stack to improve scaling inside the network is a
John> network design issue.

John> It is not a protocol design issue.

It is  possible to use  a two-layer stack  and still design a  network which
lays  down state  for each  PW in  the core  routers, and  hence is  just as
unscalable  as an ATM  network.  The  protocol design  does not  dictate the
network design. 

The primary  issue is, do we really  need a protocol which  is optimized for
unscalable network designs (uhh, I mean for network designs in which each PW
has state in the core routers)?  Anyone who thinks so should make a proposal
and should  make it clear why  the use of one  rather than two  labels is so
important.

The secondary issue is whether  any particular protocol should be considered
to be universally mandatory (for conforming products to provide), or whether
each can just be implemented within its own scope of applicability.

There is also an issue as to whether there is some realistic set of service
characteristics which  cannot be provided  unless the core  routers maintain
state per PW.   It will be hard  to get consensus on a  "one label" protocol
unless one  can first get  consensus on this.   Since the use of  two labels
does  not rule  out the  creation of  per-pseudowire state  within  the core
routers, I'm not sure why folks even want to fight this battle.

---

From: Shahram Davari <[email-masked]>
Date: Wed, 15 May 2002 12:18:36 -0700

Eric,

> Sent: Wednesday, May 15, 2002 11:57 AM
> To: Rutemiller, John
> Cc: [email-masked]'
> unscalable  as an ATM  network.
  The  protocol design  does 
> not  dictate the

1) The use of 1 label reduces the number of label lookups that needs
to be supported at PEs. Same objective as PHP!
2) If the MPLS networks must always be designed scalable, then perhaps
MPLS standard should mandate hierarchical LSPs, because all single level
LSPs have the scalability issue that you are referring to.

> each can just be implemented within its own scope of applicability.

I think each is applicable to its scope.

> state per PW.

The issue is opposite: Is there a realistic set of service characteristics which
can be provided even when the core routers maintain state per PW? 

>  will be hard  to get consensus on a  "one 
> label" protocol

The issue is not the need to keep the state at the core,
rather it is why it MUST be precluded?

-Shahram

---

From: Eric Rosen <[email-masked]>
Date: Wed, 15 May 2002 17:06:01 -0400

Shahram> 1) The use  of 1 label  reduces the  number of label  lookups that
Shahram>    needs to be supported at PEs. Same objective as PHP!

You could of course use PHP ;-)

Shahram> The issue is not the need to  keep the state at the core, rather it
Shahram> is why it MUST be precluded?  

My point was  that it is not precluded.  The signaling  protocols do not say
how one maps pseudowires to tunnels, hence you could have one pseudowire per
tunnel if that's what you want.

---

From: Shahram Davari <[email-masked]>
Date: Wed, 15 May 2002 14:31:54 -0700

Hi Eric,

> Sent: Wednesday, May 15, 2002 5:06 PM
> To: Shahram Davari
> Cc: Rutemiller, John; [email-masked]'

1) The penultimate hop may not be able to do PHP.
2) Some view PHP as a bad architecture.

> tunnel if that's what you want.

Why not make it simpler and not have any tunnel at all?

-Shahram

---

From: Eric Rosen <[email-masked]>
Date: Wed, 15 May 2002 17:51:05 -0400

Shahram> 1) The penultimate hop may not be able to do PHP. 

Are you think  of ATM-LSRs?  If you're going  to use an ATM VC  for each PW,
why use MPLS at all?

Shahram> 2) Some view PHP as a bad architecture.

Oh, please.

Shahram> Why not make it simpler and not have any tunnel at all?

Having a different protocol for every different network design isn't the way
to make things simpler.  To make things simpler, you choose a protocol which
works reaonably well in all the cases, even if it is not optimal for some of
the cases.

---

From: Sasha Vainshtein <[email-masked]>
Date: Thu, 16 May 2002 08:11:15 +0200

Eric and Shahram,

With best regards,
                                   Sasha Vainshtein

> Sent: Wednesday, May 15, 2002 11:51 PM
> To: Shahram Davari
Or even is not optimal for ANY of the cases. Having a single protocol
carries with it its own advantages, and a trade-off between these
and less than optimal coverage of any specific issue is
quite reasonable.

---

From: Maarten Vissers <[email-masked]>
Date: Thu, 16 May 2002 10:14:18 +0200

Any transport network first installs equipment and fibers based on a network
planning result. These physical resources are installed before any traffic is
being carried.

After the physical resources are installed and a first level of connectivity is
available, the next layers in the network are "installed". I.e. trails are set
up at the second layer network to provide (G.805) links to the third layer
network, trails are set up at the third layer network to provide (G.805) links
to the forth layer network, etc. In this way the topology of each layer network
is created, and the transport network is ready to support user traffic. 

So a part of the capacity of the transport network is "burned" inside the
transport network to provide for capacity in client layer networks within that
transport network. The remainder of the capacity in each of the layer networks
is to support user connections through the transport network. 

This set up of trails supporting client layer links within the transport network
is typically done before any user traffic is routed over the network. It is
based on network planning with the goal to optimise the use of the transport
network and its resources.

To support PWE3 services, a specific fully meshed PWE3-PSN (sub)layer network is
to be created (within the boundaries of a general purpose MPLS layer network),
in which PSN-tunnels (which are MPLS (sub)layer network trails) support the
PWE3-PSN links (G.805 terminology). These PWE3-PSN links terminate in the PEs,
in which the individual PWE3-PSN link connections (i.e. the PWs) are connected
to PWE3-PSN trail termination functions (i.e. the MPLS VC termination points).

As layer network topology should typically not be changed on individual
connection requests, the PWE3-PSN (sub)layer network topology must connect a PE
to all other PEs before first user traffic is accepted (assumption is that more
than one user will be connected to a PE).

In a typical transport network in Europe with some 7500 offices to which users
connect, this implies about 28 million PWE3-PSN links (i.e. PSN-tunnels) are to
be set up. Most likely however this number can not be handled administratively,
and additional grooming points in the middle of the network are required. The
single-hop-PE-to-PE approach can be maintained when a hierarchy of PSN-tunnels
is deployed, or a multi-hop-PE-to-PE approach is to be introduced in which
PSN-tunnels (PWE3-PSN links) terminate at intermediate nodes (P) and MPLS VC
signals are cross connected/routed from an ingress PSN-tunnel to an egress
PSN-tunnel. The MPLS VC label will not longer be a fixed label however, as
translation is to be supported at those points.

This latter shouldn't impact the specific PWE3 work. PWE3 defines the
encapsulation/mapping methods, which should be independent of the transport of
this signal through the PSN.

Regards,

Maarten

---

From: Sasha Vainshtein <[email-masked]>
Date: Thu, 16 May 2002 12:25:00 +0200

Maaten,
Thank you for a very nice explanation.
Please see some comments inline.
Hopefully they clarify my position.

With best regards,
                                   Sasha Vainshtein

> From: Maarten Vissers [mailto:[email-masked]]
> Sent: Thursday, May 16, 2002 10:14 AM
> To: [email-masked]'
...snipped...
> To support PWE3 services, a specific fully meshed PWE3-PSN 
> (sub)layer network is to be created (within the boundaries of a general
purpose 
> MPLS layer network), in which PSN-tunnels (which are MPLS (sub)layer
network 
> trails) support the PWE3-PSN links (G.805 terminology). 
> These PWE3-PSN links terminate in the PEs,
> in which the individual PWE3-PSN link connections (i.e. the 
> PWs) are connected to PWE3-PSN trail termination 
> functions (i.e. the MPLS VC termination points).
>
Alternatively, one could use the full mesh of PE-to-PE LSPs
created by LDP in the DU mode and, if necessary, enhance it
using, say preconfigured EXP<-->PHB mappings (if the number
of PHBs to be supported is low) to make this sub-layer
DiffServ-enabled. 
>
> As layer network topology should typically not be changed on 
> individual connection requests, the PWE3-PSN (sub)layer network topology 
> must connect a PE to all other PEs before first user traffic is accepted 
> (assumption is that more than one user will be connected to a PE).
>
The alternative mentioned above complies with this. 
>
> In a typical transport network in Europe with some 7500 
> offices to which users connect, this implies about 28 million 
PWE3-PSN links (i.e. PSN-tunnels) are to
> be set up. Most likely however this number can not be handled 
I agree. But this number, presumably, can be handled by LDP
in DU mode (noted, the LSPs thus produced would be
most probably p2mp, and would not guarantee any resource
allocation).
>
> and additional grooming points in the middle of the network 
> are required. 
>
This is another alternative. I would be interesting to compare it
with the previous one regarding level of guarantees that can be
provided.
>
> The single-hop-PE-to-PE approach can be maintained when a 
> hierarchy of PSN-tunnels is deployed, or a multi-hop-PE-to-PE 
> approach is to be introduced in which PSN-tunnels (PWE3-PSN links) 
> terminate at intermediate nodes (P) and MPLS VC
> signals are cross connected/routed from an ingress PSN-tunnel 
> to an egress PSN-tunnel. The MPLS VC label will not longer be a fixed 
> label however, as translation is to be supported at those points.
>
IMHO this approach reflects one of the fundamental problems with
G.805: the client trail-to-server trail relationship is many-no-many
and not many-to-one. The resulting complexity (from my former experience
with SONET/SDH network management systems) is enormous even if,
in some cases, you cannot do better.
So far, the PWE3 approach has been to normalize this relationship to
many (client trails = PWs)-to-one (server trail = PSN tunnel).
It would be really challenging to understand the limitations of this
approach regarding solid traffic guarantees etc.
> encapsulation/mapping methods, which should be independent of 
> the transport of this signal through the PSN.
> 
>
I agree.

---

From: Shahram Davari <[email-masked]>
Date: Thu, 16 May 2002 06:46:38 -0700

Sasha,

DU LDP can't provide the level of QoS and connectivity needed for ATM VCs,
unless massive over provisioning is done.

-Shahram

> Sent: Thursday, May 16, 2002 6:25 AM
> To: 'Maarten Vissers'
> Cc: [email-masked]'
> > > > > > >The establishment of the tunnel will affect the 
> FIB if the P
> > > > > > routers.
> > > > > > >

---

From: "Khalid Ahmad"<[email-masked]>
Date: Thu, 16 May 2002 15:17:59 -0400

Hi John,

I fully agree with your comments, and particularly in respect to this myth
about ATM "scalability".  

(By the way, I notice with some amusement that this term "scalability" is
frequently used by some IETF colleagues as a disdainfully derogatory label
for anything disliked !  What does it mean precisely ?  At least the context
in which it is used should be qualified. )

ATM was intended to be used in conjunction with signaling protocols (PNNI,
etc.), and not just for providing semi-permanent VCCs or VPCs, even though
these may be the main use nowadays. 

If people simply use PVCs, then sooner or later the n**2 limitation will
become an issue.

If SVCs are used as intended, and assuming reasonable holding times, the
n**2 limitations hardly apply.  In this respect, ATM is no more or less
scalable than any other label multiplexed, connection-oriented packet
technology being used out there, in my understanding. 

Also, while I agree label stacking is a useful concept, it simply implies
the nesting of the  virtual circuits in the multiplexed packet stream.
There is no magic here, and this nesting  of the VCs does not enable us to
get around the usual constraints of queueing traffic performance.

But I know myths die hard, since its often in the interests of some folks to
propagate them for whatever reasons !

Regards,
Khalid

-----Original Message-----
From: Rutemiller, John [mailto:[email-masked]]
Sent: Wednesday, May 15, 2002 6:40 AM
To: [email-masked]'
Subject: RE: [PWE3] PWE3 question 

John

---

From: Eric Rosen <[email-masked]>
Date: Thu, 16 May 2002 16:32:39 -0400

Since you don't understand why ATM is unscalable, a little clarification may
be in order.

Scalability  is a matter  of the  way in  which an  increase in  demand gets
reflected as an increase in the resources needed to support that demand.  To
talk sensibly of scalability, you have  to specify a particular demand and a
particular set of resources.

In the present context, "demand" is  the number of PWs that the network must
support at  some given  time, and  the number of  PEs the  network supports.
"Resources" are entries in the cross-connect tables of intermediate switches
(P  routers).  (I  thought this  context was  understood, but  perhaps not.)
Let's suppose you have n PE routers, and each terminates m PWs.

Case 1: Martini signaling, mp2p LSPs.  The amount of resources needed in each
P router is O(n).

Case 2: Martini signaling, p2p  LSP tunnels.  The amount of resources needed
in each P router is O(n**2).

Case 3: ATM style, no tunneling, each PW known to the P routers.  The amount
of resources needed in each P router is O(m*n). 

I think  it is clear  enough that for  a given set  of P routers,  the upper
bound on  the demand that can  be handled is much  larger in case  1 than in
cases 2  or 3.   So case 1  would be said  to scale  better in terms  of the
number of cross-connect  entries needed in each P router  to support a given
number of pseudowires.  Note  that in cases 2 and 3, it  does you no good to
simply add more P routers when your demand increases, as it is the resources
in each P router that are scaling  as the square of the demand. (Yes, I know
that not every PW goes through every P router, but that is a minor effect.)

A scheme in which resources in a number of intermediate switches increase as
the square  of the  demand would generally  be considered  unscalable.  What
this means in practice is that relatively small increases in demand call for
larger and  larger intermediate switches.  My understanding  is that service
providers do not generally like it when this happens.

The  fact that  each new  generation of  ATM switch  supports more  and more
cross-connect entries does not show that there is no scalability problem; on
the contrary it shows that there is a scalability problem. 

The  only  way   around  scalability  problems  is  to   use  hierarchy  and
aggregation.  The cost of using  hierarchy and aggregation of course is that
you lose granularity and optimality.   If one wants good scaling, one learns
to live with this, even if it means diffserv.

If the demand is characterized as  "PWs supported at a given time", then the
SVC/PVC distinction is irrelevant. 

Note  that the  resource is  characterized as  cross-connect entries  in the
switches, NOT as space  in the VPI/VCI field.  So the size  of that field is
also irrelevant.

I recognize that ATM does have a means of aggregation (VP-switching), though
it is limited in applicability, due  to a lack of consistency about who owns
the VPI  field.  But presumably those  who don't want  to aggregate multiple
PWs into a single LSP tunnel wouldn't use VP-switching either.  

But I'll acknowledge that it might  be more precise to say "ATM VC-switching
is unscalable" rather than "ATM is unscalable".

---

From: "David Allan"<[email-masked]>
Date: Thu, 16 May 2002 16:52:26 -0400

Eric:

This one has been nagging me for a while:

<snip>

> Case 1: Martini signaling, mp2p LSPs.  The amount of 

An mp2p LSP mesh can be said to have O(n) egress points, but it has O(n**2)
ingress points.  So if I compare the actual fabric consumed between a full
mp2p and a p2p mesh I would get O(n**2) * (average network diameter) LSHs
for p2p, and some fraction of this for mp2p (again a function of diameter in
hops reflecting on how fast the ingresses merged on the egress). mp2p
requiring uni-directionality then doubles the fraction as far as actual
administrative/signalling load is concerned (each direction requires a
separate set of transactions to establish).

So one can claim some incremental improvement in scalability via the use of
mp2p but it is not this exponential thing that everyone waves about......in
other words single level MPLS is a small fixed amount of time apart from ATM
on Moores law curve (stacking being a different story).

cheers
Dave

---

From: [email-masked]
Date: Thu, 16 May 2002 23:36:15 +0100

Agreed Dave.....Heinrich Hummel has also done some work on this (note sure
if he has posted a revised draft on this yet, I have seen one he sent me
privately) and his major conclusion is that an LSP hierachy is needed to
properly address scaling (which is not news to us operators and our trad
suppliers as we have been doing G.805 client/server stuff for years).

But why are you only telling half the story here Dave?  Scalability needs a
caveat to say that whatever parameter it is that one is scaling, it must not
break other important facets for the service it supports.  mp2p is alledged
to save something on LSPs/labels (not clear IMO) but at the expense of:
-	requiring operators to throw BW (=money) at VPN services......since
like-DS-class merging across such means we can't decouple survivability and
QoS....but still no solid gurantees are possible;
-	complex fault/QoS management problems
So seems it does not scale in these dimensions.

The root cause of these problems is 2-fold:
-	merging a blunt tool...so use it carefully
-	in co pkt-sw networks one cannot consider pkts in isolation (like
cnls networks), they only have relevance in the context of their parent
trail (LSP) object....and it is this that sets the semantics for primary
management focus.  BTW - this is also at the heart of why the so-called
'port-mode' in some of the ATMoverMPLS drafts won't work properly.

regards, Neil

-----Original Message-----
From: David Allan [mailto:[email-masked]]
Sent: 16 May 2002 21:52
To: [email-masked]; Khalid Ahmad
Cc: Rutemiller, John; [email-masked]'
Subject: RE: [PWE3] PWE3 question 

Eric:

---

From: Eric Rosen <[email-masked]>
Date: Thu, 16 May 2002 23:24:20 -0400

>> Case 1: Martini signaling, mp2p LSPs.  The amount of 

David> An mp2p LSP mesh  can be said to have O(n) egress  points, but it has
David> O(n**2) ingress points.   So if I compare the  actual fabric consumed
David> between a  full mp2p and  a p2p mesh  I would get O(n**2)  * (average
David> network diameter)  LSHs for p2p, and  some fraction of  this for mp2p
David> (again  a function of  diameter in  hops reflecting  on how  fast the
David> ingresses merged on the egress). 

I don't  follow your reasoning here.   Suppose an intermediate  switch has k
interfaces.   With  respect  to  a  given egress,  this  is  one  downstream
interface  and k-1  upstream interfaces.   With  respect to  that egress,  a
single mp2p  LSP can arrive  over each of  the k-1 upstream  interfaces, and
these k-1 entities need to be merged into a single LSP going downstream.  If
you're pointing  out that,  given n egresses,  this actually  requires state
which is O(k*n)  rather than O(n), the  point is taken, but k  is in general
much smaller than n, so that O(k*n) does not differ from O(n), and certainly
does not approach O(n**2). 

So  I don't  see the  relevance of  there being  O(n**2) ingress  points.  I
wonder if you  have a different model of how the  merging works.  Or perhaps
you have a different model of the relation between k and n?

David> mp2p requiring uni-directionality then doubles the fraction as far as
David> actual  administrative/signalling load  is concerned  (each direction
David> requires a separate set of transactions to establish). 

I did not address the issue of how well the signaling load scales, but: 

- 2*O(n) = O(n) < O(n**2)

- with DU  signaling and liberal retention the  signaling is really one-time
  only, i.e., it's a startup transient and imposes no ongoing cost; the same
  could be said of an ATM PVC model, of course

- I don't  happen to know the relative efficiency of  LDP signaling vs. PNNI
  signaling,  but I  don't see  how one  could draw  conclusions  about that
  without looking at the actual details; "one is unidirectional the other is
  bidirectional" doesn't really tell us anything.

---

From: Vijay Amrit Agrawal <[email-masked]>
Date: Fri, 17 May 2002 17:38:38 +0530

My Two cents:
This is the example I have used to understand the issue:

   A         B
     \     /
       P
     /    \
 C       D

If we want full mesh connection b/w A,B,C,D
i) Leased line for each connection. O(N**2) Physical lines. Best QOS.
       The states we have in end points (i.e. A, B, C, or D) is O(N).
        No Address/label maintenance.
ii) Logical mesh connection and one physical line for each port (X.25, Frame
relay, ATM without VP, MPLS with out hierarchical LSP/without merging,
etc.):
      O(N) physical lines.(one for each to P).
      In P we have to have O(N**2) connection state (In Frame relay , for D
there will be a DLCI for A-D, B-D, C-D  so on for other end points).
      The states we have in end points (i.e. A, B, C, or D) is O(N).
      Address/label uniqueness need to be maintained along the path from
source to destination. (i.e. B can use the same label to address C, as A
uses to reach D. )
iii) Connectionless mesh connectivity say we give an address for each node
(like in IP).
     O(N) physical lines.
     In P we have O(N) state to maintain ( i.e. for D go to this Port i.e.
one entry).
      The states we have in end points (i.e. A, B, C, or D) remains O(N).
      Address/label uniqueness needs to be maintained in the whole network (
i.e. if B uses some address to reach C, A cannot use the same address to
reach D).

      In this sense IP is MP2P.
      In some cases like this one, MPLS with LSP merging is connectionless
like IP but described differently (for example: In the above we can say to
go to D, we merge the LSP from A-D, B-D and C-D).

QOS wise i) is best. When we have to move a Network from i) to ii) there
will be issues of QOS and security and traffic getting mixed re-directed
etc.
Similarly when we move from ii) to iii).

Note: I am talking from a general theoretical perspective. I don't think
that we can just take iii) can be done only the IP way and so iii) has all
the problems that IP has. 

Regards,
Vijay

---

From: "David Allan"<[email-masked]>
Date: Fri, 17 May 2002 09:32:53 -0400

Hi Eric:

I'll try to express this a different way.

1) Can we agree that there are O(n**2) ingresses regardless of the style of
meshing (mp2p or p2p)?

2) If there are O(n**2) ingresses, there is at least O(n**2) LSHs prior to
any merging in the network.

3) After the first merge point the number of LSHs required will be some
fraction of O(n**2) and as you progress across merge points, the number of
LSHs required at each subsequent hop coverges on O(n). (at that generously
assumes each LER has only one interface facing the network)

4) If I do a sum of the LSHs that actually implement the connectivity, it
will be some fraction of O(n**2) * network diameter vs. O(n**2) times
network diameter for unmerged case.

I got the gist of your example, but being a single node and scaled to the
number of interfaces on the node, the math works out to O(k) vs. O(2(k-1)),
which as I asserted, mp2p vs p2p is a linear relationship when the sum of
LSHs is considered.

cheers
Dave

> Sent: Thursday, May 16, 2002 11:24 PM
> To: Allan, David [CAR:NS00:EXCH]
> Cc: Ahmad, Khalid [CAR:CS40:EXCH]; Rutemiller, John; [email-masked]'

---

From: Eric Rosen <[email-masked]>
Date: Fri, 17 May 2002 10:01:34 -0400

David> 1) Can we  agree that there  are O(n**2) ingresses regardless  of the
David>    style of meshing (mp2p or p2p)? 

It is true that each of the n PEs  has an LSP leading to each of the other n
PEs.  This is a total of n*(n-1) LSPs. 

However, each ingress  has only n of these.  So the  resources needed in the
PE to support either kind of mesh scale as O(n).

David> 2) If there  are O(n**2)  ingresses, there is  at least  O(n**2) LSHs
David>    prior to any merging in the network. 

What's  "LSH"?  I must  have missed  the genesis  of that  acronym somewhere
along the line. 

The question  is whether any of the  P routers has to  support O(n**2) LSPs.
With mp2p,  the LSPs merge at  every P routers,  so any given P  router only
supports O(n)  LSPs.  Without merging,  any given P router  supports O(n**2)
LSPs.  You  seem to  be disagreeing  with this, but  I don't  understand the
basis for that disagreement.

---

From: Eric Rosen <[email-masked]>
Date: Fri, 17 May 2002 10:20:07 -0400

Vijay> In this sense IP is MP2P. 

Absolutely.   That's  because IP  forwarding  is  based  on the  destination
address and is independent of the source address.

Vijay> In some cases like this  one, MPLS with LSP merging is connectionless
Vijay> like IP 

Absolutely. That's  why when folks  say "if you  use merging, you  make your
network  unmanageable",  what  they  are really  saying  is  "connectionless
networking doesn't  work".  That's a point of  view which can be  put in the
same category as "the earth is flat".

Vijay> QOS wise i) is best. 

One of  the basic design principles  of the Internet,  and of connectionless
networking in  general, is  that scalability is  the number one  factor, and
everything  else  is  subservient.   Service  characteristics  that  inhibit
scalability just don't get offered. 

Folks who  have disagreed with this  design principle have said  in the past
that "X.25 will kill IP", that "SNA  will kill IP", that "ATM will kill IP",
etc., etc.  Given what's actually happened, I think the only conclusion that
can be drawn is that,  for data networking, the service characteristics that
inhibit scalability just aren't as important in the marketplace as they were
once thought to be. 

So perhaps in theory a non-merging connection-oriented paradigm provides the
"best" QoS,  but in  practice that  level of QoS  does not  seem to  be very
important in data networking.

---

From: "David Allan"<[email-masked]>
Date: Fri, 17 May 2002 11:21:51 -0400

Hi Eric:

<snip>

> PE to support either kind of mesh scale as O(n).

Agreed.

> What's  "LSH"?  I must  have missed  the genesis  of that  

label switch hop "the hop between two MPLS nodes on which forwarding is done
using labels"

> With mp2p,  the LSPs merge at  every P routers,  so any given 
> P  router only supports O(n)  LSPs.  Without merging,  any given P router

> supports O(n**2)LSPs.  You  seem to  be disagreeing  with this, but  I
don't  
> understand the basis for that disagreement.  

Because this would only be true if the network was edge boxes and one
honking huge LSR in Kansas. I will agree that in the MP2P network will
require O(n) egresses on every LSR, but I would not assume that the O(n**2)
LSPs needs to be routed through every node in the network. The delta in
scaling is not as drastic as suggested. 

I think we are saying slightly different things. Your point is that the
number of egresses at any given node is bounded to O(n) which is a nice
property. My point is the cumulative fabric in the network required for p2p
vs. mp2p is not as different as may superficially be suggested by the O(n)
vs. O(n**2) comparison.

cheers
Dave

---

From: "David Allan"<[email-masked]>
Date: Fri, 17 May 2002 14:51:35 -0400

Zheng:

I consider state and fabric to be equivalent. Each LSH involves some state
at the LSRs in either end, therefore the cumulative number of LSHs in the
network is indicative of the amount of state involved.

cheers
Dave

> From: Zheng Wang [mailto:[email-masked]]
> Sent: Friday, May 17, 2002 1:56 PM
> To: Allan, David [CAR:NS00:EXCH]; [email-masked]
> Cc: [email-masked]
> It seems that the following holds for the state in the P routers:
> 
> 1) one label, O(n**2) where n is the number of ingresses on PE
> 2) two labels, O(n**2) where n is the numebr of egresses on PE
> 3) label merge, O(n) where n is the number of 
> ingresses/egresses for 1/2 labels
>  
> What you are saying about "fabric" is true but we are talking 
> about the
> state (size of the table) which is different (assuming that 
> the downstream
> router has control over the label assignment). Of course, 
> when you merge,
> you lose control over individual paths. It is a tardeoff. 
> 
> Cheers
> Zheng  
>

---

From: "David Allan"<[email-masked]>
Date: Fri, 17 May 2002 15:08:31 -0400

Vijay:
 
In your 'iii' , P will have more than O(n) state to maintain. It will cross
connect O(n**2) ingress LSHs to O(n) egress LSHs. Looking at the LSR MIB
(for example) that translates to O(n**2) instances of the mplsInSegmentEntry
and mplsInSegmentPerfEntry objects and O(n) instances of the XC and
OutSegment objects (or whatever their implementation translates to
statewise...). 
 
cheers
Dave

-----Original Message-----
From: Vijay Amrit Agrawal [mailto:[email-masked]]
Sent: Friday, May 17, 2002 8:09 AM
To: [email-masked]'
Subject: RE: [PWE3] PWE3 question

My Two cents: 

   A         B 
     \     / 
       P 
     /    \ 
 C       D 

Regards, 
Vijay 

-----Original Message----- 
From: Eric Rosen [ mailto:[email-masked] <mailto:[email-masked]> ] 
Sent: Friday, May 17, 2002 8:54 AM 
To: David Allan 
Cc: Khalid Ahmad; Rutemiller, John; [email-masked]' 
Subject: Re: [PWE3] PWE3 question 

_______________________________________________ 
pwe3 mailing list 
[email-masked] 
https://www1.ietf.org/mailman/listinfo/pwe3
<https://www1.ietf.org/mailman/listinfo/pwe3>

---

From: Eric Rosen <[email-masked]>
Date: Fri, 17 May 2002 15:16:23 -0400

David> I will agree  that in the MP2P network will  require O(n) egresses on
David> every LSR, but  I would not assume that the O(n**2)  LSPs needs to be
David> routed through every node in the network. 

In the case of a full mesh of  p2p circuits among n PEs, the total number of
LSPs is  exactly n*(n-1)/2 (assuming bidirectionality).  Now  it's true that
not every one of these LSPs will pass through every P router.  Let's suppose
that only one out  of every j LSPs passes through each  P router.  The value
of  j  is a  function  of  the  topology, and  for  a  given topology  is  a
constant. If  k is  the average number  of interfaces,  a P router  needs to
maintain on the average kn*(n-1)/2j LSPs.  It seems to me that this is still
O(n**2), i.e., the high order factor is n**2 times a constant.

---

From: Eric Rosen <[email-masked]>
Date: Fri, 17 May 2002 15:31:19 -0400

David> In your 'iii' , P will have more than O(n) state to maintain. It will
David> cross connect O(n**2) ingress LSHs to O(n) egress LSHs. 

Well, in  Vijay's example there is no  upstream merging going on,  and as it
happens, the  number of interfaces P has  is exactly equal to  the number of
PEs.  So in this particular case  kn (k being the number of interfaces) just
happens to be equal to n**2, and merging doesn't buy you very much. 

But  to see the  scalability implications,  what you  have to  do is  hold k
constant while allowing n to grow.

---

From: "David Allan"<[email-masked]>
Date: Fri, 17 May 2002 15:33:20 -0400

My analysis mapped onto yours suggests that 'j' is in proportion to 'n' for
a given network for both of our conclusions to be true. Not clear where 'k'
comes from. If one in 'j' LSPs passes through a 'P' router, then the number
of interfaces on a 'P' shouldn't matter.

Otherwise agree. 
Dave

> Sent: Friday, May 17, 2002 3:16 PM

---

From: "Cheng-Yin Lee" <[email-masked]>
Date: 17 May 2002 15:36:26 -0400

Ok, for what it's worth, here are 2 hypothetical examples which might help
illustrate the points being discussed.

There are n PEs in the network and each PE has an LSP to another PE.

i) all PEs are connected to one P,

PE--------P---------PE
               / \
PE------/    \-------PE

For p2p, there are O(n^2)  LSPs in P
For mp2p, there are O(n) LSPs in P

ii) all PEs are connected to a different P, Ps are fully-meshed,
LSPs are routed via directly connected Ps ( LSPs are NOT all routed via one P)

PE-------P-------P------PE
                 |   \  /  |
                 |    /\   |
                 |  /    \ |
PE-------P-------P------PE

For both p2p and mp2p, there are O(n) LSPs in P

So Eric, I think your statement below "Without merging,  any given P router
supports O(n**2) LSPs." is true for (i), but not (ii). This may be one of Dave
Allan's points.

In (ii), as the number of PEs connected to a P increases, the number of LSPs in
a P increases for p2p, but not for mp2p, so your related point on this is well
taken (but the number of LSPs on P is still O(n)). It looks to me  n^2  LSPs on
P is applicable to extreme cases as in (i) or if all LSPs are routed through one
P in (ii), but not in general.

regards
cheng-yin

---

From: [email-masked]
Date: Sat, 18 May 2002 09:59:00 +0100

Eric Rosen wrote 17 May 2002 15:20
<snip>
> same category as "the earth is flat".

Eric I think you are being rather extreme here.  I don't think anyone is
saying cnls behaviour is not something we need....I am actually a strong
supporter of it whether you believe me or not.  I am also a strong supporter
of MPLS (because its the only technology to date that offers the promise of
greater co/cnls unification), but its hard to sell this to others here (and
this is the vast majority of strategic engineering decision makers) when
they just point out that we can do all we need with ATM (for key revenue
earning services) and see little advantage in using a technology that offers
very little value-add and is far from carrier-strength in some areas.  I
think you should not dismiss this latter observation lightly, and I would be
surprised if a similar situation does not exist in other large operators
too. 

My point is that cnls behaviour alone does not completely satisfy all
applications.  This is a truth by observing the existence of ER/signalled
LSPs (against a TE application) and GMPLS (against a large-admin BW L1
application).  If we don't need these co pkt-sw and co cct-sw behaviours
then (i) why do they exist? and (ii) why are they being worked on now in
IETF?

So whilst mp2p LSP constructs may be fine for some applications (and I used
the example of 'public' services in the past, ie all look/treated alike) it
does not sit easy with VPNs IMO.  Isolation here should be more than simply
address/connectivity isolation....in my experience VPN customers expect
traffic and failure isolation too, ie the traffic behaviour in 1 VPN should
not be a function of the traffic behaviour in other VPNs under all network
states including failures.  If this is not true then its my view that this
does not really constitute the behaviour that customers would expect when
they buy a VPN service with an SLA from an operator.  You obviously don't
agree with this.

The more I think about it the less I am convinced mp2p constructs in MPLS
are really solving any problems (that are appropriate to their usage) that
cannot be solved just as well by IP directly.

The only obvious argument I can see is 1-shot address look-up......but I
understand processor speeds are now such that this is no longer a major
driver.  So what other advantages do they bring?  Well, there is the current
discussion on resource/state comsumption, but so far this does not seem to
be delivering any compelling case for mp2p AFAI can tell (I've now asked my
maths department to dig into this one for me).  They do however seem to
bring problems that as yet don't seem to be well addressed.....like how does
one do automatic fault detection/handling in such networks, and the fact
that operators have to over-engineer to compensate for the fact that pkts
don't have a 1:1 relationship with a LSP (which also may wipe-out any
resource/state savings through increased capex/opex costs....I am also
having this topic researched properly this year).

However, maybe my concerns are ultimately unfounded and perhaps customers
don't care about these matters (irrespective of the opex/capex answers) and
you are right....I guess time will tell.

regards, Neil

---

From: Vijay Amrit Agrawal <[email-masked]>
Date: Mon, 20 May 2002 12:21:00 +0530

Hi Dave,
I am not sure if we are talking about the same thing. I was talking about a
specific way of merging so I put the conditional statement "in some cases".
Let me explain:
In case iii) below, let us just take one destination D. Let us say there is
LSP from A-P with Label X1, B-P with label X2 and C-P with label X3. All
this LSP gets merged to another label say Y1 from P to D. So at first glance
it seems we will need to maintain 3 states X1, X2 and X3 for each incoming
label and it should be swapped with label Y1. Now, if I add some
intelligence and make X1=X2=X3=Z1. I only need to maintain state for Z1
indicating forward packet with label Y1 to so and so interface. I have
reduced the number of state to one for one destination and described the
forwarding in terms of LSP merging. So if there are N destination the number
of states will be O(N).
 
(As far as I know, this doesn't happen in today's world. Neither I have
thought through of any implication in real world, but it looks nice).
Regards,
Vijay

-----Original Message-----
From: David Allan [mailto:[email-masked]]
Sent: Saturday, May 18, 2002 12:39 AM
To: Vijay Amrit Agrawal; [email-masked]'
Subject: RE: [PWE3] PWE3 question

Vijay:

cheers
Dave

-----Original Message-----
From: Vijay Amrit Agrawal [mailto:[email-masked]]
Sent: Friday, May 17, 2002 8:09 AM
To: [email-masked]'
Subject: RE: [PWE3] PWE3 question

My Two cents: 

   A         B 
     \     / 
       P 
     /    \ 
 C       D 

Regards, 
Vijay

---

From: Vijay Amrit Agrawal <[email-masked]>
Date: Mon, 20 May 2002 10:22:48 +0530

Hi,
In the example below: case i) the ratio of PE:P is n:1 and in case ii) the
ratio of PE:P is n:n.  So I think both are extreme cases but helps us in
understanding.
I think we agree that in case of mp2p there is only O(N) states required.
Now coming to the case of P2P. 

If you include all the P routers, the number of states they maintain is
still O(n square) but the states are distributed across all the P routers
i.e. the P router that is not in the path from source to destination need
not keep that information. Now if the number of P routers are same as PE on
average each router has to keep (n square)/n states that reduces it to O(n).

>From this theoretical angle I think the ratio of P routers to PE matters.
But again, think of the case

PEs----P---P---P--P---PEs. In this case we should only consider the P that
are connected to PEs, and not count the P inside for calculating the ratio.

I think the below may be little controversial but will be helpful:
Another example that can be taken is from practical operations. If we think
of IP as mp2p we can see how many routes a core router needs to maintain for
a mesh connectivity b/w all hosts. Because of address aggregation (which is
similar to label merging),  we cannot have the actual number but the number
will be still O(n). With address aggregation it turns out to be just 120K. 
I think it will be difficult to find the number of DLCIs (assuming Frame
Relay) that will be required for building a network like IP, but somebody
with experience with large Frame Relay or ATM network can tell us the ratio
between the number of hosts and the P routers state. ( I understand that
Frame Relay and ATM n/w have features that are not present in IP but this
calculation is just for counting the states.)

Vijay

-----Original Message-----
From: Cheng-Yin Lee [mailto:[email-masked]]
Sent: Saturday, May 18, 2002 1:06 AM
To: [email-masked]
Cc: David Allan; Khalid Ahmad; Rutemiller, John; [email-masked]'
Subject: Re: [PWE3] PWE3 question

PE--------P---------PE
               / \
PE------/    \-------PE

ii) all PEs are connected to a different P, Ps are fully-meshed,
LSPs are routed via directly connected Ps ( LSPs are NOT all routed via one
P)

So Eric, I think your statement below "Without merging,  any given P router
supports O(n**2) LSPs." is true for (i), but not (ii). This may be one of
Dave
Allan's points.

In (ii), as the number of PEs connected to a P increases, the number of LSPs
in
a P increases for p2p, but not for mp2p, so your related point on this is
well
taken (but the number of LSPs on P is still O(n)). It looks to me  n^2  LSPs
on
P is applicable to extreme cases as in (i) or if all LSPs are routed through
one
P in (ii), but not in general.

regards
cheng-yin

---

From: "Zheng Wang" <[email-masked]>
Date: Fri, 17 May 2002 10:55:42 -0700

1) one label, O(n**2) where n is the number of ingresses on PE
2) two labels, O(n**2) where n is the numebr of egresses on PE
3) label merge, O(n) where n is the number of ingresses/egresses for 1/2 labels
 
> vs. mp2p is not as different as may superficially be 

What you are saying about "fabric" is true but we are talking about the
state (size of the table) which is different (assuming that the downstream
router has control over the label assignment). Of course, when you merge,
you lose control over individual paths. It is a tardeoff. 

Cheers
Zheng

---

From: "Bob Barrett" <[email-masked]>
Date: Tue, 21 May 2002 07:25:10 +0100

snip

snip

I agree with you on data, but that level of QoS is important to 'data' when
the data is carrying a voice service, and thus important to pwe3, if the
service providers envisage using pwe3 over a PSN to carry legacy voice
traffic.

All QoS service definitions can be mathematically reduced to a combination
of two basic types of service, fully deterministic and best effort. With
over provisioning best effort approximates to deterministic, and best effort
works very well for IP data, and will work, up to a point, for time critical
traffic.

On a general point:

One of the decisions for this group is where to draw the line on how near to
conventional circuit delivery we want to get, and how much of the transport
issue is in-scope for pwe3. I am concerned that the pwe3 list seems to be
turning into a sub-group of MPLS. Most of the recent emails have had very
little to do with circuit emulation protocol definition and much more to do
with how to transport the resulting PDUs in the PSN/MEN.

Best regards

Bob Barrett

---

From: Hummel Heinrich <[email-masked]>
Date: Tue, 21 May 2002 10:12:29 +0200

Let me add some precise numbers computed based on an example network with 300 nodes and 550 links.  n=100 nodes are PEs, 200 nodes are Ps.
The PEs are distributed randomly but fair. And so is the topology as well.
 
Case 1: Have n times (n-1) p2p LSPs, i.e. 9900 p2p LSPs, each routed according to the "shortest path" principle. 
             Computation result:  99874 hops(=links) are needed.
 
Case 2: Have n mp2p LSPs, i.e. 100 mp2p LSPs, each routed according to the "shortest path to the egress" principle.Alltogether they take 19886 hops(=links).
 
Case 3: Have n mp2p LSPs, i.e. 100 mp2p LSPs, of identical smallest-size tree structure (each time another PE is the root of this tree). 
            100 times 160 hops, i.e. 16000 hops are needed.
 
Case 4: Have n-1 pairs of mutual inverse LSPs which form a bi-directional tree of (elementary) tunnels between the n PEs. They are selected such, that the number of
            hops is minimal. 2 times 169 hops = 338 hops are needed. Additional Hierarchical LSPs, i.e.  concatenations of the elementary  LSPs, shall take care 
            that  there is a   tunnel sequence from any ingress PE to any egress PE. 
            a) [n * (n-1) ]  -    2*(n-1)  non-merging LSP-sequences (=p2p  H-LSPs) are needed
            or b) n merging (mp2p) H-LSPs are needed
           Allthough b) is more conservative on resource consumption, you may consider the cost for the H-LSPs  according to a) as well according to b to be "ZERO 
           for the network" (see may draft-hummel-mpls-hierarchical-lsp-00.txt)

Case 1 is indeed the n-suqare problem (99873 hops). Merging LSPs (see case 2 and case 3) reduce the number of hops considerable, let me say by 80 %.
Case 4 however is the way to overcome the n-square problem (338 hops instead of 99874 hops, i.e. a reduction by 99.66 %).
 
Will say: Merging LSPs do no overcome the n-square problem. I am sure, If I extended the network to let's say 1000 PEs and have similar meshes ( 1 to 5
neighbor nodes per node), the reduction due to merging LSPs stays by 80 % whereas the H-LSPs would yield 99.9966 %.
 
As you can see, the saving due to H-LSPs are immense.Therefore, why shouldn't we concatenate elementary LSPs, i.e. build H-LSPs?!
It is the same logic why we build routers instead of having a full mesh of physical links in the internet!
There are more arguments:
 
- Neither the n*(n-1) p2p LSPs nor the n mp2p LSPs can support Multicast at all (i.e.enable that some packet doesn't have to be sent multiple times over the same physical interface).H-LSPs however will reduce  that this will happen at least considerably. I really wonder why it is acceptable that a P-router may deal with
VPN-multicast, and why multicast blind traffic is acceptable (towards PEs which have no receivers for that Multicast application instance). 
 
- One can hardly affort x times n*(n-1) p2p LSPs or x times n mp2p LSPs, as to support alternate routig (traffic balancing, QoS-/Policy-sensitive routing).
  But one can easily affort  x times the number of H-LSPs. The costs for the network: x times ZERO = ZERO. Indeed, these costs are ZERO, when the
  messages for establishing the H-LSPs are tunneled thru these elementary LSPs, i.e. are completely invisible to the core network.

Regards,
Heinrich Hummel
 
Siemens AG

-----Ursprüngliche Nachricht-----
Von: David Allan [mailto:[email-masked]]
Gesendet: Donnerstag, 16. Mai 2002 22:52
An: [email-masked]; Khalid Ahmad
Cc: Rutemiller, John; [email-masked]'
Betreff: RE: [PWE3] PWE3 question 

Eric: 

<snip> 

> Case 1: Martini signaling, mp2p LSPs.  The amount of 

An mp2p LSP mesh can be said to have O(n) egress points, but it has O(n**2) ingress points.  So if I compare the actual fabric consumed between a full mp2p and a p2p mesh I would get O(n**2) * (average network diameter) LSHs for p2p, and some fraction of this for mp2p (again a function of diameter in hops reflecting on how fast the ingresses merged on the egress). mp2p requiring uni-directionality then doubles the fraction as far as actual administrative/signalling load is concerned (each direction requires a separate set of transactions to establish).

So one can claim some incremental improvement in scalability via the use of mp2p but it is not this exponential thing that everyone waves about......in other words single level MPLS is a small fixed amount of time apart from ATM on Moores law curve (stacking being a different story).

cheers 
Dave

---

From: Vinay Bannai <[email-masked]>
Date: Tue, 21 May 2002 07:53:38 -0700

Bob,

I most certainly agree with your assessment regarding sticking to circuit
emulation protocol definition and not turn this group into another sub-group
of MPLS. 

I personally do think that the issue of "connection oriented" vs "tunnel
oriented" is clearly addressed in the charter of the PWE3 group. Going
strictly by the framework I would say that the issue of scalability of the
tunnel creation, whether it is O(n) or O(n**2) is outside the purview of the
working group. 

Vinay

-----Original Message-----
From: Bob Barrett [mailto:[email-masked]] 
Sent: Monday, May 20, 2002 11:25 PM
To: [email-masked]; Vijay Amrit Agrawal
Cc: [email-masked]
Subject: RE: [PWE3] PWE3 question 

snip

snip

On a general point:

Best regards

Bob Barrett

---

From: "Khalid Ahmad"<[email-masked]>
Date: Tue, 21 May 2002 12:28:44 -0400

Hi Eric,

thank you for taking the trouble to explain to me why ATM is unscalable in
your view. Let me see if I understand your point, specifically in respect to
your case 3 below.

However, to do this I need to translate the terminology you are using into
something I can understand and which I hope is somewhat more precise.  I
will translate it to more commonly used ATM terminology, if you dont mind.

The term "demand" as described by you seems to be proportional to the number
of ATM VCCs (or VPCs)that the network must support at any given time
(assuming a given number of "PEs" or "Ps" i.e. ATM switches in this ATM
network).

The term "resources" as used by you seems to be the number of entries in the
cross-connect tables in any given ATM switch, i.e. the <VPI/VCI in : VPI/VCI
out> duples in the label switching context tables.

As is well known, ATM virtual circuit connections (VCCs) are then simply a
concatenation of ATM virtual circuit links (VCLs) between the switching
functions in the network elements. The VCLs are identified by the assigned
VPI/VCI values per transmission path (i.e. the payload of the physical link)
between the ATM switches.

Consequently, as the number of VCCs increases (i.e. demand goes up), the
number of VPI/VCI duple context table entries (i.e. resources) increases
linearly per transmission path in any given network element. I hope this is
evident.

The above considerations apply to any label multiplexing mechanism, and have
nothing to do with how networks may be configured or dimensioned to deal
with requirements for increases in capacity of individual switches.

So I am not sure I agree with your point concerning "resources" (VPI/VCI
context table entries) increasing as the square of the "demand" (VCCs). The
relationship here is linear.

To a separate point, I think another term for concept of "aggregation" is
"multiplexing", and of course I agree this is important for the efficient
dimensioning and management of networks.  Hierarchical multiplexing schemes
of various types have been long used for this purpose, so we need hardly
debate their pros and cons here.

I am at least pleased that we have laid to rest the hoary old chesnut of the
SVC/PVC distinction (I hope !).

Regards,
Khalid

-----Original Message-----
From: Eric Rosen [mailto:[email-masked]]
Sent: Thursday, May 16, 2002 4:33 PM
To: Ahmad, Khalid [CAR:CS40:EXCH]
Cc: Rutemiller, John; [email-masked]'
Subject: Re: [PWE3] PWE3 question 

Case 1: Martini signaling, mp2p LSPs.  The amount of resources needed in
each
P router is O(n).

---

From: Hummel Heinrich <[email-masked]>
Date: Thu, 23 May 2002 15:43:32 +0200

Hi all,  
Again, see the new draft-version which I submitted yesterday:  <http://www.ietf.org/internet-drafts/draft-hummel-mpls-hierarchical-lsp-01.txt> http://www.ietf.org/internet-drafts/draft-hummel-mpls-hierarchical-lsp-01.txt,
the O(n**2) problem is not elimited by merging LSP, however by Hierarchical LSPs.
 
A mathematician deals with n**2 many numbers by writing down a square-matrix. His trick: provide two indices to each (matric-) element.
A similar trick can be made here: use two labels in a label stack!
 
Remember, in my previous email on this pwe3-question thread  I compared three solutions for an example VPN network:
- (99874 hops required  for n*(n-1) p2p LSPs, 
- 19886 hops  required for n mp2p LSPs,  
- 338 hops required for 2*(n-1) LSPs assisted by H-LSPs.
 
and argued that merging LSPs do  not eliminate the O(n**2) problem.
 
Now, ATM is viewed being unscalable. As easily as CR-LDP may be extended as to  create H-LSPs (see the small section 6 in above I-D), as easily could
PNNI-signalling  be extended as to create H-SVCs. As I said above: 2 labels are needed. ATM does have a stack of two labels. They are called VPI and VCI.
All it takes is, to define a well-known ATM EXPLICIT NULL value, and to use the existing capability as to replace an incoming VCI resp. VPI with a new
outgoing VPI7VCI pair. 
 
As a result,  ATM Forum could  also provide a tunneling solution for the mentioned example VPN network, which only requires 338 hops  rather than 19886 hops.
19886 / 338 =58,8. Will say: The ATM Forum could easily come up with a solution which is 58 times better than current MPLS.
 
 Heinrich

 -----Ursprüngliche Nachricht-----
Von: Khalid Ahmad [mailto:[email-masked]]
Gesendet: Dienstag, 21. Mai 2002 18:29
An: [email-masked]
Cc: Rutemiller, John; [email-masked]'
Betreff: RE: [PWE3] PWE3 question 

Hi Eric, 

thank you for taking the trouble to explain to me why ATM is unscalable in your view. Let me see if I understand your point, specifically in respect to your case 3 below.

However, to do this I need to translate the terminology you are using into something I can understand and which I hope is somewhat more precise.  I will translate it to more commonly used ATM terminology, if you dont mind.

The term "demand" as described by you seems to be proportional to the number of ATM VCCs (or VPCs)that the network must support at any given time (assuming a given number of "PEs" or "Ps" i.e. ATM switches in this ATM network).

The term "resources" as used by you seems to be the number of entries in the cross-connect tables in any given ATM switch, i.e. the <VPI/VCI in : VPI/VCI out> duples in the label switching context tables.

As is well known, ATM virtual circuit connections (VCCs) are then simply a concatenation of ATM virtual circuit links (VCLs) between the switching functions in the network elements. The VCLs are identified by the assigned VPI/VCI values per transmission path (i.e. the payload of the physical link) between the ATM switches.

Consequently, as the number of VCCs increases (i.e. demand goes up), the number of VPI/VCI duple context table entries (i.e. resources) increases linearly per transmission path in any given network element. I hope this is evident.

The above considerations apply to any label multiplexing mechanism, and have nothing to do with how networks may be configured or dimensioned to deal with requirements for increases in capacity of individual switches.

So I am not sure I agree with your point concerning "resources" (VPI/VCI context table entries) increasing as the square of the "demand" (VCCs). The relationship here is linear.

To a separate point, I think another term for concept of "aggregation" is "multiplexing", and of course I agree this is important for the efficient dimensioning and management of networks.  Hierarchical multiplexing schemes of various types have been long used for this purpose, so we need hardly debate their pros and cons here.

I am at least pleased that we have laid to rest the hoary old chesnut of the SVC/PVC distinction (I hope !). 

Regards, 
Khalid

---

From: [email-masked]
Date: Thu, 23 May 2002 21:39:16 +0100

Hi Heinrich......please see in-line.  regards Neil

-----Original Message-----
From: Hummel Heinrich [mailto:[email-masked]]
Sent: 23 May 2002 14:44
To: 'Khalid Ahmad'; [email-masked]
Cc: Rutemiller, John; [email-masked]'
Subject: AW: [PWE3] PWE3 question 

Hi all,  
Again, see the new draft-version which I submitted yesterday:
<http://www.ietf.org/internet-drafts/draft-hummel-mpls-hierarchical-lsp-01.t
xt>
http://www.ietf.org/internet-drafts/draft-hummel-mpls-hierarchical-lsp-01.tx
t,
the O(n**2) problem is not elimited by merging LSP, however by Hierarchical
LSPs. 
NH=> I am not (as yet) convinced by this.  Hierarchical LSPs reduce
hop-counts IMO not the connectivity requirements, ie motorways vs side
roads.
 
A mathematician deals with n**2 many numbers by writing down a
square-matrix. His trick: provide two indices to each (matric-) element.
A similar trick can be made here: use two labels in a label stack!
 
Remember, in my previous email on this pwe3-question thread  I compared
three solutions for an example VPN network:
- (99874 hops required  for n*(n-1) p2p LSPs, 
- 19886 hops  required for n mp2p LSPs,  
- 338 hops required for 2*(n-1) LSPs assisted by H-LSPs.
 
and argued that merging LSPs do  not eliminate the O(n**2) problem. 
NH=> I could not agree with you more.  If X (arbitrary) want to communicate
with Y (arbitrary) there has to exist 'something' that makes this possible.
If mp2p LSPs are used as some server layer there simply has to be something
higher that is p2p between X and Y.....this is not magic or sleight of hand,
just obvious stuff really.  However, the *effect* of the mp2p constructs
that lie underneath create some hard problems to address.....esp wrt to
fault management of said entities. 

Now, ATM is viewed being unscalable. 
NH=> This almost does not matter, though I agree its a fallacy.....ATM (and
FR) is critical to most large operators, and suppliers ignore at their
peril.
 
  As easily as CR-LDP may be extended as to  create H-LSPs (see the small
section 6 in above I-D), as easily could
PNNI-signalling  be extended as to create H-SVCs. 
NH=> PNNI signalling is 'good' signalling as it is based on Q.2931 which
makes much sense to us.
 
  As I said above: 2 labels are needed. ATM does have a stack of two labels.
They are called VPI and VCI.
All it takes is, to define a well-known ATM EXPLICIT NULL value, and to use
the existing capability as to replace an incoming VCI resp. VPI with a new
outgoing VPI7VCI pair. 
 
As a result,  ATM Forum could  also provide a tunneling solution for the
mentioned example VPN network, which only requires 338 hops  rather than
19886 hops.
19886 / 338 =58,8. Will say: The ATM Forum could easily come up with a
solution which is 58 times better than current MPLS.
 
 Heinrich

 -----Ursprüngliche Nachricht-----
Von: Khalid Ahmad [mailto:[email-masked]]
Gesendet: Dienstag, 21. Mai 2002 18:29
An: [email-masked]
Cc: Rutemiller, John; [email-masked]'
Betreff: RE: [PWE3] PWE3 question 

Hi Eric, 

Regards, 
Khalid 

-----Original Message----- 
From: Eric Rosen [ mailto:[email-masked] <mailto:[email-masked]> ] 
Sent: Thursday, May 16, 2002 4:33 PM 
To: Ahmad, Khalid [CAR:CS40:EXCH] 
Cc: Rutemiller, John; [email-masked]' 
Subject: Re: [PWE3] PWE3 question 

be in order. 

also irrelevant.

---

From: Maarten Vissers <[email-masked]>
Date: Mon, 27 May 2002 13:01:34 +0200

All,

I had a closer look yesterday onto this issue of best network architecture. Some
considerations/thoughts...

1) there are 4 (at least so far) network architectures, which I will refer to
as:
	- MESH
	- MERGING
	- MULTIPLEXING
	- HIERARCHY

2) MESH: a MPLS tunnel is present between any two PEs in the network
   MERGING: each PE is the endpoint of a mp2p merging connection, which
            connects all other PEs with this PE
   MULTIPLEXING: regular multiplexing is performed in P nodes instead of 
                 merging
   HIERARCHY: a hierarchy of MPLS tunnels is present in the network to
              reduce amount of administration in more core parts of network

3) MERGING (at N) is a kind of "PASSIVE MULTIPLEXING" (at N+1)... 

              --------------------
                  Level N+1
              --------------------
                   Level N
              --------------------

   Assume two levels N and N+1 with N+1 being the client level of N...
   Merging at level N is performed in level N fabrics... 
   It causes multiplexing at level N+1 by the virtue of the presence
   non-overlapping level N+1 tributary slot identifier (i.e. label) sets
   within the level N signal payloads; i.e. no level N+1 trib slot id 
   interchange (i.e. label translation) is required at the merging points
   (only level N label translation is to be performed)...
   This thus reduces the number of switch table entries to maintain 
   significantly...

4) MERGING requires that each PE has 1 (logical) ingress link at level N+1
   and M-1 (logical) egress links at level N+1; these latter are part of
   the M-1 mp2p connections to the other M-1 PEs. Level N label translation
   is to be performed at the P nodes.

5) MULTIPLEXING requires that each PE has 1 (logical) ingress link and
   1 (logical) egress link at level N+1. Level N+1 label translation is
   to be performed at the P nodes.

6) MERGING and MULTIPLEXING have the same set of level N trails. Level N 
   trails run between two merging P nodes. As long as no MPLS OAM is
   used, one may (incorrectly) think that the level N trails are merged...
   But when MPLS OAM is deployed, it becomes obvious that level N trails
   start at the egress of PE and P nodes and end at the ingress of P and PE
   nodes... the level N MPLS OAM is terminated (inserted, extracted) at 
   those points... only the level N+1 MPLS signals are forwarded from a P 
   node's ingress to a P node's egress port/interface.

                            Level N Fabric
                    forwarding level N+1 signals
                                ------
              |<--trail @ N-->| |    |
        PE/P --------->---------|    |
                                |    |
          |<----trail @ N---->| |    | |<-trail @ N->|
    PE/P ---------------->------|  P |------->--------- P/PE
                                |    |
               |<-trail @ N ->| |    |
         PE/P --------->--------|    |
                                |    |
                                ------
                      Figure 1 - Merging node

                            Level N+1 Fabric
                    forwarding level N+1 signals
                                ------
              |<--trail @ N-->| |    |
        PE/P --------->---------|    |
                                |    |
          |<----trail @ N---->| |    | |<-trail @ N->|
    PE/P ---------------->------|  P |------->--------- P/PE
                                |    |
               |<-trail @ N ->| |    |
         PE/P --------->--------|    |
                                |    |
                                ------
                      Figure 2 - Multiplexing node

   In MERGING approach the payload of the level N signal is treated as a 
   single stream. In MULTIPLEXING application the payload of the level N 
   signal is treated a a set of K independent level N+1 client signals.

                        ---------------
                        | Level N     |
                        | Trail       |
              ----->----| Termination |-->-- to Fabric
                        | Point       | (one aggregate level N+1 stream)
                        |             |
                        ---------------
                     Figure 3 - Merging

                        ---------------
                        | Level N    1|-->--
                        | Trail      2|-->-- to Fabric
              ----->----| Termination |-->-- (K level N+1 signals)
                        | Point      :|-->--
                        |            K|-->--
                        ---------------
                     Figure 3 - Multiplexing

6) In either MERGING and MULTIPLEXING cases, a defect condition detected at
   the level N trail termination point in the ingress port/interface of a P 
   node must result in the appropriate consequent actions at its client
   level N+1... i.e. MPLS-FDI OAM packets must be inserted at level N+1.

   In MULTIPLEXING case, the level N+1 client information is known by 
   default while it is required to switch/cross connect/forward the 
   level N+1 signals individually.

   In MERGING case, the level N+1 client information is unknown by
   default, while the level N fabric isn't concerned about/using it.
   Therefore in this case the level N+1 client information (i.e. the
   set of active labels) is to be included in the level N trail
   termination point, where it is to be used to generate MPLS-FDI OAM
   signals when the level N trail has a trail signal fail condition
   detected.

7) Assume that a P node has a physical port in which one level of MPLS
   tunnel is terminated and the tributary MPLS signals inside that
   tunnel forwarded to an egress tunnel within an egress port.
   The switching/forwarding in the Fabric will be controlled by some
   <In:Out> tuples:

   MULTIPLEXING:
   <[Port X, level N label A, level N+1 label R] : 
     [Port Y, level N label B, level N+1 label S]>

   MERGING: 
   <[port X, level N label A] : [port Y, level N label B]>
   I.e. merging is multiplexing with "R = S". A further reduction of
   information to store in P node could be realised when in addition to
   "R = S" also "A = B" is used.

   The disadvantage of merging is that within the complete mp2p connection
   a unique set of level N+1 labels is to be used. This requires a mp2p-wide
   administration of labels.
   With multiplexing, the label administration is local to the link.
   I.e. the reduction of complexity in one area cause an increase of
   complexity in another area...

8) The number of level N+1 p2p connections in either network architecture
   case hasn't changed at all... level N+1 p2p connections are invariant
   to MESH, MERGING, MULTIPLEXING, HIERARCHY.

9) Variation associated with these network architectures comes in:
   - number of level N+1 links and associated level N trails
   - number of level N+1 switch table entries
   - number of level N switch table entries
   - number of ports on P nodes
   - number of level N links
   - number of ports that must be aware of level N+1

   It would be nice to build a table providing an overview of these
   variation elements related to MESH, MERGING, MULTIPLEXING, HIERARCHY.

   E.g. the number of bidirectional level N+1 links in a MESH is 
   "m(m-1)/2". The number of unidirectional level N+1 links in a MESH is
   "m(m-1)". (Assume that there are M PEs in the network.)

   The number of level N+1 ingress links on a PE is
   MESH: (m-1), MERGING: 1, MUXING: 1, HIERARCHY: 1
   The number of level N+1 egress links on a PE is
   MESH: (m-1), MERGING: (m-1), MUXING: 1, HIERARCHY: 1

   etc...

10) As we were able to build a voice network, scaling issues are very
    relative... I still assume there are today more telephones in the
    network than data endpoints, and there are more voice connections
    active than data connections... As we could scale this voice network
    we should be able to scale the data network as well...

    The telephony network has grown to its current size over many many
    years... and it currently contains (correct me if wrong) 
    - local exchanges (10000..100000 subscribers and few trunk ports/exhange)
    - trunk/toll exchanges (no subscribers, only trunk ports)
    - international exchanges.
    Furthermore a very hierarchical transport network to interconnect
    those exchanges.
 
    Perhaps this is also the future MPLS network architecture... but in 
    the mean time, there will be many different optimisations required.
    No doubt we will see MESH, MERGE, MULTIPLEXING and HIERARCHY be
    present in the coming years in the MPLS networks. As such, any
    service definition intending to run through/over the MPLS network
    must be capable to run over the mentioned architectures. If we would
    define a service X which can only run on one of the four, it will no
    doubt be replaced by a modified version X' which supports all four and
    provide interworking.

    PWE3 must thus make sure that any XXXoverMPLS service can run over any
    of the four mentioned MPLS network architectures. 
	Note - the level N+1 can be interpreted as the MPLS VC level.
        The level N can be interpreted as the first MPLS tunnel level.

Regards,

Maarten

---

From: "Zheng Wang" <[email-masked]>
Date: Mon, 20 May 2002 15:44:53 -0700

Zheng: 

I consider state and fabric to be equivalent. Each LSH involves some state at the LSRs in either end, therefore the cumulative number of LSHs in the network is indicative of the amount of state involved.

Dave;

The totoal number of LSHs is not an important metric - it is the number of state you keep in each box that matters. In other word, the max size of the table in your router is the issue here. 

Cheers

Zheng

---

From: [email-masked]
Date: Thu, 30 May 2002 10:44:07 +0100

Zheng...I'd be happy to agree with you provided it broke nothing else.  But
I have explained on the list several times that it is no good saying
something is scalable in 1-dimension if it is broken in others.  mp2p LSPs
create very hard fault management problems and they are not good constructs
on which to offer VPN services IMO.  VPN services need more than isolation
wrt addressing, they also need strong fault/performance isolation from each
other.  Like-DS-class merging across VPNs on mp2p E-LSPs does not provide
this per VPN fault/performance isolation.
 
regards, Neil

-----Original Message-----
From: Zheng Wang [mailto:[email-masked]]
Sent: 20 May 2002 23:45
To: David Allan; erosen
Cc: pwe3
Subject: RE: [PWE3] PWE3 question 

Zheng: 

Dave;

The totoal number of LSHs is not an important metric - it is the number of
state you keep in each box that matters. In other word, the max size of the
table in your router is the issue here. 

Cheers

Zheng

---

From: "Zheng Wang" <[email-masked]>
Date: Thu, 30 May 2002 12:10:30 -0700

Zheng...I'd be happy to agree with you provided it broke nothing else.  But I have explained on the list several times that it is no good saying something is scalable in 1-dimension if it is broken in others.  mp2p LSPs create very hard fault management problems and they are not good constructs on which to offer VPN services IMO.  VPN services need more than isolation wrt addressing, they also need strong fault/performance isolation from each other.  Like-DS-class merging across VPNs on mp2p E-LSPs does not provide this per VPN fault/performance isolation.

Neil
 
I dont disagree with you. To quote a paragraph from my previous message:
 
> Of course, when you merge, you lose control over individual paths. It is a tardeoff. 
 
Cheers
Zheng

---

From: Hummel Heinrich <[email-masked]>
Date: Fri, 28 Jun 2002 16:04:52 +0200

I submitted the following I-D, which reports about 4 different methods on how to
accomplish a full mesh connectivity, i.e. on how to tackle the N-square problem.
The reported results are based on a sample network with N=200 PEs.

http://search.ietf.org/internet-drafts/draft-hummel-mpls-n-square-investigations-00.txt 

This draft is a companion draft to 
http://search.ietf.org/internet-drafts/draft-hummel-mpls-hierarchical-lsp-01.txt

The topic of both I-Ds relates to what is being discussed on this email thread.

Your comments are appreciated.

Regards,
Heinrich Hummel
Siemens AG