Subject: Re: draft-ietf-l3vpn-mvpn-considerations-05

From: Eric Rosen <[email-masked]>
Date: Thu, 03 Dec 2009 11:39:16 -0500

Appendix A of the "considerations" draft purports to provide an analysis
showing that BGP C-multicast routing provides a more scalable MVPN PE-PE
control plane than does PE-PE PIM.  However, in addition to errors of fact,
the analysis has lots of methodological problems; as a result it does not
serve its intended function, which is to support the recommendations of
section 3.3.1.

Due to the large number of methodological problems, I believe the Appendix
should be removed before the draft progresses.  I also have to say that I
find it peculiar for a PIM scalability analysis to progress without having
been last called in any of the multicast WGs.  The topic itself seems out of
scope for the L3VPN WG.  Nevertheless I will try to explain as clearly as I
can why I think the analysis is defective.

A scalability analysis must show how the utilization of some resource or set
of resources on some system increases as some demand or set of demands
increases.  The appendix proposes two resources to consider:

   Though the type of processing, messages and states, may vary with the
   different approaches, we propose here a rough estimation of the load
   of PEs, in terms of number of messages processed and number of
   control plane states maintained.  A "message processed" being a
   message being parsed, a lookup being done, and some action being
   taken (such has updating a control plane or data plane state).

This is okay as one of the resources considered.  If I understand this
correctly, this omits consideration of any messages received that do not
result in any "action being taken", and thus omits consideration of any
acknowledgment messages or other messages that fall into the category of
"transport overhead".  It also omits consideration of messages transmitted,
thus does not distinguish actions that cause a new message to be sent from
actions that do not.  So it is not a complete measure of the amount of
"messaging", but it is a factor worth considering.

   A "state maintained" being a multicast state kept in the control plane
   memory of a PE, related to a interface or a PE being subscribed to a
   multicast stream.

This is also worth considering as an abstract measure of memory resources.
However, as we will see, the analysis is not very precise about what it
counts as a "state", and this leads to some mistakes.

The real methodological problems begin when the analysis attempts to be
quantitative:

   The estimation unit used is the "message.equipment" (or "m.e"): one
   "message.equipment" corresponding to "one equipment processing one
   message" (10 m.e being "10 equipments processing each one message",
   or "5 messages each processed by 2 equipments", or "1 message
   processed by 10 equipment", etc.).  Similarly, for the amount of
   control plane state, the unit used is "state.equipment" or "s.e".
   This allow to take into account the fact that a message (or a state)
   can have be processed (or maintained) by more than one node.

This "estimation unit" has no meaning whatsoever, as it does not reflect
resource utilization on any system.  Suppose there are N systems, each
running an algorithm that scales as O(N).  Contrast this with an alternative
in which a single system does the work of all N, running an algorithm that
scales as O(N^2).  Other things being equal, the former would be considered
to be the more scalable alternative.  However, if you use the "estimation
unit" of "work.equipment", you would conclude that in both schemes the work
scales as O(N^2), and hence that the schemes are equally scalable.  If this
were a valid methodology, no one would ever have heard of such notions as
"distributed routing" or "datagram networks".  Use of this peculiar
"estimation unit" will always bias the results towards a centralized system
and away from a distributed system.

Any subsequent revision of Appendix A should eliminate the notion of "s.e."
or "m.e." and focus solely on the scaling of the individual systems.

Another methodological error is the fact that the analysis focuses heavily
on the Source-Specific Multicast mode, even though the number of VPN
customer deployments of SSM is quite small when compared to the number of
VPN customer deployments of ASM.  To support the recommendations of section
3.3, the analysis needs to focus on scenarios that relevant to actual
deployments. Some of the analysis of SSM also applies to the ASM procedures
for joining/pruning source trees, but this is not made clear, and it's
difficult for a reader to know how much of the analysis is relevant to
actual deployments.

In the most recent rev of the draft, a new section devoted to ASM has been
introduced, and this section has its own set of problems, as I will discuss
shortly.

In addition to the time spent on the virtually irrelevant SSM scenario, much
time is spent on examining a "PIM with explicit tracking" scheme, even
though no such scheme has been proposed.  It's difficult to understand why
this straw man is discussed, while other actual proposals for using PIM are
not.

Another methodological error is that only the messaging needed to join a
single (S,G) flow is considered.  Thus the analysis ignores any scaling
issues that show up as the number of (S,G) flows increases.  Since the
number of (S,G) flows handled by a Route Reflector will far exceed the
number handled by any PE router, failure to consider this particular scaling
factor tends to bias the results in favor of a scheme that uses Route
Reflectors.  Also omitted from consideration are scaling issues that show up
as the rate of change of multicast states increases.

Another methodological error is that the analysis takes no account of the
increased amount of state that must be maintained when C-multicast trees are
set up using two different protocols, rather than just one.  When PIM is
used as the PE-PE control plane, the (C-S,C-G) and (C-*,C-G) states that
drive the inter-PE control plane are the same as the one as the drive the
CE-PE control plane.  When BGP is used, a given multicast state has to be
represented both as a PIM state and as a BGP state (C-multicast route).
This should really be counted as a doubling (at least) of the number of
states that need to be maintained.  This is a very conservative estimate, as
the BGP states tend to contain much more information (e.g., RTs) than the
corresponding PIM states.  It's clear that appendix A does not consider both
the BGP states and the PIM states, but it's never quite clear just what
states it is considering.

Another methodological error is that the PE-CE interactions are not taken
into account.  In many scenarios the PE-PE interactions are just "noise"
when compared to the real bottleneck, the PE-CE interactions.  The total
number of PIM messages processed by a PE includes PIM messages from CEs as
well as from PEs.  Suppose that by adding BGP, you could reduce this total
number of PIM messages by 1%, at the cost of doubling the amount of state
you need to maintain.  Would that be a good tradeoff?  In order to support
the recommendations of section 3.3, Appendix A has to show that the
tradeoff is a lot better than this.  Yet it doesn't even attempt to do so.

It's possible to "prove" any conclusion you want by carefully scoping the
set of things you look at.  For example, if one wanted to prove that BGP is
less scalable, one could say "let's compare the two schemes to see how the
total number of TCP connections in the network scales as the number of PEs
increases".  Of course, to produce a valid analysis, one has to look at more
than one issue, and one has to demonstrate that one is actually looking at
the bottlenecks.

This collection of methodological errors is more than enough to send
Appendix A  "back to the drawing board".  If the goal is to make a
choice between two protocols by considering which scales best, then all  the
scaling factors must be considered.  

Let's take a look now at some of the more detailed issues.

Consider the messaging needed by a PE to maintain an (S,G) state in the
"steady state".  By "steady state", I mean during periods (a) that are
longer than the PIM refresh period and (b) in which the PE does not join or
prune (S,G).  (I think this is similar to what Appendix A calls "baseline
processing".)  Let's also suppose that the PE has k VRF interfaces over
which there are (S,G) receivers.

Assuming a typical one-minute PIM refresh time, the number of Join(S,G) PIM
messages per minute that the PE receives from the other PEs of the same MVPN
in the steady state is either 0 or 1.  (Due to PIM Join Suppression, during
the steady state only one PE on the LAN sends a Join(S,G), the others
receive it.  A PE with (S,G) state that sees a Join(S,G) from another PE
does not send one itself.)  The number of Join(S,G) messages the PE receives
over its VRF interfaces is k per minute.

So the messages received per minute to support the (S,G) state is either k
or k+1.  Note that this has no dependence whatsoever on the number of PEs.

What if BGP is used instead of PIM for PE-PE control?  In that case, no
PE-PE message needs to be sent at all, so the number of messages received
per minute to support the (S,G) state is k per minute.  Even if k is 1, the
most one can get is a 50% reduction; if k is 10 that goes down to 10%, if k
is 100 that goes down to 1%.  Considering the extra state needed to maintain
the (S,G) in both PIM form and BGP form, these reductions seem rather
modest, to say the least.

The methodological problem here is that Appendix A does not consider the PIM
overhead on the VRF interfaces.  But when BGP C-multicast routing is added,
it does not replace the C-PIM instance for a given MVPN, it just reduces by
1 the number of PIM interfaces supported by that C-PIM interface.  By
ignoring this fact (and by ignoring the cost of adding a second multicast
protocol without removing PIM) one makes a small difference look much more
significant than it really is.  It is not valid for a scalability analysis
to focus on just one aspect of the protocols unless it has first shown that
that one aspect is the bottleneck.

(Actually, adding BGP does not even fully reduce the number of PIM
interfaces by even 1, as C-PIM on a PE still needs to keep state for the its
"interface" to the other PEs; see, e.g., section 13.2.1 of
draft-ietf-l3vpn-2547bis-mcast-bgp.)

Appendix A talks of "order of magnitude" differences; that simply doesn't
exist.  In fact, the analysis of Appendix A fails to demonstrate any real
scaling advantage, in the steady state, for one protocol over the other.

Some amount of the analysis in Appendix A is devoted to the startup and
shutdown transients, when a PE first joins an (S,G) or prunes an (S,G).  A
rather big deal is made of the "prune override" procedures.  I have focused
this critique on the steady state, because if the steady state (as I defined
it above) is actually reached, the startup/shutdown transients are not very
important.  I suppose one might disagree with that, but certainly thee has
been no analysis showing that the transients are more important than the
steady state.

It would certainly be interesting to analyze the case where there is so much
multicast churn that steady state is never reached.  However, that cannot be
done by focusing on a single (S,G) over a limited time period.

Now let's look specifically at the ASM case.

One of the features of the BGP solution is that there is no way, in BGP,
to represent the PIM operation of "prune source S off the (*,G) tree".
To compensate for this, every PE in a given VPN is forced to keep track of
every (S,G) stream for which any PE in that VPN has receivers.  This is done
by means of the Source Active A-D routes.

There are two options that can be used.  In one option, where "inter-site
shared trees are not used", each PE functions as a C-RP, and then all the PE
of a given VPN must exchange information on all the (S,G) streams that they
see locally.  This option uses BGP to do something that is very similar to
what MSDP is used for today.  Let me quote from Pekka Savola's review of
this procedure:

         Active source BGP messages.  This is a duplication of a
         similar mechanism in MSDP (RFC3618) which has caused
         much grief in Internet.  Does this meant that when a
         host does 'nmap -sU 224.0.0.0/4' at a VPN site, this
         will result in about 268 million BGP active source
         updates being sent (2^28) in the SP backbone?

The only answer to this was to beef up the Security Considerations with
some required procedures that will prevent the PE from melting down entirely
in this case.  Of course, Appendix A's methodology of only considering a
single (S,G) prevents it from seeing that this particular BGP option
uses a mechanism whose scalability is already known to be poo,r and which
the multicast WGs have decided not even to bring forward to IPv6.

In the other, more sensible option, a PE originates a Source Active A-D
route for an (S,G) only when it receives a C-multicast Join route for that
(S,G).  However, the Source Active A-D route goes to all the PEs in the VPN,
not just to those that have receivers for that (S,G).  I believe this means
that the amount of state a given PE maintains in the BGP scheme is
proportional to the number of (S,G) flows for which any PE in the VPN has
receivers.  Actually, since the SA route contains the RD of the originating
PE, the situation is a bit worse; if (S,G) traffic is sent over the backbone
by more than one PE, the other PEs in the same VPN must maintain an (S,G) SA
A-D route for each such PE.

In the BGP solution, even if a PE has only (*,G) receivers, or even if it
has no receivers for G at all, it must maintain state for all the (S,G)s in
the MVPN, multiplied by the average number of upstream PEs per source.

Strangely, this scaling issue is considered to be insignificant by Appendix
A, for the following reasons:

        when PIM-based C-multicast routing is used there are
        ... possible control plane state transitions triggered by
        the reception of (S,G) packets ; such events would induce
        processing on all PEs joined to G

Control plane state transitions triggered by the reception of data packets
have not even been mentioned to this point, and no analysis has been given
to show whether or not they cause a scaling issue, or whether any such
scaling issue is mitigated by the use of BGP.  I also really don't see what
this has to do with the issue at hand, namely the added state overhead of
the Source Active A-D routes.  Certainly there is no analysis provided that
would show the SA A-D routes trade off in a favorable manner against the
data-driven state changes.

      when PIM-based C-multicast routing is used there are ...
      possible PIM Assert messages specific to (S,G) ; this would
      induce a message processing on each PE of the VPN for each
      PIM Assert message

It is true that the Source Active A-D routes eliminate the need for Assert
messages.  The implication is that this is favorable for BGP; however, there
does not appear to be any analysis showing that.

The new material on ASM does not appear to contain any analysis at all, just
unsupported assertions.

Let me also mention a few other, less major, problems in the Appendix:

        for each PIM Prune message, all PE routers on the LAN
        work to let the upstream PE determine the answer to the
        "did the last receiver leave?" question.

I believe that only one PE router needs to send a Join in order to override
the Prune, as once the others see that Join they reset their Join timers.
Of course, actual behavior varies depending on the value of the timer and
on the random jitter.

        We can see that answering the "last receiver leaves"
        question is a significant proportion of the work that the
        C-multicast routing building block has to make, and where
        the approaches differ most.

This assertion is made without support.

        the case where the RT-Constraint mechanisms [RFC4684] is
        not used is not covered;

Since RT-constraint is not a mandatory to implement mechanism, I don't see
why this is not covered.  Note though that it doesn't really matter in the
steady state.

>From A.1.1.1.  "PIM LAN procedures, by default"

   +------------+------------+---------------+----------+--------------+
   |            | upstream   | other PEs     | RR       | total across |
   |            | PE (1)     | (total across | (none)   | all          |
   |            |            | (#mvpn_PE-1)  |          | equipments   |
   |            |            | PEs)          |          |              |
   +------------+------------+---------------+----------+--------------+
   | first PE   | 1 m.e      | #MVPN_PE-1    | /        | #MVPN_PE m.e |
   | joins      |            | m.e           |          |              |
   +------------+------------+---------------+----------+--------------+

If the above row implies that the "other PEs" must process the join, where
"process" is defined (as it is in this Appendix) as "a message being parsed,
a lookup being done, and some action being taken (such has updating a
control plane or data plane state)", then one the next row is incorrect:

   +------------+------------+---------------+----------+--------------+   
   | for *each* | 1 m.e      | #mvpn_PE-1    | /        | #mvpn_PE m.e |
   | additional |            | m.e           |          |              |
   | PE joining |            |               |          |              |
   +------------+------------+---------------+----------+--------------+

If a second PE has "processed" the Join(S,G) from the first, then it doesn't
need to send a Join if it decides to join (S,G) itself; join suppression
will be in effect.

The third and fifth columns above are examples of cases where the Appendix
misleadingly multiplies by #mvpn_PE.  In fact, the number of messages
received by a given PE does not increase in proportion to #mvpn_PE.  This
problem infects the entire set of tables in the Appendix.

Section A.1.1.3

The RR has to maintain all the multicast states of all the clients (PEs), as
long as those multicast states have a PMSI as input interface.  That's way
more multicast states than any PE has to maintain. The analysis in this
Appendix simply ignores the negative scaling effects due to the large number
of RR clients.

The claim is made that when a second PE joins (S,G), the upstream PE does
not receive a new BGP route.  That may or may not be the case.  The second
PE will always need to send a C-multicast Join route to the RRs; in PIM the
second PE might not need to send any message.  Whether the RRs will forward
the route originated by the second PE depends upon whether that route is
more preferable (according to normal BGP procedures) than the route
originated by the first PE.  A similar observation applies when C-multicast
routes are withdrawn.

---

From: Eric Rosen <[email-masked]>
Date: Thu, 03 Dec 2009 11:40:03 -0500

I believe that comparing segmented inter-AS P-tunnels with non-segmented
inter-AS P-tunnels gives the following comparison matrix:

- Scalability for MI-PMSIs: Segmented more scalable

- Scalability for S-PMSIs: Comparable scalability

- Complexity at ASBRs: segmented more complex (requires ASBRs to have more
  VPN-specific knowledge, and some options increased configuration at ASBRs)

- Independence of tunneling technology from one AS to another: segmented
  provides this, non-segmented does not.

- Compatibility with existing implementations: non-segmented provides this,
  segmented does not.

Looking at these considerations, I would not choose "segmented" as a
required feature of the minimal implementation, because I don't think anyone
will ever use MI-PMSIs instantiated as segmented inter-AS P-tunnels, and I
think the minimal implementation should emphasize simplicity over
generality.  But whatever the document recommends, it should take all the
considerations into account, not just the ones that best support its
conclusion.

---

From: Yakov Rekhter <[email-masked]>
Date: Tue, 8 Dec 2009 07:08:19 -0800

Eric,

> - Scalability for MI-PMSIs: Segmented more scalable

This is correct.

> - Scalability for S-PMSIs: Comparable scalability

This is correct, *but only* when C-multicast routing is done with PIM.

This is *incorrect* when C-multicast routing is done with BGP, as
when BGP is used for C-multicast routing, segmented P-tunnels are
more scalable than non-segmented due to the ability to aggregate
segmented S-PMSIs with I-PMSIs.

>   VPN-specific knowledge, and some options increased configuration at ASBRs)

To be more precise, depending on whether inter-AS I-PMSI auto-discovery
routes are at the granularity of <source ASBR, MVPN> or <source AS,
MVPN>, the segmented P-tunnels would require either the same or
additional configuration on ASBRs as the non-segmented P-tunnels.

(Note that the issue of configuration has been already discussed on the mailing
list - see http://www.ietf.org/mail-archive/web/l3vpn/current/msg02420.html) 
  
>   provides this, non-segmented does not.

This is correct.

>From Merriam-Webster:

   Main Entry: compatible

      1 : capable of existing together in harmony <compatible theories> 
          <compatible people>
      2 : capable of cross-fertilizing freely or uniting vegetatively
      3 : capable of forming a homogeneous mixture that neither separates 
          nor is altered by chemical interaction
      4 : capable of being used in transfusion or grafting without 
          immunological reaction (as agglutination or tissue rejection)
      5 : designed to work with another device or system without 
          modification; especially : being a computer designed to operate 
          in the same manner and use the same software as another computer

Given the above definition of "compatible", your claim is *incorrect*,
as having non-segmented P-tunnels, by itself, does not provide
compatibility with "existing implementations".

This is because an implementation of non-segmented inter-AS I-PMSI
P-tunnels with BGP-based auto-discovery, as specified in
draft-ietf-l3vpn-mcast-bgp is *not* compatible with "existing
implementations" of non-segmented inter-AS I-PMSI P-tunnels with
BGP-based auto-discovery based on MDT-SAFI, even if both of these
implementations use non-segmented P-tunnels.

Likewise, an implementation of non-segmented S-PMSI P-tunnels with
BGP-based auto-discovery, as specified in draft-ietf-l3vpn-mcast-bgp
is *not compatible* with an "existing implementation" of non-segmented
S-PMSI P-tunnels with UDP-based auto-discovery, even if both these
implementations use non-segmented P-tunnels.

As a side comment, I would also suggest to revise the following
from section 6 of draft-ietf-l3vpn-mvpn-considerations-05:

   -  that implementations support segmented inter-AS tunnels and
      consider supporting non-segmented inter-AS tunnels (in order to
      maintain backwards compatibility and for migration) ;

to better aling it with the (correct) text in section 3.5 of
draft-ietf-l3vpn-mvpn-considerations-05.

> considerations into account, not just the ones that best support its

The document should take all the *valid* considerations into account.
Moreover, these considerations should reflect the requirements,
as specified in rfc4834. The document does all this.

Yakov.

---

From: Thomas Morin <[email-masked]>
Date: Tue, 08 Dec 2009 17:38:42 +0100

Hello Eric, Yakov,

Given your exchange, I would propose to add to section 3.5, a modified 
version of Eric's points taking also into account Yakov's suggestions 
and the discussion in June/July (text to be added before the last 
paragraph of section 3.5, except of course text not included):

    The following is a summary comparison matrix between the "segmented
    inter-AS P-tunnels" and "non-segmented inter-AS P-tunnels" approaches:

    - Scalability for MI-PMSIs: "Segmented inter-AS P-tunnels" more scalable

    [unmodified]

    - Scalability for S-PMSIs: Comparable scalability when the "Full
    per-MVPN PIM peering across an MI-PMSI" approach is used for
    C-multicast routing; "segmented inter-AS P-tunnels" more scalable
    when BGP is used for C-multicast routing (due to the ability to
    aggregate S-PMSIs with I-PMSI when BGP is used for C-multicast routing)

    [modified to take into account S-PMSI aggregation as discussed in
    June/July and reminded below]

    - Configuration at ASBRs: depending on whether segmented P-tunnels
    have a granularity of source ASBR or source AS, the "segmented
    inter-AS P-tunnels" approach would require respectively the same or
    additional configuration on ASBRs as the "non-segmented inter-AS
    P-tunnels" approach

    [modified to take into account the discussion in June/July and
    reminded below]

    - Independence of tunneling technology from one AS to another: the
    "segmented inter-AS P-tunnels" approach provides this, the
    "non-segmented inter-AS P-tunnels" approach does not.

    [unmodified]

    - Facilitated co-existence with existing deployments, and providing
    a lighter engineering in some scenarios : the "non-segmented
    inter-AS P-tunnels" approach provides this, the "segmented inter-AS
    P-tunnels" approach does not.

    [unmodified to avoid the inexact "compatibility" term]

Moreover, the sixth item in section 6 "Summary of recommendation" can be 
rewritten to avoid the inexact term of "backward compatibility":

    that implementations support segmented inter-AS tunnels and consider
    supporting non-segmented inter-AS tunnels (in order to facilitate
    co-existence with existing deployments, and provide a lighter
    engineering in some scenarios) ;

Please send feedback on these proposed changes.

Thanks,

-Thomas

Yakov Rekhter:
> routes are at the granularity of<source ASBR, MVPN>  or<source AS,
> list - seehttp://www.ietf.org/mail-archive/web/l3vpn/current/msg02420.html)
>
>    
>        1 : capable of existing together in harmony<compatible theories>
>            <compatible people>

---

From: Eric Rosen <[email-masked]>
Date: Mon, 14 Dec 2009 10:24:19 -0500

Eric> Scalability for S-PMSIs: Comparable scalability

Yakov> when C-multicast routing is done with BGP, as when BGP is used for
Yakov> C-multicast routing, segmented P-tunnels are more scalable than
Yakov> non-segmented due to the ability to aggregate segmented S-PMSIs with
Yakov> I-PMSIs.

I don't see the relevance of this point.  What does "the ability to
aggregate segmented S-PMSIs with I-PMSIs" have to do with the scalability of
S-PMSIs.

Eric> Complexity at ASBRs: segmented more complex (requires ASBRs to have
Eric> more VPN-specific knowledge, and some options increased configuration
Eric> at ASBRs)

Yakov> To be more precise, depending on whether inter-AS I-PMSI auto-discovery
Yakov> routes are at the granularity of <source ASBR, MVPN> or <source AS,
Yakov> MVPN>, the segmented P-tunnels would require either the same or
Yakov> additional configuration on ASBRs as the non-segmented P-tunnels.

To be even more precise, the more scalable option requires per-MVPN
configuration on each ASBR, and even the less scalable option requires more
complexity on the ASBRs.

Eric> Compatibility with existing implementations: non-segmented provides
Eric> this, segmented does not.

Yakov> having non-segmented P-tunnels, by itself, does not provide
Yakov> compatibility with "existing implementations".

It would indeed be more precise to say that non-segmented P-tunnels is a
necessary but not always sufficient feature for interworking new deployments
with old.

Yakov> an implementation of non-segmented inter-AS I-PMSI P-tunnels with
Yakov> BGP-based auto-discovery, as specified in draft-ietf-l3vpn-mcast-bgp
Yakov> is *not* compatible with "existing implementations" of non-segmented
Yakov> inter-AS I-PMSI P-tunnels with BGP-based auto-discovery based on
Yakov> MDT-SAFI, even if both of these implementations use non-segmented
Yakov> P-tunnels.

Actually, I believe these two implementations will interwork correctly, as
is, when there is an "option C" interconnect, PIM/MI-PMSI is used as the
PE-PE control protocol, S-PMSI Joins are used, and the MI-PMSI is set up
using a sparse mode group address.

In an "option B" interconnect, new implementations would have to include and
parse the "connector attribute" as well as the "RT VRF Import Attribute" in
order to interwork with old implementations; a very simple and
straightforward interworking technique that is not available with segmented
inter-AS P-tunnels.

Thomas proposes to change the phrase "in order to maintain backwards
compatibility and for migration" to "in order to facilitate co-existence
with existing deployments, and provide a lighter engineering in some
scenarios".  I think it would be better to say "in order to facilitate
interworking with and migration from existing deployments, and to provide a
simpler solution when the complexity of segmented inter-AS P-tunnels is not
needed."

---

From: Yakov Rekhter <[email-masked]>
Date: Mon, 14 Dec 2009 08:05:15 -0800

Eric,

> aggregate segmented S-PMSIs with I-PMSIs" have to do with the scalability of

For one thing, in the multi-AS scenario it allows (downstream) ASes
to reduce the overhead due to S-PMSIs by aggregating them into
I-PMSIs.

In addition, in the multi-AS scenario it allows (downstream) ASes
to aggregate P-tunnels of the intra-AS segments of S-PMSIs, thus
reducing the overhead due to S-PMSIs.

Yakov.

---

From: Yakov Rekhter <[email-masked]>
Date: Mon, 14 Dec 2009 08:23:40 -0800

Eric,

> using a sparse mode group address.

Incorrect.

Even if "PIM/MI-PMSI is used as the PE-PE control protocol, S-PMSI
Joins are used and the MI-PMSI is set up using a sparse mode group
address", an implementation that follows draft-ietf-l3vpn-* will
*not* interwork with "existing implementations" as according to
*your own draft* (draft-rosen-vpn-mcast) the existing implementation
"REQUIRES the use of the MDT-SAFI NLRI (even if PIM-SSM is not used
to set up the default MDT)" (while an implemendation that follows
draft-ietf-l3vpn-* is not required to use MDT-SAFI NLRI).

Yakov.

---

From: Eric Rosen <[email-masked]>
Date: Tue, 15 Dec 2009 11:05:14 -0500

Eric> What does "the ability to aggregate segmented S-PMSIs with I-PMSIs"
Eric> have to do with the scalability of S-PMSIs.

Yakov> For one thing, in the multi-AS scenario it allows (downstream) ASes
Yakov> to reduce the overhead due to S-PMSIs by aggregating them into
Yakov> I-PMSIs.

This means that a downstream provider can use I-PMSIs even if an upstream
provider uses S-PMSIs.  That may or may not be a useful feature, but it
doesn't make the use of S-PMSIs more scalable.

Furthermore, this cannot be done using only the purported mandatory features
of the considerations draft, it requires optional features.  

Yakov> In addition, in the multi-AS scenario it allows (downstream) ASes
Yakov> to aggregate P-tunnels of the intra-AS segments of S-PMSIs, thus
Yakov> reducing the overhead due to S-PMSIs.

I don't think the ability to aggregate multiple S-PMSIs into a single
P-tunnel has anything to do with I-PMSIs.  In addition, non-segmented
S-PMSIs can also be aggregated into a single P-tunnel.

In a transit AS, the aggregation of multiple S-PMSI intra-AS segments into a
single P-tunnel across that transit AS cannot be done using only the
purported mandatory features of the considerations draft.

I think it is somewhat misleading to say that a certain feature should be
mandatory because it has certain properties, if those properties cannot be
realized without the implementation of optional features.

Yakov> according to *your own draft* (draft-rosen-vpn-mcast) the existing
Yakov> implementation "REQUIRES the use of the MDT-SAFI NLRI

That just means that an implementation without MDT-SAFI is not compliant
with that draft.  I don't think I ever said otherwise.

---

From: Yakov Rekhter <[email-masked]>
Date: Tue, 15 Dec 2009 08:55:39 -0800

Eric,

> doesn't make the use of S-PMSIs more scalable.

Sure it does.

> P-tunnel has anything to do with I-PMSIs.  

This has to do with scalability of segmented vs non-segmented S-PMSIs.

> In addition, non-segmented

By virtue of being non-segmented, non-segmentd S-PMSIs do *not*
provide the ability to aggregate P-tunnels of the intra-AS *segments*
of S-PMSIs (for one thing, non-segmented S-PMSIs do not have "intra-AS 
segments").

Yakov.

---

From: Thomas Morin <[email-masked]>
Date: Wed, 16 Dec 2009 09:30:35 +0100

Eric, working group,

Eric Rosen:
[...]
 > [...] as we will see, the analysis is not very precise about what it
[...]
 > states it is considering.

In revision -02 of the draft, section A.3 (which is now A.1.1.3) 
included a change to properly take
into account what you describe above, after a similar comment by Maria 
Napierala we received February 23th 2009.

http://tools.ietf.org/rfcdiff?url2=draft-ietf-l3vpn-mvpn-considerations-02.txt#part-l12

I notice now that the correction was lost when A.1.1.3 was revised in -05.

Given this:
- the table in A.1.1.3 will be corrected, so that a factor 2 is added in 
columns 2 and 3, and the "total" updated accordingly.
- I will also add a note in the introduction to indicate that a state is 
counted twice when both present in PIM and BGP, and a note  to A.1.1.3 
indicating that both PIM and BGP states have been accounted for on the 
receiver PEs and upstream PE.

(So, this is in fact not a "methodological error", but a simple counting 
error.)

-Thomas

---

From: Thomas Morin <[email-masked]>
Date: Wed, 16 Dec 2009 09:33:07 +0100

Eric, working group,

 > Of course, actual behavior varies depending on the value of the timer and

The text was already clarified in version -04, after your comments from 
2009-06-11, and I provided an explanation on 2009-06-19.

  Previous text:
      The "did the last receiver leave?" question is thus implicitly 
replied to by all PE
      routers, for each PIM Prune message.

You had claimed in June that the text "lead one to believe that a 
Prune(S,G) immediately causes lots of Join(S,G) messages to be sent", 
which could be argued sense because the phrase "replied to" could be 
understood a "a message is sent", even though it was not the intent of 
the text.

The text now reads as follows:
      Thus, for each PIM Prune message, all PE routers work to let the 
upstream PE
      determine the answer to the "did the last receiver leave?" question.

Indeed, when a PE sends a Prune(S,G):
   - all PEs parse this message, and try to find a matching state in 
their TIB ==> reflected by "all PE routers on the LAN work..."
   (- if a PE finds one, it setup a short timer to send a Prune to 
override the Join)
   - if no PE find any, after some delay, the upstream PE can determine 
that the last receiver has left ==> reflected by "... to let the 
upstream PE determine the answer to the ``did the last receiver 
leave?``question"

I don't think that the new text carries any wrong or misleading term, 
since "work" properly designate what the PEs of the MVPN do, without 
carrying a misleading idea of multiple Join message being sent.

If you can explain why you think the revised text above does not 
properly reflect the PIM Prune override procedures, feel free to suggest 
a better text (just repeating your comment from June is not helpful).

 > This assertion is made without support.

Ok, the introduction of appendix A is not supposed to carry quantitative 
conclusions, so let's just say "is part of the work" instead of "is a 
significant proportion of the work" in this introduction.

Thanks,

-Thomas

---

From: Thomas Morin <[email-masked]>
Date: Wed, 16 Dec 2009 09:39:39 +0100

Eric, working group,

Eric Rosen:
 > [...] much time is spent on examining a "PIM with explicit tracking" 
scheme, even
 > though no such scheme has been proposed. It's difficult to understand 
why this straw
 > man is discussed, while other actual proposals for using PIM are not.

"PIM-SM" specifications (RFC4601, section 4.3.3, page 36):

    Although the mechanisms are not specified in this document, it is
    possible for upstream routers to explicitly track the join membership
    of individual downstream routers if Join suppression is disabled.  A
    router can advertise its willingness to disable Join suppression by
    using the T bit in the LAN Prune Delay Hello option.  Unless all PIM
    routers on a link negotiate this capability, explicit tracking and
    the disabling of the Join suppression mechanism are not possible.

Nothing, as far as I know, exclude this modality of PIM from the "Full 
Per-MVPN PIM Peering Across a MI-PMSI" procedures described in 
draft-ietf-l3vpn-2547bis-mcast.  So we are definitely not "discussing a 
straw-man".  I wonder if you are possibly confusing "PIM-SM 
specifications" and "a PIM implementation".

However, if nobody thinks it is useful to look at this use of PIM with 
Explicit tracking, then we can certainly remove it.

As far as "other actual proposal for PIM", I guess you may be referring 
to proposals in draft-rosen-l3vpn-mvpn-mspmsi: of course, they are not 
in the scope of this document, which is restricted to the solutions 
described in draft-ietf-l3vpn-2547bis-mcast and 
draft-ietf-l3vpn-2547bis-mcast-bgp.

-Thomas

PS: Let me note that you did not bring up this up during the LC in June, 
and that this comment is not related to any new content in the revisions 
made since this LC.

---

From: Thomas Morin <[email-masked]>
Date: Wed, 16 Dec 2009 09:39:55 +0100

Eric, working group,

Eric Rosen:
 > why this is not covered.

You omitted the first part of the sentence you quoted, which actually 
provides the answer:

       following the recommendation of [I-D.ietf-l3vpn-2547bis-mcast-bgp]
       the case where the RT-Constraint mechanisms [RFC4684] is not used 
is not
       covered;

Here is the corresponding quote in draft-ieft-2547bis-mcast-bgp (which 
you happen co-author) :

    16. Scalability Considerations

    A PE should use Route Target Constrain [RT-CONSTRAIN] to advertise
    the Route Targets that the PE uses for the VRF Route Imports extended
    community (note that doing this requires just a single Route Target
    Constraint advertisement by the PE). This allows each C-multicast
    route to reach only the relevant PE, rather than all the PEs
    participating the an MVPN.

Hence the considerations document does not cover the case where the 
RT-Constraint mechanism is not used.

-Thomas

---

From: Thomas Morin <[email-masked]>
Date: Wed, 16 Dec 2009 09:42:23 +0100

Eric, working group,

 > "messaging", but it is a factor worth considering.

Yes, you could always think about refining and distinguishing more, but 
with a goal of looking at the orders of magnitude, I think the document 
already balances this aspect finely.

 > This is also worth considering as an abstract measure of memory 
resources.

Ok.

 > or "m.e." and focus solely on the scaling of the individual systems.

You already brought up a slightly different form of this argument six 
month ago during WG LC (June 11th).

You could certainly claim that your argument "has not been addressed" 
and thus deserves being repeated, but the issue is that you did not care 
about answering me in the thread that followed on this particular point 
(my last email from July 10th), thus artificially delaying the 
resolution of this point.

But, for the sake of completeness, here are elements that justify how 
the study is currently done:
- as explained in revision -04, the choice of using these units is 
motivated; we need some efficient way to compare BGP and PIM on a common 
ground, even though the type and amount of "invididual systems" 
concerned by the processing of one message are not the same (sometimes 
only the upstream PE, sometimes only RRs, sometimes all PEs, sometimes a 
mix)
- your argument is abstract and does not prove that the comparison done 
in our specific case are irrelevant
- contrary to the imaginary example you take, we are looking at two 
distributed systems, and we observe orders of magnitude of difference 
(in that sense, you are beating a strawman with your example)
- your argument of "no meaning whatsoever" would hold if we were looking 
at individual numbers like O(N**2) and saying "this is big" in the 
absolute, but we don't do and only do comparisons based on these 
estimation units
- in last revisions, we reviewed the text and tables of Appendix A, and 
other sections, so that when a counting is mentioned and refers to a 
total across multiple equipement, we make sure that the reader will not 
be mislead into believing that we talk about a count for a given PE

I can concede to you that there may be more than one way to analyze the 
scalability of C-multicast routing options (and Appendix A.1 proposes 
one of them).  But the opportunity was missed to explore what an 
alternative way of doing the analysis would bring, as after the document 
went for the LC this summer you did not answer me in the thread that 
followed on this particular point (my last email from July 10th) ; you 
did not suggest a different approach until today (~6 month after).  
Moreover, other than from you, there were no objections to the way the 
analysis is done. Given all that, it is too late to request that the 
whole analysis be redone.

-Thomas

---

From: Thomas Morin <[email-masked]>
Date: Wed, 16 Dec 2009 09:47:37 +0100

Eric, working group,

 > From A.1.1.1.  "PIM LAN procedures, by default"

In the sequence above the first PE joining results in all PEs processing 
a message (except the PE joining):
  - the upstream PE processes the Join and ends up propagating the state 
upstream and setting up forwarding state
  - the other PEs parse the message, and lookup their TIB to possibly 
find an (S,G) state; they find none and discard the information in this 
Join) ; note that they don't have any (S,G) state at this point

For each subsequent PE joining (second line in the table, quoted above):
   - the joining PE is transitioning its Upstream (S,G) state machine 
from NotJoined to Joined, and thus sends a Join (RFC 4601, section 
4.5.7, p.73)
   - for other PEs that are already joined to (S,G) : parse the message, 
and lookup their TIB to find an (S,G) state, find one, and reset their 
timer to further delay the moment at which they would send their Join
   - for the other PEs that are not joined to (S,G) : parse the message, 
and lookup their TIB to try find an (S,G) state, find none, and discard 
the information in this Join)

[...]
 > need to send a Join if it decides to join (S,G) itself;

As PIM procedures are specified (RFC 4601, section 4.5.7, p.73), even if 
the second PE had initially processed a message (parsed, done a lookup, 
discarded) from the first joining PE, this PE will still send a Join 
when it starts to need to send (S,G) traffic towards a CE.

 >
 > If the [first] row implies that the "other PEs" must process the 
join, where
 > control plane or data plane state)", then one the [second] row is 
incorrect:

Let's look at your reasoning, and how it could hold:
- if you want to ignore that the definition of "process" includes 
"discarding a message" as "an action being taken", then yes, your 
reasoning would hold ("if first [row] implies, [..] then second row is 
incorrect"), but your reasoning would actually be of low interest, 
because in that case the first row would be invalid too...
- if we count as one m.e when a PE parses a Join or Prune for which it 
is not the upstream, does a lookup for the corresponding (S,G) to 
finally discard the message, then the *two* rows are both valid

The "(such has updating a control plane or data plane state)" phrase is, 
obviously, a non-exhaustive list of examples. When assessing the 
processing done by PEs, we have to reflect the work done by a PE to 
identify it can discard a message because it does not have any (S,G) 
state, after parsing the Join or Prune and doing a lookup in its TIB.

We can certainly add "or discard the information in the message" in the 
list of examples.
I see this sentence has a typo ("has"' instead of "as"), which possibly 
was the source of confusion : it will be fixed too.

So, to avoid propagating any confusion any further, we can update the 
document as follows:

    A "message processed" being a message being parsed, a lookup being done, and some action being
    taken (such as, for instance, updating a control plane or data plane state or discarding the
    information in the message).

 > misleadingly multiplies by #mvpn_PE.
 >

As we see above, the #mVPN_PEs order of magnitude is definitely correct 
for each PE joining.

Now, if we let's look at when PE leave:
- each PE leaving causes a Prune to be sent (RFC 4601, section 4.5.7, 
end of p.73)
- PEs that are joined to (S,G), parse the message, try to find an (S,G) 
state in their TIB, find one, and then setup a short timer to send a 
Prune to override the Join
- PEs that are not joined to (S,G), parse the message, try to find an 
(S,G) state in their TIB, find none, and decide they can discard the 
information
- the upstream PE parses the message, matches it with its (S,G) state, 
setups a timer and :
   - in the "last PE leaving" case, after a delay stop forwarding traffic
   - else, will not stop forwarding because of the Join sent by another 
PE to override the Prune

The #mVPN_PEs factor here also correctly reflects the amount of work 
done by PEs.

 >
 > In fact, the number of messages received by a given PE does not 
increase in proportion to #mvpn_PE.

The content of Appendix A certainly does not imply that "the number of 
messages
received by a given PE increases in proportion to #mvpn_PE".

The #mVPN_PE factor is for the total across all PEs, not for a given PE.

( this exact same comment was in your email from June 11th, and I 
already replied to you June 19th : you deliberately ignored my reply, I 
don't think it is fair to again further propagate such irrelevant 
implicit claims on what the document says...)

 >
 > This problem infects the entire set of tables in the Appendix.
 >

See above, your point is only right if you tweak what "processing a 
message" means up to the point of being irrelevant and not accounting 
for all the processing done by PEs on a LAN.

Let me now highlight that the points about "prune override" and "join 
suppression" in your recent email where forming a large part of your 
comments during the LC in June, and where fueling your claims of 
"incorrectness" of the analysis (such as the claim that the "join 
suppression" was not taken into account).  By not providing a precise 
quote of what exactly would have been "incorrect" until last week -- 
despite multiple requests to you to provide them, and despite the 
multiple clarifications I provided on the fact that "join suppression" 
and "prune override" where relevantly accounted for -- you have 
artificially delayed the clarification of this issue for roughly six 
months.

-Thomas

---

From: Thomas Morin <[email-masked]>
Date: Wed, 16 Dec 2009 09:50:54 +0100

Eric, working group,

Eric Rosen:
 > on the Source-Specific Multicast mode,
[...]

You made this comment in June, and the document has since precisely been 
revised to cover ASM (section A.1.2).
Still claiming that "the analysis focuses heavily on the Source-Specific 
Multicast mode" is just misleading.

The only reason why A.1.2 (on ASM) it is smaller than A.1.1 (on SSM) is 
because it avoids being redundant, for the portion of the procedures for 
the setup and maintenance of one (S,G) SPT state or one (*,G), that are 
common with one (S,G) SSM state.

 > [...] To support the recommendations of section

I think that the introduction of A.1.2 is clear on this:

   PEs will generally have to maintain one shared tree, plus one source
   tree for each source sending toward G; each tree resulting in an
   amount of processing and state maintenance similar to what is
   described in the scenario in Appendix A.1.1,

We can make it even clearer by adding :
- in the start of section A.1.2 (ASM):

      The conclusion in A.1.1 are reused in this section, for the parts 
that are
      common to the setup and maintenance of states related to a source 
tree
      or a shared tree.

- in the introduction of section A.1.1 (on SSM):

       For all options provided for C-multicast routing, the procedures 
to setup
       and maintain a shortest path tree toward the source of an SSM 
group are
       the same than the procedures used to setup and maintain a 
shortest path
       tree toward an RP or a non-SSM source ; the results of this 
section are
       thus re-used in section A.1.2.

 >
 > [...] In addition to the time spent on the virtually irrelevant SSM 
scenario, [...]

This statement looks to me as totally abusive:
- SSM by itself is certainly not an "irrelevant scenario" ; 
one-to-many-receivers applications certainly are part the applications 
of enterprise VPN customers (look at section 4.1 and 4.1.1 of RFC4834) ; 
at the very least you'd need to support such a claim with actual data...
- as explained above, many results from the SSM case are reused in the 
ASM case, the behavior related to the maintenance of a tree on the the 
shortest path to a source or RP being common

So, this "time" is not spent unwisely...

 > uses a mechanism whose scalability is already known to be poor and which

Let me note that the potential security issues of this way of supporting 
ASM with BGP-based C-multicast routing, is one of the reasons identified 
in section 4 of the document that lead us to make the recommendation 
that this approach should not be the only one implemented.

Do you want us to remind this in Appendix A.1.2 ?

( The other claim you have that the scaling analysis in Appendix A 
should be extended to look at more than a single (S,G) : I will address 
this in a separate email. )

 > the MVPN, multiplied by the average number of upstream PEs per source.

You are correct that for each source sending to a group, there may be 
one S-A A-D route per PE upstream for this source.

I will update the text accordingly.

Let's also note that this won't change the orders of magnitude.

(What you say in the rest of your comment is already mentioned in A.1.2.)

 > A, for the following reasons:

Your claim that this scaling issue would be "considered insignificant" 
is not supported.  Your "following reasons" is only followed by stuff 
related to PIM, and I find nothing in A.1.2 indicating we would downplay 
the dependency of BGP of the number of sources  (and not either for PIM 
actually).

 > the Source Active A-D routes.

When saying "I also really don't see what this has to do with the issue 
at hand" and mentioning Source Active A-D routes you are attempting to 
redefine the scope of the section you comment.

To avoid confusion, I will address your claims on the overhead of 
per-source routes or messages in a separate message.

 > Certainly there is no analysis provided that

I don't share with you that what we state implies that the comparison 
with the corresponding mechanisms in PIM would be "favorable for BGP".

The goal for this section is to thoroughly assess the load put on BGP or 
PIM as C-multicast routing protocols in the ASM case.

The hard part is to find a way to compare very distinct mechanisms in 
PIM and BGP (as you said, one relies on dataplane driven events where 
the other does not, and one relies on an Assert mechanisms where the 
other does not, and one does not rely on BGP S-A A-D messages where the 
other does not). And the key is to get something meaningful in the 
context of A.1 (scalability wrt. number of PEs) taking into account the 
mechanisms in play for both PIM and BGP protocols, with hypothesis that 
are relevant for an operator trying to choose between the two.

I think that the section properly explains why the orders of magnitude 
for a increased number of PEs are essentially the same for ASM than the 
one found for SSM.

That said, precise suggestions to amend this subsection are still welcome.

( However, note that we would be wrong to give too much weight to the 
scenario your focus on right above ("if a PE has only (*,G) receivers, 
or if it has no receivers for G at all") as it is not that useful for an 
operator willing chose a C-multicast routing option for a deployment, 
and looking at the PE-PE scalability and dimensioning aspect of the 
issue, because you cannot count on some PEs on the MVPN not having  
(*,G) state or having (*,G) state but no (S,G) state, since this depends 
on customer multicast policy and activity.  )

 > The new material on ASM does not appear to contain any analysis at 
all, just

I don't think this is true, and your comments certainly don't support 
this claim.
(But without anything more explicit from you, your claim is just not 
refutable...)

That said, we can possibly make A.1.2 better, and all suggestions are 
welcome.
(easier if we don't get an artificial 6 month delay before we get some 
usable feedback)

-Thomas

---

From: Thomas Morin <[email-masked]>
Date: Wed, 16 Dec 2009 15:01:51 +0100

Eric, working group,

Eric Rosen:
> long as those multicast states have a PMSI as input interface.  That's way more multicast states than any PE has to maintain.

More exactly, each RR, has to keep the states of all the PEs in its cluster.
Note that this is not specific to multicast, but already true with unicast.

> The analysis in this Appendix simply ignores the negative scaling effects due to the large number
> of RR clients.
>    

I think the analysis in A.1.1.3 properly takes into account the "number 
of RR clients": the "total state maintained" for RRs (last line of the 
table, fourth column) includes a 2x#R_PE factor, corresponding to each 
PE being a client of two RRs.

As far as "negative scaling effects" are concerned, this is your own 
"conclusion". Based on the analysis,  section A.1.1 concludes that the 
different approaches for C-multicast all have to overall maintain an 
amount of state comparable in order of magnitude (linear with the number 
of PEs), but spread differently on the different types of equipments.

(let me also note that this specific comment above could have been done 
during the LC in June, but wasn't)

> PE will always need to send a C-multicast Join route to the RRs;

> in PIM the second PE might not need to send any message.

This assertion on PIM is incorrect:
RFC4601, section 4.5.7, end of p.73 : when JoinDesired transitions to 
true a Join is always sent.

> Whether the RRs will forward
> originated by the first PE.

This is incomplete :
- for a re-advertisement to happen, the route need to be more preferable 
*and* have different attributes
- we have to take into account the aggregation done by RRs (described in 
section 11.4 of draft-ietf-l3vpn-2547bis-mcast-bgp)
- for each RR, whether or not he might have to re-advertise will depend 
on which clusters the "second PE" and "upstream PE" are in

(More on this below...)

> A similar observation applies when C-multicast
> routes are withdrawn.

(yes, similar, and especially with the same correction to make on the 
PIM side : a Prune is always sent when JoinDesired transitions to False 
for a said (S,G) because the last CE has left this stream, RFC4601 
section 4.5.7, end of p.73)

Now, to go back on the computation in  section A.1.1.3 on BGP, you are 
correct that the current text does not yet properly reflects the cases 
where an RR may re-advertises a route they were already advertising. 
Let's look practically at the attributes that could vary :

   - next-hop : updates of the value of this attribute do not "follow" 
PE advertisements, being rewritten by the RR when aggregating 
C-multicast routes having same NLRI (section 11.4 of 
draft-ietf-l3vpn-2547bis-mcast-bgp) ; the advertisements made by 
different RRs may lead to updates of the value of this attribute

   - ORIGINATOR_ID : updates of the value of this attributes do not "follow" PE advertisements, as the RR rewrite it when aggregating C-multicast routes having same NLRI; the advertisements made by different RRs may lead possibly to updates of the value of this attribute, given that each RR would set ORIGINATOR_ID to a value of its own

   - CLUSTER_LIST : the value of this attribute only varies between clusters, changes of the value of this attributes does not "follow" PE advertisements, and only advertisements made by different RRs may lead possibly to updates of the value of this attribute

- LOCAL_PREF : the value of this attribute is determined locally, this 
is true both for the routes advertized by each PE (which could all be 
configured to use the same value) and for a route that results from the 
aggregation by an RR of the route with the same NLRI advertised by the 
PEs of his cluster (the RRs could also be configured to use a local pref 
independent from the local_pref of the routes advertised to him) ; 
updates of the value of this attribute don't "follow" PE or RR 
advertisements

Let's keep in mind that these attributes don't play any active role in 
C-multicast routing (contrary to the role they play in unicast).  Other 
BGP attributes do not have, AFAIK, any particular reason to be set for 
C-multicast routes, and if they were, an RR (or ASBR for attributes 
relevant for inter-AS) could overwrite these values when aggregating 
these routes.

Given the above, for a said C-multicast Source Tree Join (S,G) NLRI, 
what may force an RR to re-advertise the route with different attributes 
to the upstream PE would be the case of an RR of another cluster 
advertising a route better than its current best route.

If we consider our (#R_PE -1 ) joining a said (S,G), one after the other 
after the first PE joining, some of these events may thus lead to a 
re-advertisement to the upstream PE, but the number of times this can 
happen is at worse the number of RRs in clusters having receivers ; plus 
one because of the same route for the local cluster.  Given that we look 
at scalability with an increased number of PEs, we need to consider the 
possibility where all clusters have a receiver PE.

With these elements in mind, it seems to me that the section A.1.1.3 
need to be updated, along with an explanation in the lines of the one 
provided above, so that the second column reads like this:

+------------+----------
|            | upstream
|            | PE (1)
|            |
|            |
|            |
+------------+----------
| first PE   | 2 m.e
| joins      |            (unchanged)
+------------+----------
| for *each* | between 0
| additional | and 2 m.e         (was zero)
| PE joining |
|            |
+------------+----------
| baseline   | 0
| processing | (unchanged)
| over a     |
| period T   |
+------------+----------
| for *each* | between 0
| PE leaving | and 2 m.e       (was zero)
|            |
+------------+----------
| the last   | 2 m.e (unchanged)
| PE leaves  |
+------------+----------
| total for  | at most
| #R_PE PEs  | #RRs m.e    (was 4 m.e)
|            |
|            |
|            |
|            |
+------------+----------

(for clarity, the cells in lines 2,4 and 6 would also need to point to 
the explanation)

Note that between "4 m.e" and "at most #RRs", we do not change the order 
of magnitude when the number of PE increases. This is not a "surprise" 
given the aggregation made by RRs.

-Thomas

PS: Let me also note, again, that your comment could have been done 
during the LC in June, but wasn't.

---

From: Thomas Morin <[email-masked]>
Date: Wed, 16 Dec 2009 16:12:29 +0100

Eric, working group,

Eric Rosen:
> well as from PEs.

This is a comment you had already made during the LC in June, a short 
thread followed on this point, and I provided reasons explaining why 
covering PE-CE aspects would not be a relevant extension to Appendix A.  
Unfortunately, you left my email from July 10th without a reply.

> Suppose that by adding BGP, you could reduce this total
> you need to maintain.  Would that be a good tradeoff?

Come on, Eric, who do you think will buy such a cheap tradesman trick 
?...  ;-)

> In order to support

The scope of Appendix A is to provide insight to the arguments in 
section 3.3 which aim at comparing the C-multicast PE-PE multicast 
routing options, by looking at scalability with an increased number of 
PEs, and the cost of a PE joining or leaving.

What you only did in your recent comments is build a specific scenario 
where you show that PE-CE is "the bottleneck". It certainly does not 
mean that this is always the case.   And the concluding recommendations 
do address the case when PE-CE would be the bottleneck (as explained below).

Thus, I stand that what is said in section 3.3 or Appendix A for these 
matters does not mandate extending the scope of the PE-PE scaling 
analysis to the PE-CE overhead.

> that one aspect is the bottleneck.

I have already explained in the past, and again in the beginning of this 
email, why PE-CE is not part of the analysis in Appendix A.

What *you* are doing is focusing on "steady state" (with a definition of 
steady state that is probably not relevant by the way) to convince us 
that PE-CE is the bottleneck.

While the analysis in A.1 is precisely *not* focusing on steady state or 
non-steady state, but looking at both, so I don't think why we would 
have to show that any is the bottleneck.

> In fact, the analysis of Appendix A fails to demonstrate any real

Appendix A does not "fail" to do what you suggest, since it does not 
need to do so and not claim doing so.

Now, when you say that PE-CE may sometimes be a significant factor of 
the overall load due to multicast routing, I think this is reasonable, 
which is why the following text was added to a new sub -section of 
section 3.3 (section 3.3.2):

    The overhead associated with the PE-CE exchange of C-multicast
    routing is independent of the choice of the mechanism used for the
    PE-PE C-multicast routing.  Therefore, the impact of the PE-CE
    C-multicast routing overhead on the overall system scalability is
    independent of the protocol used for PE-PE signaling, and therefore
    is not relevant when comparing the different approaches proposed for
    the PE-PE C-multicast routing.  This is true even if in some
    operational contexts the PE-CE C-multicast routing overhead is a
    significant factor in the overall system overhead.

Furthermore, to reflect your point that PE-CE may in some cases be an important factor, and to also reflect the fact that actually a provider will have to pounder each argument in its own deployment context, I propose to replace the following from 3.3.7:

    In the meantime, this recommendation would enable service providers
    to choose between the first and the fourth mechanism, without
    this choice being constrained by vendors implementation choices.

  with:

    In the meantime, this recommendation would enable a service
    provider to choose between the first and the fourth mechanism,
    without this choice being constrained by vendors implementation
    choices, and taking into account the peculiarities of its own
    deployment context by pondering the weight of the different
    factors into account.

-Thomas

---

From: Thomas Morin <[email-masked]>
Date: Wed, 16 Dec 2009 16:20:09 +0100

Eric, working group,

Let me first note that these comments quoted below are not specific to 
new content added in releases -04/-05 and should have been done during 
the LC in June.

Eric Rosen:
[...]
> Also omitted from consideration are scaling issues that show up

No, section A.2 describes the cost of one individual PE joining and 
leaving, which provide the required insight to compare the behavior of 
the different C-multicast routing protocols for different rates of 
change of multicast subscription by PEs.

(This section details what was briefly mentioned in the end of section 
A.4 since version -00 of the WG draft)

> shutdown transients, when a PE first joins an (S,G) or prunes an (S,G). 

Indeed...

> A rather big deal is made of the "prune override" procedures.

( I don't think that the analysis focuses more on the "prune override" 
procedures than on any other PIM or BGP procedures.  Please be more 
explicit. )

> I have focused this critique on the steady state, because if the 
> steady state (as I defined

The statements above are "surprising":
- first, you imply that "a steady state" may be "reached" : this 
certainly does not take into account that PEs joining and leaving is 
something that, unlike unicast routing changes, happens following 
customer multicast application use patterns and customer routing 
policy   (by the way, "transient" is probably not the best term, since 
it is most often used for unicast routing which has specific dynamics)
- second, you state that "thee has been no analysis showing that the 
transients are more important than the steady state" :
     * this is certainly true, but at the same time, there is nothing is 
the analysis that claim that
     * and the reverse (steady state more important than dynamic events) 
is the basis for your claim that we do not take into account the PE-CE 
side, but certainly has not been shown either

Given the above, let me remind you of the following : Appendix A looks both at the transient scenario with some join/leave ("for each additional PE joining/leaving")*and*  the steady state scenario ("baseline processing over a period T"), where for some period no PE joins or leaves.

If we were to constrain the analysis to*only*  the scenario "where no PE joins or leaves" (that is purely imaginary by the way, as such a period must at least start with a period where some PE join a streams, and would have an end too), we would precisely loose generality and irrelevantly narrow the scope.

> done by focusing on a single (S,G) over a limited time period.

(The above statement is unsupported.)

-Thomas

---

From: Yakov Rekhter <[email-masked]>
Date: Wed, 16 Dec 2009 11:23:09 -0800

Thomas,

> when it starts to need to send (S,G) traffic towards a CE.
  
Just to add, from 4.5.7 of rfc4601:

   If a router wishes to propagate a Join(S,G) upstream, it must also
   watch for messages on its upstream interface from other routers on
   that subnet, and these may modify its behavior.  If it sees a
   Join(S,G) to the correct upstream neighbor, it should suppress its
   own Join(S,G).

which means that in order to suppress its own Join, the router
should be in the state where it "wishes to propagate a Join (S,G)
upstream". Clearly, that until the second PE receives Join
from downstream, the second PE is *not* in the state where it
"wishes" to propagate a Join upstream, and thus will *not*
suppress its first Join.

Yakov.

---

From: Yakov Rekhter <[email-masked]>
Date: Wed, 16 Dec 2009 11:35:39 -0800

Thomas,

> change of multicast subscription by PEs.

On the topic of scaling "as the rate of change of multicast states
increases" that Eric brough in his e-mail, let's keep in mind that
the rate of change of multicast state is of practical significance
only when it starts to impose a non-neglidgeable load on the control
plane. You covered this in section 3.3.5 already. So, Eric's
claim that this is "omitted from consideration" is incorrect.

Yakov.

---

From: "NAPIERALA, MARIA H (ATTLABS)" <[email-masked]>
Date: Wed, 16 Dec 2009 14:52:12 -0500

Does this mean that BGP C-multicast signaling should never be deployed without RT-Constraint because otherwise it would not work correctly/would not scale?

Maria

> -----Original Message-----
> From: [email-masked] [mailto:[email-masked]] On Behalf
> Of Thomas Morin
> Sent: Wednesday, December 16, 2009 3:40 AM
> To: L3VPN; Eric Rosen
> Subject: Re: draft-ietf-l3vpn-mvpn-considerations-05 / Appendix A /
> RTConstraint recommendation
> 
> Eric, working group,
> 
> Eric Rosen:

---

From: "NAPIERALA, MARIA H (ATTLABS)" <[email-masked]>
Date: Wed, 16 Dec 2009 16:16:36 -0500

Thomas,

regarding my question: "In section A.1.1.3, why the 'each additional PE joining' row for the 'upstream PE' column shows "0"?", you pointed me to this e-mail.
Please see my replies in-line.

Maria

> different RRs may lead to updates of the value of this attribute

This does not help in route aggregation because RR applies the ORIGINATOR_ID to each client's/egress PE route.

>    - ORIGINATOR_ID : updates of the value of this attributes do not
> "follow" PE advertisements, as the RR rewrite it when aggregating C-
> multicast routes having same NLRI; the advertisements made by different
> RRs may lead possibly to updates of the value of this attribute, given
> that each RR would set ORIGINATOR_ID to a value of its own
> 

Regarding route *aggregation* the draft-ietf-l3vpn-2547bis-mcast-bgp-08 only states the following:

   "To further reduce the routing churn due to C-multicast routes changes
   a Route Reflector that re-advertises a C-multicast route SHOULD set
   the Next Hop field of the MP_REACH_NLRI attribute of the route to an
   IP address of the Route Reflector. Likewise, an ASBR that re-
   advertises a C-multicast route SHOULD set the Next Hop field of the
   MP_REACH_NLRI attribute of the route to an IP address of the ASBR."

It states nothing about setting ORIGINATOR_ID to self on RR. 

Maria

---

From: Robert Raszuk <[email-masked]>
Date: Thu, 17 Dec 2009 00:28:28 +0100

Hi Maria,

Excellent point. Looking back at the archive of this WG I have been 
pointing out this issue in the past, but it was always never addressed.

It seems that for this mvpn solution to work classic "bgp route 
reflectors" would not do the right job.

They should be replaced with "multicast control aggregation" devices 
which would have to support it's own set of requirements and it's own 
advertisement rules. In fact draft already recommends to use such 
devices as a standalone control plane boxes.

Therefor before proceeding in this WG I would like to either see all 
references to BGP route reflectors use removed. Alternatively they could 
be replaced with the new term describing their newly defined and 
apparently required functionality for any form of sensible operation of 
the proposed herein solution.

Cheers,
R.

---

From: "NAPIERALA, MARIA H (ATTLABS)" <[email-masked]>
Date: Wed, 16 Dec 2009 19:39:02 -0500

Thomas,

Let me restate some of the comments from my original e-mail since you said that some of them you addressed in the replies to Eric. This is (fragments) of one of those e-mails.

Maria

>  >    |            |            | PEs)          |          |
> |
>  >    | joins      |            | m.e           |          |
> |
>   - the upstream PE processes the Join and ends up propagating the
>     state upstream and setting up forwarding state

The processing described under the first bullet above also means setting up (s,g) state in the control plane.  I assume this is what you mean by "processing" vs. "parsing" under the second bullet. (BTW, if this needs to be analyzed down to the level of *parsing* of the messages, let's be it..).
Clearly, workload under the first bullet is not the same amount of work as that described under the second bullet. The second bullet is just *discarding* the massage based on two things: "I am not the upstream router" and "I don't have (S,G) state". It cannot be claimed that this is equal with what the upstream PE has to do.

Let's compare it with the first row "first PE joins" of table in section A.1.1.3. i.e., with the workload on the upstream PE when BGP signaling is used. All the work that upstream PE needs to do in PIM signaling case also needs to be done in BGP signaling case. But with BGP signaling, upstream PE has also to perform the BGP processing (and *parsing*), including the best path selection. I don't see this being accounted for at all on PEs.
If you worry so much about the work a PE has to do to *discard* a message, why would you ignore a significant amount of BGP related work on a PE?

>  >    | PE joining |            |               |          |
> |

The workload on the upstream PE (and other PEs that already have (S,G) state) in this case is significantly less than in the case of "the first PE joining" since the PEs already have the (S,G) state. But it is estimated as the same. On those PEs that do not have (S,G) there is even less work. Yet, all is estimated as the same amount of work. 

> Sent: Wednesday, December 16, 2009 3:48 AM
> To: L3VPN; Eric Rosen
> Subject: Re: draft-ietf-l3vpn-mvpn-considerations-05 / Appendix A / PIM
> LANprocedures
> 
> Eric, working group,
> 
> Eric Rosen:
>  >
>  >    |            |            | PEs)          |          |
> |
>  >    | joins      |            | m.e           |          |
> |
>  >    | PE joining |            |               |          |
> |

---

From: Yakov Rekhter <[email-masked]>
Date: Thu, 17 Dec 2009 06:14:42 -0800

Maria,

> Does this mean that BGP C-multicast signaling should never be deployed 
> without RT-Constraint because otherwise it would not work correctly/would 
> not scale?

BGP C-multicast signaling works correctly without RT-Constraint.
 
The scale at which a service provider can start wanting RT constraint
is very much dependent on the scale of deployment and performance
of the equipments used.

Yakov.

---

From: "NAPIERALA, MARIA H (ATTLABS)" <[email-masked]>
Date: Thu, 17 Dec 2009 11:13:41 -0500

That is what I thought, in which case the consideration draft should
consider both BGP C-multicast signaling scenarios: with and without
using RT-Constraint.

Maria

> From: Yakov Rekhter [mailto:[email-masked]]
> Sent: Thursday, December 17, 2009 9:15 AM
> To: NAPIERALA, MARIA H (ATTLABS)
> Cc: Thomas Morin; L3VPN; Eric Rosen
> > >  > Since RT-constraint is not a mandatory to implement mechanism,
I
> > > don't see
> > >        following the recommendation of [I-D.ietf-l3vpn-2547bis-
> mcast-
> > > bgp]
not
> > > used

---

From: "Drake, John E" <[email-masked]>
Date: Thu, 17 Dec 2009 08:25:55 -0800

Would it be possible to call the question? 

> From: [email-masked] [mailto:[email-masked]] 
> On Behalf Of NAPIERALA, MARIA H (ATTLABS)
> Sent: Thursday, December 17, 2009 8:14 AM
> To: Yakov Rekhter
> Cc: Eric Rosen; Thomas Morin; L3VPN
> Subject: RE: draft-ietf-l3vpn-mvpn-considerations-05 / 
> Appendix A /RTConstraint recommendation 
> 
> That is what I thought, in which case the consideration draft 
> should consider both BGP C-multicast signaling scenarios: 
> with and without using RT-Constraint.
> 
> Maria
> 
> > is very much dependent on the scale of deployment and 
> performance of 
> > the equipments used.
> > 
> > Yakov.
> > 
> > 
> > > Maria
> > >
> > > > Subject: Re: draft-ietf-l3vpn-mvpn-considerations-05 / 
> Appendix A
> /
> > > >     the Route Targets that the PE uses for the VRF 
> Route Imports 
> > > > extended
> > > > PS: Let me note that you did not bring up this up 
> during the LC in 
> > > > June, and that this comment is not related to any new 
> content in 
> > > > the revisions made since this LC.
> > > >
> > >
> > >
>

---

From: Thomas Morin <[email-masked]>
Date: Fri, 18 Dec 2009 16:02:17 +0100

Eric, working group,

In different parts of the document (quoted below) you claim that the 
scaling analysis should consider scalability with an increased number of 
sources sending to a group.

Before looking at the relevance question, let's see how it relates to 
the current state of the document.
Section 3.3 of the document relates to scalability with an increased 
number of PEs, and refers to the scaling analysis in Appendix A.1 
"Scalability with an increased number of PEs". Given this, claiming that 
not doing a scaling analysis with an increased number of (S,G) is "a 
methodological error" in this scaling analysis is not correct.

That "scaling with an increased number of (S,G)" should be studied is 
something that noone has proposed to do yet, as far as I can tell.

Now, that said, let's try to see how interesting it would be to consider 
looking more at this aspect:
- in your comments, you insist on the linear dependency of BGP 
C-multicast routing with the number of (S,G), but scaling linearly with 
the number of sources sending to a group is not a peculiarity of the BGP 
Source Active auto-discovery mechanism, but is also common for PIM-SM 
SPTs and SSM trees (whether they are carried across the provider core 
through PIM and BGP, by the way)
- "the number of sources sending to a group", or the number of SPTs or 
SSM trees, is not something typically under the control of the provider, 
as it completely depends on the multicast routing policy and 
subscriptions patterns of each MVPN customer; so it should be obvious to 
anyone that the provider will have to prepare its network to scale with 
these unknowns in mind and put bounds to the amount of per-source state 
on his PEs (and RRs) ; this is true independently of whether PIM-SM or 
BGP is used for C-multicast routing (and true whether or not MSDP is 
used in that deployment)
- note also also that "scaling with reference to the number of sources" 
is trivially extrapolated from the single (S,G) case
- as far as RR are concerned (a point on which you focus below): having 
an amount of state to maintain on RRs that scales linearly with the 
scale of deployments and customer routes is not unusual, especially when 
it offers scaling advantages

Given the above, I don't think it is obvious that looking at scalability 
with the number of sources sending to a group, will be helpful for a 
provider to choose the protocol he will use for PE-PE C-multicast routing.

Speaking for myself as a contributor, these elements explain why I did 
not contribute a scaling study for an increased number of sources.   
"//Your mileage may vary", but we are six month after WG LC....

-Thomas

PS: a few more comments below

Eric Rosen:
> number handled by any PE router,

(This is not exact, as a said PE could use different RRs for different 
VPNs.)

>   failure to consider this particular scaling
> Reflectors.  [...]

I disagree: as said, above, PIM approaches are subject to the same type 
of linear dependency with the number of (S,G).

[...]
> [...] let's look specifically at the ASM case.
>

>           updates being sent (228) in the SP backbone?
>
> the multicast WGs have decided not even to bring forward to IPv6.[...]
>    

Your are generalizing Pekka's comment on a *security* issue MSDP *in the 
Internet*, to the issue of *scaling* in a multicast VPN context.  I 
think this is an abusive generalization of his quote, and that it does 
not help much seeing why draft-ietf-l3vpn-mvpn-considerations should 
look at scaling with reference to the number of sources.   Nobody 
objects that security issues related to the amount of "per source" state 
is a legitimate concern, but that does not mechanically make it an 
interesting scalability question in our context.

Second, let me note that you replied yourself to Pekka Savola to his 
comment, by referring to security mechanisms common to PIM and BGP, that 
allows to control the amount of multicast state in PEs.

[...]
> A [...]

(See start of this email)

---

From: Thomas Morin <[email-masked]>
Date: Fri, 18 Dec 2009 21:49:57 +0100

Maria,

> It states nothing about setting ORIGINATOR_ID to self on RR.

It also says:

      [...] C-multicast routes for a given (S,G) of a given MVPN 
originated by PEs that are clients of
      a given Route Reflector are aggregated by the Route Reflector. [...]

A BGP aggregated route is a route originated by the BGP peer doing the 
aggregation, why would it carry another ORIGINATOR_ID than the one of 
the BGP peer doing the aggregation (here the RR) ?

-Thomas

---

From: Thomas Morin <[email-masked]>
Date: Fri, 18 Dec 2009 22:02:23 +0100

Hello Robert,

> reflectors" would not do the right job.

You challenge the fact that BGP MVPN would not work with "classic" bgp 
route reflectors: why do it after the specs are long gone out of the 
working group ?  how does it relate to the current discussion ?

(supposing it is even relevant to claim that, and pending some 
definition of "classic"...)

> advertisement rules.

(Your claim is vague enough to not be even refutable, but:)
What is true is that the aggregation done by RR for mVPN following 
section 11.4 of draft-ietf-l3vpn-2547bis-mcast-bgp relies on mechanisms 
already defined in other specs.

> In fact draft already recommends to use such devices as a standalone 
> control plane boxes.

What in fact is written in the draft is this:

    In the higher scale scenarios, it may be
    required to adapt the route reflector infrastructure to the MVPN
    routing load by using, for example:

    o  a separation of resources for unicast and multicast VPN routing:
       using dedicated MVPN Route Reflector(s) (or using dedicated MVPN
       BGP sessions or dedicated MVPN BGP instances) ;

    o  the deployment of additional route reflector resources, for
       example increasing the processing resources on existing route
       reflectors or deployment of additional route reflectors.

    Among the above, the most straightforward approach is to consider the
    introduction of route reflectors dedicated to the MVPN service and
    dimension them accordingly to the need of that service (but doing so
    is not required and is left as an operator engineering decision).

So :
- this text refers to the use of RRs following 
draft-ietf-l3vpn-2547bis-mcast-bgp, but *not* to anything like 
"multicast control aggregation devices" which, as you say, would "have 
to support it's own set of requirements and it's own advertisement rules"
- the reasons for suggesting the use of RRs or dedicated RRs are purely 
*quantitative* (scale) and *not qualitative* as you suggest.

Cheers,

-Thomas

>> Thomas,
>>
>> regarding my question: "In section A.1.1.3, why the 'each additional 
>> PE joining' row for the 'upstream PE' column shows "0"?", you pointed 
>> me to this e-mail.
>> This does not help in route aggregation because RR applies the 
>> ORIGINATOR_ID to each client's/egress PE route.
>>
>> Regarding route *aggregation* the 
>> draft-ietf-l3vpn-2547bis-mcast-bgp-08 only states the following:
>>

---

From: Robert Raszuk <[email-masked]>
Date: Fri, 18 Dec 2009 22:12:02 +0100

Hi Thomas,

> the BGP peer doing the aggregation (here the RR) ?

Because RR behavior is very well already defined in RFC 4456  which says:

    ORIGINATOR_ID is a new optional, non-transitive BGP attribute of Type
    code 9.  This attribute is 4 bytes long and it will be created by an
    RR in reflecting a route.  This attribute will carry the BGP
    Identifier of the originator of the route in the local AS.

Here RR is doing _aggregation_ and not _origination_ of the route. Sorry 
but this is not the same.

C-multicast routes are originated by the PEs and to be compliant to RFC 
4456 each time best path shifts from one PE to the other the RR should 
reflect it to all of it's clients with new corresponding to the new best 
path ORIGINATOR_ID.

Cheers,
R.

---

From: Robert Raszuk <[email-masked]>
Date: Fri, 18 Dec 2009 22:19:30 +0100

Thomas,

My comments were concerning scaling the solution to be able to support 
real network's demands. Not some theoretical ones.

And the proposed solution of using RRs as aggregators of c-multicast 
routes is not what RR RFC 4456 specifies for reflection rules.

Cheers,
R.

---

From: Yakov Rekhter <[email-masked]>
Date: Mon, 21 Dec 2009 06:13:23 -0800

Thomas,

> BGP is used for C-multicast routing (and true whether or not MSDP is 

Just to add to the point above, the following from Security
Considerations section of draft-ietf-l3vpn-2547bis-mcast-09.txt:

   In particular, an implementation SHOULD provide mechanisms that allow
   a SP to place limitations on the following:

     - total number of (C-*,C-G) and/or (C-S,C-G) states per VRF

And as you mentioned above, "this is true independently of whether
PIM-SM or BGP is used for C-multicast routing (and true whether or
not MSDP is used in that deployment)"

Yakov.

---

From: Yakov Rekhter <[email-masked]>
Date: Mon, 21 Dec 2009 06:16:25 -0800

Robert,

> but this is not the same.

As the RR performs the aggregation, it is *the RR* that originates
the *aggregated route*.

Yakov.

---

From: Robert Raszuk <[email-masked]>
Date: Mon, 21 Dec 2009 15:54:59 +0100

Yakov,

The spec started with just talking about setting the next hop to self on 
the RR, then is was discovered that this is not sufficient and 
ORIGINATOR_ID was added.

Now we have a case where this is not really a route reflector as 
definition of reflection means to receive a path and *reflect* such path 
to the clients/non-clients.

Now this "RR" is an originator of the C-multicast routes ... I think it 
would be much better to call it correctly ... as already proposed to 
call it "C-multicast route aggregator" or perhaps just "C-multicast 
route proxy" - in any case it is clear that IBGP aggregation and route 
origination by a control plane only RR is a new network function - not 
that common in IP or even in VPN networks.

Now regardless of the name where can one find a definition of such IBGP 
aggregation ?

As an example: should such "IBGP aggregation device" remove local 
preference if it is send by the PEs along with c-multicast routes ? If 
not having two such IBGP aggregation devices which is likely to be the 
practical case may still generate unnecessary churn on all PEs involved.

Robert.

---

From: Yakov Rekhter <[email-masked]>
Date: Mon, 21 Dec 2009 07:05:39 -0800

Robert,

> the proposed herein solution.

Let me remind you that modifying the ORIGINATOR_ID and Next_hop
attributes by Route Reflectors (by setting up the ORIGINATOR_ID
attribute to self) is by no means new to C-multicast routing. From
rfc4684, section 3.2:

  i.   When advertising RT membership NLRI to a route-reflector client,
       the Originator attribute shall be set to the router-id of the
       advertiser, and the Next-hop attribute shall be set of the local
       address for that session.

Interestingly enough at the time this rfc was published, you did
*not* raise any objections/concerns to Route Reflectors modifying
the ORIGINATOR_ID and the Next_Hop attributes (by setting both of
them to self).

Moreover, at that time you did *not* request that in rfc4684 "all
references to BGP route reflectors use removed". You did *not*
request to replace all references to BGP route reflectors in that
rfc "with the new term".

All this is especially interesting since you are one of the co-authors
of rfc4684.

Yakov.

---

From: Robert Raszuk <[email-masked]>
Date: Mon, 21 Dec 2009 16:20:10 +0100

Yakov,

The goals of overwriting Originator_ID by the RR in RFC4684 are totally 
different then those described in mvpn spec.

To quote:

    The first of these route advertisement rules is designed such that
    the originator of an RT membership NLRI does not drop an RT
    membership NLRI that is reflected back to it, thus allowing the route
    reflector to use this RT membership NLRI in order to signal the
    client that it should distribute VPN routes with the specific target
    towards the reflector.

So it has no bearing on limiting the churn of RTs ... contrary it makes 
sure that reflected NLRI is just not dropped.

On the other hand in current mvpn draft the goal is to stabilize the 
control plane behavior to make it comparable with alternative solutions 
to the same application space.

Cheers,
R.

---

From: "NAPIERALA, MARIA H (ATTLABS)" <[email-masked]>
Date: Mon, 21 Dec 2009 13:16:35 -0500

Thomas,

>       a given Route Reflector are aggregated by the Route Reflector.

You omitted very important text from the same section 11.4 that precedes the one you quoted, namely:

  "Note that C-multicast routes are "de facto" aggregated by BGP. This
   is because the MCAST-VPN NLRIs advertised by multiple PEs, for a C-
   multicast route for a particular C-S and C-G (or a particular C-* and
   C-G) of a given MVPN are identical."

This is the route aggregation the text you quoted refers to. 

Regarding the routing *churn*, section 11.4 says further down that:

  ”To further reduce the routing churn due to C-multicast routes changes
   a Route Reflector that re-advertises a C-multicast route SHOULD set
   the Next Hop field of the MP_REACH_NLRI attribute of the route to an
   IP address of the Route Reflector."

Which, in fact, doesn’t reduce routing churn. 

Maria

---

From: Rahul Aggarwal <[email-masked]>
Date: Mon, 21 Dec 2009 10:54:28 -0800

Maria,

<snipped>

> Which, in fact, doesn’t reduce routing churn.

Incorrect. The text you quoted *does* reduce routing churn due to
C-multicast route changes. This is because it ensures that a Route
Reflector advertises a C-multicast route, for a given (C-S/*, C-G) only
when it receives the first route for the (C-S/*. C-G) from a client. It
also ensures that a Route Reflector doesn't withdraw a C-multicast route
for a given (C-S/*, C-G) as long as it has at least one route from a
client for the (C-S/*, C-G).

rahul

---

From: Robert Raszuk <[email-masked]>
Date: Mon, 21 Dec 2009 20:11:15 +0100

Rahul,

Incorrect.

As Maria have said correctly *the* quoted text does not reduce routing 
churn.

The procedure you are spelling out are not reflected in the above 
paragraph. At least not in version 8 of the 
draft-ietf-l3vpn-2547bis-mcast-bgp.

Thx,
R.

---

From: Rahul Aggarwal <[email-masked]>
Date: Mon, 21 Dec 2009 11:20:36 -0800

Robert,

> draft-ietf-l3vpn-2547bis-mcast-bgp.

That is an incorrect interpretation of the text in section 11.4. The
procedure I spelled out follows from the text in section 11.4.

rahul

---

From: "NAPIERALA, MARIA H (ATTLABS)" <[email-masked]>
Date: Mon, 21 Dec 2009 14:52:00 -0500

Rahul,

the Route Reflector advertizes the first C-multicast (S/*,G) Join route and any subsequent C-multicast (S/*,G) Join route if it is better than previously selected, with the corresponding to the new best route ORIGINATOR_ID.

Which text in section 11.4 states otherwise?

Maria

> Of Rahul Aggarwal
> Sent: Monday, December 21, 2009 2:21 PM
> To: Robert Raszuk
> Cc: Thomas Morin; L3VPN
> Subject: Re: draft-ietf-l3vpn-mvpn-considerations-05 / Appendix A
> /sectionA.1.1.3
> 
> 
> Robert,
>

---

From: Robert Raszuk <[email-masked]>
Date: Mon, 21 Dec 2009 20:56:02 +0100

Rahul,

> procedure I spelled out follows from the text in section 11.4.

Unfortunately not. Route reflector advertises *best* path not the 
*first* path. The text as stated today does not accomplish C-multicast 
routes aggregation. Also the concept of path virtualization you are 
essentially recommending at the withdraw is new.

Moreover it makes the use of route reflector "aggregators" a mandatory 
device in the solution.

Btw how service provider interested in the service and having full mesh 
of IBGP provide any c-multicast aggregation ?

How about service provider which uses confederations ?

I think this thread has reached it's limits. The discussion about route 
reflection as described in section 11.4 should take place on IDR list.

I bet vast majority of the members of L3VPN WG do not even read emails 
when the subject contains "mvpn" part :).

Cheers and Happy Holidays,
R.

---

From: Rahul Aggarwal <[email-masked]>
Date: Tue, 22 Dec 2009 15:03:31 -0800

Maria,

On Mon, 21 Dec 2009, NAPIERALA, MARIA H (ATTLABS) wrote:

> Rahul,
>
> the Route Reflector advertizes the first C-multicast (S/*,G) Join route and any
> subsequent C-multicast (S/*,G) Join route if it is better than previously
> selected,

No. As specified in section 11.4 the Route Reflector advertises an
aggregated C-multicast route.

> with the corresponding to the new best route ORIGINATOR_ID.
>

The entire text in section 11.4 describes how a Route Reflector
aggregates C-multicast routes.

At least two vendors with an inter-operable implementation have had no
trouble interpreting section 11.4. So despite your mis-interpretations
the spec is serving its purpose.

And finally, I am done repeating myself.

rahul

---

From: "NAPIERALA, MARIA H (ATTLABS)" <[email-masked]>
Date: Tue, 22 Dec 2009 18:49:56 -0500

Rahul,

> > with the corresponding to the new best route ORIGINATOR_ID.

You conveniently omitted this statement..

> aggregates C-multicast routes.

Well, this doesn't answer my question.

> the spec is serving its purpose.

I never said that section 11.4 (as it is written) produces non-interoperable implementations.
  
> And finally, I am done repeating myself.

Same for me.

Maria

> > > > >>   â€To further reduce the routing churn due to C-multicast
> routes
> > > changes
> > > > >>    a Route Reflector that re-advertises a C-multicast route
> SHOULD
> > > set
> > > > >>    the Next Hop field of the MP_REACH_NLRI attribute of the
> route
> > > to an
> > > > >> Which, in fact, doesnâ€™t reduce routing churn.
> > > > >>
> > > > >
> > > > > Reflector advertises a C-multicast route, for a given (C-S/*,
> C-G)
> > > only
> > > > > also ensures that a Route Reflector doesn't withdraw a C-
> multicast
> > > route
> > > > > for a given (C-S/*, C-G) as long as it has at least one route
> from
> > > a

---

From: Eric Rosen <[email-masked]>
Date: Wed, 23 Dec 2009 11:03:31 -0500

Yakov> By virtue of being non-segmented, non-segmented S-PMSIs do *not*
Yakov> provide the ability to aggregate P-tunnels of the intra-AS *segments*
Yakov> of S-PMSIs (for one thing, non-segmented S-PMSIs do not have "intra-AS 
Yakov> segments").

Well, by that logic one could say that non-segmented S-PMSIs are more
scalable on the grounds that they require fewer segments ;-)  This would fit
right in with the logic of Appendix A ;-)

Note that there are Carrier's Carrier techniques that can be used to
aggregate S-PMSIs through a transit AS when the S-PMSIs are not segmented.  

Eric> a downstream provider can use I-PMSIs even if an upstream provider
Eric> uses S-PMSIs.  That may or may not be a useful feature, but it doesn't
Eric> make the use of S-PMSIs more scalable.

Yakov> Sure it does.

I can't wait to see the multi-provider deployment in which one provider
decides to improve scale by using I-PMSIs and the other provider decides to
improve scale by using S-PMSIs ;-)

---

From: Eric Rosen <[email-masked]>
Date: Wed, 23 Dec 2009 14:31:11 -0500

> As specified in section 11.4 the Route Reflector advertises an aggregated
> C-multicast route.

Well, section 11.4 says:

   C-multicast routes are "de facto" aggregated by BGP. This
   is because the MCAST-VPN NLRIs advertised by multiple PEs, for a C-
   multicast route for a particular C-S and C-G (or a particular C-* and
   C-G) of a given MVPN are identical.

The use of "de facto" means that no new procedures are required, and that
the "aggregation" is the same as BGP automatically does in all cases where
two different routes have the same NLRI.  It is a very creative
interpretation to suggest this requires the RR to originate a new route
rather than to distribute a route received from a PE. 

> two vendors with an inter-operable implementation have had no
> trouble interpreting section 11.4

Note that different interpretations of section 11.4 will still
interoperate.  It's a bit odd to say that no one has had trouble
interpreting it when at the same time you're telling folks that they are
misinterpreting it ;-)

I think what we're really talking about here is an implementation
optimization that is interesting, but that is not required for
interoperability, and that might or might not be worth doing.  Ordinarily,
this sort of optimization would be considered to be a local matter, not
something about which there has to be WG consensus.  It's only the presence
of Appendix A that forces us to get bogged down in unproductive discussions
of such details, as we now have to argue about whether the so-called
analysis there is to be based only on what the specifications require or
also on what implementations are likely to do.

---

From: "NAPIERALA, MARIA H (ATTLABS)" <[email-masked]>
Date: Thu, 24 Dec 2009 09:56:31 -0500

> It's only the presence
> analysis there is to be based only on what the specifications require
> or also on what implementations are likely to do.
 
Ditto!

Maria