Subject: [Softwires] [softwire]comments on stateless 4v6 domain

From: Qiong <[email-masked]>
Date: Thu, 28 Jul 2011 14:28:32 +0800

Dear all,

Following the recent discussion on stateless 4v6 solutions, I'm still
wondering how to define or plan a reasonable stateless 4v6 domain in
reality. My main concern is how to distinguish between multiple different
domains with lower cost ?

I have noticed that we can define multiple stateless 4v6 domains in
draft-dec-stateless-4v6, 4rd, etc., within the same local-area. This can be
very helpful to apply different strategies for our customers. For example,
our legacy users can just have non-shared public IPv4 address while our new
customers would have shared-IPv4 address, or even with different multiplex
ratio.  In this situation, multiple domains can be defined for different
type of users and need to be distinguished accordingly.

Here, the ideal way is to define one IPv4 pool for one stateless 4v6 prefix.
However, our current situation is quite different. Not only does it has been
too difficult to retrieve the existing IPv4 addresses and re-plan them
again, but also the remaining IPv4 prefix pool is quite scattered. As a
result, the larger the domain covers, the more IPv4 prefix becomes. So, it
might be normal to have up to thousands of rules indicating specific IPv4
prefix pool and corresponding IPv6 prefix in stateless gateway.  Every
packet arriving at the stateless gateway has to firstly lookup the rule
table and find its 4v6 domain. This seems have turned a totally stateless
solution into partial stateful.

Please correct me if I missed something. :-) Thanks in advance.

Best wishes

Qiong Sun

---

From: Satoru Matsushima <[email-masked]>
Date: Thu, 28 Jul 2011 03:08:09 -0400

Hello Qiong,

On 2011/07/28, at 2:28, Qiong wrote:

> Dear all,
> 
> Following the recent discussion on stateless 4v6 solutions, I'm still wondering how to define or plan a reasonable stateless 4v6 domain in reality. My main concern is how to distinguish between multiple different domains with lower cost ?
> 
> I have noticed that we can define multiple stateless 4v6 domains in draft-dec-stateless-4v6, 4rd, etc., within the same local-area. This can be very helpful to apply different strategies for our customers. For example, our legacy users can just have non-shared public IPv4 address while our new customers would have shared-IPv4 address, or even with different multiplex ratio.  In this situation, multiple domains can be defined for different type of users and need to be distinguished accordingly. 
> 
> Here, the ideal way is to define one IPv4 pool for one stateless 4v6 prefix. However, our current situation is quite different. Not only does it has been too difficult to retrieve the existing IPv4 addresses and re-plan them again, but also the remaining IPv4 prefix pool is quite scattered. As a result, the larger the domain covers, the more IPv4 prefix becomes. So, it might be normal to have up to thousands of rules indicating specific IPv4 prefix pool and corresponding IPv6 prefix in stateless gateway.  Every packet arriving at the stateless gateway has to firstly lookup the rule table and find its 4v6 domain. This seems have turned a totally stateless solution into partial stageful.

Routing always supports us. You can choose a tunnel which have appropriate mapping rule by your routing table. Even it is 'totally stateless solution', you have to lookup routing table as far as it is routing network. Hope it helps your network design.

Best regards,
--satoru

---

From: Qiong <[email-masked]>
Date: Thu, 28 Jul 2011 16:31:45 +0800

Hi, Satoru,

Thanks a lot for your suggestion and I understand it can work. But in my
understanding, it still lies some differences with the so-called "totally
stateless solution".

Suppose there is only one domain, when an IPv4 downstream packet arrives at
stateless gateway, it would firstly forward to the virtual tunnel interface
(take 4V6 Mapped Tunnel for example) with only one default route. Then after
encapsulated with a uniform IPv6 prefix and IPv4 prefix, etc, it will lookup
IPv6 routing table and forward to 4v6 CE. Here, actually there is no need to
do routing lookup for IPv4 address expect one default route. However, if we
introduce multiple domains with multiple IPv4 prefix pools ( the number of
each domain is N1, N2, N3,... Nm) , then the entry number of IPv4 routing
table will be N1+N2+N3+..+Nm. In current situation, Nm would not be a small
number anymore. So maybe some more optimization work will still be needed to
make it more "stateless" ?

Best wishes

On Thu, Jul 28, 2011 at 3:08 PM, Satoru Matsushima <
[email-masked]> wrote:

> Hello Qiong,
>
>
> Routing always supports us. You can choose a tunnel which have appropriate
> mapping rule by your routing table. Even it is 'totally stateless solution',
> you have to lookup routing table as far as it is routing network. Hope it
> helps your network design.
>
> Best regards,
> --satoru

---

From: Qiong <[email-masked]>
Date: Thu, 28 Jul 2011 17:53:00 +0800

Hi, Remi,

Thanks for your comments. I agree with you that in this situation, stateless
solution still superior to stateful ones from scalability aspect. It is "per
independent IPv4 prefix" mapping rule database, rather than
customer-based. But it still may have some tradeoff for domain realm. The
larger the domain it covers, the more independent IPv4 prefix there will be.
And also the whole domain has to synchronize these mapping rules.

I think this problem would be quite common for large broadband service
providers who already have a huge amount of customers. Do we have to
transfer all these legacy customers with public IPv4 address into
shared-mode directly? In our consideration, we think it might be better to
offer shared-mode solution for new customers only and leave the legacy
customers with non-shared-mode solution to avoid complaint from address
sharing. As a result, we have to deal with the co-existence scenario for at
least shared-mode and non-shared-mode.

I agree that this mapping specification can be applied to all stateless
solutions, separated from specific solution. Thanks for your suggestion.

Best wishes

Qiong Sun

---

From: Satoru Matsushima <[email-masked]>
Date: Thu, 28 Jul 2011 05:55:44 -0400

Hi Qiong,

On 2011/07/28, at 4:31, Qiong wrote:

> Hi, Satoru,
> 
> Thanks a lot for your suggestion and I understand it can work. But in my understanding, it still lies some differences with the so-called "totally stateless solution". 

I don't mean that 'totally stateless solution' isn't a part of routing system. 

> Suppose there is only one domain, when an IPv4 downstream packet arrives at stateless gateway, it would firstly forward to the virtual tunnel interface (take 4V6 Mapped Tunnel for example) with only one default route. Then after encapsulated with a uniform IPv6 prefix and IPv4 prefix, etc, it will lookup IPv6 routing table and forward to 4v6 CE. Here, actually there is no need to do routing lookup for IPv4 address expect one default route.

You already did routing when you found default route.

> However, if we introduce multiple domains with multiple IPv4 prefix pools ( the number of each domain is N1, N2, N3,... Nm) , then the entry number of IPv4 routing table will be N1+N2+N3+..+Nm. In current situation, Nm would not be a small number anymore. So maybe some more optimization work will still be needed to make it more "stateless" ?  

What do you mean 'the entry number of IPv4 routing table'? 
Do you presume that your IGP routing has almost 300K routes as same number of current internet full routes?

Best regards,
--satoru

---

From: Tina TSOU <[email-masked]>
Date: Thu, 28 Jul 2011 11:10:47 +0000

Matsushima-san,
In this draft, we may need to refer or define the definition of 
- Stateful operation
- Stateless operation
- Dynamic port allocation
- Static port allocation

Hope in this way, the discussion could be easier.

Best Regards,
Tina TSOU
http://tinatsou.weebly.com/contact.html

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of Satoru Matsushima
Sent: Thursday, July 28, 2011 5:56 AM
To: Qiong
Cc: [email-masked]
Subject: Re: [Softwires] [softwire]comments on stateless 4v6 domain

Hi Qiong,

Best regards,
--satoru

_______________________________________________
Softwires mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/softwires

---

From: Satoru Matsushima <[email-masked]>
Date: Thu, 28 Jul 2011 09:03:26 -0400

Thank you Tina, it is also pointed out by Dave yesterday.

Interestingly, Qiong pointed out another issue, which is routing scalability.
I agree with Qiong that routing scalability is significant issue.
FWIW, I would recommend something, LISP for example, could help him.

Best regards,
--satoru

---

From: Rémi Després <[email-masked]>
Date: Thu, 28 Jul 2011 15:37:30 +0200

Hi, all,

How to use the 4rd stateless address mapping to support, in a provider network having multiple IPv4 prefixes, both exclusive and shared IPv4 addresses, is an typical question.

Starting from a comment from Quiong Sun, here are some views on the subject. 

Le 28 juil. 2011 à 11:53, Qiong a écrit :

> Hi, Remi,
> 
> Thanks for your comments. I agree with you that in this situation, stateless solution still superior to stateful ones from scalability aspect. It is "per independent IPv4 prefix" mapping rule database, rather than customer-based. But it still may have some tradeoff for domain realm. The larger the domain it covers, the more independent IPv4 prefix there will be. And also the whole domain has to synchronize these mapping rules. 
> 
> I think this problem would be quite common for large broadband service providers who already have a huge amount of customers. Do we have to transfer all these legacy customers with public IPv4 address into shared-mode directly? In our consideration, we think it might be better to offer shared-mode solution for new customers only and leave the legacy customers with non-shared-mode solution to avoid complaint from address sharing.

> As a result, we have to deal with the co-existence scenario for at least shared-mode and non-shared-mode.  

Yes, I think this will prove a frequent requirement.

Complementary comments:

1.
A possible approach, to progressively reduce the number of used IPv4 prefixes, consists in proposing two different tariffs:
- one for length L of IPv6 prefixes, and full IPv4 address per customer,
- the other, slightly lower, for length L+K of IPv6 prefixes, and shared IPv4 address per customer.

Thus the number of full-address customers can be expected to decrease, and the longest IPv4 prefixes can be expected to progressively be reclaimed.

2.
With a dihcotomic search for a prefix match in BR's and CE', the performance impact of having a large number of rules can remain quite limited, both in CE's and in BR's.
(Scalability of the BR function is in any case ensured by the fact that BR's are, with respect to individual-customer states, completely stateless (and also, of course, with respect to transport-connection states).

3.
By configuring appropriately as many mapping rules as there are IPv4 prefixes, all IPv4 prefixes can be used without any impact on IPv6 routing (no "IPv4 entropy" exported to IPv6).

This is illustrated in sec 3.2 of tools.ietf.org/html/draft-despres-softwire-sam-01 (not too easy to read, sorry for that, but the subject had some intrinsic sophistication and was new).
(Incidentally this 4rd founding draft wasn't included Ole's 4rd-history slide yesterday, but is IMHO worth looking at for anyone interested in the genesis of 4rd.) 

Hope it helps.

Regards,
RD

> I agree that this mapping specification can be applied to all stateless solutions, separated from specific solution. Thanks for your suggestion.
> 
> Best wishes
> 
> Qiong Sun 
>  
>

---

From: Jacni Qin <[email-masked]>
Date: Thu, 28 Jul 2011 22:19:09 +0800

hi Remi,

On Thu, Jul 28, 2011 at 9:37 PM, Rémi Després <[email-masked]>wrote:

> Hi, all,
>
> How to use the 4rd stateless address mapping to support, in a provider
> network having multiple IPv4 prefixes, both exclusive and shared IPv4
> addresses, is an typical question.
>

Jacni>: Yes, it is. As I mentioned in another thread, maybe in different
way.
I'd suggest some discussions about this should be added in the document.

> > Thanks for your comments. I agree with you that in this situation,
> stateless solution still superior to stateful ones from scalability aspect.
> It is "per independent IPv4 prefix" mapping rule database, rather than
> sharing.
> A possible approach, to progressively reduce the number of used IPv4
> prefixes, consists in proposing two different tariffs:
> Thus the number of full-address customers can be expected to decrease, and
> the longest IPv4 prefixes can be expected to progressively be reclaimed.
>
> Jacni>: The addresses are reclaimed progressively, that is one reason why
we get fragmented prefixes.

2.
> With a dihcotomic search for a prefix match in BR's and CE', the
> performance impact of having a large number of rules can remain quite
> limited, both in CE's and in BR's.
> (Scalability of the BR function is in any case ensured by the fact that
> BR's are, with respect to individual-customer states, completely stateless
> (and also, of course, with respect to transport-connection states).
>
> 3.
> By configuring appropriately as many mapping rules as there are IPv4
> prefixes, all IPv4 prefixes can be used without any impact on IPv6 routing
> (no "IPv4 entropy" exported to IPv6).
>

Jacni>: In favor of this one, if I could. The Access gateway can be where
the first aggregation happens. And we can have a *big* BR serving numerous
Access Gateways belonging to seperate domains.

Cheers,
Jacni

> This is illustrated in sec 3.2 of
> tools.ietf.org/html/draft-despres-softwire-sam-01 (not too easy to read,
> sorry for that, but the subject had some intrinsic sophistication and was
> new).
> (Incidentally this 4rd founding draft wasn't included Ole's 4rd-history
> slide yesterday, but is IMHO worth looking at for anyone interested in the
> genesis of 4rd.)
>
> Hope it helps.
>
> Regards,
> RD
>
>

---

From: Rémi Després <[email-masked]>
Date: Thu, 28 Jul 2011 16:50:46 +0200

Hi, Jacni,

Le 28 juil. 2011 à 16:19, Jacni Qin a écrit :
...

> Jacni>: In favor of this one, if I could.

AFAIK, you can.

> The Access gateway can be where the first aggregation happens. And we can have a *big* BR serving numerous Access Gateways belonging to seperate domains.

It is also possible to have several different BR sets, each set having its own BR anycast address and its set of IPv4 prefixes.
Thus, these BR's can be placed close to where these IPv4 prefixes enter the provider network. 

Cheers,
RD

---

From: Cameron Byrne <[email-masked]>
Date: Thu, 28 Jul 2011 08:10:06 -0700

> FWIW, I would recommend something, LISP for example, could help him.

My advice is that you do not want to tie your solution to LISP.

Cb.

> >> Thanks a lot for your suggestion and I understand it can work. But in
my understanding, it still lies some differences with the so-called "totally
stateless solution".
> >> Suppose there is only one domain, when an IPv4 downstream packet
arrives at stateless gateway, it would firstly forward to the virtual tunnel
interface (take 4V6 Mapped Tunnel for example) with only one default route.
Then after encapsulated with a uniform IPv6 prefix and IPv4 prefix, etc, it
will lookup IPv6 routing table and forward to 4v6 CE. Here, actually there
is no need to do routing lookup for IPv4 address expect one default route.
> >> However, if we introduce multiple domains with multiple IPv4 prefix
pools ( the number of each domain is N1, N2, N3,... Nm) , then the entry
number of IPv4 routing table will be N1+N2+N3+..+Nm. In current situation,
Nm would not be a small number anymore. So maybe some more optimization work
will still be needed to make it more "stateless" ?

---

From: "Peng Wu" <[email-masked]>
Date: Fri, 29 Jul 2011 02:10:38 +0800

Hi Satoru,

>Do you presume that your IGP routing has almost 300K routes as same number of current internet full routes?

If I get Qiong correctly, she means that, suppose the IPv4 IGP addresses blocks are quite de-aggregated already. If the ISP upgrade the network to IPv6, and support IPv4 using stateless 4v6 with the same IPv4 address distribution, then the IPv4 IGP kind of introduce the IPv4 IGP routing table into IPv6.

If this situation does exists, then it's an interesting operation problem.

---

From: "Peng Wu" <[email-masked]>
Date: Fri, 29 Jul 2011 02:32:14 +0800

+1
The first category is stateless encapsulation, no table lookup, something like that; the second category is port allocation manner.
Now the motivation draft kind of mix them together. I guess Classification is needed. Maybe describe both in separated parts? Depending on what you guys really want.

------------------				 
Peng Wu
PhD candidate
Department of Computer Science & Technology
Tsinghua University, Beijing, China

-------------------------------------------------------------
From：Tina TSOU
Date：2011-07-28 19:04:47
To：Satoru Matsushima; Qiong
CC：[email-masked]
Subject：Re: [Softwires] [softwire]comments on stateless 4v6 domain

---

From: Tina TSOU <[email-masked]>
Date: Thu, 28 Jul 2011 10:46:38 +0000

Matsushima-san,

Best Regards,
Tina TSOU

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of Satoru Matsushima
Sent: Thursday, July 28, 2011 5:56 AM
To: Qiong
Cc: [email-masked]
Subject: Re: [Softwires] [softwire]comments on stateless 4v6 domain

Hi Qiong,

Best regards,
--satoru

---

From: Qiong <[email-masked]>
Date: Fri, 29 Jul 2011 10:56:29 +0800

Hi Peng, Satoru,

Thanks for your comments and suggestions. Actually, I think there are two
kinds of routing issues around here, and we should take appropriate planning
for our stateless 4v6 routing system.

The first one is what Peng has mentioned above, that de-aggregated IPv4 IGP
addresses blocks together   with embedded non-continuous port set would have
impact on IPv6 routing system. This situation has no relationship with
shared-mode or exclusive-mode, and I personally think we can reduce this
impact with careful IPv4 address planning.

Here are two possible ways:
-- Guarantee that one shared-IPv4 address would only be allocated within the
same BNG (say, BRAS, etc). Thus, non-continous port info will no longer be
needed to shown up in IPv6 IGP and all the packets with the same shared-IPv4
address will be destined to the same BNG.
-- Select the same IPv6 stateless prefix with native IPv6 prefix in a
certain area. For example, if 2001:c68:1122::/48 is the prefix for one MAN,
we can also use it for stateless IPv6 prefix , followed by shared IPv4
address/port set. In this way, de-aggregated IPv6 routings can be further
aggregated in the upper level (say, backbone network). However, in this way,
this IPv6 prefix is relatively long and it would be difficult to ensure the
whole stateless IPv6 prefix (including IPv4 address/port set) to be less
than /64 (easier for SLLAC).

The second one is to deal with the co-existence scenario for shared-mode and
exclusive-mode (or other differentiated situation). As suggested by Satoru
and Remi, it can either be accomplished by IPv4 routing in stateless GW, or
by mapping rule specification. Anyway, it still have to do some kind of
 lookup, no matter whether it is a routing table or other kinds of database.
But I also agree that the performance would not be a big problem to handle
these additional prefix-based rules.

Best wishes

Qiong Sun

2011/7/29 Peng Wu <[email-masked]>

> If I get Qiong correctly, she means that, suppose the IPv4 IGP addresses
> blocks are quite de-aggregated already. If the ISP upgrade the network to
> IPv6, and support IPv4 using stateless 4v6 with the same IPv4 address
> distribution, then the IPv4 IGP kind of introduce the IPv4 IGP routing table
> into IPv6.
>

---

From: Qiong <[email-masked]>
Date: Fri, 29 Jul 2011 11:25:57 +0800

Hi, Tina,

Agree.

>From the perspective of routing issues discussed these days, I think
stateful ones would store customer (lightweight AFTR) /session
(DS-Lite/NAT44/NAT64) information, while stateless ones would just store
*location-based information* just as traditional routing system.

Actually, since stateless ones have combined some kind of user info into
these locators (e.g. shared-IPv4 address, port multiplex ratio, etc), it
would have some impact on routing system. That's why these configurations in
stateless solutions would be a little more complicated than stateful ones.
Maybe this is also a tradeoff between different solutions.

Best regards

Qiong Sun

On Thu, Jul 28, 2011 at 7:10 PM, Tina TSOU <[email-masked]>wrote:

> > Suppose there is only one domain, when an IPv4 downstream packet arrives
> at stateless gateway, it would firstly forward to the virtual tunnel
> > However, if we introduce multiple domains with multiple IPv4 prefix pools
> ( the number of each domain is N1, N2, N3,... Nm) , then the entry number of
> IPv4 routing table will be N1+N2+N3+..+Nm. In current situation, Nm would
> not be a small number anymore. So maybe some more optimization work will
> still be needed to make it more "stateless" ?
>

---

From: Qiong <[email-masked]>
Date: Fri, 29 Jul 2011 12:24:29 +0800

Hi Remi,

Thank you very much for your comments and your reference. It is exactly what
I want. Please see my comments inline.

1.
> [Qiong]: I'm not quite clear of the reason why "the longest IPv4 prefixes
can be expected to progressively be reclaimed". The situation here is that
these IPv4 prefixes (indicated as D1, D2, D3 in your draft) are disjoint
prefixes with different lengths. Maybe we can firstly choose a shorter
prefix (say D1 in your example). But after we have run out of this address
pool, we have to choose another one from the remaining pools, and it is
rather difficult to re-allocate existing IPv4 address pool unless this full
IPv4 address customers do not exist anymore. Would you please explain it a
bit more ? Thanks.

> (and also, of course, with respect to transport-connection states).
[Qiong]: Agree. The performance impact would not be a big problem for BR's.
But we should still take our plan carefully with the region it covers.

> (no "IPv4 entropy" exported to IPv6).
[Qiong]: As I have mentioned in another thread, indeed, special
configuration should be applied to avoid potential non-continuous IPv4
prefix impacting on IPv6 routing.

Best regards

Qiong Sun