Subject: [Softwires] review of draft-operators-softwire-stateless-4v6-motivation-02.txt

From: Jari Arkko <[email-masked]>
Date: Sat, 23 Jul 2011 10:36:14 -0400

I have reviewed this draft in preparation for discussions in the 
upcoming meeting.

Thanks for writing this. In general, I agree with many of the points 
raised in the document. I do have a number of issues with the document 
as well, however. In some cases listed below, the document appears to be 
claiming a little bit too broad benefit where it should have argued for 
a specific difference between stateful and stateless solutions. 
Secondly, and more, seriously, I wish the document would contain a 
description of the actual trade-offs, including some of the issues that 
seem apparent in the stateless solution. To name some:

- difficulty in supporting widely different numbers of ports that 
different users employ ("cookie-cutter")
- operational impacts of having to make changes to port allocations for 
some users (does it change my addressing plan?)
- port randomization

Note: I'm not saying that these things are showstoppers for the the 
deployment of stateless solutions, but I think the document would come 
across are more balanced if it actually did consider both the negative 
and positive aspects.

Finally, I think the document would benefit from describing differences 
between stateful solutions with dynamic mappings (traditional NAT44-like 
CGNs), network-based solutions with a static mappings, and approaches 
that are based on static/stateless mappings at the protocol level. 
Understanding the difference between these three I think is key to 
making a decision on moving ahead with stateless solutions work.

Technical:

> Service providers need to consider
> optimizing the form of packet processing when encapsulation is used.
> Since existing header compression techniques are stateful, it is
> expected that stateless solution minimize overhead introduced by the
> solution.
>   

I'm not sure I understand this. First off, both stateless and stateful 
solutions appear to be sending relatively static data from CPEs to the 
central network equipment (same source addresses etc). Is you argument 
that the stateless approach is even more static in this sense, and that 
this would be beneficial from the point of commonly used header 
compression tools? Can you be more specific about the exact differences 
between the two approaches in this case?

FWIW, my experience is that fixed wireless solutions -- such as those 
involving a home access point communicating over 3G/LTE towards an 
operator network -- do not today use header compression as much as they 
theoretically could. I think we'll see more header compression 
deployment in coming years, and that would also provide an opportunity 
to push the right compression technology.

> 3.1.4. No Additional Protocol for Port Control is Required
>
> The deployment of stateless solution does not require the deployment
> of new dynamic signaling protocols to the end-user CPE in addition to
> those already used.  In particular, existing protocols (e.g., UPnP
> IGD:2 [UPnP-IGD]) can be used to control the NAT mapping in the CPE.

I think this point, while valid, needs some reformulation. As I can see:

* A new protocol will be required between CPE and network equipment. The 
question is if this protocol needs both a data path and control path 
component or not. And its not clear to me how provisioning works, 
somehow the CPE needs to learn what port ranges to use. So _some_ amount 
of control protocols are needed.

* Stateless/full aspect does not seem to matter with regards to whether 
you can use existing protocols to talk to the closest NAT, whether its 
at the CPE or in the network. Where you need new protocols and new code 
is if (a) there are multiple layers of NATs, and e.g., the home NAT 
needs to talk to the service provider NAT or (b) security and resource 
model of the NAT changes from "I have all ports" to "I can only give you 
some ports".

> 3.2.1. Preserve Current Practices
>
>
> Service Providers require as much as possible to preserve the same
> operations as for current IP networking environments.
>
> If stateless solutions are deployed, common practices are preserved.
> In particular, the maintenance and operation of the network do not
> require any additional constraints such as: path optimization
> practices, enforcing traffic engineering policies, issues related to
> traffic oscillation between stateful devices, load-balancing the
> traffic or load sharing the traffic among egress/ingress points can
> be used, etc.  In particular:
>
> o  anycast-based schemes can be used for load-balancing and
>      redundancy purposes.
>
> o  asymmetric routing to/from the IPv4 Internet is natively supported
>     and no path-pinning mechanisms have to be additionally
>     implemented.

I think the issues at the bottom are valid requirements. However, the 
general point seems a bit overstated, IMO. The network is changing. Both 
with stateless and stateful solutions. I would suggest reformulating 
this point to be more about the specific issues than the general point.

> 3.2.5. Simplification of Qualification Procedures

IMHO, this point is perhaps a little bit overstated as well. The general 
argument is that stateless mechanisms are simpler and therefore easier 
to test. I think the reality is that any solution involving major 
fraction of traffic from the ISP and involving significant multi-vendor 
interoperability requirements (CPE - network node) is something that I 
would like to test very, very carefully. And remember that this is not 
just TCP port forwarding to static port ranges per subscriber. Its UDP, 
too. And ICMP. And some provisioning mechanism so that CPEs learn what 
ranges they can use.

> 3.3.2. No Organizational Impact
>
> Stateless solutions rely on IP-related techniques to share and to
> deliver IPv4 packets over an IPv6 network.  In particular, IPv4
> packets are delivered without any modification to their destination
> CPE.  As such there is a clear separation between the IP/transport
> layers and the service layers; no service interference is to be
> observed when a stateless solution is deployed.  This clear
> separation:
>
> Facilitates service evolution:  Since the payload of IPv4 packets is
> not altered in the path, services can evolve without requiring any
> specific function in the Service Provider's network;
>
> Limits vendor dependency:  The upgrade of value-added services does
> not involve any particular action from vendors that provide
> devices embedding the stateless IPv4/IPv6 interconnection
> function.
>
> No service-related skills are required for network operators who
> manage devices that embed the IPv4/IPv6 interconnection function:  IP
> teams can be in charge of these devices; there is a priori no need
> to create a dedicated team to manage and to operate devices
> embedding the stateless IPv4/IPv6 interconnection function.  The
> introduction of stateless capabilities in the network are unlikely
> to degrade management costs.

I think the point about less involvement between transport and service 
functions is a valid one. That being said, I think the introduction of 
either stateless or stateful solutions does have an organizational 
impact. For instance, now the capabilities and configuration of your CPE 
equipment are very relevant with respect to what you can deploy in the 
network side.

I would suggest focusing on the specific point and not arguing the 
general point.

Editorial:

> most sensitive problems 

... most pressing? sensitive doesn't seem like the right word.

> stateless solutions optimizes CPE-to-CPE
>   

... optimize ...

Jari

---

From: Rémi Després <[email-masked]>
Date: Sat, 23 Jul 2011 19:27:33 +0200

Hi, Jari,

Good to have substantial comments on this milestone document.

Le 23 juil. 2011 à 16:36, Jari Arkko a écrit :

> I have reviewed this draft in preparation for discussions in the upcoming meeting.
> 
> Thanks for writing this. In general, I agree with many of the points raised in the document. I do have a number of issues with the document as well, however. In some cases listed below, the document appears to be claiming a little bit too broad benefit where it should have argued for a specific difference between stateful and stateless solutions. Secondly, and more, seriously, I wish the document would contain a description of the actual trade-offs,

> including some of the issues that seem apparent in the stateless solution.

What "seems apparent" would be easier to discuss if we would know which stateless solutions we talk about .
Making progress on these specifications is key, IMHO, to avoid unverifiable comments on unspecified designs.

> To name some:
> 
> - difficulty in supporting widely different numbers of ports that different users employ ("cookie-cutter")

If different customers need different service levels in IPv4, a simple approach is to also give them different service levels in IPv6.
With the 4rd mapping rules, an IPv6 prefix shorter by 1 bit can give twice as many IPv4 ports.
This can be done locally in provider-edge routers.
It would be useful, IMHO, to make it clear in the specification of 4rd mapping rules.

> - operational impacts of having to make changes to port allocations for some users (does it change my addressing plan?)

As just explained, this can be done by a change of IPv6-prefix length. 

> - port randomization

I agree that sec. 5.3 could be somewhat expanded, but it does discuss the issue.
Points worth to add, IMHO, include:
- With an provider DNS server that is IPv6 accessible, port randomization isn't restricted.
- The increasing risk of port-guessing attacks in IPv4, as IPv4 addresses have more and more to be shared, is a good reason activate dual-stack operation as generally and as quickly and as possible.

> Note: I'm not saying that these things are showstoppers for the the deployment of stateless solutions, but I think the document would come across are more balanced if it actually did consider both the negative and positive aspects.

1. The document does mention negative aspects.
2. The motivation document for stateless solutions isn't the right place to list all motivations for stateful solutions. 
A separate document could be written for that, in particular concerning the need for PCP (not obvious IMHO). 

> Finally, I think the document would benefit from describing differences between stateful solutions with dynamic mappings (traditional NAT44-like CGNs), network-based solutions with a static mappings, and approaches that are based on static/stateless mappings at the protocol level. Understanding the difference between these three I think is key to making a decision on moving ahead with stateless solutions work.

A tutorial on stateful solutions would be nice to have, I agree, but these solutions are described in other documents (DS-lite, PCP...)

This should in no way delay real work on proposed stateless solutions.

There are subjects that lack volunteers to contribute but this isn't the case for stateless solutions. They have volunteers from very significant actors of the Internet.
Delaying their work would be counterproductive for the final decision to be made.    
   
> I'm not sure I understand this. First off, both stateless and stateful solutions appear to be sending relatively static data from CPEs to the central network equipment (same source addresses etc). Is you argument that the stateless approach is even more static in this sense, and that this would be beneficial from the point of commonly used header compression tools? Can you be more specific about the exact differences between the two approaches in this case?
> 
> FWIW, my experience is that fixed wireless solutions -- such as those involving a home access point communicating over 3G/LTE towards an operator network -- do not today use header compression as much as they theoretically could.

I share this understanding.
A possible reason is that the benefit is insufficient today in fixed wireless solutions.

> I think we'll see more header compression deployment in coming years, and that would also provide an opportunity to push the right compression technology.

Same view.

Now, ignoring header compression for a while, there isn't much difference of transmission efficiency between translation-based and encapsulation based IPv4/IPv6 solutions.
I plan to have a separate comment on this point of Wojciech's comparison of 4V6T and 4V6E. 

> * A new protocol will be required between CPE and network equipment. The question is if this protocol needs both a data path and control path component or not. And its not clear to me how provisioning works, somehow the CPE needs to learn what port ranges to use. So _some_ amount of control protocols are needed.

With the 4rd address mapping (common to 4V6T and 4V6E), the CPE needs only parameters that can be statelessly delivered in DHCPv6 or other parameter provisioning protocols.
No new protocol is therefore needed.
 
> * Stateless/full aspect does not seem to matter with regards to whether you can use existing protocols to talk to the closest NAT, whether its at the CPE or in the network. Where you need new protocols and new code is if (a) there are multiple layers of NATs, and e.g., the home NAT needs to talk to the service provider NAT or (b) security and resource model of the NAT changes from "I have all ports" to "I can only give you some ports".

Unless I miss something, 
DS-lite specialists can answer why PCP (new protocol) is needed for port reservation in CGN's, but the point remains that with statelessly shared addresses no new protocol is needed.

> I think the issues at the bottom are valid requirements. However, the general point seems a bit overstated, IMO. The network is changing. Both with stateless and stateful solutions. I would suggest reformulating this point to be more about the specific issues than the general point.

Experience with 6rd gives an idea of how much stateless solutions can be non-intrusive in operation practices.
More specific answers will better be given when stateless solutions are better specified, IMHO.

> IMHO, this point is perhaps a little bit overstated as well. The general argument is that stateless mechanisms are simpler and therefore easier to test. I think the reality is that any solution involving major fraction of traffic from the ISP and involving significant multi-vendor interoperability requirements (CPE - network node) is something that I would like to test very, very carefully. And remember that this is not just TCP port forwarding to static port ranges per subscriber. Its UDP, too. And ICMP.

That is orthogonality of functions that facilitates qualification.
Having on ALG concern of any kind is a great simplification.
ICMP needs a specific treatment, but the solution of draft-murakami-softwire-4rd sec.9 is in my understanding simple to implement and qualify.

> And some provisioning mechanism so that CPEs learn what ranges they can use.

See above (same parameter provisioning protocol as for other parameters).

Best regards,
RD

> I think the point about less involvement between transport and service functions is a valid one. That being said, I think the introduction of either stateless or stateful solutions does have an organizational impact. For instance, now the capabilities and configuration of your CPE equipment are very relevant with respect to what you can deploy in the network side.
> 
> I would suggest focusing on the specific point and not arguing the general point.
> 
> Editorial:
> 
>> most sensitive problems 
> 
> _______________________________________________
> Softwires mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/softwires

---

From: Jari Arkko <[email-masked]>
Date: Sat, 23 Jul 2011 13:53:23 -0400

Rémi,

> Making progress on these specifications is key, IMHO, to avoid unverifiable comments on unspecified designs.

That's a fair comment. I was thinking in terms of a hypothetical 
solution that I had in my mind, but you are right that there may be 
differences between different approach. In the last IETF I was worried 
about making ICMP to the outside work without adding state, and someone 
helped me understand that it can be made to work. There may be similar 
eye-opening experiences to come.

Jari

---

From: Satoru Matsushima <[email-masked]>
Date: Tue, 26 Jul 2011 11:19:08 -0400

Hi Jari, 

Thank you for your comments. Please see inline:

On 2011/07/23, at 10:36, Jari Arkko wrote:

> Thanks for writing this. In general, I agree with many of the points raised in the document. I do have a number of issues with the document as well, however. In some cases listed below, the document appears to be claiming a little bit too broad benefit where it should have argued for a specific difference between stateful and stateless solutions. Secondly, and more, seriously, I wish the document would contain a description of the actual trade-offs, including some of the issues that seem apparent in the stateless solution.

Thanks, actually we had some trade-off explanation in previous version. But due to focus more on the motivation, current version doesn't mention the trade-off. Since the purpose of this document is to describe motivations, and kick-off to develop stateless solution, the authors focus to motivation otherwise analysis work. I agree that trade-off analysis is significant, but it would be better to be another document outside of which the motivation draft bears.

> Note: I'm not saying that these things are showstoppers for the the deployment of stateless solutions, but I think the document would come across are more balanced if it actually did consider both…

My thoughts for these question:

(1) stateless solution is not exclusive with other solution in theory. CPE can use statefull solution for such purpose.
(2) 4rd as for example, multiple mapping rule can be helpful to change port allocation without address plan changing and renumbering.
(3) Port randomization still preserve host side. Restrict ports should be in CPE. Appropriate NAT session filtering is helpful to avoid the issues of port scan threats. (i.e., address and address-and-port dependent filtering)

> Finally, I think the document would benefit from describing differences between stateful solutions with dynamic mappings (traditional NAT44-like CGNs), network-based solutions with a static mapping…

As Remi pointed out, we couldn't assume any stateless solution in the motivation draft. So again, I think that it would be better to be another document to do it if we needed.

Then, let me try to make technical discussion:

> FWIW, my experience is that fixed wireless solutions -- such as those involving a home access point communicating over 3G/LTE towards an operator network -- do not today use header compression as much as they theoretically could. I think we'll see more header compression deployment in coming years, and that would also provide an opportunity to push the right compression technology.
> 

Yes, it is not header compression technology itself. It is operational effort to be by choosing translation method in tunneling network architecture, which I meant that within same address-family communication.

> * A new protocol will be required between CPE and network equipment. The question is if this protocol needs both a data path and control path component or not. And its not clear to me how provision…

Thanks, we'll clarify it, and improve the text.

> * Stateless/full aspect does not seem to matter with regards to whether you can use existing protocols to talk to the closest NAT, whether its at the CPE or in the network. Where you need new proto…

Agreed. But point (b), current NAT implementation already can control to port range/number to force its port pool. As long as we utilize it, I don't think we need new code to do (b).

> I think the issues at the bottom are valid requirements. However, the general point seems a bit overstated, IMO. The network is changing. Both with stateless and stateful solutions. I would suggest…

I think we can clarify it. But I don't understand your point clearly. Can you give some example about 'network changing'.

> IMHO, this point is perhaps a little bit overstated as well. The general argument is that stateless mechanisms are simpler and therefore easier to test. I think the reality is that any solution involving major fraction of traffic from the ISP and involving significant multi-vendor interoperability requirements (CPE - network node) is something that I would like to test very, very carefully. And remember that this is not just TCP port forwarding to static port ranges per subscriber. Its UDP, too. And ICMP. And some provisioning mechanism so that CPEs learn what ranges they can use.

Thanks, we'll revisit about it. In general, we usually try to keep current procedure when we change something. From my tiny experience of deployment stateless solution into any area, qualification procedure of these are lightweight than statefull.

> I would suggest focusing on the specific point and not arguing the general point.

Thanks, we'll revisit it.

>> stateless solutions optimizes CPE-to-CPE

Thanks,

cheers,
--satoru

---

From: Cameron Byrne <[email-masked]>
Date: Tue, 26 Jul 2011 08:50:50 -0700

> Yes, it is not header compression technology itself. It is operational effort to be by choosing translation method in tunneling network architecture, which I meant that within same address-family c…

Since i have seen 3G/LTE on a few of these related threads, i will
state that it is unlikely that mobile wireless providers and the
3GPP/GSMA will adopt yet another tunneling protocol or user equipment
modification.

cb

---

From: "Rajiv Asati (rajiva)" <[email-masked]>
Date: Tue, 26 Jul 2011 15:50:46 -0500

Cameron,

> 3GPP/GSMA will adopt yet another tunneling protocol or user equipment

Thanks for sharing your opinion, as most of us may not be caring about the 3GPP environment while thinking about various transition options.

Cheers,
Rajiv

> -----Original Message-----
> From: [email-masked] [mailto:[email-masked]] On Behalf
> Of Cameron Byrne
> Sent: Tuesday, July 26, 2011 11:51 AM
> To: Satoru Matsushima
> Cc: [email-masked]
> Subject: Re: [Softwires] review ofdraft-operators-softwire-stateless-4v6-
> motivation-02.txt
> 
> >> Thanks for writing this. In general, I agree with many of the points raised
> in the document. I do have a number of issues with the document as well,
> however. In some cases listed below, the document appears to be claiming a
> little bit too broad benefit where it should have argued for a specific
> difference between stateful and stateless solutions. Secondly, and more,
> seriously, I wish the document would contain a description of the actual
> trade-offs, including some of the issues that seem apparent in the stateless
> solution.
> >
> > Thanks, actually we had some trade-off explanation in previous version. But
> due to focus more on the motivation, current version doesn't mention the
> trade-off. Since the purpose of this document is to describe motivations, and
> kick-off to develop stateless solution, the authors focus to motivation
> otherwise analysis work. I agree that trade-off analysis is significant, but
> it would be better to be another document outside of which the motivation
> draft bears.
> >
> >
> >> To name some:
> >>
> deployment of stateless solutions, but I think the document would come across
> are more balanced if it actually did consider both the negative and positive
> aspects.
> >
> > (1) stateless solution is not exclusive with other solution in theory. CPE
> can use statefull solution for such purpose.
> > (2) 4rd as for example, multiple mapping rule can be helpful to change port
> allocation without address plan changing and renumbering.
> > (3) Port randomization still preserve host side. Restrict ports should be in
> CPE. Appropriate NAT session filtering is helpful to avoid the issues of port
> scan threats. (i.e., address and address-and-port dependent filtering)
> >
> >
> >>
> CGNs), network-based solutions with a static mappings, and approaches that are
> based on static/stateless mappings at the protocol level. Understanding the
> difference between these three I think is key to making a decision on moving
> ahead with stateless solutions work.
> >
> > As Remi pointed out, we couldn't assume any stateless solution in the
> motivation draft. So again, I think that it would be better to be another
> document to do it if we needed.
> >
> solutions appear to be sending relatively static data from CPEs to the central
> network equipment (same source addresses etc). Is you argument that the
> stateless approach is even more static in this sense, and that this would be
> beneficial from the point of commonly used header compression tools? Can you
> be more specific about the exact differences between the two approaches in
> this case?
> >>
> involving a home access point communicating over 3G/LTE towards an operator
> network -- do not today use header compression as much as they theoretically
> could. I think we'll see more header compression deployment in coming years,
> and that would also provide an opportunity to push the right compression
> technology.
> >>
> >
> > Yes, it is not header compression technology itself. It is operational
> effort to be by choosing translation method in tunneling network architecture,
> which I meant that within same address-family communication.
> >
> >
> 
> question is if this protocol needs both a data path and control path component
> or not. And its not clear to me how provisioning works, somehow the CPE needs
> to learn what port ranges to use. So _some_ amount of control protocols are
> needed.
> >
> >> * Stateless/full aspect does not seem to matter with regards to whether you
> can use existing protocols to talk to the closest NAT, whether its at the CPE
> or in the network. Where you need new protocols and new code is if (a) there
> are multiple layers of NATs, and e.g., the home NAT needs to talk to the
> service provider NAT or (b) security and resource model of the NAT changes
> from "I have all ports" to "I can only give you some ports".
> >
> > Agreed. But point (b), current NAT implementation already can control to
> port range/number to force its port pool. As long as we utilize it, I don't
> think we need new code to do (b).
> >
> >
> >>
> general point seems a bit overstated, IMO. The network is changing. Both with
> stateless and stateful solutions. I would suggest reformulating this point to
> be more about the specific issues than the general point.
> >
> > I think we can clarify it. But I don't understand your point clearly. Can
> you give some example about 'network changing'.
> >
> >
> >>
> argument is that stateless mechanisms are simpler and therefore easier to
> test. I think the reality is that any solution involving major fraction of
> traffic from the ISP and involving significant multi-vendor interoperability
> requirements (CPE - network node) is something that I would like to test very,
> very carefully. And remember that this is not just TCP port forwarding to
> static port ranges per subscriber. Its UDP, too. And ICMP. And some
> provisioning mechanism so that CPEs learn what ranges they can use.
> >
> > Thanks, we'll revisit about it. In general, we usually try to keep current
> procedure when we change something. From my tiny experience of deployment
> stateless solution into any area, qualification procedure of these are
> lightweight than statefull.
> >
> >
> >>
> functions is a valid one. That being said, I think the introduction of either
> stateless or stateful solutions does have an organizational impact. For
> instance, now the capabilities and configuration of your CPE equipment are
> very relevant with respect to what you can deploy in the network side.
> >>

---

From: "Lee, Yiu" <[email-masked]>
Date: Wed, 27 Jul 2011 01:40:23 +0000

I want to eacho Satoru's comment that this draft wasn't intended to
compare stateless and stateful transition mechanisms. This draft only
describes the motivations.

On 7/26/11 11:19 AM, "Satoru Matsushima" <[email-masked]>
wrote:

>Thanks, actually we had some trade-off explanation in previous version.
>But due to focus more on the motivation, current version doesn't mention
>the trade-off. Since the purpose of this document is to describe
>motivations, and kick-off to develop stateless solution, the authors
>focus to motivation otherwise analysis work. I agree that trade-off
>analysis is significant, but it would be better to be another document
>outside of which the motivation draft bears.

---

From: Jari Arkko <[email-masked]>
Date: Wed, 27 Jul 2011 08:08:59 -0400

Rémi,
>> describes the motivations.

And I don't want a full blown comparison either, but if the argument is 
that we need to do this because is better/cheaper/more efficient than 
something else, I think we do need to explain the main differences to 
that something else (positive and negative).

Jari

P.S. I hope you'll have a productive discussion about this today, but 
unfortunately I'm presenting in another working group so I can't be at 
the SOFTWIRE meeting.

---

From: Rémi Després <[email-masked]>
Date: Fri, 29 Jul 2011 14:08:04 +0200

Le 27 juil. 2011 à 14:08, Jari Arkko a écrit :

> And I don't want a full blown comparison either, but if the argument is that we need to do this because is better/cheaper/more efficient than something else, I think we do need to explain the main differences to that something else (positive and negative).

Fair enough.
Actually I sometimes confused what was in version -O2 and what was in version -00, more detailed on several points.  

Kind regards,
RD