Subject: [alto] Comments on provisioned bandwidth and ALTO

From: "Woundy, Richard" <[email-masked]>
Date: Thu, 18 Mar 2010 02:42:20 -0400

Folks,

I had provided some offline comments about the inclusion of provisioned bandwidth in ALTO. Enrico asked me to re-send them to the mailing list. I can also discuss in next week's ALTO session.

Here are my privacy concerns.

1. Depending on the ISP's pricing and rollout of bandwidth tiers, there may be relatively few subscribers within a particular tier. Therefore a third party consuming ALTO provisioned bandwidth information can make a good guess about the identity of a subscriber within a "rarely used" bandwidth tier.

2. Separately, a third party consuming ALTO provisioned bandwidth information may be able to make an informed guess about the economic status of a subscriber based on the bandwidth tier, which may not be desirable to the subscriber.

3. The subscriber may not intend to use *all* provisioned bandwidth for a particular application (e.g. P2P). For example, perhaps the subscriber intends to use provisioned uplink bandwidth for telecommuting, telepresence, online storage backups, etc. A third party consuming ALTO provisioned bandwidth information should be aware that the subscriber's provisioned bandwidth may be reserved for different applications.

Here are my thoughts on dynamic address re-allocation.

ISPs reallocate IPv4 subnets within their infrastructure from time to time, partly to ensure the efficient usage of IPv4 addresses (a scarce resource), and partly to enable efficient route tables within their network routers. The frequency of these "renumbering events" depend on the growth in number of subscribers and the availability of address space within the ISP. As a result, a subscriber's household device could retain an IPv4 address for as short as a few minutes, or for months at a time or even longer.

Some folks have suggested that ISPs providing ALTO services could sub-divide their subscribers' devices into different IPv4 subnets (or certain IPv4 address ranges) based on the purchased service tier, as well as based on the location in the network topology. The problem is that this sub-allocation of IPv4 subnets tends to decrease the efficiency of IPv4 address allocation. A growing ISP that needs to maintain high efficiency of IPv4 address utilization may be reluctant to jeopardize their future acquisition of IPv4 address space.

Therefore, consumers of per-user ALTO information should assume that subscribers retain IPv4 addresses for only a relatively short period of time, e.g. minutes, and that subscribers of different service tiers will co-exist in some ISP's IPv4 subnets.

-- Rich

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of alto issue tracker
Sent: Wednesday, February 17, 2010 5:14 AM
To: [email-masked]; [email-masked]
Cc: [email-masked]
Subject: [alto] #4: Provisioned bandwidth

#4: Provisioned bandwidth
---------------------------------------------+------------------------------
 Reporter:  enrico.marocco@…                 |       Owner:  Sebastian Kiesel <ietf-alto@…>         
     Type:  enhancement                      |      Status:  new                                    
 Priority:  major                            |   Milestone:                                         
Component:  reqs                             |     Version:                                         
 Severity:  -                                |    Keywords:                                         
---------------------------------------------+------------------------------
 The document should track (and ideally stimulate discussion to reach
 consensus) the arguments about providing information about provisioned
 bandwidth, as it may have non trivial impact on the protocol design. The
 topic was discussed in [wiki:Ietf76 Hiroshima] and previously on the
 mailing list:
 [http://www.ietf.org/mail-archive/web/alto/current/msg00322.html]
 [http://www.ietf.org/mail-archive/web/alto/current/msg00476.html]

-- 
Ticket URL: <https://svn.tools.ietf.org/wg/alto/trac/ticket/4>
alto <http://tools.ietf.org/alto/>

_______________________________________________
alto mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/alto

---

From: Salman Abdul Baset <[email-masked]>
Date: Thu, 18 Mar 2010 03:18:11 -0400 (EDT)

As you point out, allowing third party to query ALTO servers to determine 
the provisioned bandwidth of *any user* has privacy implications.

However, the same privacy implications do not apply if a user wants to 
query the provisioned bandwidth of the broadband connection it purchased.
There could be access limitations, however. For example, a user 
that is connected to Starbucks WiFi is likely to be limited from querying 
the connection bandwidth.

Also, an application may want to query its gateway or cable modem to 
determine the current load at the modem. Such a mechanism does not require 
querying any ISP managed ALTO server.

It will be helpful to clarify these two issues.

Salman

---

From: "Martin Stiemerling" <[email-masked]>
Date: Thu, 18 Mar 2010 11:01:45 +0100

Hi Rich, 

> From: [email-masked] [mailto:[email-masked]] On Behalf Of
> Woundy, Richard
> Sent: Thursday, March 18, 2010 7:42 AM
> To: [email-masked]
> Subject: [alto] Comments on provisioned bandwidth and ALTO
> 
> Folks,
> 
> I had provided some offline comments about the inclusion of provisioned
> bandwidth in ALTO. Enrico asked me to re-send them to the mailing list.
> I can also discuss in next week's ALTO session.
> 
> 1. Depending on the ISP's pricing and rollout of bandwidth tiers, there
> may be relatively few subscribers within a particular tier. Therefore a
> third party consuming ALTO provisioned bandwidth information can make a
> good guess about the identity of a subscriber within a "rarely used"
> bandwidth tier.
> 
> 2. Separately, a third party consuming ALTO provisioned bandwidth
> information may be able to make an informed guess about the economic
> status of a subscriber based on the bandwidth tier, which may not be
> desirable to the subscriber.

Why should I wait for ALTO if I can do this with reasonable effort by today with today's means?

You can use the geo information provided for a particular IP address out of the whois and combine this with some other available data (google maps/street view) to see if people are living in wealthy area or not. Try this with the IP addresses indicated for this email as smtp sender...( 195.37.70.41). Unfortunately, there is no street view for this location, but in other places this will help even better to determine if the residents are wealthy or not. ;)

http://maps.google.com/maps?q=49.405839,8.684582&num=1&t=h&sll=49.409988,8.699925&sspn=0.006295,0.006295&ie=UTF8&ll=49.406129,8.685808&spn=0.006828,0.012746&z=16&iwloc=A

You may also get much better data if you get access to some consumer rating agency's database. At least in Germany they can tell in most areas house by house the income to be expected. Used by direct marketing...

I see the point you are making, but I would opt for including the possibility to query to provisioned bandwidth in the ALTO protocol, but leave it to the discretion of the ISP deploying the service to enable this or not. Including documenting the privacy risks if enabled. 

My use case for this:
The provisioned bandwidth may be used by p2p streaming systems to rule out peers that will anyway not help in streaming (e.g., an peer with 128kbit/s uplink intended to serve full HDTV streams).

> 
> 3. The subscriber may not intend to use *all* provisioned bandwidth for
> a particular application (e.g. P2P). For example, perhaps the
> subscriber intends to use provisioned uplink bandwidth for
> telecommuting, telepresence, online storage backups, etc. A third party
> consuming ALTO provisioned bandwidth information should be aware that
> the subscriber's provisioned bandwidth may be reserved for different
> applications.

True, this should be documented as an issue, but as said above this is helpful in ruling out peers anyway not useful. Second, the actual usable bandwidth needs to be tested in the following operational steps of the p2p system anyway. 

> ISPs reallocate IPv4 subnets within their infrastructure from time to
> time, partly to ensure the efficient usage of IPv4 addresses (a scarce
> resource), and partly to enable efficient route tables within their
> network routers. The frequency of these "renumbering events" depend on
> the growth in number of subscribers and the availability of address
> space within the ISP. As a result, a subscriber's household device
> could retain an IPv4 address for as short as a few minutes, or for
> months at a time or even longer.
> 
> Some folks have suggested that ISPs providing ALTO services could sub-
> divide their subscribers' devices into different IPv4 subnets (or
> certain IPv4 address ranges) based on the purchased service tier, as
> well as based on the location in the network topology. The problem is
> that this sub-allocation of IPv4 subnets tends to decrease the
> efficiency of IPv4 address allocation. A growing ISP that needs to
> maintain high efficiency of IPv4 address utilization may be reluctant
> to jeopardize their future acquisition of IPv4 address space.
> 
> Therefore, consumers of per-user ALTO information should assume that
> subscribers retain IPv4 addresses for only a relatively short period of
> time, e.g. minutes, and that subscribers of different service tiers
> will co-exist in some ISP's IPv4 subnets.

I will reply to this in a separate email.

Thanks,

  Martin

[email-masked]

NEC Laboratories Europe - Network Research Division
NEC Europe Limited | Registered Office: NEC House, 1 Victoria Road, London W3 6BL | Registered in England 2832014

---

From: "Martin Stiemerling" <[email-masked]>
Date: Thu, 18 Mar 2010 12:17:34 +0100

Rich,

Here I go with my comments on the dynamic IP address re-allocation.

I commented on this during the IETF-76 on (taken from the minutes)
Martin: how static is a network map in reality?
and also by email:
http://www.ietf.org/mail-archive/web/alto/current/msg00519.html

My point was that the proposed network map/cost map in the ALTO protocol would be less beneficially, if the ISP are exactly doing what you have described and what I have commented on during the last IETF. 

However, I've had the impression that most people in the room didn't agree to this view:

The network maps (and the associated cost map) is fine as long as the IP prefix allocation in an ISP's network is stable. Where stable means changes of the IP prefix allocation are happening in the range of weeks. 

Now the caveats of network maps:
It is up to the ISP to change the IP prefix allocation in its own network at anytime. Including using some dynamic re-allocation schemes, for instance, supported by using Cisco's ODAP feature, but also shuffling them manually. If now, a dynamic reallocation scheme is used (and I guess some ISP will need to do this soon, as there not too many free ISP prefix blocks left), the network maps have to be updated in much shorter time frame than actually envisioned by the ALTO protocol.

This takes away the advantage of having network maps and would clearly argue for enhanced oracle approach.

Coming now to provisioned bandwidth and dynamic IP address re-allocation:
I don't see a particular reason why the information gained from provisioned bandwidth is not subject to the same rules as topology information delivered for a peer's IP address.

Both are subject to change and have more less only a good meaning at the time of the query. With time evolving, the information where a peer is topology wise close or if the peer has a particular provisioned bandwidth will anyhow change (e.g., home gateway or cable modem is rebooting, causing to get a new IP address assigned and the old one being re-assigned to some other customer access line).

Thanks,

  Martin

---

From: Reinaldo Penno <[email-masked]>
Date: Thu, 18 Mar 2010 10:39:38 -0400

Hello,

On 3/18/10 4:17 AM, "Martin Stiemerling" <[email-masked]>
wrote:

> My point was that the proposed network map/cost map in the ALTO protocol would
> be less beneficially, if the ISP are exactly doing what you have described and
> what I have commented on during the last IETF.
> 
> The network maps (and the associated cost map) is fine as long as the IP
> prefix allocation in an ISP's network is stable. Where stable means changes of
> the IP prefix allocation are happening in the range of weeks.
> 
> It is up to the ISP to change the IP prefix allocation in its own network at
> anytime. Including using some dynamic re-allocation schemes, for instance,
> supported by using Cisco's ODAP feature, but also shuffling them manually. If
> now, a dynamic reallocation scheme is used (and I guess some ISP will need to
> do this soon, as there not too many free ISP prefix blocks left), the network
> maps have to be updated in much shorter time frame than actually envisioned by
> the ALTO protocol.

There is already plenty of data suggesting what ISPs will do when they run
out of IPv4 addresses and I do not think what you suggest will happen in any
significant way. ISPs will use 10/8 for their internal network (some already
do) in combination with NAT44/VRFs, or will use IPv6 and perform NAT64. The
internal network addressing will be fairly stable.

As far as what the external routing domain will see, this is something that
might be interesting point. Sharing of IP addresses in a CGN will need
special considerations within the ALTO protocol.

> for enhanced oracle approach.

I do not agree. But I would point out that in a NATed Internet the location
of the ALTO server handling the requests in relation to the NAT boxes will
play an important role. You cannot mix address family (as in private vs.
public vs. IPv6) in a query and hope for it to work.

> I don't see a particular reason why the information gained from provisioned
> bandwidth is not subject to the same rules as topology information delivered
> for a peer's IP address.

Sure, provisioned bandwidth could be included as an attribute of the
endpoint service (or old ranking service) . I do not think the protocol puts
any limitation on it today.

> 
> Both are subject to change and have more less only a good meaning at the time
> of the query. With time evolving, the information where a peer is topology
> wise close or if the peer has a particular provisioned bandwidth will anyhow
> change (e.g., home gateway or cable modem is rebooting, causing to get a new
> IP address assigned and the old one being re-assigned to some other customer
> access line).
> 
> Thanks,
> 
>   Martin
> 
>>

---

From: Sebastian Kiesel <[email-masked]>
Date: Thu, 18 Mar 2010 22:42:10 +0100

Hi,

a technical concern regarding the provisioned bandwidth attribute:

If a P2P app uses this information for a peer selection strategy like
"try the highest bandwidth tier peers first" or "try the highest
bandwidth tier peers with higher probability", and if there is a swarm
with many peers but very few of them are in the highest bandwidth tier,
the user experience of these premium customers might be worse than for a
customer with lower provisioned bandwidth (even if the P2P app uses flow
control and congestion control, the initial syn packets alone could
overwhelm the premium customer's links).

So we should be very clear what we want to do with this kind of
information in the application.

Thinking about this for a while, this problem is not limited to the
provisioned bandwidth.  Maybe we need to distinguish between rating
criteria that are commutative (i.e., peer B would be a "good" neighbor
for peer A according to that criterion implies that A would be a "good"
neighbor for B) and those criteria that are not.  Using non-commutative
rating criteria might be risky.

-- Sebastian

---

From: "Woundy, Richard" <[email-masked]>
Date: Thu, 18 Mar 2010 17:48:04 -0400

I agree.

Here's a slightly different example. The common tier for ISP A may have
provisioned bandwidth of 12 Mbps, and 10 Mbps for ISP B. Assuming that
an application doesn't need 11 Mbps, it may make more sense for the app
to pick an ISP A client 12 times out of 22 (55%), and an ISP B client 10
times out of 22 (45%), rather than an ISP A client 100% of the time.

-- Rich

-----Original Message-----
From: Sebastian Kiesel [mailto:[email-masked]] 
Sent: Thursday, March 18, 2010 5:42 PM
To: Woundy, Richard
Cc: [email-masked]
Subject: Re: [alto] Comments on provisioned bandwidth and ALTO

Hi,

-- Sebastian

---

From: Richard Alimi <[email-masked]>
Date: Thu, 18 Mar 2010 18:06:57 -0400

Hi Sebastian,

> information in the application.

I think this is a very good point.  My personal opinion is that it would be 
cleaner to avoid third-party inquiries for provisioned bandwidth. For example, 
this would permit a peer to locally make some estimation (perhaps its "spare" 
capacity), and advertise that instead.

That said, avoiding third-party inquiries also has a couple of drawbacks:

1) In order for a third-party to determine "provisioned" bandwidth, there must
   be an ALTO Client running at the target IP address.  Then, to distribute
   the provisioned bandwidth to another ALTO Client, it either must be
   delivered directly directly or through an intermediary using an
   application-specific protocol (e.g., a P2P tracker, peer exchange, or a
   DHT).

2) An ALTO Client can "lie" about its provisioned bandwidth. Actually, this
   could also be seen as an advantage, since it may not be "lying" for
   malicious reasons.  Instead, it may be performing reasonable "adjustments"
   based on other applications that are also sucking up some of the bandwidth.

   Using the ALTO redistribution mechanism (where the value is signed by the
   ALTO Server) would avoid the possibility of lying, but it also means that
   no such "adjustments" would be possible. Of course, an ALTO Client still
   has the option to not redistribute its own provisioned bandwidth value in
   the first place.

-- 
Richard Alimi
Department of Computer Science
Yale University

---

From: Sebastian Kiesel <[email-masked]>
Date: Thu, 18 Mar 2010 23:20:44 +0100

Rich,

On Thu, Mar 18, 2010 at 05:48:04PM -0400, Woundy, Richard wrote:
> times out of 22 (45%), 

what you describe here sounds very reasonable. But how would you
implement it as an algorithm that works for arbitrary BW distributions
in the swarm?

My first thought,

(Probability of picking peer X) := (Prov.BW of peer X) / (Average prov.BW) 

would exactly cause the problem I described, if there were very many
peers with a BW slightly below the average and very few ones with a BW
far above the average.

I'm not saying that the provisioned bandwidth attribute is useles,
but I think we must be very careful in designing algorithms that make
use of it.

-- Sebastian

---

From: Reinaldo Penno <[email-masked]>
Date: Thu, 18 Mar 2010 19:14:53 -0400

Trying (to get) the peers with the highest actual bandwidth is exactly what
P2P protocols do today. The difference is that they take some time to select
the better peers through probing, optimistic unchoke, etc.

Therefore I would think and agree that if bandwidth (specially provisioned
bandwidth as opposed to actual) is part of the decision process, it can
throw localization askew and create a security issue.

But this is for P2P swarms client based. For CDN and caches this seems to
have its uses but I would think that actual bandwidth (some kind of average)
vs. provisioned would be more useful.

Thanks,

Reinaldo