Subject: [Tsvwg] Re: Diffserv codepoint seems strange for Re-ECN <draft-briscoe-tsvwg-re-ecn-tcp-00>

From: Bob Briscoe <[email-masked]>
Date: Thu, 26 Jan 2006 16:49:55 +0000

Sally,

[BB] Thanks for your detailed and helpful comments.

[BB] I promised I would respond to your e-mail after xmas, but sorry it has 
taken so long. I'm afraid I'm the sort of person who prefers to go away and 
think about something and only writes to mailing lists (or indeed approach 
the IETF in the first place) when I think I've got something worth saying, 
which makes my output rather bursty.

[BB] Responses inline (except one, which was so long I started another thread)

At 21:02 15/12/2005, Sally Floyd wrote:
>Bob -
>
>[SF]My apologies for the long delay in responding to your email.  I
>have finally read draft-briscoe-tsvwg-re-ecn-tcp-00, and will answer
>your email as best I can.
>
> >[BB] Your assumption/question about confining the re-ECN semantics for the
> >ECT(0) & ECT(1) codepoints solely to certain DSCPs threw me - I've realised
> >why - it doesn't make much sense (at least to me).
>
>[SF]My disagreemenet at the WG meeting was about what I understood to
>be your delaying the move of RFC 3540, Robust ECN Signaling with
>Nonces, to Proposed Standard, and instead proposing the mechanism
>in draft-briscoe-tsvwg-re-ecn-tcp-00 for the default semantics of
>the ECT(0) and ECT(1) codepoints.

[BB] Understood.

[BB] In the last couple of days we've come up with a new re-ECN marking 
scheme that doesn't need to conflict with the ECN Nonce. But I want to 
manage expectations, as we are trying to break its security, so I can't say 
we definitely want to move to it. I will write it up ASAP.

[BB] If we assume for now this new scheme is OK, we can have an adult 
debate about whether we should use up IP header real-estate for the ECN 
Nonce, for re-ECN (perhaps) or for both (perhaps). Rather than just one or 
the other.

[BB] However, I'm still not convinced we should standardise the ECN Nonce 
into IP. See later...

> >[BB]The network /middleboxes currently don't read the nonce codepoints, 
> only
> >the receiver. Diffserv codepoints are for reading by the network.
>
>[SF]Yep, but the entire point of your semantics for ECT(0) and ECT(1) is
>for them to be used by droppers in the network, as I understand it.

[BB] I see what you meant now. If one use of the codepoints was going to be 
for the network  to understand (re-ECN) and the other for the receiver 
(Nonce), you thought the sender should negotiate their meaning with both 
the network (Diffserv) and the receiver (TCP options). I used the CR flag, 
not Diffserv.

[BB] Our logic was that re-ECN was designed to apply to all DSCPs---we 
didn't want the sender negotiating whether it would be policed or not. So 
we worked out a scheme on the assumption that the network layer 
functionality of the ECN Nonce was redundant. And we used the CR flag to 
tell the network re-ECN was being used, but only once feedback was 
established (because re-ECN wasn't really being used until then).

[BB] But this may all be moot now, if our new marking scheme works.

[BB] more...

> >[BB] So I
> >assume your ECN-alternate-semantics draft only applied to marking
> >semantics, not to the (current) nonce ECT codepoint semantics which are e2e.
>
>[SF]Yep, this is not about the ECN-alternate-semantics draft, this is
>about the delaying of moving RFC 3540 to Proposed Standard.
>
> >[BB]So, rather than a Diffserv codepoint, in the re-ECN draft I used the 
> TCP
> >handshake to negotiate re-ECN capability between sender and receiver. I
> >give a compatibility mode where the re-ECN sender can work with a nonce
> >receiver, or basic ECN for that matter. It's optional, so if an implementer
> >doesn't do the compatibility mode a re-ECN sender falls back to basic ECN.
>
>[SF]That is fine by me.
>
> >[BB]But the re-ECN draft does suggest that the network/middleboxes read and
> >have semantics for these two codepoints. In the incremental deployment
> >section I explain how this would work with nonce senders and receivers. The
> >basic idea is that a nonce sender appears to be declaring 50% path
> >congestion to the network. So policers should be configured permissively on
> >first deployment. Then, they can be gradually tightened up to encourage
> >upgrade to re-ECN.
> >
> >[BB]Of course the word 'upgrade' assumes re-ECN is 'better' than the nonce.
>
>[SF]Yep.  And that is my disagreement.
>
>[SF]If re-ECN used diff-serv codepoints to signal to use of re-ECN to
>droppers in the network, then RFC 3540 could still be used for the
>default semantics for the ECN Nonce, and re-ECN could be the alternate
>semantics.  This would not mean that the use of re-ECN would be
>restricted to a single diffserv codepoint;  there could be a diffserv
>codepoint for best-effort traffic that uses re-ECN, and other
>diffserv codepoints for various diffserv traffic classes that wish
>to use re-ECN.  (I could imagine that some diffserv traffic classes
>might not want to be subject to packet dropping by re-ECN droppers in
>the network, and other diff-serv traffic classes might want to use
>re-ECN, or to have two variants with two codepoints, one with and
>one without re-ECN.)

[BB] I see the interaction of Diffserv and re-ECN something like you've 
outlined, but I wouldn't put it as if the source could somehow negotiate 
about how it is policed.

[BB] A network might assert (not negotiate) that it would police the 
response to congestion differently for different DSCPs. Which might well 
mean that some DSCPs weren't policed at all so re-ECN would not be needed 
for these (perhaps because there was not meant to be a closed loop response 
to congestion, however I would expect codepoints needing no response to 
congestion to be exceptional).

[BB] I think your wording implies an operator MUST only use a DSCP that 
hasn't been used before to introduce rate policing. My wording allows a 
network operator to introduce rate policing for DSCPs that have already 
been defined. I am especially thinking of BE, where dropping is the norm, 
so a policer that drops in a more principled way wouldn't materially change 
the PHB. For instance, I don't think you would expect traffic ticketing 
policers to not be allowed to police BE traffic.

[BB] It is true that the IETF hasn't defined a congestion response for any 
of the current DSCPs (and I don't think that is the IETF's role). But it 
doesn't make sense to have a mechanism for introducing policing that 
REQUIRES that introducing policing MUST only be done by using a new DSCP. 
Policing is something an operator should chose to do, aiming to keep within 
their per domain behaviour. Indeed, one can think of a per domain behaviour 
as the combination of what the routers do for the traffic of app A and what 
the policers prevent all the other traffic doing to app A's traffic.

>[SF]The question is, is the current ECN nonce in RFC 3540 of sufficient
>value to be saved as the default semantics for ECT(0) and ECT(1)?
>I think it is.

[BB] I think it would be wrong to set aside the ECN nonce codepoints in the 
IP header for a use that relies on a trust model that only applies in a 
subset of cases. The ECN nonce is only relevant where the source acts in 
the interests of the network. The IP header should solely contain 
capabilities that are necessary for all higher layer apps....

[BB] ...We can agree to break this architectural principle, but we have 
principles to warn us to think long and hard when we want to break them. 
I'm not convinced the argument here is at all strong enough. I haven't even 
seen any particularly strong pressure to use the ECN nonce for the one use 
it does have (I guess not surprising given ECN isn't widely deployed).

[BB] Much as I am impressed by the engineering elegance of the ECN Nonce, 
and much as I believe we should put in protection before (not after) 
unresponsive flows become a problem, on balance I don't think the ECN nonce 
is a sufficiently generic solution to add to IP.

>[SF]It is hard to evaluate draft-briscoe-tsvwg-re-ecn-tcp-00 completely,
>since parts of it is incomplete.  However, in my view, the main
>difference between the ECN-Nonce and re-ECN is that the ECN-Nonce
>is used by the sender to prevent lying by the receiver, while re-ECN
>is used by the ingress and egress droppers together to prevent
>congestion control cheating by either or both of the end nodes.
>That is, as I understand it, re-ECN is only of use with the droppers
>in the network.
>
>[SF]My experience to far in working on congestion control mechanisms
>that require many interacting players is to have a significant
>distrust of arguments of why all of the players will have an incentive
>to act their part.  E.g., the SIGCOMM 2005 article on re-ECN outlines,
>why the end-nodes will have an incentive to use re-ECN, and the
>network operators will have an incentive to deploy ingress policing
>and egress dropping.  I have to say that I am skeptical, and
>personally, I have more confidence on mechanisms that require less
>global consistency, when that is possible.  (Unfortunately, it is
>not always possible, as we all know.)

[BB] I (nearly) agree with your skepticism. I too don't like the reliance 
of re-ECN on deployment of a dropper at the egress. But I do see a couple 
of clear first steps on the deployment path, either of which might be first.

[BB] a) An edge to edge scenario as in 
<draft-briscoe-tsvwg-cl-architecture-01.txt> using ECN for pre-congestion 
notification to drive admission control. I see this as a way for operators 
to get first experience with ECN deployment without requiring endpoint 
deployment. Re-ECN has a policing role here too. As the admission 
controlled Diffserv region grows from single operator to multi-operator, 
re-ECN policing allows congested downstream networks to ensure that 
upstream networks are doing admission control correctly. I have written a 
first (currently internal) draft about this - it will be one of the 
application scenarios for the re-ECN draft.

[BB] b) I see cellular operators as obvious takers for congestion policing.
   i) It is expensive for them to reduce congestion in their networks
   ii) Unresponsive apps haven't really taken hold in their market yet 
(because GPRS usage charges are so high).
   iii) They have a lot to lose from allowing unresponsive apps to take 
hold (they either have to suppress the GPRS market by keeping usage charges 
high, or lose much of their voice revenues).
   iv) They can ask for new capabilities in cellular devices to protect 
their own interests, because the device manufacturers' interests are tied in.

[BB] Re-ECN deployment makes sense for one (cellular) operator alone, who 
could deploy it at the ingress and the egress of their own network. And of 
course, the ingress in one direction is the egress in the other, so 
equipment vendors would make both ingress policing and egress dropping 
available at the same time on each edge router product.

[BB] I would also expect the cellular operators to act in concert, prob 
through 3GPP, to standardise congestion policing, based on what we do at 
the IETF.

[BB] Finally, re-ECN has a border mechanism for networks that have deployed 
re-ECN policers and droppers, so they can penalise neighbouring networks 
that dump congestion into their networks. We have only written up the bare 
bones of this - it will be in draft-01. The penalties are greater if these 
neighbours don't deploy re-ECN than if they do. However, these mechanisms 
rely on the downstream network having the market power to dictate terms. 
But again, the cellular operators, particularly acting in concert, 
represent a formidable power bloc.

[BB] Clearly, until I have written these various deployment scenarios up 
properly I don't expect you to be able to pick over the details. I am 
trying hard to get all this writing up done ASAP.

[BB] I do also want to point out tho that policing congestion response is 
just one application of re-ECN. It is intended to be a general mechanism to 
introduce accountability for causing congestion. Other identified 
applications are: QoS, DoS-mitigation, traffic engineering. That's why I 
believe the IP layer is the correct place for re-ECN, because 
accountability in the use of resources is a general requirement across all 
apps.

>[SF]In particular, I have more
>confidence in the use of the ECN Nonce by the sender to ensure that
>the receiver is not lying about congestion feedback,

[BB] Understood (but no help against all the senders that gain from being 
seen to serve their clients faster).

>[SF]and the use
>of policers in the network that rely on locally available information
>(e.g., packet drops at that point or upstream, ECN marks at that
>point or upstream, etc.) to check that connections are using
>appropriate end-to-end congestion control.

[BB] See following mail: "Traffic ticketing considered ineffective or harmful".

>[SF]That does not mean that I am opposed to re-ECN - I think it is
>interesting, and has a fair amount of promise.  However, I don't
>have sufficient confidence in it to think it superior to the ECN
>Nonce,

[BB] That's understandable, given we haven't done a comprehensive job of 
documenting it yet.

>[SF]so my own preference would be for the use of re-ECN to be
>signalled to policers and droppers by diff-serv codepoints, rather
>than by a re-definition of the default semantics of the ECN Nonce.

[BB] As explained, I don't like tying re-ECN to Diffserv in this way. It 
isn't necessary anyway with the new marking scheme we are writing up. 
Indeed, it wasn't necessary with the old coding scheme, which flagged 
whether re-ECN was in use by setting bit 48 of the IP header.

[BB] We need a proper debate before potentially ending up setting aside two 
different sets of IP header codepoints for similar functions done two 
different ways.

[BB] I understand we can't assume re-ECN will happen. So I would argue that 
the ECN Nonce should stay experimental for longer. That doesn't stop 
implementers if they want to do it.

[BB] Responses to your specific review comments on our draft follow...

>[SF]More specific feedback on draft-briscoe-tsvwg-re-ecn-tcp-00 is below.
>
> >[BB]This is all in the re-ECN draft - pls read the details there which I 
> think
> >you'll find is less strange than using DSCPs. Perhaps strange is a little
> >too strong. But certainly confining re-ECN to certain DSCPs is
> >unnecessarily over-constraining.
> >
> >(The essence of this backwards compatibility was also in the re-feedback
> >SIGCOMM paper - don't know where the Diffserv idea came from).
>
>[SF]It came from me, as described above.  An alternative mechanism,
>proposed below, would be to uses the proposed CR flag in the IP
>header for this.  Either one would allow the ECN Nonce to
>proceed as is to Proposed Standard.
>
>Regards,
>
>- Sally
>http://www.icir.org/floyd/
>
>
>--------------------------------------------------------------------
>[SF]Feedback on draft-briscoe-tsvwg-re-ecn-tcp-00:
>
>[SF]P. 4:
>"Policing is only truly effective at the first ingress into an
>internetwork, but path congestion is only visible at the last
>egress."
>
>[SF]I disagree.  Policing, in the form of `traffic tickets' that discourage
>flows from cheating about end-to-end congestion control, is effective
>wherever it is done in the netowrk.  And path congestion is visible
>at the congestion point, or anywhere downstream.  (My understanding
>is that congestion these days is typically not in the core, but on
>access links, exchange points, edge networks, and the like.)

[BB] See following mail: "Traffic ticketing considered ineffective or harmful"

[BB] More...

>[SF]P. 7:
>"The assumption behind the ECN nonce is that a sender will want to
>detect whether a receiver is suppressing congestion feedback.  This
>is only true if the sender's interests are aligned with the network's,
>or with the community of users as a whole.  This may be true for
>certain large senders, who are under close scrutiny and have a
>reputation to maintain."
>
>[SF]The ECN nonce would be useful for more than just "certain large
>senders".  For example, our measurement study of 80.000 web servers
>("Measuring the Evolution of Transport Protocols in the Internet",
>April 2005 CCR, "http://www.icir.org/tbit/") found only almost all
>of the web servers use appropriate congestion contron in TCP
>connections, halving the congestion window in response to packet
>drops.  I would imagine that most of these 80.000 web servers would
>also find the ECN Nonce useful to verify compliance from receivers.

[BB] There's a big difference between servers complying with the TCP 
standard (not being bothered to try to game it) and servers being willing 
to police the compliance of others to the standard. So I don't think your 
last sentence follows at all.

[BB] There may be a role for a nonce at the transport layer (I would be 
willing to help define one) to protect senders against the attack where 
receivers can fake early ACKs.
<http://www-cse.ucsd.edu/~savage/papers/CCR99.pdf>
<http://www-cse.ucsd.edu/~savage/papers/ICNP01.pdf>
But I don't think there's sufficient justification for using IP header bits 
to allow the sender (but not the network) to tell if a receiver is lying 
about congestion that the network has signalled.

[BB] Whether or not there's a problem with web servers, doesn't say whether 
the network layer should have safety mechanisms against congestion 
collapse. It's a bit like building a square castle with only three walls 
and saying there's no attacks through any of the walls.

[BB] The Tbit study was useful proof that there wasn't a problem with Web 
sessions complying with TCP. But conclusions shouldn't be stretched further 
than appropriate. We don't know why there wasn't a problem or what might 
trigger a problem. We don't know whether there is a problem with p2p 
filesharing or p2p streaming (we have done some predictions in the latter 
case). Indeed, we _can't_ prove that a network without safety mechanisms is 
safe just by showing it's not being attacked. We can only predict what 
conditions might trigger attacks.

>[SF]P. 15:
>[SF]TCP connections with an ACK acking eight or more packets: As you
>say in your proposal, when an ACK arrives acking eight or more
>packets, re-ECN is unable to know whether none of the intervening
>packets were ECN-marked, or if eight of the intervening packets
>were ECN-marked.  As you say, this could use more investigation;
>both the ultra-conservative response of assuming packets were
>ECN-marked, and the optimistic response of assuming that no packets
>were ECN-marked, could be problematic.  If it relied on past history,
>the TCP sender could perhaps at least make a plausible inference.
>[E.g.  it is unlikely that 8 packets will be marked in a RTT, when
>it has never been the case that 4, 5, 6, or 7 packets have been
>marked in a single RTT.]

[BB] Agreed more investigation needed.

[BB] However, we chose the ultra-conservative assumption both because it 
was the safe one, and it would only cause the sender to lower throughput 
when throughput already had to be very low.

[BB] If the average forward marking rate is p, there would have to be 8/p 
ACK losses in a row and they would all have to be pure ACKs, for re-TCP not 
to know if it had missed 8 marks. The conservative assumption only has any 
chance of being mistaken for very high marking rates. And at very high 
marking rates, if the source assumes 8 more marks than there actually were, 
its rate will not become much slower than it already is. Specifically, L 
lost ACKs in a row would only lose track of 8 echoes of CE if p > 8/L.

L, ACKS lost in a row            p, marking rate
________________________________________________
10                               >80%
50                               >16%
80                               >10%
800                              >1%

[BB] The values of L that we are ever likely to see are below say 80 (and 
that is stretching credulity). If we do experience such sequences of ACK 
losses, the conservative assumption will only be mistaken for marking 
rates > 10%. When the marking rate is 10%, the source will be going so slow 
already that assuming 8 more CE marks won't make it go much slower.

[BB] On the 1 in a billion occasions someone might see 800 ACK losses in a 
row, the source will have to halve its rate when it shouldn't have, if it 
assumes it may have missed 8 CEs. It will just be "one of those 
things"---lost in the noise. I personally wouldn't recommend complicating a 
TCP implementation by doing inference from past history, just to eliminate 
such safe, innocuous and extremely rare mistakes. I'll clarify this in the 
draft-01.

>[SF]P. 15:
>"We expect Re-ECN would be deployed for some time before policers
>and droppers start to enforce it."
>
>[SF]Yep, from my experience, I would expect it to be a long time indeed.
>And if re-ECN used a diff-serv codepoint to signal its use to
>policers and droppers, then other traffic could proceed with the use
>of the ECN Nonce.
>
>[SF]P. 20:
>[SF]The proosed CR flag in the IP header:
>[SF]It would be good to clarify in this document exactly what the CR
>flag would entail.  The document says that "it will imply that the
>transport understands and is using re-feedback of other fields in the
>IP header, such as the TTL".  It would be nice if the CR flag
>(if this last free flag in the IP header gets allocated to re-ECN)
>would be used to signal to policers and droppers that this flow
>was using re-ECN.  Then you would not have to use diff-serv
>codepoints for this, and the ECN Nonce could advance to Proposed
>Standard as the default semantics for ECT(0) and ECT(1).

[BB] In section 5 I already said "When set, the CR flag also serves as an 
indication that the transports are re-ECN capable (Re-ECT)." But if this 
new marking scheme that we've recently come up with works out, this all 
changes anyway.

[BB] Cheers, and thanks again for your careful review.

Bob

____________________________________________________________________________
Notice: This contribution is the personal view of the author and does not 
necessarily reflect the technical nor commercial direction of BT plc.
____________________________________________________________________________
Bob Briscoe,                           Networks Research Centre, BT Research
B54/130 Adastral Park,Martlesham Heath,Ipswich,IP5 3RE,UK.   +44 1473 645196 

_______________________________________________
tsvwg mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/tsvwg

---

From: Bob Briscoe <[email-masked]>
Date: Thu, 26 Jan 2006 16:54:29 +0000

Sally,

I've started a new thread to respond to your first detail point on traffic 
ticketing (TT) policers. It was originally in the thread "Re: Diffserv 
codepoint seems strange for Re-ECN <draft-briscoe-tsvwg-re-ecn-tcp-00> ".

[BTW, I'm making a big assumption that by TT you mean approaches like 
XCHOKe, Approx Fair Dropping (AFD) etc.]

Until yesterday I thought my argument against TT policing just concerned 
their inaccuracy (ie they are fairly ineffective as the bad guys can fly 
under their radar). Having written this e-mail, I now understand better a 
number of much stronger arguments against them, which I don't think have 
been articulated before.

To summarise the whole mail, even if TT policers were universally deployed:
- they would only stop dumb overuse (ie fairly ineffective),
- they would cause an arms race by spreading congestion over multiple paths 
that would take us back to where we started (again, ineffective)
- and they would make evolution to new congestion control very complicated, 
if not impossible (harmful to the good of evolution).

My response to your specific point about multiple bottlenecks is only a 
small part of what follows, but I do address it---indeed I argue that 
re-ECN is (perhaps considerably) better than TT on its own terms. But most 
of my arguments are really directed at the wider list---for general 
discussion---you don't need to feel that you have to answer for the whole 
TT approach that I am questioning.

Details below.

At 21:02 15/12/2005, Sally Floyd wrote:
>Feedback on draft-briscoe-tsvwg-re-ecn-tcp-00:
>
>P. 4:
>I disagree.  Policing, in the form of `traffic tickets' that discourage

[BB] I agree one can do a fair bit in the short term to deal with 
unresponsive flows by deploying 'traffic ticketing' (TT). But I think the 
aims of TT are different to ours.

I think the aim of TT is to catch extreme offenders who aren't going to 
take evasive action. We want to be able to discriminate flows that would 
fly under the radar of TT. We also want to nail the whole accountability 
problem of IP once and for all.

TT polices source-dest address pairs. If deployed, TT would simply 
encourage a malicious/selfish/careless endpoint (A) wanting to transfer 
data from A to Z to multiplex through bit-torrent-like app-layer relays 
M,N,O,P etc (these might be offering a relaying service p2p around the 
Internet, or they may be within A's local network control). Or A could just 
spoof multiple source addresses.

The problem is that TT is address-based, whereas re-ECN can control the 
congestion caused by traffic crossing the physical interface at any trust 
boundary (which generalises to a network of hosts behind the interface, or 
a network of networks). I think you would be happy if all the sub-flows 
A-M, M-Z, A-N, N-Z etc were TCP friendly. For us, that's not seeing the 
wood for the trees. Encouraging the spread of congestion over multiple 
paths is fine if one host does it. But once enough hosts do it, it hasn't 
solved anything.

Below is a list of problems with TT, all of which re-ECN is designed to 
deal with in what he hope is a more strategic change to the Internet's 
network layer. Each point is expanded on below:
- Paths through multiple bottlenecks
- Deploying policers at any potential bottleck vs ingress & egress only
- TT policers considered harmful to congestion response evolution
- Policing individual packets
- Per customer (interface) policing (parallel flows, heavy users, DDoS, etc.)
- Policing whole networks and network deployment incentives

Paths through multiple bottlenecks
----------------------------------
All network paths traverse two edges. And as you say, congestion tends to 
arise more at the access edges. Traffic traversing one bottleneck is a 
special case. The general and increasingly typical case is two (or more).

Sandvine stats from Sep 2005 [Sandvine05] give the following P2P traffic 
volumes as a proportion of bandwidth consumed on broadband provider networks
         up      down
eu      85%     60%
uk&us   76%     48%
fr      90%     85%
de      83%     65%

Sandvine's figures conflict with CAIDA's stats ranging over 10-28% measured 
in Jan 04 [Karag04b], but CAIDA warn "the purpose of our study is not to 
precisely quantify the percentage of P2P traffic in the backbone, but 
instead to affirm or refute claims on the trends of P2P file-sharing usage 
during the past few years." Further CAIDA's stats were gathered on a T1 
provider's OC48, and they admit the limitation that some P2P apps use ping 
distance as well as available bandwidth to rank which hosts to peer with. 
On the other hand Sandvine's stats don't stand up to academic analysis as 
they give little about methodolgy (except that they do payload inspection). 
Further, one might worry that it is in Sandvine's interests to 'big up' the 
impact of P2P.

Other studies very broadly support these stats, but whatever the exact 
numbers, that potentially means at least two bottlenecks is becoming much 
more common (even given ISPs are trying to serve p2p traffic from caches by 
redirection). When path congestion accumulates through more than one 
bottleneck, TT can only sanction greedy traffic _after_ it has caused 
problems through the earlier bottlenecks on the path.

As an example, assume half the path congestion is contributed by each of 
two nodes at the ingress and egress edges. Further assume TT is precise 
(which it isn't) and is willing to cause false negatives by setting its 
lower bound on RTT equal to the misbehaving flow's RTT. A misbehaving 
source could then get sqrt(2) = 1.4 times the correct rate through the 
first node. I don't know which paper on TT is considered the best (please 
advise), but taking XCHOKe as indicative, due to the uncertainties 
introduced, it seems it has to allow leeway for any traffic up to 15x the 
correct rate (later experiments in the XCHOKe paper do better, but only 
because the UDP sources are excessively greedy, so they get identified more 
quickly). So we have to allow 1.4x15 = 21x the correct rate in the 2 
bottleneck scenario.

Allowing 21x (or 15x for that matter) just isn't good enough to be worth 
deploying. Malicious sources can just fly beneath the radar at 20x the 
correct rate. Alternatively, if TT were intended to discourage apps that 
needed to be unresponsive but didn't want to ask for QoS, from our 
calculations TT would hardly catch any with such a wide leeway.

I would add that edge congestion is not just a phase the Internet is going 
through --- it is due to the cost economics of fan-out which should be the 
general rule for all time --- core congestion only predominated in the 
early Internet because it was a network between campuses, rather than 
predominantly homes and mobile devices.

Whether p2p traffic is just a trend and we will return to big server 
predominance, we don't know. But the network layer behaviour is meant to be 
generic for all possible app behaviours.

In response to the similar concerns you expressed from the mic at 
sigcomm'05, we prepared a paper comparing the limits of TT with 
re-feedback. We've just had some useful comments back from reviewers about 
poor clarity, so you can either wait for us to improve that, or have a go 
at reading it now:
"A path-aware rate policer: design and comparative evaluation"
<http://www.cs.ucl.ac.uk/staff/B.Briscoe/pubs.html#repol>
We focuse particularly on how quickly a misbehaving flow can be detected.

Deploying policers at any potential bottleck vs ingress & egress only
---------------------------------------------------------------------
I agree re-ECN suffers from needing coordinated deployment at egress and 
ingress. I'd prefer it didn't, but it does. However, in a multi-bottleneck 
world 'traffic ticketing' (TT) suffers from exactly the same problem.

For why, let's take the 2-bottleneck example above with an idealised TT 
that doesn't need to allow any leeway for false negatives (ie it can reduce 
an unresponsive flow to x1 of a TCP-friendly flow's goodput).

But first I'm going to suggest an improvement on the existing TT designs to 
help them deal with multiple bottlenecks (then show they still aren't as 
good as re-ECN). A TT design could treat ECN marked packets arriving from 
upstream in the same way it treats packets it is about to mark (or drop).

Let's say there are two types of sources we are concerned about:
- selfish: wanting max goodput
- malicious: wanting to cause max congestion to others

The second policer might prevent a selfish source from sending more than x1 
of well-behaved, assuming it reduced its sending rate to the achievable 
goodput.

But the malicious source wouldn't care about goodput. The first TT policer 
would only know about half the congestion, so it would only reduce the rate 
to x1.4. Only the second policer would reduce it to x1. So the malicious 
source could congest the first bottleneck by exploiting its lack of 
knowledge of downstream congestion.

BTW, there is no TT story for why an egress network should help an ingress 
by taking account of arriving upstream congestion marking. Whereas there is 
for re-ECN (tho not fully articulated in the I-D yet).

Further, TT must be deployed at all potential bottlenecks to be effective, 
not just the very edge nodes like re-ECN. That's more smarts overall => 
higher cost.

TT policers considered harmful to congestion response evolution
---------------------------------------------------------------
One of our goals with re-ECN is that the ingress network can evolve to 
allow a sender a different congestion response from TCP-friendly. TT 
policers throughout the Internet wouold embed the TCP-friendly assumption 
across the Internet, and hugely complicate evolution from TCP.

We see re-ECN as a way to evolve the DCCP framework to allow 
non-TCP-friendly rate control modules. We believe the current activity in 
DCCP to deal with interactive streaming will inexorably prove that you 
can't stay TCP-friendly for interactive streaming apps where users are 
extremely averse to rate-changes more frequent than timescales of minutes 
(we did human factors expts on this back in 2002).

Once TT policing is scattered around the Internet, it prevents evolution 
from TCP-friendliness, embedding it as the only response to congestion for 
the whole Internet, ever. To evolve from that gets really difficult because 
you need to signal to remote networks what sort of rate response you want.

Re-ECN is designed so that a new congestion response can be agreed 
bilaterally---just between the sender and the ingress. To us, this is the 
golden egg, because it is effectively a QoS mechanism that only requires 
local signalling neetween source and ingress, not along the path. Re-ECN's 
bulk inter-domain policing is why the ingress doesn't need to specifically 
ask networks further downstream if a different congestion response is OK. 
Kelly has proved the network would remain stable if different congestion 
responses were allowed.

Per customer (interface) policing (parallel flows, heavy users, DDoS, etc.)
---------------------------------------------------------------------------
As I said, solely policing TCP friendliness of each flow is not seeing the 
wood for the trees. We already talked about TT policers encouraging hosts 
to spread congestion across paths. If a source is causing congestion on 
multiple paths, it would be infeasible for TT policers around the Internet 
to co-ordinate with each other, to get an aggregated view of the congestion 
caused by traffic from each source. Particularly given they only know 
sufficient about path congestion when they are near the egress for each 
flow. But Re-ECN co-ordinates across flows naturally, because it works at 
the ingress interface.

We have designed a policer that takes a rounded view of a whole user, not 
just per flow. So the user wouldn't gain from using strategies like 
creating parallel flows, or sending continuously 24x7. The idea is that an 
ISP gives each user a monthly quota of congestion it is allowed to 
cause---because congestion is the fundamental cost of network usage. If the 
user is heading towards consuming more than the quota, traffic is throttled 
in proportion to the congestion it causes to keep the user within quota.

This moves away from the 'rule' that each flow should be TCP-friendly. The 
point is that multiple TCP-friendly flows can still be Internet-harmful, in 
terms of network economics (implicitly questioning the basis of the 
TCP-friendly rule by considering all flows together).

This per-interface policer has only been described in outline (last para of 
Sec 3.3.2 of the re-feedback paper in SIGCOMM), but we're working on a full 
write-up.

Policing whole networks and network deployment incentives
---------------------------------------------------------
TT policers can't police the rate response to congestion of whole networks. 
Re-ECN provides the right metric at a network boundary for a network to 
encourage its neighbours to police their users. That's because it's 
measured per physical interface, not per address pair.

Yes, re-ECN needs egress networks to deploy droppers first. But networks 
can protect themselves against other networks that don't deploy re-ECN 
policers, and puts the right metrics in place to encourage them to do so 
through contractual agreements.

Policing individual packets
---------------------------
TT policers take a while to detect a new misbehaving flow. Re-ECN has been 
designed to encourage good behaviour right from the first packet of a flow. 
So it is inherently immune to source address spoofing. And it allows the 
network to control its congestion even if the traffic mix becomes dominated 
by short flows.

It would even also incentivise proxies to aggregate congestion control 
across multiple short flows. So even if the world moved to some strange 
future where all the flows in the Internet finished within a single window, 
I believe we would still be OK.

References:
-----------
@INPROCEEDINGS{Karag04b:p2p_vol,
   author = {Thomas Karagiannis and Andre Broido and Nevil Brownlee and {kc}
            {claffy} and Michalis Faloutsos},
   affiliation = {1,5 UC Riverside; 2-4: CAIDA},
   month = nov # "--" # dec,
   year = 2004,
   title = {Is {P2P} Dying or Just Hiding?},
   booktitle = {Proc Globecom'04},
   address = {URL: \url{http://www.caida.org/outreach/papers/2004/p2p-dying/}},
   keywords = {Data Communication, Networks, Internet, Peer-to-Peer, P2p,
              Topology, Traffic Characterisation},
   entryby = {RJB},
   entry = {18 Jan 2006}
}
@MISC{Sandvine05:p2p_vol,
   author = {{Sandvine inc}},
   affiliation = {Sandvine},
   month = sep,
   year = 2005,
   title = {{EDonkey} --- {Still} King of {P2P} in {France} and {Germany}},
   howpublished = {URL \url{http://www.sandvine.com/news/pr_detail.asp?ID=88}},
   keywords = {Data Communication, Networks, Internet, Peer-to-Peer, P2p,
              Topology, Traffic Characterisation},
   entryby = {RJB},
   entry = {18 Jan 2005}
}

Cheers

Bob

____________________________________________________________________________
Bob Briscoe, <[email-masked]>      Networks Research Centre, BT Research
B54/130 Adastral Park,Martlesham Heath,Ipswich,IP5 3RE,UK.   +44 1473 645196