Subject: [tsvwg] Codel AF results

From: Greg White <[email-masked]>
Date: Sun, 26 Apr 2020 16:07:04 +0000

Jonathan, et al.,

In the slides for tomorrow’s TSVWG interim.  I noticed that slide 9 indicates that for SCE, Dual Queue (Codel AF) provides both fairness and ultra-low delay.  That seems a bit surprising based on the results that have been shared publicly.

I believe that there have only been three scenarios that have been shared: 50Mbps/20ms, 50Mbps/80ms, 50Mbps/160ms (all with a single Cubic flow and a single SCE flow).

The result shared at the February TSVWG interim (50Mbps/80ms), slide 19 here:
https://datatracker.ietf.org/meeting/interim-2020-tsvwg-01/materials/slides-interim-2020-tsvwg-01-sessa-some-congestion-experienced
shows that the SCE sender sees the same latency result as the Cubic sender, but suffers from throughput degradation.

In the recently posted results which cover all three cases:
http://sce.dnsmgr.net/results/ect1-2020-04-23-final/sce-s2-twoflow/sce-s2-twoflow-ns-cubic-vs-cubic-sce-twin_codel_af-20ms_tcp_delivery_with_rtt.svg
http://sce.dnsmgr.net/results/ect1-2020-04-23-final/sce-s2-twoflow/sce-s2-twoflow-ns-cubic-vs-cubic-sce-twin_codel_af-80ms_tcp_delivery_with_rtt.svg
http://sce.dnsmgr.net/results/ect1-2020-04-23-final/sce-s2-twoflow/sce-s2-twoflow-ns-cubic-vs-cubic-sce-twin_codel_af-160ms_tcp_delivery_with_rtt.svg

The 50Mbps/80ms result is the same as was shown in February (no latency benefit for SCE, only throughput degradation).
The 50Mbps/160ms shows the same characteristic.

Only the 50Mbps/20ms result shows some indication that the SCE sender can achieve lower latency than the Cubic sender.

So I don’t think there is sufficient evidence to support the claim that Codel AF can provide ultra-low delay in the context that it is meant here.  In fact two of the three results seem to refute it.  Are you meaning that it only supports low latency in that one specific scenario, or are you meaning that it can provide low latency only to sparse traffic that is not seeking capacity?

Also since all three of the published results show degradation of SCE throughput (as compared to Cubic), it raises the question as to whether Codel AF can provide fairness across a broad range of operating conditions.  Perhaps it can, but it seems premature to claim that it does.

Maybe I missed seeing some other published results.  If so, could you point them out to me?

-Greg

---

From: Jonathan Morton <[email-masked]>
Date: Sun, 26 Apr 2020 20:08:31 +0300

> Only the 50Mbps/20ms result shows some indication that the SCE sender can achieve lower latency than the Cubic sender.

Correctly noted, and this is entirely explainable by the experiment setup.  As we were focused on testing the L4S claims at the time, we neglected to describe these particular results on the SCE side, so I'll try to remedy that here.  Normally we have been able to do so by making a standalone presentation about new SCE work.

Elsewhere in our report, we noted that most Internet paths exhibit varying amounts of jitter, and are usually not as smooth and quiet as we can all achieve in laboratory conditions.  On a typical 80ms path, our use of a 2.5ms target delay is sufficiently close to zero to count as "ultra low" for practical purposes, but we allow short-term excursions above that target in order to keep utilisation acceptably high.  But where a quiet 20ms path does exist, for example between a CDN and a wired-in CPE device, there may be applications where a further reduction in peak delay can usefully be accommodated.

Our approach to this is to use a DSCP to explicitly mark traffic that is established to be following such a suitable path, and to use the "Twin CodelAF" approach, described in the latter part of the CodelAF I-D, to implement a low-latency PHB without conferring favour in terms of throughput.  In keeping with this approach, only SCE flows on the 20ms simulated path are marked with the DSCP we have chosen for internal experiments (which is one of the legacy TOS codepoints).  So that is why you only see a difference in the 20ms runs.

In the single-flow test at 20ms, specifically with twin_codel_af, you can clearly see the SCE flow settling on a queue delay in the vicinity of 1ms, possibly less (the graph scale isn't really detailed enough to distinguish that).  It should be easy to see the difference with the 80ms and 160ms runs which settle at 2.5ms.  The exception is when the path capacity is reduced to 5Mbps, in which the serialisation time of individual packets already exceeds these targets.

In the two-flow test at 20ms, you can also see some artefacts caused by some remaining bugs in the SCE implementation.  We are actively working on those behind the scenes, but in between them you can see that the SCE flow is given equal throughput to its competitor but at lower latency; this would be even more obvious if we gave the competing flow an 80ms path.  This is a property that the L4S proposal's DualQ promises but, according to discussions in another thread, doesn't seem to provide to everyone's satisfaction.  In Twin-CodelAF, we use a simple round-robin mechanism, weighted according to flow counts in each class.

As for the imbalance in the higher-RTT runs, this is indeed unchanged from what we showed in February, but it is also a significant improvement over what we were able to show at Singapore.  The use of AF techniques is still something we are actively developing, and we have ideas on how to resolve this remaining imbalance.  For example, we could add an integrating controller to the AF mechanism, rather than relying only on an instantaneous proportional weight.  Once we establish something that works better, we'll publish a description and test results accordingly.

Time pressures prevented us from exploring more complex traffic scenarios in any detail.  Perhaps next time.

 - Jonathan Morton

---

From: Greg White <[email-masked]>
Date: Sun, 26 Apr 2020 21:03:09 +0000

Ok, thanks for the clarification.  Based on this I suppose we should consider the "Y" in both columns (Fairness and Ultra-Low Delay) for Dual Queue (Codel AF) as being aspirational?

I'm a little unclear how you envision the DSCP mechanism would work.    Would the sender first try to ascertain whether the path has stable and low RTT characteristics (and maybe also whether the bottleneck implements Codel AF?) before it would begin marking its packets with the chosen DSCP, and then once it began marking, it would get a latency benefit from Codel AF in some cases?  Or, are you anticipating that path measurements would be done a priori by the network operator, and then senders would be configured to use (or not) the selected DSCP on a case-by-case basis? 

-Greg

    

    

    
    
     - Jonathan Morton

---

From: Jonathan Morton <[email-masked]>
Date: Mon, 27 Apr 2020 02:41:53 +0300

> On 27 Apr, 2020, at 12:03 am, Greg White <[email-masked]> wrote:
> 

Only to the same extent as the corresponding cells in the L4S table.

In terms of Fairness, the 2:1 ratio observed is consistent over a reasonably wide range of conditions, including differences of RTT between the conventional and SCE flows - so, unlike DualQ, it is RTT independent.  There are cases where imprecision on TCP Prague's heuristic results in much worse ratios than that.

The 2:1 ratio arises because the AF mechanism adjusts the delay targets for each flow in proportion to queue occupancy, so the resulting queue occupancies and therefore throughputs are proportional to the targets for each marking type - which, by default, are 5ms for CE and 2.5ms for SCE.  So the only aspirational point, as far as we are concerned, is to resolve that 2:1 ratio by further improving the AF mechanism.

In terms of Ultra Low Delay, that is actually demonstrated, not aspirational.  The throughput isn't as high as we'd like due to the bugs we haven't yet fixed at the sender, but the qdisc is working just fine in this respect, and the latency characteristics look good.

> I'm a little unclear how you envision the DSCP mechanism would work.    Would the sender first try to ascertain whether the path has stable and low RTT characteristics (and maybe also whether the b…

We haven't yet explored the operational aspects of this very deeply; this is more of a tech demo.

However, we anticipate this improved latency performance (from a 2.5ms to a 0.5ms delay target) will be required and viable only in a limited range of circumstances, so deploying it would likely be a deliberate operation, not automatic.  It would involve cooperation between the traffic source and the networks it traverses, much like any other Diffserv PHB; since it is applicable mainly to short paths, the number of networks involved is likely small.  By default, SCE traffic shouldn't need Diffserv assistance, so can traverse Diffserv domains as best-effort traffic.

 - Jonathan Morton