Subject: [ippm] I-D Action: draft-ietf-ippm-capacity-metric-method-08.txt

From: [email-masked]
Date: Mon, 29 Mar 2021 14:31:20 -0700

A New Internet-Draft is available from the on-line Internet-Drafts directories.
This draft is a work item of the IP Performance Measurement WG of the IETF.

        Title           : Metrics and Methods for One-way IP Capacity
        Authors         : Al Morton
                          Ruediger Geib
                          Len Ciavattone
	Filename        : draft-ietf-ippm-capacity-metric-method-08.txt
	Pages           : 37
	Date            : 2021-03-29

Abstract:
   This memo revisits the problem of Network Capacity metrics first
   examined in RFC 5136.  The memo specifies a more practical Maximum
   IP-layer Capacity metric definition catering for measurement
   purposes, and outlines the corresponding methods of measurement.

The IETF datatracker status page for this draft is:
https://datatracker.ietf.org/doc/draft-ietf-ippm-capacity-metric-method/

There are also htmlized versions available at:
https://tools.ietf.org/html/draft-ietf-ippm-capacity-metric-method-08
https://datatracker.ietf.org/doc/html/draft-ietf-ippm-capacity-metric-method-08

A diff from the previous version is available at:
https://www.ietf.org/rfcdiff?url2=draft-ietf-ippm-capacity-metric-method-08

Please note that it may take a couple of minutes from the time of submission
until the htmlized version and diff are available at tools.ietf.org.

Internet-Drafts are also available by anonymous FTP at:
ftp://ftp.ietf.org/internet-drafts/

---

From: Magnus Westerlund <[email-masked]>
Date: Tue, 30 Mar 2021 08:54:04 +0000

Hi,

I have reviewed the latest versions changes and have some comments.

Section 8.1

   If no status feedback messages arrive at the sender for the interval	
   greater than the Lost Status Backoff timeout = w*FT, beginning when	
   the last message (of any type) was successfully received at the  sender:

"w" is used here and paragraphs below. But it is not really defined. Based
on what is written below it is an algorithm constant, so maybe it should be
given a name. 

So the backoff happens after w*FT. And I agree that with a regular feedback
interval that will be working independent of RTT, starting the time when the
first feedback has been received. What I thin this paragraph could be
clearer on is that w*FT need to be considered in relation to the one-way
delay jitter. And the most likely cause here is buffer depth in the
bottleneck. It must also be related to upper delay threshold. What occurs if
upper delay threshold is higher than w*FT? Is there a potential that a step
increases is sufficient to push the buffer occupancy more than w*FT longer,
i.e. causing the feedback to be delayed sufficient so that it triggers the
lost feedback rather than the upper threshold? I haven't calculated what
traffic increases that are possible and if that excess traffic can cause
this to happen without additional cross traffic over bottleneck. I would
assume this is fine to happen in the initial fast ramp-up, but it should not
really happen in the regular ramp-up case.

   The RECOMMENDED initial value for w is 3, taking Round Trip Time	
   (RTT) less than FT into account.  A test with RTT longer than FT is a

   valid reason to increase the initial value of w appropriately.	
   Variable w SHALL be incremented by 1 whenever the Lost Status Backoff

   timeout is exceeded.  So with FT = 50ms, a status feedback message	
   loss would be declared at 150ms following a successful message, again

   at 50ms after that (200ms total), and so on.

I still think that Section 8 could be more explicit about the goal of the
load algorithm. 

To accurately measure the capacity of the path the load algorithm strive to
load the path so that the one way latency increases to indicate that the
bottleneck buffer is sufficiently filled to achieve high utility. This load
is not intended to be fair against other applications, it is intended to
provide as accurate measurement as possible, potentially pushing other
applications out of the way.

The goal of being explicit about this is so that there is no doubt to anyone
what this algorithm does, and that it is not a generic congestion control
algorithm.  So Section 14.1 discuss this, but I think the document can be
more honest here and state it explicitly in Section 8.

Now, I am no longer an AD, but I don't have a problem with a limited scope
algorithm that just is open with its intention of being unfair. I think
having an algorithm that people may copy without understanding what it does
is more a problem. 

Cheers

Magnus Westerlund

> -----Original Message-----
> From: I-D-Announce <[email-masked]> On Behalf Of
> [email-masked]
> Sent: den 29 mars 2021 23:31
> To: [email-masked]
> Cc: [email-masked]
> Subject: I-D Action: draft-ietf-ippm-capacity-metric-method-08.txt
> 
> 
> _______________________________________________
> I-D-Announce mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/i-d-announce
> Internet-Draft directories: http://www.ietf.org/shadow.html or
> ftp://ftp.ietf.org/ietf/1shadow-sites.txt

---

From: <[email-masked]>
Date: Tue, 30 Mar 2021 10:48:44 +0000

Hi Magnus,

[MW] To accurately measure the capacity of the path the load algorithm strive to load the path so that the one way latency increases to indicate that the bottleneck buffer is sufficiently filled to achieve high utility. This load is not intended to be fair against other applications, it is intended to provide as accurate measurement as possible, potentially pushing other applications out of the way.

[RG] The sender rate adaption mechanism operates by congestion feedback, as do many standard transport protocols. The purpose of the algorithm is not to determine, whether a queue or a policer burst size "is sufficiently filled to achieve high utility", the purpose is to accurately determine the access speed which can be reached without queue-build up and/or packet loss. After feedback indicating a sender rate which has caused congestion has been received, the sender rate is adapted until the maximum access speed not causing congestion is determined. To decide on a sender rate fulfilling this this criterium, a sender rate causing a congestion which can be accurately identified as congestion by a measurement system is required. If you are aware of a method reaching this goal without causing limited congestion, please share.

[RG] I don't understand whether your feedback is focused on improvements of parametrization or on the basic behaviour of the algorithm. The algorithm is not designed to be unfair.

Regards,

Ruediger

-----Ursprüngliche Nachricht-----which
Von: ippm <[email-masked]> Im Auftrag von Magnus Westerlund
Gesendet: Dienstag, 30. März 2021 10:54
An: [email-masked]
Betreff: Re: [ippm] I-D Action: draft-ietf-ippm-capacity-metric-method-08.txt

Hi,

Section 8.1

"w" is used here and paragraphs below. But it is not really defined. Based on what is written below it is an algorithm constant, so maybe it should be given a name. 

So the backoff happens after w*FT. And I agree that with a regular feedback interval that will be working independent of RTT, starting the time when the first feedback has been received. What I thin this paragraph could be clearer on is that w*FT need to be considered in relation to the one-way delay jitter. And the most likely cause here is buffer depth in the bottleneck. It must also be related to upper delay threshold. What occurs if upper delay threshold is higher than w*FT? Is there a potential that a step increases is sufficient to push the buffer occupancy more than w*FT longer, i.e. causing the feedback to be delayed sufficient so that it triggers the lost feedback rather than the upper threshold? I haven't calculated what traffic increases that are possible and if that excess traffic can cause this to happen without additional cross traffic over bottleneck. I would assume this is fine to happen in the initial fast ramp-up, but it should not really happen in the regular ramp-up case.

I still think that Section 8 could be more explicit about the goal of the load algorithm. 

To accurately measure the capacity of the path the load algorithm strive to load the path so that the one way latency increases to indicate that the bottleneck buffer is sufficiently filled to achieve high utility. This load is not intended to be fair against other applications, it is intended to provide as accurate measurement as possible, potentially pushing other applications out of the way.

The goal of being explicit about this is so that there is no doubt to anyone what this algorithm does, and that it is not a generic congestion control algorithm.  So Section 14.1 discuss this, but I think the document can be more honest here and state it explicitly in Section 8.

Now, I am no longer an AD, but I don't have a problem with a limited scope algorithm that just is open with its intention of being unfair. I think having an algorithm that people may copy without understanding what it does is more a problem. 

Cheers

Magnus Westerlund

> https://www.ietf.org/rfcdiff?url2=draft-ietf-ippm-capacity-metric-meth
> od-
> 08
> 
>

---

From: J Ignacio Alvarez-Hamelin <[email-masked]>
Date: Tue, 30 Mar 2021 08:42:01 -0300

Hi Rüdiger,

There is a family of different congestion avoidance algorithms, but the goal is to achieve the maximum throughput. Nevertheless, it is clear that some part of the network along the path from sender and server is congested; any of these algorithms will start to work. In other words, you have to put your server close enough to your link under measurement. The popular M-Lab uses at least three TCP flows to determine the rate as a way to trying to avoid this kind of problem, but in that case, servers are far away. 
Among the possible algorithms to maximize the uses vs. congestion in a short time, there is BBR used by QUIC (if I remember well):

 http://dl.ifip.org/db/conf/networking/networking2018/2B1-1570416152.pdf
 https://www.sciencedirect.com/science/article/pii/S0140366419303470?casa_token=ggSPL6OXQH0AAAAA:zXNHOPgQeydZo3XBJRbeAApYrGQ7iZPocVZHCu11P19Hs_KdwRO9sMEse-HhKgiRGHsJeR4_MK0

I hope this answer the question and give a better understanding, cheers,

	Ignacio

_______________________________________________________________

Dr. Ing. José Ignacio Alvarez-Hamelin
CONICET and Facultad de Ingeniería, Universidad de Buenos Aires
Av. Paseo Colón 850 - C1063ACV - Buenos Aires - Argentina
+54 (11) 5285 0716 / 5285 0705
e-mail: [email-masked]
web: http://cnet.fi.uba.ar/ignacio.alvarez-hamelin/
_______________________________________________________________

> On 30 Mar 2021, at 07:48, <[email-masked]> <[email-masked]> wrote:
> 
> Hi Magnus,
> 
> ippm mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/ippm

---

From: <[email-masked]>
Date: Tue, 30 Mar 2021 12:37:30 +0000

Hi Ignacio,

Thanks, I'm aware of these protocols. The capacity metric method design aim is measurement, not transport. That's not a license to ignore other purposes, of course. I'd consider some different constraints to apply, like a slower ramp-up and brief operation as close to the (congestion free) access rate as possible, if the feedback indicates stable performance. And then turn off and stay silent (and stay silent earlier if networking conditions aren't stable - the design aim isn't to annoy persons on either end of the measurement). It's certainly not designed to exchange information on a level of minutes or hours in a fair and reliable manner. I wouldn't expect anybody to run a speed test for minutes or hours, no matter whether the implementation is based on transport or on a measurement protocol (I'd also expect the performance of my other to decay while running a TCP based speed test on my access).

I understand that long RTTs are an issue. A transport getting rare feedback should act conservative. A measurement protocol shouldn't create persistent or annoying congestion either, if RTT is high. To me, reaching the design aim of accurate access speed measurement as independent from RTT as possible is important, as the performance of "speed tests" based on TCP is depending on RTT, and causes them to be less accurate.

Regards,

Rüdiger

-----Ursprüngliche Nachricht-----
Von: J Ignacio Alvarez-Hamelin <[email-masked]> 
Gesendet: Dienstag, 30. März 2021 13:42
An: Geib, Rüdiger <[email-masked]>
Cc: magnus.westerlund=[email-masked]; [email-masked]
Betreff: Re: [ippm] I-D Action: draft-ietf-ippm-capacity-metric-method-08.txt

Hi Rüdiger,

	Ignacio

CONICET and Facultad de Ingeniería, Universidad de Buenos Aires Av. Paseo Colón 850 - C1063ACV - Buenos Aires - Argentina

> Betreff: Re: [ippm] I-D Action: 
> draft-ietf-ippm-capacity-metric-method-08.txt
> 
> Hi,
> 
>> https://datatracker.ietf.org/doc/draft-ietf-ippm-capacity-metric-meth
>> o
>> d/
>> 
>> https://datatracker.ietf.org/doc/html/draft-ietf-ippm-capacity-metric
>> -
>> method-08
>>

---

From: "MORTON, ALFRED C (AL)" <[email-masked]>
Date: Tue, 30 Mar 2021 16:31:27 +0000

Hi Magnus,

Thanks for your review of 08. You have some questions to answer in the thread below.

However, I read your review as saying you only need two points addressed now:

1. Warn that people must not use the Section 8.1 Load rate adjustment algorithm as a general Congestion Control Algorithm (CCA).  We will add that in Section 8.1. We didn't call the algorithm a CCA anywhere, and we already make our goal clear (again and again).

2. The w*FT Timeout needs a better description. Your discussion of this wandered into one-way delay jitter and buffer depth. We might be able to base the timeout on the setting of the high delay range threshold (default 90ms) with the same recurring timeout property for repeated lost messages as described now. Something like 

           HDRT + FT + (w*FT) = Lost Status Backoff timeout

where: 
HDRT = high delay range threshold (default 90ms)
FT   = feedback time interval (default 50ms)
w    = index for repeated timeouts (w=0 initially, w++ on each timeout)

So here's our status: 
We've prepared two new versions of the draft with a large amount of new material predominantly in response to your comments, which quite frankly, have expanded tremendously over the last month (starting with a few short paragraphs in your DISCUSS).  We need to hear that addressing 1. and 2. now will satisfy your needs completely, then take next steps.

Thanks,
Al  

> From: ippm [mailto:[email-masked]] On Behalf Of
> [email-masked]
> Sent: Tuesday, March 30, 2021 8:38 AM
> To: [email-masked]
> Cc: [email-masked]
> Subject: Re: [ippm] I-D Action: draft-ietf-ippm-capacity-metric-method-
> 08.txt
> 
> Hi Ignacio,
> 
> Thanks, I'm aware of these protocols. The capacity metric method design
> aim is measurement, not transport. That's not a license to ignore other
> purposes, of course. I'd consider some different constraints to apply,
> like a slower ramp-up and brief operation as close to the (congestion
> free) access rate as possible, if the feedback indicates stable
> performance. And then turn off and stay silent (and stay silent earlier if
> networking conditions aren't stable - the design aim isn't to annoy
> persons on either end of the measurement). It's certainly not designed to
> exchange information on a level of minutes or hours in a fair and reliable
> manner. I wouldn't expect anybody to run a speed test for minutes or
> hours, no matter whether the implementation is based on transport or on a
> measurement protocol (I'd also expect the performance of my other to decay
> while running a TCP based speed test on my access).
> 
> I understand that long RTTs are an issue. A transport getting rare
> feedback should act conservative. A measurement protocol shouldn't create
> persistent or annoying congestion either, if RTT is high. To me, reaching
> the design aim of accurate access speed measurement as independent from
> RTT as possible is important, as the performance of "speed tests" based on
> TCP is depending on RTT, and causes them to be less accurate.
> 
> Regards,
> 
> Rüdiger
> 
> There is a family of different congestion avoidance algorithms, but the
> goal is to achieve the maximum throughput. Nevertheless, it is clear that
> some part of the network along the path from sender and server is
> congested; any of these algorithms will start to work. In other words, you
> have to put your server close enough to your link under measurement. The
> popular M-Lab uses at least three TCP flows to determine the rate as a way
> to trying to avoid this kind of problem, but in that case, servers are far
> away.
> Among the possible algorithms to maximize the uses vs. congestion in a
> short time, there is BBR used by QUIC (if I remember well):
> 
> 
> https://urldefense.com/v3/__http://dl.ifip.org/db/conf/networking/networki
> ng2018/2B1-
> 1570416152.pdf__;!!BhdT!yVLn0Ivu5MvPahTyoYRvk8VD3mI_hYOlYzLC0AGB1C50nCNjQ6
> cxSqflIg18$
> 
> https://urldefense.com/v3/__https://www.sciencedirect.com/science/article/
> pii/S0140366419303470?casa_token=ggSPL6OXQH0AAAAA:zXNHOPgQeydZo3XBJRbeAApY
> rGQ7iZPocVZHCu11P19Hs_KdwRO9sMEse-
> HhKgiRGHsJeR4_MK0__;!!BhdT!yVLn0Ivu5MvPahTyoYRvk8VD3mI_hYOlYzLC0AGB1C50nCN
> jQ6cxSlQZad4t$
> 
> CONICET and Facultad de Ingeniería, Universidad de Buenos Aires Av. Paseo
> Colón 850 - C1063ACV - Buenos Aires - Argentina
> web: https://urldefense.com/v3/__http://cnet.fi.uba.ar/ignacio.alvarez-
> hamelin/__;!!BhdT!yVLn0Ivu5MvPahTyoYRvk8VD3mI_hYOlYzLC0AGB1C50nCNjQ6cxSuun
> ueiD$
> > [MW] To accurately measure the capacity of the path the load algorithm
> strive to load the path so that the one way latency increases to indicate
> that the bottleneck buffer is sufficiently filled to achieve high utility.
> This load is not intended to be fair against other applications, it is
> intended to provide as accurate measurement as possible, potentially
> pushing other applications out of the way.
> >
> > [RG] The sender rate adaption mechanism operates by congestion feedback,
> as do many standard transport protocols. The purpose of the algorithm is
> not to determine, whether a queue or a policer burst size "is sufficiently
> filled to achieve high utility", the purpose is to accurately determine
> the access speed which can be reached without queue-build up and/or packet
> loss. After feedback indicating a sender rate which has caused congestion
> has been received, the sender rate is adapted until the maximum access
> speed not causing congestion is determined. To decide on a sender rate
> fulfilling this this criterium, a sender rate causing a congestion which
> can be accurately identified as congestion by a measurement system is
> required. If you are aware of a method reaching this goal without causing
> limited congestion, please share.
> >
> > [RG] I don't understand whether your feedback is focused on improvements
> of parametrization or on the basic behaviour of the algorithm. The
> algorithm is not designed to be unfair.
> >
> > Regards,
> >
> > Ruediger
> >
> > "w" is used here and paragraphs below. But it is not really defined.
> Based on what is written below it is an algorithm constant, so maybe it
> should be given a name.
> >
> > So the backoff happens after w*FT. And I agree that with a regular
> feedback interval that will be working independent of RTT, starting the
> time when the first feedback has been received. What I thin this paragraph
> could be clearer on is that w*FT need to be considered in relation to the
> one-way delay jitter. And the most likely cause here is buffer depth in
> the bottleneck. It must also be related to upper delay threshold. What
> occurs if upper delay threshold is higher than w*FT? Is there a potential
> that a step increases is sufficient to push the buffer occupancy more than
> w*FT longer, i.e. causing the feedback to be delayed sufficient so that it
> triggers the lost feedback rather than the upper threshold? I haven't
> calculated what traffic increases that are possible and if that excess
> traffic can cause this to happen without additional cross traffic over
> bottleneck. I would assume this is fine to happen in the initial fast
> ramp-up, but it should not really happen in the regular ramp-up case.
> >
> > To accurately measure the capacity of the path the load algorithm strive
> to load the path so that the one way latency increases to indicate that
> the bottleneck buffer is sufficiently filled to achieve high utility. This
> load is not intended to be fair against other applications, it is intended
> to provide as accurate measurement as possible, potentially pushing other
> > The goal of being explicit about this is so that there is no doubt to
> anyone what this algorithm does, and that it is not a generic congestion
> control algorithm.  So Section 14.1 discuss this, but I think the document
> can be more honest here and state it explicitly in Section 8.
> >
> > Now, I am no longer an AD, but I don't have a problem with a limited
> scope algorithm that just is open with its intention of being unfair. I
> think having an algorithm that people may copy without understanding what
> it does is more a problem.
> >
> > Cheers
> >
> > Magnus Westerlund
> >
> >
> >> https://urldefense.com/v3/__https://datatracker.ietf.org/doc/draft-
> ietf-ippm-capacity-metric-
> meth__;!!BhdT!yVLn0Ivu5MvPahTyoYRvk8VD3mI_hYOlYzLC0AGB1C50nCNjQ6cxSk8Syqb0
> $
> >> o
> >> d/
> >>
> >> https://urldefense.com/v3/__https://tools.ietf.org/html/draft-ietf-
> ippm-capacity-metric-method-
> 08__;!!BhdT!yVLn0Ivu5MvPahTyoYRvk8VD3mI_hYOlYzLC0AGB1C50nCNjQ6cxSndfx2ub$
> >>
> https://urldefense.com/v3/__https://datatracker.ietf.org/doc/html/draft-
> ietf-ippm-capacity-
> metric__;!!BhdT!yVLn0Ivu5MvPahTyoYRvk8VD3mI_hYOlYzLC0AGB1C50nCNjQ6cxSjV8vJ
> r1$
> >> -
> >> method-08
> >>
> >> https://urldefense.com/v3/__https://www.ietf.org/rfcdiff?url2=draft-
> ietf-ippm-capacity-metric-
> met__;!!BhdT!yVLn0Ivu5MvPahTyoYRvk8VD3mI_hYOlYzLC0AGB1C50nCNjQ6cxSrJUzesA$
> >> h
> >> od-
> >> 08
> >>
> >>
> >> https://urldefense.com/v3/__ftp://ftp.ietf.org/internet-
> drafts/__;!!BhdT!yVLn0Ivu5MvPahTyoYRvk8VD3mI_hYOlYzLC0AGB1C50nCNjQ6cxSiqz4
> ahP$
> >>
> >>
> >> https://urldefense.com/v3/__https://www.ietf.org/mailman/listinfo/i-d-
> announce__;!!BhdT!yVLn0Ivu5MvPahTyoYRvk8VD3mI_hYOlYzLC0AGB1C50nCNjQ6cxSgqh
> X936$
> >> Internet-Draft directories:
> https://urldefense.com/v3/__http://www.ietf.org/shadow.html__;!!BhdT!yVLn0
> Ivu5MvPahTyoYRvk8VD3mI_hYOlYzLC0AGB1C50nCNjQ6cxSk0bEN04$  or
> >> https://urldefense.com/v3/__ftp://ftp.ietf.org/ietf/1shadow-
> sites.txt__;!!BhdT!yVLn0Ivu5MvPahTyoYRvk8VD3mI_hYOlYzLC0AGB1C50nCNjQ6cxSuY
> fdqPq$
> >
> https://urldefense.com/v3/__https://www.ietf.org/mailman/listinfo/ippm__;!
> !BhdT!yVLn0Ivu5MvPahTyoYRvk8VD3mI_hYOlYzLC0AGB1C50nCNjQ6cxSnKRhcmW$
> 
> https://urldefense.com/v3/__https://www.ietf.org/mailman/listinfo/ippm__;!
> !BhdT!yVLn0Ivu5MvPahTyoYRvk8VD3mI_hYOlYzLC0AGB1C50nCNjQ6cxSnKRhcmW$

---

From: Magnus Westerlund <[email-masked]>
Date: Wed, 31 Mar 2021 07:43:02 +0000

Hi Ruediger,

On Tue, 2021-03-30 at 10:48 +0000, [email-masked] wrote:
> Hi Magnus,
> 
> [MW] To accurately measure the capacity of the path the load algorithm strive
> to load the path so that the one way latency increases to indicate that the
> bottleneck buffer is sufficiently filled to achieve high utility. This load is
> not intended to be fair against other applications, it is intended to provide
> as accurate measurement as possible, potentially pushing other applications
> out of the way.
> 
> [RG] The sender rate adaption mechanism operates by congestion feedback, as do
> many standard transport protocols. The purpose of the algorithm is not to
> determine, whether a queue or a policer burst size "is sufficiently filled to
> achieve high utility", the purpose is to accurately determine the access speed
> which can be reached without queue-build up and/or packet loss. After feedback
> indicating a sender rate which has caused congestion has been received, the
> sender rate is adapted until the maximum access speed not causing congestion
> is determined. To decide on a sender rate fulfilling this this criterium, a
> sender rate causing a congestion which can be accurately identified as
> congestion by a measurement system is required. If you are aware of a method
> reaching this goal without causing limited congestion, please share.

Okay, I might be slightly conflating two things. So the indication that the path
bottleneck is filled to capacity is the fact that there are some queue buildup.
Loss and queue build up are the two signals that the algorithm has defined to
indicate load and congestion, as it isn't defining a ECN response. However, I
would argue on a number of link technologhies you will not find maximum capacity
unless you attempt to transmit enough data and actually build up some queue in
the relevant buffer. That is definitely true for a number of radio technologies.
The underlying reason can be both efficiency in inter user scheduling as well as
deal with short term radio fading effects. So capacity and high utility are
necessarly linked to me.

> 
> [RG] I don't understand whether your feedback is focused on improvements of
> parametrization or on the basic behaviour of the algorithm. The algorithm is
> not designed to be unfair.

I am not after improvements of the algorithms I after accruate representation of
the algorithm and intended usage. From my perspective this algorithm does not
need to be either self-fair or fair against any other congestion control
algorithms. And I am fine with that as long as it is clear on the box what it
does. Have anyone run any tests of how this beahves when competing with other
congestion controls, like CUBIC, BBR(1/2), LEDBAT etc? I am not asking for it
but it if it has been run it would be enlightening.

Cheers

Magnus

---

From: <[email-masked]>
Date: Wed, 31 Mar 2021 09:30:16 +0000

Hi Magnus,

Thanks for clarifying. 

The scheduling behaviour of wireless shared media is an issue and first results I saw indicated an increased RTT and a rather high access capacity. Store until there's sufficient load to use a "bearer" and then send at line rate. I think that's one of the reasons why a measurement "preamble" is part of the method. As far as I can tell, the above behaviour turned (often at least) into a more stable forwarding, as long as measurement traffic arrived regularly afterwards.

We can't get around inducing congestion to recon that we are "over the top". I'd be happy to work with ECN marks too, but ECN isn't widespread at devices scheduling customer and server traffic with operator connectivity. The overshooting carefully is a requirement, and that's a reason why the algorithm measures queue-build up by increasing delay and classifies it as a congestion indication (as I think do some or all of the latest transport protocols too, without comparing the detailed algos). Fairness against competing transport hasn't been tested. In my eyes, such a test needs to compare commodity speed test vs. competing transport and then IP-capacity metric traffic vs. competing transport. Both, commodity speed-test and IP-capacity metric traffic decrease the bandwidth of competing transport. One of both might have a significantly stronger impact. I don't judge here, as I'm not aware of tests, but I also don't think it's ok to state commodity speed tests are fair, as they are TCP based. Their design goal is to reliably congest the receiver access until drops occur.

Udpst is open source. In general, and not addressing you personally, I'd appreciate all improvements to reduce total test load and test duration, and dealing with background traffic or (more generic) dealing with feedback indicating varying IP-capacity during a single test session at the receiver. If the result is "available capacity" because of such a time-varying receiver IP capacity, then that's the result and the test stops. I agree, that a test should not try to determine IP access capacity "by all means" (and I think neither does the text provided, nor does the implementation). 

Regards,

Rüdiger

-----Ursprüngliche Nachricht-----
Von: Magnus Westerlund <[email-masked]> 
Gesendet: Mittwoch, 31. März 2021 09:43
An: Geib, Rüdiger <[email-masked]>
Cc: [email-masked]
Betreff: Re: AW: I-D Action: draft-ietf-ippm-capacity-metric-method-08.txt

Hi Ruediger,

Cheers

Magnus

---

From: Magnus Westerlund <[email-masked]>
Date: Wed, 31 Mar 2021 14:27:40 +0000

Hi,

Yes, so I think ECN is definitely something that can wait for future
improvements. I was simply trying to be as clear as possible that from an
endpoint you don't have that many basic indicators of congestions. 

So I am primarily concerned that this one has the right description for what we
expect it to do. From my perspective I have no issue with this being published
with a clear limited applicability and clear indication of what it is expected
to do. For the intention I am fine with you authors reword it to something
similar that is more consistent with the existing text. I just want it to be
clear that due to the nature of the intent to measure capacity it may push
existing elastic traffic out of the way in the probing to determine maximum
capacity. 

Cheers

Magnus

On Wed, 2021-03-31 at 09:30 +0000, [email-masked] wrote:
> Hi Magnus,
> 
> Thanks for clarifying. 
> 
> The scheduling behaviour of wireless shared media is an issue and first
> results I saw indicated an increased RTT and a rather high access capacity.
> Store until there's sufficient load to use a "bearer" and then send at line
> rate. I think that's one of the reasons why a measurement "preamble" is part
> of the method. As far as I can tell, the above behaviour turned (often at
> least) into a more stable forwarding, as long as measurement traffic arrived
> regularly afterwards.
> 
> We can't get around inducing congestion to recon that we are "over the top".
> I'd be happy to work with ECN marks too, but ECN isn't widespread at devices
> scheduling customer and server traffic with operator connectivity. The
> overshooting carefully is a requirement, and that's a reason why the algorithm
> measures queue-build up by increasing delay and classifies it as a congestion
> indication (as I think do some or all of the latest transport protocols too,
> without comparing the detailed algos). Fairness against competing transport
> hasn't been tested. In my eyes, such a test needs to compare commodity speed
> test vs. competing transport and then IP-capacity metric traffic vs. competing
> transport. Both, commodity speed-test and IP-capacity metric traffic decrease
> the bandwidth of competing transport. One of both might have a significantly
> stronger impact. I don't judge here, as I'm not aware of tests, but I also
> don't think it's ok to state commodity speed tests are fair, as they are TCP
> based. Their design goal is to reliably congest the receiver access until
> drops occur.
> 
> Udpst is open source. In general, and not addressing you personally, I'd
> appreciate all improvements to reduce total test load and test duration, and
> dealing with background traffic or (more generic) dealing with feedback
> indicating varying IP-capacity during a single test session at the receiver.
> If the result is "available capacity" because of such a time-varying receiver
> IP capacity, then that's the result and the test stops. I agree, that a test
> should not try to determine IP access capacity "by all means" (and I think
> neither does the text provided, nor does the implementation). 
> 
> Regards,
> 
> Rüdiger
>  
> 
> > strive
> > is

---

From: "MORTON, ALFRED C (AL)" <[email-masked]>
Date: Wed, 31 Mar 2021 23:37:46 +0000

Hi Magnus, Rüdiger, and all,

Grabbing a couple of recent comments for reply:

> From: ippm [mailto:[email-masked]] On Behalf Of Magnus Westerlund
> Sent: Wednesday, March 31, 2021 10:28 AM
<snip>
> endpoint you don't have that many basic indicators of congestions.
[acm] 
OK, we don't mention ECN right now.

> existing elastic traffic out of the way in the probing to determine
> maximum capacity.
[acm] 
Right.

In Version 08, we have this statement in the Scope/Applicability section regarding the 
algorithm:

   The load rate adjustment algorithm's goal is to determine the Maximum
   IP-Layer Capacity in the context of an infrequent, diagnostic, short
   term measurement.  It is RECOMMENDED to discontinue non-measurement
   traffic that shares a subscriber's dedicated resources while testing.

We can express the limitations more clearly in the first sentence, 
and augment the second sentence as follows:

The load rate adjustment algorithm's scope is limited to helping determine the Maximum IP-Layer Capacity in the context of an infrequent, diagnostic, short term measurement. It is RECOMMENDED to discontinue non-measurement traffic that shares a subscriber's dedicated resources while testing: measurements may not be accurate and throughput of competing elastic traffic may be greatly reduced.

> On Wed, 2021-03-31 at 09:30 +0000, [email-masked] wrote:
...
> > Gesendet: Mittwoch, 31. März 2021 09:43
...
> > Hi Ruediger,
...
> > I am not after improvements of the algorithms I after accruate representation
> > of the algorithm and intended usage. From my perspective this algorithm does not
> > algorithms. And I am fine with that as long as it is clear on the box
> > what it does. Have anyone run any tests of how this beahves when competing with
> > other congestion controls, like CUBIC, BBR(1/2), LEDBAT etc? I am not asking for it
[acm] 
Len has run some tests using the running code (version 4.5 at that time) with 
10 competing TCP (iperf) flows using CUBIC CCA on a 500Mbps rate-limited  access.
This is a rather extreme scenario for competing traffic. 

With the default load rate adjustment parameters, it wasn't possible to starve all 
the elastic traffic in a test using 10 second duration. However, an algorithm 
modification to retain fast rate increases following congestion produced more 
accurate test results during a 10 second test. This modification has shown some value
in other extreme circumstances, such as inter-continental testing [see Ref Y.Sup60].

Hope this helps,
Al