Subject: [alto] Comment on draft-ietf-alto-performance-metric-09

From: Martin Duke <[email-masked]>
Date: Fri, 17 Apr 2020 12:45:15 -0700

This is not a full review. I'm late to this and I apologize if I'm covering
ground which has been fully resolved before. With all hats off...

It has come up in the working group before that some of the cost metrics
can vary quite a lot over time, much more quickly than the scale at which
ALTO would update them.

Have we considered changing the metrics to have more long-lived properties
while still being useful? For example
1. we could split the various delay metrics into min and max versions. Min
and Max delay should be relatively constant over a given topology. Max
delay maps quite nicely to something like an SLA.
2. packet loss -- can we focus this on link layer losses rather than queue
drops? Most simply, this could just be a boolean that says "this path
frequently has losses not related to congestion" but could also be expanded
into a link packet error rate, if valuable. Congestion based losses are of
course dependent on the application.

Thanks
Martin

---

From: "Y. Richard Yang" <[email-masked]>
Date: Fri, 17 Apr 2020 16:47:11 -0400

Hi Martin,

Thank you so much for the review! The authors (Luis, Sabine, ...) just had
a review meeting this morning and your review is very timely.

Please see below.

On Fri, Apr 17, 2020 at 3:45 PM Martin Duke <[email-masked]> wrote:

> This is not a full review. I'm late to this and I apologize if I'm
> covering ground which has been fully resolved before. With all hats off...
>

Indeed. We discussed more stable statistics (e.g., min can be stable if it
reflects propagation delay and no major reroute, 95-percentile can be more
stable, ...).

> course dependent on the application.

This is an interesting point! I am not an expert on many link layer
technologies, but I assume that the loss (more like error) rate is more
stable in many settings. Early this week during the NSF Huge Data workshop,
one discussion item was that we are approaching a new state where many more
files can be corrupted but not detected by TCP checksum for large-scale
data transfers. It is not clear where the corruption happens (link link, a
buggy switch, ...), but I assume that it is more stable. It is a good
suggestion.

Back up at a slightly higher level, one key issue that we want to resolve
before we are finally happy to wrap up this document is whether we can
reduce the number of metrics, instead of having a plethora of metrics:

set of metrics (delay, loss, bandwidth, ...) X set of statistics (min, max,
percentile, avg) X set of measurement settings (ping each x sec)

>From your comments, one direction I see is to define a small subset of the
above cross-product space, as a first step. One we define a larger set but
make sure to include the more stable/more-likely-quickly-usable settings?

Thanks again for the timely feedback!

Richard