Subject: Re: [alto] I-D Action: draft-wu-alto-te-metrics-02.txt

From: Qin Wu <[email-masked]>
Date: Sat, 28 Jun 2014 03:20:29 +0000

Hi, all:
Here is the update to draft-wu-alto-te-metrics
http://tools.ietf.org/html/draft-wu-alto-te-metrics-02
The diff is:
http://www.ietf.org/rfcdiff?url2=draft-wu-alto-te-metrics-02
The main changes are
1. Remove the Cost Mode part as these metrics are intended to be used in various cost modes.
2. Move Collection method part to new section(i.e., section 2) since they are common part for these metrics.
3. Other editorial changes, e.g., remove duplicated part in each new defined metrics.

There is one remaining issue in this draft, i.e., do we need to specify the measurement interval for each metrics
In details in this draft.  

In the current draft, we assume using a fixed, small measurement interval.
However this may be not enough for ALTO setting. we may need to support more complicated use case,e.g., bandwidth calendaring.

In the last IETF meeting, we gave an example to show how bandwidth calendaring look like.
We may need to specify the more semantics for calendaring attribute, e.g., measurement period.
We may need to extend bandwidth calendaring to other metrics, e.g., delay.

After a few discussion, authors feels we may need to split calendaring part from this draft and put it into the separate draft.
Let us know what do you think about this?

Regards!
-Qin
 
Calendaring and multicost metrics should be defined in a separate document. Hence, we have a more modular approach. What do you think?
- We, as as the metrics are intended to be used in various cost modes. 
- It looks to me that the Collection Method part is common. Hence, we can move them to be common.
-----邮件原件-----
发件人: I-D-Announce [mailto:[email-masked]] 代表 [email-masked]
发送时间: 2014年6月28日 10:57
收件人: [email-masked]
主题: I-D Action: draft-wu-alto-te-metrics-02.txt

A New Internet-Draft is available from the on-line Internet-Drafts directories.

        Title           : ALTO Traffic Engineering Cost Metrics
        Authors         : Qin Wu
                          Y. Richard Yang
                          Young Lee
                          Dhruv Dhody
                          Sabine Randriamasy
	Filename        : draft-wu-alto-te-metrics-02.txt
	Pages           : 26
	Date            : 2014-06-27

Abstract:
   Cost Metric is a basic concept in Application-Layer Traffic
   Optimization (ALTO).  It is used in both the Cost Map Service and the
   Endpoint Cost Service.  Future extensions to ALTO may also use Cost
   Metric.

   Different applications may benefit from different Cost Metrics.  For
   example, a Resource Consumer may prefer Resource Providers that have
   low delay to the Resource Consumer.  However the base ALTO protocol
   [ALTO] has defined only a single cost metric, i.e., the generic
   "routingcost" metric (Sec. 14.2 of ALTO base specification [ALTO]).

   In this document, we define eleven Cost Metrics, derived from OSPF-TE
   and ISIS-TE, to measure network delay, jitter, packet loss, hop
   count, and bandwidth.  The metrics defined in this document provide a
   relatively comprehensive set of Cost Metrics for ALTO focusing on
   traffic engineering (TE).  Additional Cost Metrics such as financial
   cost metrics may be defined in other documents.

The IETF datatracker status page for this draft is:
https://datatracker.ietf.org/doc/draft-wu-alto-te-metrics/

There's also a htmlized version available at:
http://tools.ietf.org/html/draft-wu-alto-te-metrics-02

A diff from the previous version is available at:
http://www.ietf.org/rfcdiff?url2=draft-wu-alto-te-metrics-02

Please note that it may take a couple of minutes from the time of submission until the htmlized version and diff are available at tools.ietf.org.

Internet-Drafts are also available by anonymous FTP at:
ftp://ftp.ietf.org/internet-drafts/

_______________________________________________
I-D-Announce mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/i-d-announce
Internet-Draft directories: http://www.ietf.org/shadow.html or ftp://ftp.ietf.org/ietf/1shadow-sites.txt

---

From: "Y. Richard Yang" <[email-masked]>
Date: Fri, 27 Jun 2014 23:46:59 -0400

Thanks for posting the new version. Let me add my view on the issue when
defining the metrics. Please see below.

On Fri, Jun 27, 2014 at 11:20 PM, Qin Wu <[email-masked]> wrote:

> There is one remaining issue in this draft, i.e., do we need to specify
> the measurement interval for each metrics

However this may be not enough for ALTO setting. we may need to support
> more complicated use case,e.g., bandwidth calendaring.
>
>
I like it that you raise the issue on defining the semantics of revealed
metrics. For example, when an ALTO sever says that routingcost = 10 (or
delay = 10 ms using a new metric defined in the document), what does it
mean?  The charter of ALTO is that it does not deal with round-trip level
dynamic information. Hence, one should interpret the number as a
longer-time scale number. But how long? Does it reflect diurnal pattern?
Does it mean that the value is 10 in the last 15 sec, 30 sec, 1 min, 1
hour, 1 day, 1 week, etc.

Given the increasing interest in handling calendaring (e.g., there is a
paper in the coming SIGCOMM on WAN Calendaring), I feel that it is time to
be slightly more explicit. Long-term is not equivalent to be fuzzy.

Specifically, I feel that a reasonable metric should include the following:

- measurement start time, measurement finish time, the statistics operator
(e.g., avg, vs mean, vs x-percentile) or

- measurement window (assuming finish time is around the query time) and
the statistics operator.

Declaring such info can be tricky, as it may need to expand the grammar of
the IRD capabilities, but it adds clarify.

Make sense?

Richard

> After a few discussion, authors feels we may need to split calendaring
> part from this draft and put it into the separate draft.
> Calendaring and multicost metrics should be defined in a separate
> document. Hence, we have a more modular approach. What do you think?
> Please note that it may take a couple of minutes from the time of
> submission until the htmlized version and diff are available at
> tools.ietf.org.
>
> Internet-Draft directories: http://www.ietf.org/shadow.html or
> ftp://ftp.ietf.org/ietf/1shadow-sites.txt
>

-- 
-- 
 =====================================
| Y. Richard Yang <[email-masked]>   |
| Professor of Computer Science       |
| http://www.cs.yale.edu/~yry/        |
 =====================================

---

From: Qin Wu <[email-masked]>
Date: Sat, 28 Jun 2014 05:58:52 +0000

发件人: [email-masked] [mailto:[email-masked]] 代表 Y. Richard Yang
发送时间: 2014年6月28日 11:47
收件人: Qin Wu
抄送: [email-masked]; RANDRIAMASY, SABINE (SABINE)
主题: Re: I-D Action: draft-wu-alto-te-metrics-02.txt

Thanks for posting the new version. Let me add my view on the issue when defining the metrics. Please see below.

On Fri, Jun 27, 2014 at 11:20 PM, Qin Wu <[email-masked]<mailto:[email-masked]>> wrote:
Hi, all:

I like it that you raise the issue on defining the semantics of revealed metrics. For example, when an ALTO sever says that routingcost = 10 (or delay = 10 ms using a new metric defined in the document), what does it mean?  The charter of ALTO is that it does not deal with round-trip level dynamic information. Hence, one should interpret the number as a longer-time scale number. But how long? Does it reflect diurnal pattern? Does it mean that the value is 10 in the last 15 sec, 30 sec, 1 min, 1 hour, 1 day, 1 week, etc.

[Qin]:  I realized we should distinguish measure interval from how often the value is reported from the server to the client
Usually the value is measured at fixed measurement interval. When the server receive these values, they can do aggregation and report in the array of several measured value or report the latest value.

Given the increasing interest in handling calendaring (e.g., there is a paper in the coming SIGCOMM on WAN Calendaring), I feel that it is time to be slightly more explicit. Long-term is not equivalent to be fuzzy.

- measurement start time, measurement finish time, the statistics operator (e.g., avg, vs mean, vs x-percentile) or

- measurement window (assuming finish time is around the query time) and the statistics operator.

[Qin]: These are very closed to what we proposed in last London meeting when we presented this draft.
What is different, the measurement period is replaced with measurement window and you add a lot of statistics operators which looks good to me.

Declaring such info can be tricky, as it may need to expand the grammar of the IRD capabilities, but it adds clarify.

[Qin]: Agree.
Make sense?

[Qin]: Yes. So we address this issue in the separate calendaring document. Do we need to add reference to calendaring document in this draft?

Richard

Regards!
-Qin

-----邮件原件-----
发件人: I-D-Announce [mailto:[email-masked]<mailto:[email-masked]>] 代表 [email-masked]<mailto:[email-masked]>
发送时间: 2014年6月28日 10:57
收件人: [email-masked]<mailto:[email-masked]>
主题: I-D Action: draft-wu-alto-te-metrics-02.txt

Abstract:

Please note that it may take a couple of minutes from the time of submission until the htmlized version and diff are available at tools.ietf.org<http://tools.ietf.org>.

[email-masked]<mailto:[email-masked]>

--
--
 =====================================
| Y. Richard Yang <[email-masked]<mailto:[email-masked]>>   |
| Professor of Computer Science       |
| http://www.cs.yale.edu/~yry/        |
 =====================================

---

From: Qin Wu <[email-masked]>
Date: Sat, 28 Jun 2014 06:43:45 +0000

A minor update to v(-02) to fix some nits.
http://tools.ietf.org/html/draft-wu-alto-te-metrics-03

Regards!
-Qin
-----邮件原件-----
发件人: alto [mailto:[email-masked]] 代表 Qin Wu
发送时间: 2014年6月28日 11:20
收件人: [email-masked]
主题: Re: [alto] I-D Action: draft-wu-alto-te-metrics-02.txt

Hi, all:

There is one remaining issue in this draft, i.e., do we need to specify the measurement interval for each metrics In details in this draft.  

Regards!
-Qin
 
-----邮件原件-----
发件人: I-D-Announce [mailto:[email-masked]] 代表 [email-masked]
发送时间: 2014年6月28日 10:57
收件人: [email-masked]
主题: I-D Action: draft-wu-alto-te-metrics-02.txt

Abstract:

alto mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/alto

---

From: "RANDRIAMASY, SABINE (SABINE)" <[email-masked]>
Date: Mon, 30 Jun 2014 11:30:07 +0000

Hi Qin, Richard,

Thank you for this discussion and the ALTO Cost value description attributes. Please see below,
Best regards
Sabine

De : Qin Wu [mailto:[email-masked]]
Envoyé : samedi 28 juin 2014 07:59
À : Y. Richard Yang
Cc : [email-masked]; RANDRIAMASY, SABINE (SABINE)
Objet : RE: I-D Action: draft-wu-alto-te-metrics-02.txt

发件人: [email-masked]<mailto:[email-masked]> [mailto:[email-masked]] 代表 Y. Richard Yang
发送时间: 2014年6月28日 11:47
收件人: Qin Wu
抄送: [email-masked]<mailto:[email-masked]>; RANDRIAMASY, SABINE (SABINE)
主题: Re: I-D Action: draft-wu-alto-te-metrics-02.txt

[Qin]:  I realized we should distinguish measure interval from how often the value is reported from the server to the client
Usually the value is measured at fixed measurement interval. When the server receive these values, they can do aggregation and report in the array of several measured value or report the latest value.
[    SR ]
[   SR  ] Indeed, each ALTO Server may decide to aggregate measured values over intervals that it specifies and are usually longer than the RFC6390  « measurement timing ».  At least because ALTO is no real time information service and will unveil TE metric information only if abstracted. On the other hand, TE values implicitly call for some reliability and accuracy and an ALTO Client does not want to make too frequent requests and should be able to evaluate the level of accuracy and reliability of the provided ALTO TE  values. This positions  « ALTO TE metric”  w.r.t  classical « TE metrics » specified e.g. in other [RFC3630], [RFC3784] or [BGP-LS].

[Qin]: These are very closed to what we proposed in last London meeting when we presented this draft.
What is different, the measurement period is replaced with measurement window and you add a lot of statistics operators which looks good to me.
[     ]
[     ] agree, this is necessary for the ALTO Client to interpret the value it gets. In addition to such information, I would like to propose the following to the list and ask for feedback:

Each metric described in the draft has a set of sections on Metric Description Fields (MDF for short): name, string, description, unit etc... I suggest to add for each metric, a MDF field called for example "ALTO sample interval". This field would give a clue on the accuracy of the measurement.  Some text to start with could be for example:

"ALTO Sample interval:
The ALTO Server may collect values from the sources measurements done e.g. every 1 second  over a period of 30 seconds. The ALTO Server may then aggregate these values over ALTO Sample intervals of at least 30 seconds and provide updates every hour. The ALTO Client may then assume that the ALTO TE values provided in an ALTO Sample interval are stationary.". For values with “ short stationarity” and also to minimize ALTO transactions, an alternative may be that the ALTO Server supports the ALTO Cost Calendaring service.

What do you think?

[Qin]: Agree.
[     ] There may be indeed the necessity to enrich the IRD grammar with the ALTO Sample interval field and another field called for example "ALTO update interval". The latter field though is not specific to ALTO TE metrics as it applies to most other ALTO information.

Make sense?

Richard

Regards!
-Qin

-----邮件原件-----
发件人: I-D-Announce [mailto:[email-masked]<mailto:[email-masked]>] 代表 [email-masked]<mailto:[email-masked]>
发送时间: 2014年6月28日 10:57
收件人: [email-masked]<mailto:[email-masked]>
主题: I-D Action: draft-wu-alto-te-metrics-02.txt

Abstract:

---

From: Qin Wu <[email-masked]>
Date: Mon, 30 Jun 2014 12:08:52 +0000

[Qin]: I think it is better not to invent too many new terminologies. If my understanding is correct, ALTO sample interval is equivalent to time window proposed by Richard.

"ALTO Sample interval:
The ALTO Server may collect values from the sources measurements done e.g. every 1 second  over a period of 30 seconds.

[Qin]: If my understanding is correct, 1 second is measurement interval while  a period of 30 seconds is time windows.
The ALTO Server may then aggregate these values over ALTO Sample intervals of at least 30 seconds and provide updates every hour.

[Qin]: when you say provide update every hour, are you saying ALTO server provide update to ALTO client even thought ALTO client didn’t send request to ALTO server or
ALTO server update the existing cost values with the new ones, I believe it is the latter.
So here the one hours can be called as update interval, I agree.

The ALTO Client may then assume that the ALTO TE values provided in an ALTO Sample interval are stationary.". For values with “ short stationarity” and also to minimize ALTO transactions, an alternative may be that the ALTO Server supports the ALTO Cost Calendaring service.

[Qin]: I am confused when you say there is alternative, i.e., ALTO cost calendaring service, how ALTO cost calendaring is different from the example you give above.
Can I understand in the cost calendaring service, update interval is not needed?

What do you think?

[Qin]: Agree.

===============