Subject: Re: [alto] ALTO Draft ReCharter WG review

From: [email-masked]
Date: Wed, 3 Mar 2021 21:39:30 +0800 (GMT+08:00)

Dear all,

Below are some comments on the 2nd item in the recharter text.

As far as I know, the ALTO incremental update extension (RFC 8895) already provides a mechanism to enable the "pub-sub" of ALTO information, using Server-Sent Events (SSE). I see there are multiple directions indicated by the new charter item:

1. Decouple the "pub-sub" protocol with the underlying mechanism.

Besides SSE, other mechanisms can also be used to realize the "pub-sub" of ALTO information, such as HTTP/2, HTTP/3 or the methods mentioned in the charter text. Thus, a direct extension is to define the abstract format of control messages and data messages (i.e., WHAT information should be provided but not HOW), and allow different underlying protocols to use protocol-specific encodings.

For example, SSE encodes the metadata (e.g., content-type and stream id) and the content of an event using "event:" and "data:" prefixes at the beginning of each line, and uses empty lines to indicate the end of a message, while HTTP/2 (RFC 7540) may encode the metadata and the content of an event using PUSH_PROMISE/HEADERS and DATA frame .

I think this requirement may help integrating ALTO in network management platforms (such as OpenDaylight, Kubernetes, and ETSI ZSM*) which design their own pub-sub systems for reasons such as consistency or ease of development. It would be great if there is an interest in this direction from companies/organizations.

* The integration fabric of ETSI ZSM provides pub-sub support but ZSM also allows services to use their own pub-sub mechanisms.

2. Enable more fine-grained control of pub-sub.

In RFC 8895, there are two types of commands which only defines WHAT information to subscribe:

- add: Make one or more new requests to receive the incremental updates.

- remove: Terminate the subscription of one or more previously-made requests.

In the meantime, the updates will be continuously sent to the client whenever a server sees fit.

The charter text proposes to enable ALTO clients to request and receive "a diverse types (such as event-triggered/sporadic, continuous), continuous, customized feed of publisher-generated information". It seems to me that the new extension wants to allow clients to specify not only WHAT information to be subscribed but also WHEN/HOW the information should be delivered (e.g., Notify me the latest value every 5 second.).

Personally I find both directions to be interesting and useful. It would be great if they can be supported by real use cases.

Just my two cents.

Best,

Kai

-----Original Messages-----
From:"Qin Wu" <[email-masked]>
Sent Time:2021-02-22 21:50:44 (Monday)
To: "IETF ALTO" <[email-masked]>
Cc: "[email-masked]" <[email-masked]>, "[email-masked]" <[email-masked]>
Subject: [alto] ALTO Draft ReCharter WG review

Hi, :

We have requested one hour session for ALTO WG meeting in the upcoming IETF 110, which is arranged on Friday, March 12, 14:30-15:30(UTC).

The goal is to boil down ALTO recharter and have consensus on charter contents in IETF 110.

To get this goal, an updated inline draft charter text for ALTO has just been posted to this list,

This charter has received a couple of rounds of informal review from WG members, chairs and our Ads from brief to deep thorough, 5 new chartered items have been listed.

We would like to solicit feedback on these new chartered items and your use case, deployment, idea corresponding to these new chartered items.

Sharing your past deployment story will also be appreciated.

============================================================================================

The ALTO working group was established in 2008 to devise a request/response protocol to allow a host to benefit from a server that is more cognizant of the network infrastructure than the host is.

The working group has developed an HTTP-based protocol and recent work has reported large-scale deployment of ALTO based solutions supporting applications such as content distribution networks (CDN).

ALTO is now proposed as a component for cloud-based interactive applications, large-scale data analytics, multi-cloud SD-WAN deployment, and distributed

computing. In all these cases, exposing network information such as abstract topologies and network function deployment location helps applications.

To support these emerging uses, extensions are needed, and additional functional and architectural features need to be considered as follows:

o Protocol extensions to support a richer and extensible set of policy attributes in ALTO information update request and response. Such policy attributes may indicate information dependency (e.g., ALTO path-cost/QoS properties with dependency on real-time network  indications), optimization criteria (e.g., lowest latency/throughput network performance objective), and constraints (e.g., relaxation bound of optimization criteria, domain or network node to be traversed, diversity and redundancy of paths).

o Protocol extensions for facilitating operational automation tasks and improving transport efficiency. In particular, extensions to provide "pub/sub" mechanisms to allow the client to request and receive a diverse types (such as event-triggered/sporadic, continuous), continuous, customized feed of publisher-generated information. Efforts developed in other working groups such as MQTT Publish / Subscribe Architecture, WebSub, Subscription to YANG Notifications will be considered, and issues such as scalability (e.g., using unicast or broadcast/multicast, and periodicity of object updates) should be considered.

o The working group will investigate the configuration, management, and operation of ALTO systems and may develop suitable data models.

o Extensions to ALTO services to support multi-domain settings. ALTO is currently specified for a single ALTO server in a single administrative domain, but a network may consist of

multiple domains and the potential information sources may not be limited to a certain domain. The working group will investigate extending the ALTO framework to (1) specify multi-ALTO-server protocol flow and usage guidelines when an ALTO service involves network paths spanning multiple domains with multiple ALTO servers, and (2) extend or introduce ALTO

services allowing east-west interfaces for multiple ALTO server integration and collaboration. The specifications and extensions should use existing services whenever possible. The specifications and extensions should consider realistic complexities including incremental deployment, dynamicity, and security issues such as access control, authorization (e.g., an ALTO server provides information for a network that the server has no authorization), and privacy protection in multi-domain settings.

o The working group will update RFC 7971 to provide operational considerations for recent protocol extensions (e.g., cost calendar, unified properties, and path vector) and new extensions that the WG develops. New considerations will include decisions about the set of information resources (e.g., what metrics to use), notification of changes either in proactive or reactive mode (e.g., pull the backend, or trigger just-in-time measurements), aggregation/processing of the collected information  (e.g., compute information and network information )according to the clients’ requests, and integration with new transport mechanisms (e.g., HTTP/2 and HTTP/3).

When the WG considers standardizing information that the ALTO server could provide, the following criteria are important

to ensure real feasibility:

- Can the ALTO server realistically provide (measure or derive) that information?

- Is it information that the ALTO client cannot find easily some other way?

- Is the distribution of the information allowed by the operator of the network? Does the exposure of the information introduce privacy and information leakage concerns?

Issues related to the specific content exchanged in systems that make use of ALTO are excluded from the WG's scope, as is the issue of dealing with enforcing the legality of the content. The WG will also not propose standards on how congestion is signaled, remediated, or avoided.

-Qin Wu (on behalf of chairs)

---

From: [email-masked]
Date: Wed, 3 Mar 2021 21:53:17 +0800 (GMT+08:00)

By the way, below are the links to some documents that are mentioned in the previous email.

[1] https://tools.ietf.org/html/rfc8895

RFC 8895: Application-Layer Traffic Optimization (ALTO) Incremental Updates Using Server-Sent Events (SSE)

[2] https://tools.ietf.org/html/rfc7540

RFC 7540: Hypertext Transfer Protocol Version 2 (HTTP/2)

[3] https://www.etsi.org/deliver/etsi_gs/ZSM/001_099/002/01.01.01_60/gs_ZSM002v010101p.pdf

ETSI ZSM Architecture (See B.4 and B.5 for pub-sub in ZSM)

[4] https://docs.opendaylight.org/projects/controller/en/latest/dev-guide.html#messaging-patterns

OpenDaylight MD-SAL

[5] https://kubemq.io/

KubeMQ: the native message broker for Kubernetes

[2]

-----Original Messages-----
From:[email-masked]
Sent Time:2021-03-03 21:39:30 (Wednesday)
To: "Qin Wu" <[email-masked]>
Cc: "[email-masked]" <[email-masked]>, "[email-masked]" <[email-masked]>, "IETF ALTO" <[email-masked]>
Subject: Re: [alto] ALTO Draft ReCharter WG review

Dear all,

Just my two cents.

Best,

Kai

-----Original Messages-----

Hi, :

---

From: Jensen Zhang <[email-masked]>
Date: Wed, 3 Mar 2021 23:33:06 +0800

Dear all,

I would like to make some comments on the 3rd recharter item.

This item is going to propose YANG data models for ALTO configuration and
management. Most of this kind of YANG data models for communication
protocols like PCEP [1] and HTTP [2] will support both client and server
configuration. One of the open issues for ALTO is:

*whether we should also provide the data model for the ALTO client
configuration*.

For some of the traditional ALTO use cases like p2p, I think the YANG data
model only for the ALTO server is enough. It can help the network operator
easily configure and manage ALTO services. The data model for the ALTO
client may not be necessary because the client is usually not under the
control of the network operator. However, there are two cases where people
may be interested in the data model for the ALTO client:

1. The multi-domain setting is a potential use case. But it depends on how
we are going to design the server-to-server communication.

  (a) If we are going to reuse the current ALTO framework, then the
architecture could be similar to the PCE-based architecture, i.e., each
ALTO domain should initiate an entity that can be an alto-server,
alto-client, or alto-server-and-client. And for the alto-client or
alto-server-and-client entity, the operator could configure the list of
peered alto-server/alto-server-and-client entity directly, or how to
discover the peers. The operator could also configure which information the
client entity is interested in and would like to fetch from the peers.

  (b) If we are going to completely redesign the communication protocol
among ALTO servers, we may need specific data models for the configuration
of this new protocol. The traditional roles of the ALTO client and server
may no longer be applicable.

2. The other use case could come from the network-application integration.
More specifically, the multi-service operator (e.g., Comcast, Telefonica)
who can offer both application service (e.g., TV, CDN) and network service
can be an example. In such a case, the application service operator may
have some collaboration with the network operator on the protocol
configuration level.

  For example, in a CDN-ISP collaboration setting (@Luis can comment on
it), the CDN operator may request the network operator to install a new
ALTO service to compute on-demand ALTO information resources based on new
parameters dynamically. And in the meantime, the CDN operator may also
configure its own ALTO client to periodically send requests to the new ALTO
service or use the pub/sub mechanism (e.g. SSE). And the CDN operator may
want the ALTO client to report some operational status/statistics like when
the last request is done, whether the last response is out-dated, how many
versions are updated for the current information resource (Not quite sure
if this info should be in the scope of the ALTO data model). It makes more
sense to do these kinds of things via the configuration protocol instead of
another ALTO protocol extension.

It would be great if people can share further comments or their own
interesting use cases.

[1] https://tools.ietf.org/html/draft-ietf-pce-pcep-yang-15
[2] https://tools.ietf.org/html/draft-ietf-netconf-http-client-server-06

Best regards,
Jensen

On Mon, Feb 22, 2021 at 9:51 PM Qin Wu <[email-masked]> wrote:

> Hi, :
>
> We have requested one hour session for ALTO WG meeting in the upcoming
> IETF 110, which is arranged on Friday, March 12, 14:30-15:30(UTC).
>
> We would like to solicit feedback on these new chartered items and your
> use case, deployment, idea corresponding to these new chartered items.
>
> The ALTO working group was established in 2008 to devise a
> request/response protocol to allow a host to benefit from a server that is
> more cognizant of the network infrastructure than the host is.
>
>
>
> The working group has developed an HTTP-based protocol and recent work has
> reported large-scale deployment of ALTO based solutions supporting
> applications such as content distribution networks (CDN).
>
>
>
> ALTO is now proposed as a component for cloud-based interactive
> applications, large-scale data analytics, multi-cloud SD-WAN deployment,
> and distributed
>
> computing. In all these cases, exposing network information such as
> abstract topologies and network function deployment location helps
> applications.
>
>
>
> To support these emerging uses, extensions are needed, and additional
> functional and architectural features need to be considered as follows:
>
>
>
> o Protocol extensions to support a richer and extensible set of policy
> attributes in ALTO information update request and response. Such policy
> attributes may indicate information dependency (e.g., ALTO path-cost/QoS
> properties with dependency on real-time network  indications), optimization
> criteria (e.g., lowest latency/throughput network performance objective),
> and constraints (e.g., relaxation bound of optimization criteria, domain or
> network node to be traversed, diversity and redundancy of paths).
>
>
>
> o Protocol extensions for facilitating operational automation tasks and
> improving transport efficiency. In particular, extensions to provide
> "pub/sub" mechanisms to allow the client to request and receive a diverse
> types (such as event-triggered/sporadic, continuous), continuous,
> customized feed of publisher-generated information. Efforts developed in
> other working groups such as MQTT Publish / Subscribe Architecture, WebSub,
> Subscription to YANG Notifications will be considered, and issues such as
> scalability (e.g., using unicast or broadcast/multicast, and periodicity of
> object updates) should be considered.
>
>
>
> o The working group will investigate the configuration, management, and
> operation of ALTO systems and may develop suitable data models.
>
>
>
> o Extensions to ALTO services to support multi-domain settings. ALTO is
> currently specified for a single ALTO server in a single administrative
> domain, but a network may consist of
>
> multiple domains and the potential information sources may not be limited
> to a certain domain. The working group will investigate extending the ALTO
> framework to (1) specify multi-ALTO-server protocol flow and usage
> guidelines when an ALTO service involves network paths spanning multiple
> domains with multiple ALTO servers, and (2) extend or introduce ALTO
>
> services allowing east-west interfaces for multiple ALTO server
> integration and collaboration. The specifications and extensions should use
> existing services whenever possible. The specifications and extensions
> should consider realistic complexities including incremental deployment,
> dynamicity, and security issues such as access control, authorization
> (e.g., an ALTO server provides information for a network that the server
> has no authorization), and privacy protection in multi-domain settings.
>
>
>
> o The working group will update RFC 7971 to provide operational
> considerations for recent protocol extensions (e.g., cost calendar, unified
> properties, and path vector) and new extensions that the WG develops. New
> considerations will include decisions about the set of information
> resources (e.g., what metrics to use), notification of changes either in
> proactive or reactive mode (e.g., pull the backend, or trigger just-in-time
> measurements), aggregation/processing of the collected information  (e.g.,
> compute information and network information )according to the clients’
> requests, and integration with new transport mechanisms (e.g., HTTP/2 and
> HTTP/3).
>
>
>
> When the WG considers standardizing information that the ALTO server could
> provide, the following criteria are important
>
> - Is the distribution of the information allowed by the operator of the
> network? Does the exposure of the information introduce privacy and
> information leakage concerns?
>
>
>
> Issues related to the specific content exchanged in systems that make use
> of ALTO are excluded from the WG's scope, as is the issue of dealing with
> enforcing the legality of the content. The WG will also not propose
> standards on how congestion is signaled, remediated, or avoided.
>
>
>
> _______________________________________________
> alto mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/alto
>

---

From: Dhruv Dhody <[email-masked]>
Date: Thu, 4 Mar 2021 11:37:19 +0530

Hi Jensen,

Thanks for starting this thread. It is great that you are thinking of
similar questions as I was :)

On Wed, Mar 3, 2021 at 9:03 PM Jensen Zhang <[email-masked]> wrote:
> This item is going to propose YANG data models for ALTO configuration and management. Most of this kind of YANG data models for communication protocols like PCEP [1] and HTTP [2] will support both client and server configuration. One of the open issues for ALTO is:
>
> whether we should also provide the data model for the ALTO client configuration.
>

If the ALTO client is a function that can be placed in an entity that
uses YANG-based techniques for configuration, status check, and
monitoring. It makes sense for us to provide support for ALTO-client
in YANG model. Some of the use-cases you are already highlighting
below. For some like P2P tracker not soo much!

> For some of the traditional ALTO use cases like p2p, I think the YANG data model only for the ALTO server is enough. It can help the network operator easily configure and manage ALTO services. The data model for the ALTO client may not be necessary because the client is usually not under the control of the network operator. However, there are two cases where people may be interested in the data model for the ALTO client:
>
> 1. The multi-domain setting is a potential use case. But it depends on how we are going to design the server-to-server communication.
>
>   (a) If we are going to reuse the current ALTO framework, then the architecture could be similar to the PCE-based architecture, i.e., each ALTO domain should initiate an entity that can be an alto-server, alto-client, or alto-server-and-client. And for the alto-client or alto-server-and-client entity, the operator could configure the list of peered alto-server/alto-server-and-client entity directly, or how to discover the peers. The operator could also configure which information the client entity is interested in and would like to fetch from the peers.
>
>   (b) If we are going to completely redesign the communication protocol among ALTO servers, we may need specific data models for the configuration of this new protocol. The traditional roles of the ALTO client and server may no longer be applicable.
>

Yes the YANG needs to follow whatever decision is taken regarding the
ALTO server-to-server communication. There is also a high-level
decision that if we have a single YANG model for ALTO that can be used
by both client and server or have independent yang models: one for
client and another for server.

BTW RFC 7971 includes this -

       Cascaded servers: An ALTO server may itself include an ALTO
       client and query other ALTO servers, e.g., for certain
       destinations.  This results is a cascaded deployment of ALTO
       servers, as further explained below.

> 2. The other use case could come from the network-application integration. More specifically, the multi-service operator (e.g., Comcast, Telefonica) who can offer both application service (e.g., TV, CDN) and network service can be an example. In such a case, the application service operator may have some collaboration with the network operator on the protocol configuration level.
>
>   For example, in a CDN-ISP collaboration setting (@Luis can comment on it), the CDN operator may request the network operator to install a new ALTO service to compute on-demand ALTO information resources based on new parameters dynamically. And in the meantime, the CDN operator may also configure its own ALTO client to periodically send requests to the new ALTO service or use the pub/sub mechanism (e.g. SSE). And the CDN operator may want the ALTO client to report some operational status/statistics like when the last request is done, whether the last response is out-dated, how many versions are updated for the current information resource (Not quite sure if this info should be in the scope of the ALTO data model). It makes more sense to do these kinds of things via the configuration protocol instead of another ALTO protocol extension.
>

Using the YANG model for monitoring purposes (status, error,
statistics) at the ALTO client/server is quite useful.

Thanks,
Dhruv

> It would be great if people can share further comments or their own interesting use cases.
>

---

From: Jensen Zhang <[email-masked]>
Date: Thu, 4 Mar 2021 15:17:29 +0800

Hi Dhruv,

Thanks a lot for your feedback.

On Thu, Mar 4, 2021 at 2:07 PM Dhruv Dhody <[email-masked]> wrote:

> > This item is going to propose YANG data models for ALTO configuration
> and management. Most of this kind of YANG data models for communication

Thanks for your help to clarify the requirement on when we need the YANG
model.

>
> > For some of the traditional ALTO use cases like p2p, I think the YANG
> data model only for the ALTO server is enough. It can help the network
> operator easily configure and manage ALTO services. The data model for the
> ALTO client may not be necessary because the client is usually not under
> the control of the network operator. However, there are two cases where
> people may be interested in the data model for the ALTO client:
> >
> > 1. The multi-domain setting is a potential use case. But it depends on
> how we are going to design the server-to-server communication.
> >

My personal feeling is that the configuration requirements for the ALTO
server and client are very different. e.g., The ALTO server needs to be
configured which services it can provide, while the ALTO client needs to be
configured which servers and services it would like to request. So I'm not
sure if we should define a single YANG model for both client and server.
But I notice that PCEP defines a single model for both PCE and PCC. From
your experience, which decision is better for ALTO?

>        servers, as further explained below.
Yes, this is a simple or special case of the multi-domain setting. In this
case, the ALTO server can use the legacy ALTO protocol to communicate with
other ALTO servers. But there are some limitations in more practical
settings. So the 4th recharter item is proposed to try to address them.

>
> > 2. The other use case could come from the network-application
> integration. More specifically, the multi-service operator (e.g., Comcast,
> Telefonica) who can offer both application service (e.g., TV, CDN) and
> network service can be an example. In such a case, the application service
> operator may have some collaboration with the network operator on the
> protocol configuration level.
> >

Thanks for your suggestion. From your experience, which kind of information
should not be monitored using the YANG model? For example, if the client
wants to record all the historical requests, should we use the YANG model
to do it? Or we should only use the YANG model to track the immediate
status?

Thanks,
Jensen

> >> This charter has received a couple of rounds of informal review from WG
> members, chairs and our Ads from brief to deep thorough, 5 new chartered
> items have been listed.
> >>
> >> The working group has developed an HTTP-based protocol and recent work
> has reported large-scale deployment of ALTO based solutions supporting
> >> multiple domains and the potential information sources may not be
> limited to a certain domain. The working group will investigate extending
> the ALTO framework to (1) specify multi-ALTO-server protocol flow and usage
> >> When the WG considers standardizing information that the ALTO server
> could provide, the following criteria are important
> >>
> >> Issues related to the specific content exchanged in systems that make
> use of ALTO are excluded from the WG's scope, as is the issue of dealing
> with enforcing the legality of the content. The WG will also not propose

---

From: Dhruv Dhody <[email-masked]>
Date: Thu, 4 Mar 2021 23:51:23 +0530

Hi Jensen,

<snip>

> your experience, which decision is better for ALTO?

Dhruv: To me, this would be dependent on what we do with ALTO server-to-server
communication. If one server is acting as a client and would need
management in the same way as a stand-alone ALTO client, then a single
model helps avoid duplication.

<snip>

> Thanks for your suggestion. From your experience, which kind of
> information should not be monitored using the YANG model? For example, if
> the client wants to record all the historical requests, should we use the
> YANG model to do it? Or we should only use the YANG model to track the
> immediate status?
>
>
Dhruv: I doubt the ALTO client remembers the past request. YANG cant model
data that doesn't exist. Correct me if I am mistaken. The immediate status,
the statistics, the last error, is what you would usually find!

Thanks!
Dhruv

Thanks,
>> https://tools.ietf.org/html/draft-ietf-netconf-http-client-server-06
>> >
>> > Best regards,
>> > Jensen
>> >
>> >
>> >> This charter has received a couple of rounds of informal review from
>> WG members, chairs and our Ads from brief to deep thorough, 5 new chartered
>> items have been listed.
>> >>
>> >> We would like to solicit feedback on these new chartered items and
>> your use case, deployment, idea corresponding to these new chartered items.
>> >>
>> >> o Protocol extensions for facilitating operational automation tasks
>> and improving transport efficiency. In particular, extensions to provide
>> >> o The working group will investigate the configuration, management,
>> and operation of ALTO systems and may develop suitable data models.
>> >>
>> >>
>> >>
>> >> o Extensions to ALTO services to support multi-domain settings. ALTO
>> is currently specified for a single ALTO server in a single administrative
>> >> - Is the distribution of the information allowed by the operator of
>> the network? Does the exposure of the information introduce privacy and