Subject: [alto] Fwd: ALTO at IETF-108: Finishing the current milestones and discussion on re-chartering the WG

From: Danny Alex Lachos Perez <[email-masked]>
Date: Tue, 14 Jul 2020 12:28:36 -0300

Dear all,

Following the suggestion of the AD (Martin Duke) and the WG chairs (Jan and
VIjay) [0], many of us started excellent discussions on ALTO recharter.
In particular, this thread discusses the multi-domain aspects of ALTO
(please, take a look below).
We are moving the thread to the WG mailing list for more open discussions.

Any thoughts/comments/suggestions are greatly appreciated,

Thank you so much!

Best regards,

Danny Lachos

[0] https://mailarchive.ietf.org/arch/msg/alto/bPUxI6xmbms7i6_eoPiOy9Bpxro/

---------- Forwarded message ---------
From: Ingmar Poese <[email-masked]>
Date: Tue, Jul 14, 2020 at 8:33 AM
Subject: Re: [alto] ALTO at IETF-108: Finishing the current milestones and
discussion on re-chartering the WG
To: Sebastian Kiesel <[email-masked]>, Danny Alex Lachos Perez <
[email-masked]>
Cc: Y. Richard Yang <[email-masked]>

Hi all,

please see inline.

Ingmar

Am 14.07.20 um 11:56 schrieb Sebastian Kiesel:
> Hi Danny, all,
>
> On Mon, Jun 29, 2020 at 08:18:04AM -0300, Danny Alex Lachos Perez wrote:
>> Hello Sebastian,
>>
>> Thanks a lot for your comments
>> Please, see inline:
>>
>> Even the very first ALTO use case was multi-domain in some sense:
>>> the bittorrent clients in the access networks of Comcast and Sprint,
>>> the pirate bay tracker in Sweden, that could be seen as three domains.
>>> The examples in the drafts you have listed are obviously different and
>>> more complex, but I am still struggling to tell what exactly is new and
>>> needs to be done.
>>
>> Yes, I remember your question. I will try to clarify our multi-domain
>> approach.
>>  From your P2P example, we have the next figure:
>> An ALTO server in each domain will provide only local information to ALTO
>> clients. In the example, the tracker (ALTO client) receives
>> topology-/policy-related information of a single domain (AS1 or AS2).
Let's
>> call it "single-domain information exposure".
>
> While your observation is probably true in most deployments we have seen
> so far, I do not completely agree on an architectural level.
>
> We have designed the ALTO protocol such that an ALTO client may ask one
> ALTO server *any* question, e.g., "give me network map and cost map for
> the whole Internet address space" or ECS(IP1, IP2) where IP1 and IP2
> are arbitrary IP addresses in whatever domain.
> The idea that there might be more than one ALTO server, and that one
> ALTO server might give better answers to some specific questions while
> another ALTO server might give better answers to other questions,
> was in our mind when we worked on the protocol, but it did not result
> in any specific feature of the base protocol that would deal with that
> situation.
I would actually like to go a little further on this point. At the early
stages, if i remember correctly, there was the discussion on "my view on
the Internet". I would like to point this out again and the fact that
"The Internet" looks different from different vantage points. Depending
on where you are, decisions are different. Furthermore, a client is not
capable of deciding which information to use from what ALTO server - and
it if get multiple answers, which one is "more" correct.

If the ALTO Server now only has information about the local network, or
only gives out information about the local network, matching this to
interAS client (or even clients in the same confederation) almost
impossible - and this gets worse if not all Networks offer an ALTO service.
I think an ALTO server should always give an indication on the entire
Internet from it's point of view, even if much of it is aggregated. This
reduces the problems to "what ALTO server to ask" for the client and can
be much easier deployed and maintained on a large scale.

>
> This is different for the discovery mechansims: there we had the idea
> that an ALTO server that is "near" to the endpoint of the data
> transmissions to be optimized, typically announced by the operator of
> the access network, will probably give good answers.
> If the "peer selection" (or any other routing decision in the overlay)
> is done in the endpoint of the data transmission itself, use RFC 7286;
> if it is done at a remote entity such as a tracker, use RFC 8686 to find
> ALTO server that is nearby the actual endpoint.
>
>
>> On the other hand, if we want an ALTO client to receive merged
information
>> from multiple domains, ALTO servers need to be able to exchange
>> information. In the example, the tracker will receive AS1’s and AS2’s
>> merged information. Let's call it "multi-domain information exposure".
>> As we identified [0], different use cases could benefit from multi-domain
>> information exposure using ALTO. However, it also places requirements
that
>> the current ALTO design do not satisfy [1].
>
> There is definitively room for extensions. We have so many architectural
> options (side note: what is now just the "ALTO protocol" has been called
> in RFC 5693 and 6708 the "ALTO client protocol", in anticipation that
> there might be an ALTO server-to-server protocol in the future).
>
>
> First area of questions: Data model.  If every ALTO server defines the
> network map on its own and uses the default smaller-value-is-better
> routingcost metric, which does not have a well-defined base unit, it
> will be hard to impossible to merge information from different servers.
> Deprecate them?  Replace with what? Or go for an architectural option
> (see below) that does not require information merging on the ALTO
> protocol side.
This is an interesting problem that i would like to go into further. But
then, i have a very narrow Agenda on our use case, so maybe i am
overshooting here.

>
> Second area of questions: placement of entities and protocols between
> them. This is also not completely new, see Sec. 2.2.3 of RFC 7971
>
> Ideas:
>
> 1. Define a new server discovery mechanism.  An ALTO client would
> discover several ALTO servers from different domains, ask each of them,
> and then aggregate the information on the client-side.
>
> 2. Mesh-style ALTO server-to-server communication.  Do we need a new
> s2s protocol?  Is a small extension to the ALTO protocol sufficient?
>
> 3. Hierarchical structure with backend information sources and frontend
> servers towards the ALTO clients. Aggregation of the information from
> different backends takes place in the frontends. How to ensure
> consistency? Do we need a new protocol for backend to frontend
> communication? Is a small extension to the ALTO protocol sufficient?
>
>
>  From an academic perspective, I'd love to investigate and specify each
> of these options and many more.  But from an IETF perspective: who is
> willing to deploy anything on a scale that requires and justifies the
> effort for standardization?  If there is one: what is their preference
> which way to go?  Is your group planning a deployment?
We (BENOCS) have had that on our ToDo list for about a year now, but we
have not come around to implementing it so far. The deployment would
encase about 100 million subscribers towards a few Hypergiants across
Europe.

I have a fairly good plan on how to do it - although we were not going
to use ALTO for cross domain ALTO (we call it cascading internally), but
rather implement something ourselves that fits our use case of
Hypergiant<->end-user mapping (we do not do P2P).

I would be open to discussing this in a broader sense to standardize the
approach.

---------- Forwarded message ---------
From: Sebastian Kiesel <[email-masked]>
Date: Tue, Jul 14, 2020 at 6:56 AM
Subject: Re: [alto] ALTO at IETF-108: Finishing the current milestones and
discussion on re-chartering the WG
To: Danny Alex Lachos Perez <[email-masked]>
Cc: Ingmar Poese <[email-masked]>, Y. Richard Yang <[email-masked]>

Hi Danny, all,

On Mon, Jun 29, 2020 at 08:18:04AM -0300, Danny Alex Lachos Perez wrote:
> Hello Sebastian,
>
> Thanks a lot for your comments
> Please, see inline:
>
> Even the very first ALTO use case was multi-domain in some sense:
> > the bittorrent clients in the access networks of Comcast and Sprint,
> > the pirate bay tracker in Sweden, that could be seen as three domains.
> > The examples in the drafts you have listed are obviously different and
> > more complex, but I am still struggling to tell what exactly is new and
> > needs to be done.
>
> Yes, I remember your question. I will try to clarify our multi-domain
> approach.
> From your P2P example, we have the next figure:
> An ALTO server in each domain will provide only local information to ALTO
> clients. In the example, the tracker (ALTO client) receives
> topology-/policy-related information of a single domain (AS1 or AS2).
Let's
> call it "single-domain information exposure".

While your observation is probably true in most deployments we have seen
so far, I do not completely agree on an architectural level.

We have designed the ALTO protocol such that an ALTO client may ask one
ALTO server *any* question, e.g., "give me network map and cost map for
the whole Internet address space" or ECS(IP1, IP2) where IP1 and IP2
are arbitrary IP addresses in whatever domain.
The idea that there might be more than one ALTO server, and that one
ALTO server might give better answers to some specific questions while
another ALTO server might give better answers to other questions,
was in our mind when we worked on the protocol, but it did not result
in any specific feature of the base protocol that would deal with that
situation.

This is different for the discovery mechansims: there we had the idea
that an ALTO server that is "near" to the endpoint of the data
transmissions to be optimized, typically announced by the operator of
the access network, will probably give good answers.
If the "peer selection" (or any other routing decision in the overlay)
is done in the endpoint of the data transmission itself, use RFC 7286;
if it is done at a remote entity such as a tracker, use RFC 8686 to find
ALTO server that is nearby the actual endpoint.

> On the other hand, if we want an ALTO client to receive merged information
> from multiple domains, ALTO servers need to be able to exchange
> information. In the example, the tracker will receive AS1’s and AS2’s
> merged information. Let's call it "multi-domain information exposure".
> As we identified [0], different use cases could benefit from multi-domain
> information exposure using ALTO. However, it also places requirements that
> the current ALTO design do not satisfy [1].

There is definitively room for extensions. We have so many architectural
options (side note: what is now just the "ALTO protocol" has been called
in RFC 5693 and 6708 the "ALTO client protocol", in anticipation that
there might be an ALTO server-to-server protocol in the future).

First area of questions: Data model.  If every ALTO server defines the
network map on its own and uses the default smaller-value-is-better
routingcost metric, which does not have a well-defined base unit, it
will be hard to impossible to merge information from different servers.
Deprecate them?  Replace with what? Or go for an architectural option
(see below) that does not require information merging on the ALTO
protocol side.

Second area of questions: placement of entities and protocols between
them. This is also not completely new, see Sec. 2.2.3 of RFC 7971

Ideas:

1. Define a new server discovery mechanism.  An ALTO client would
discover several ALTO servers from different domains, ask each of them,
and then aggregate the information on the client-side.

2. Mesh-style ALTO server-to-server communication.  Do we need a new
s2s protocol?  Is a small extension to the ALTO protocol sufficient?

3. Hierarchical structure with backend information sources and frontend
servers towards the ALTO clients. Aggregation of the information from
different backends takes place in the frontends. How to ensure
consistency? Do we need a new protocol for backend to frontend
communication? Is a small extension to the ALTO protocol sufficient?

>From an academic perspective, I'd love to investigate and specify each
of these options and many more.  But from an IETF perspective: who is
willing to deploy anything on a scale that requires and justifies the
effort for standardization?  If there is one: what is their preference
which way to go?  Is your group planning a deployment?

---------- Forwarded message ---------
From: Danny Alex Lachos Perez <[email-masked]>
Date: Mon, Jun 29, 2020 at 8:18 AM
Subject: Re: [alto] ALTO at IETF-108: Finishing the current milestones and
discussion on re-chartering the WG
To: Sebastian Kiesel <[email-masked]>
Cc: Ingmar Poese <[email-masked]>, Y. Richard Yang <[email-masked]>

Hello Sebastian,

Thanks a lot for your comments
Please, see inline:

Even the very first ALTO use case was multi-domain in some sense:
> the bittorrent clients in the access networks of Comcast and Sprint,
> the pirate bay tracker in Sweden, that could be seen as three domains.
> The examples in the drafts you have listed are obviously different and
> more complex, but I am still struggling to tell what exactly is new and
> needs to be done.

Yes, I remember your question. I will try to clarify our multi-domain
approach.
>From your P2P example, we have the next figure:
https://drive.google.com/file/d/1GjoxeliHJjw7rp9eXpGxHB5m98EXl2Vc/view

An ALTO server in each domain will provide only local information to ALTO
clients. In the example, the tracker (ALTO client) receives
topology-/policy-related information of a single domain (AS1 or AS2). Let's
call it "single-domain information exposure".
On the other hand, if we want an ALTO client to receive merged information
from multiple domains, ALTO servers need to be able to exchange
information. In the example, the tracker will receive AS1’s and AS2’s
merged information. Let's call it "multi-domain information exposure".
As we identified [0], different use cases could benefit from multi-domain
information exposure using ALTO. However, it also places requirements that
the current ALTO design do not satisfy [1].

A second thought of mine is: how much extensions (and added complexity)
> do we need in the ALTO protocol?

We have identified an initial set of requirements/extensions such as (i)
conceptual query interfaces and data representation (e.g., unified resource
representation, resource query language), (ii) communication mechanisms
(e.g., ALTO servers communication, multi-domain connectivity discovery,
multi-domain ALTO server discovery), (iii) and computation model (e.g.,
computation complexity optimization, security & privacy) [1].

Would it make sense to start
> standardizing the "provisioning protocol" (see RFC 5693, Fig. 1) for
> multi-domain scenarios and put some (or all) of the additional
> complexity there?

Very interesting your point on the provision protocol for multi-domain
scenarios. RFC7285 also mentions an external interface for populating the
ALTO server and the possibility to exchange information with other ALTO
servers. This would make sense with the multi-ALTO server communication
requirement to allow exchanging network information from multiple
domains either using a hierarchical or mesh architectural design.

Best regards,

Danny Lachos

[0] ANRW19: https://dl.acm.org/doi/10.1145/3340301.3341126 (slides:
https://irtf.org/anrw/2019/slides-anrw19-final28.pdf)
[1] slides:
https://datatracker.ietf.org/meeting/interim-2020-alto-01/materials/slides-interim-2020-alto-01-sessa-supporting-multi-domain-use-cases-with-alto

---------- Forwarded message ---------
From: Sebastian Kiesel <[email-masked]>
Date: Fri, Jun 26, 2020 at 1:59 PM
Subject: Re: [alto] ALTO at IETF-108: Finishing the current milestones and
discussion on re-chartering the WG
To: Danny Alex Lachos Perez <[email-masked]>
Cc: Ingmar Poese <[email-masked]>, Y. Richard Yang <[email-masked]>

Hi Danny,

Thank you for the heads up!

My first question would be: what are the domains in multi-domain?

Even the very first ALTO use case was multi-domain in some sense:
the bittorrent clients in the access networks of Comcast and Sprint,
the pirate bay tracker in Sweden, that could be seen as three domains.
The examples in the drafts you have listed are obviously different and
more complex, but I am still struggling to tell what exactly is new and
needs to be done.

A second thought of mine is: how much extensions (and added complexity)
do we need in the ALTO protocol?  Would it make sense to start
standardizing the "provisioning protocol" (see RFC 5693, Fig. 1) for
multi-domain scenarios and put some (or all) of the additional
complexity there?

Thanks
Sebastian

---------- Forwarded message ---------
From: Danny Alex Lachos Perez <[email-masked]>
Date: Fri, Jun 26, 2020 at 12:17 PM
Subject: Re: [alto] ALTO at IETF-108: Finishing the current milestones and
discussion on re-chartering the WG
To: Ingmar Poese <[email-masked]>, Sebastian Kiesel <[email-masked]
Cc: Y. Richard Yang <[email-masked]>

Dear Ingmar and Sebastian,

In the last ALTO virtual meeting (Apr, 21), we had a very brief discussion
about our on-going work on multi-domain.
Since multi-domain is to be considered as a major re-charter item, I am
using this thread to resume the discussion and engage key experts like you
two.

Here are some points to start our discussion:

   - Potential initial drafts to consider could be those that show how the
   use cases in multi-domain need a solution that currently is not enough in
   the ALTO design. The main idea in those documents should be to provide
   study cases and design requirements that are more in the scope of
   informational drafts (Use cases/Problem statement/Design requirements).
   - For a long time, we have had different personal
   drafts/papers/presentations associated with multi-domain [0-6]. Therefore,
   one action point could be to collect such pieces and write an initial draft
   regarding multi-domain use cases and ALTO design issues.
   - On the other hand, while we also have an initial scope on some
   potential solutions, I think, such solutions need more interactions and
   probably we should consider them as proposed standard drafts.

Please, any feedback/comment/suggestion on the above points are really
welcome.

I hope to start a fruitful discussion,
Thanks in advance and extend this thread to others if you think it could be
convenient

@Richard: Please, feel free to complement my initial thoughts

Best regards,

Danny Lachos
[0] ANRW19: https://dl.acm.org/doi/10.1145/3340301.3341126 (slides:
https://irtf.org/anrw/2019/slides-anrw19-final28.pdf)
[1] slides:
https://datatracker.ietf.org/meeting/interim-2020-alto-01/materials/slides-interim-2020-alto-01-sessa-supporting-multi-domain-use-cases-with-alto

[2]
https://datatracker.ietf.org/doc/draft-lachos-alto-multi-domain-use-cases/
[3]
https://datatracker.ietf.org/doc/draft-xiang-alto-unified-representation/
[4] https://datatracker.ietf.org/doc/draft-xiang-alto-multidomain-analytics/
[5] https://datatracker.ietf.org/doc/draft-lachosrothenberg-alto-md-e2e-ns/
[6] https://datatracker.ietf.org/doc/draft-lachosrothenberg-alto-brokermdo/

---

From: Sebastian Kiesel <[email-masked]>
Date: Tue, 14 Jul 2020 20:38:53 +0200

Hi Ingmar, all,

On Tue, Jul 14, 2020 at 01:33:37PM +0200, Ingmar Poese wrote:
> stages, if i remember correctly, there was the discussion on "my view on the
> Internet".

You remember correctly. 

And if I remember correctly, we also had the idea that (at least for the
P2P use case), the ALTO information generally should flow in the
opposite direction of the money:  If, for example, I am a paying
customer of a specific ISP, I should listen to my ISP's guidance
(disseminated through my ISP's ALTO server), in order to reduce the
ressource consumption in their network and their costs for their
peerings or upstream, so they won't have to increase my monthly bill
and/or enforce volume caps or bandwidth throttling.  

As long as the assumption "bigger pipes = lower cost per byte = less
congestion = higher per-flow throughput" holds true, this strategy will
also give better application performance to me, the famous win-win
situation (or assumption). But there are also counter-examples: If I am
a mobile subscriber, my mobile network operator might be tempted to say
(for the P2P use case): any destination peer outside of my network is
better than pushing the same data twice through my radio access network,
to a another customer in my network running the P2P application. If
the ALTO server gives such simplistic guidance (low preference for local
network prefixes, uniformly higher preference for all other prefixes),
I might end up contacting a mobile user in a different part of the
world, thus going through 2 wireless links plus a transcontinental link.

So, multi-domain optimization might be challenging. And I am still not
so sure what are the domains in the multi-domain draft ...

> I would like to point this out again and the fact that "The
> Internet" looks different from different vantage points. Depending on where
> you are, decisions are different. Furthermore, a client is not capable of
> deciding which information to use from what ALTO server - and it if get
> multiple answers, which one is "more" correct.

The notion of "more correct" is difficult as this indeed highly depends
on the point of view.

> If the ALTO Server now only has information about the local network, or only
> gives out information about the local network, matching this to interAS
> client (or even clients in the same confederation) almost impossible - and
> this gets worse if not all Networks offer an ALTO service.
> reduces the problems to "what ALTO server to ask" for the client and can be
> much easier deployed and maintained on a large scale.

I agree.

> We (BENOCS) have had that on our ToDo list for about a year now, but we have
> not come around to implementing it so far. The deployment would encase about
> 100 million subscribers towards a few Hypergiants across Europe.

This really sounds impressive!

> I have a fairly good plan on how to do it - although we were not going to
> use ALTO for cross domain ALTO (we call it cascading internally), but rather
> implement something ourselves that fits our use case of

Do you already have an idea which parts of your solution require
standardization and are understood well enough so that we can put
them on the deliverables list for our rechartering discussion?
 
> I would be open to discussing this in a broader sense to standardize the

We definitively should have a venue in the context of IETF or IRTF
for discussing this, but is is already ripe for standardization?

Thanks,
Sebastian

---

From: Jensen Zhang <[email-masked]>
Date: Fri, 17 Jul 2020 19:08:13 -0400

Hi Sebastian, Ingmar, Danny, all,

First, thanks for all your discussions. I also have several comments.
Please see inline.

> so sure what are the domains in the multi-domain draft ...

I agree. The term "domain" is not defined clearly. There have been
multiple (active/expired) multi-domain related drafts in WG. They may
talk about different use cases but all claim they are "multi-domain".
But in general, I think a "domain" means a partial network that
provides an ALTO server. Those domains can be ASes of the same ISP,
IXPs, the backbone networks of different ISPs, different access
networks, or even mobile edge clouds. Each domain may only have a
partial view (routing and performance) of the user-intended traffic
(not the complete view of partial traffic). So they need some
server-to-server information exchange to get a more accurate view of
traffic for their end-users. But for different kinds of "domains",
there may be different requirements. e.g., the internal routing
information can be shared among domains of the same ISP, but may not
for different ISPs.

> > much easier deployed and maintained on a large scale.

I also agree. I think the term "local network" is equal to the term
"domain" that I used above, right? But the problem is how to make the
ALTO server for a domain (local network) have enough information to
give an indication on the entire Internet? Do you think existing
routing protocols and measurement tools are enough?

> > then, i have a very narrow Agenda on our use case, so maybe i am

Thanks for pointing out this problem. Personally, I think simply using
the same data model of the ALTO client protocol is not good enough for
the server-to-server information exchange. But we may not need to
fully deprecate them. But we do need some extensions. e.g., include
the well-defined base unit. And I still remember the "ingress point"
issue that Ingmar mentioned in the last IETF meeting. This information
should also be considered.

> This really sounds impressive!

Very impressive. I'm fully from academia. So I would like to see the
real requirements and issues coming from real industrial deployment.
But one debate is whether we use a hierarchical structure like DNS, or
a collaborative structure like BGP, or a delegated structure like the
master-slave cluster?

> for discussing this, but is is already ripe for standardization?

I agree. I cannot say it is already ripe. But I believe there are
already many people interested in this topic. But to understand its
feasibility, we need more discussions.

Ingmar, Sebastian, and Danny, many thanks for your great discussions.
As WG chairs suggest, would some of you like to give a single slide
presentation to discuss this potential rechartering item during the
IETF 108 ALTO session?

Thanks,
Jensen

---

From: chunshxiong(熊春山) <[email-masked]>
Date: Mon, 20 Jul 2020 03:34:57 +0000

Hello all,

The discussions are so interesting.

I provide my view on ALTO as below.

Normally, for the routing procedure, there are two types of routing protocols/procedures, one is interior-domain routing and another is inter-domain routing.
For the interior-domain routing, the routers in the path selects the next router based on the internal calculated lowest "cost".
For the inter-domain routing, the routers in the path selects the next router based on the local domain configured "policies" or "rules".

The current ALTO server provides the similar function of the "router" in the interior domain: router is select next destination "router" based on the lowest cost, while the ALTO server provides the best destination "resource provider" based on lowest cost.

So for the ALTO to support inter-domain, the similar routing technologies can be used, e.g. the BGP-like technologies can be reused. i.e. the ALTO servers from different domains need to talk each to exchange the " resource provider and related cost" information, but the " resource provider and related cost" information are provided in the ALTO server as BGP as policies and rules configured by the domain manager.

For the ALTO client, it must not access directly to the inter-domain ALTO server, it can access its interior-domain ALTO server, this interior-domain ALTO server will talk with its domain border ALTO server and provides the final destination to the ALTO client.

So there are type communications :1) interior ALTO server to domain border ALTO server; 2) inter domain ALTO server

Here is my basic understanding on the inter-domain ALTO, it is just my personal view.

I find that SDN-based WAN is proposed in some paper, the SDN controller selects a routing path based on the global view on the WAN. Maybe the SDN-based WAN technologies can be investigated to identify whether the SDN controller helps to select a best destination based on the global view of the WAN. In such case, the interior ALTO server needs to communicate with the SDN controller. The communication between two SDN controller is outside of ALTO. With this method, it will simplify the standard work in ALTO, but we need to cooperate with SD-WAN work (it is still unclear in which organization).  It is just a draft idea, I need more time to study this possibility, but I hope people also to notice this direction.

BRs,
Chunshan Xiong

-----Original Message-----
From: alto <[email-masked]> On Behalf Of Jensen Zhang
Sent: Saturday, July 18, 2020 7:08 AM
To: Sebastian Kiesel <[email-masked]>
Cc: Ingmar Poese <[email-masked]>; IETF ALTO <[email-masked]>
Subject: Re: [alto] ALTO at IETF-108: Finishing the current milestones and discussion on re-chartering the WG(Internet mail)

On Tue, Jul 14, 2020 at 2:39 PM Sebastian Kiesel <[email-masked]> wrote:
> > > > > the bittorrent clients in the access networks of Comcast and 
> > > > > Sprint, the pirate bay tracker in Sweden, that could be seen as three domains.
> > > > > The examples in the drafts you have listed are obviously 
> > > > > different and more complex, but I am still struggling to tell 
> > > > > what exactly is new and needs to be done.
> > > >
> > > > Yes, I remember your question. I will try to clarify our 
> > > > multi-domain approach.
> > > > An ALTO server in each domain will provide only local 
> > > > information to ALTO clients. In the example, the tracker (ALTO 
> > > > client) receives topology-/policy-related information of a 
> > > > single domain (AS1 or AS2). Let's call it "single-domain information exposure".
> > >
> > > While your observation is probably true in most deployments we 
> > > have seen so far, I do not completely agree on an architectural level.
> > >
> > > We have designed the ALTO protocol such that an ALTO client may 
> > > ask one ALTO server *any* question, e.g., "give me network map and 
> > > cost map for the whole Internet address space" or ECS(IP1, IP2) 
> > > where IP1 and IP2 are arbitrary IP addresses in whatever domain.
> > > The idea that there might be more than one ALTO server, and that 
> > > one ALTO server might give better answers to some specific 
> > > questions while another ALTO server might give better answers to 
> > > other questions, was in our mind when we worked on the protocol, 
> > > but it did not result in any specific feature of the base protocol 
> > > that would deal with that situation.
> >
> > I would actually like to go a little further on this point. At the 
> > early stages, if i remember correctly, there was the discussion on 
> > "my view on the Internet".
>
> You remember correctly.
>
> And if I remember correctly, we also had the idea that (at least for 
> the P2P use case), the ALTO information generally should flow in the 
> congestion = higher per-flow throughput" holds true, this strategy 
> will also give better application performance to me, the famous 
> win-win situation (or assumption). But there are also 
> counter-examples: If I am a mobile subscriber, my mobile network 
> operator might be tempted to say (for the P2P use case): any 
> destination peer outside of my network is better than pushing the same 
> data twice through my radio access network, to a another customer in 
> my network running the P2P application. If the ALTO server gives such 
> simplistic guidance (low preference for local network prefixes, 
> uniformly higher preference for all other prefixes), I might end up 
> contacting a mobile user in a different part of the world, thus going through 2 wireless links plus a transcontinental link.
>

I agree. The term "domain" is not defined clearly. There have been multiple (active/expired) multi-domain related drafts in WG. They may talk about different use cases but all claim they are "multi-domain".
But in general, I think a "domain" means a partial network that provides an ALTO server. Those domains can be ASes of the same ISP, IXPs, the backbone networks of different ISPs, different access networks, or even mobile edge clouds. Each domain may only have a partial view (routing and performance) of the user-intended traffic (not the complete view of partial traffic). So they need some server-to-server information exchange to get a more accurate view of traffic for their end-users. But for different kinds of "domains", there may be different requirements. e.g., the internal routing information can be shared among domains of the same ISP, but may not for different ISPs.

> > Internet" looks different from different vantage points. Depending 
> > on where you are, decisions are different. Furthermore, a client is 
> > not capable of deciding which information to use from what ALTO 
> > server - and it if get multiple answers, which one is "more" correct.
>
> The notion of "more correct" is difficult as this indeed highly 
> depends on the point of view.
>
> > If the ALTO Server now only has information about the local network, 
> > or only gives out information about the local network, matching this 
> > to interAS client (or even clients in the same confederation) almost 
> > I think an ALTO server should always give an indication on the 
> > entire Internet from it's point of view, even if much of it is 
> > aggregated. This reduces the problems to "what ALTO server to ask" 
> > for the client and can be much easier deployed and maintained on a large scale.
>
> I agree.

I also agree. I think the term "local network" is equal to the term "domain" that I used above, right? But the problem is how to make the ALTO server for a domain (local network) have enough information to give an indication on the entire Internet? Do you think existing routing protocols and measurement tools are enough?

> > > This is different for the discovery mechansims: there we had the 
> > > idea that an ALTO server that is "near" to the endpoint of the 
> > > data transmissions to be optimized, typically announced by the 
> > > operator of the access network, will probably give good answers.
> > > If the "peer selection" (or any other routing decision in the 
> > > overlay) is done in the endpoint of the data transmission itself, 
> > > use RFC 7286; if it is done at a remote entity such as a tracker, 
> > > use RFC 8686 to find ALTO server that is nearby the actual endpoint.
> > >
> > >
> > > > information from multiple domains, ALTO servers need to be able 
> > > > to exchange information. In the example, the tracker will 
> > > > receive AS1’s and AS2’s merged information. Let's call it "multi-domain information exposure".
> > > > As we identified [0], different use cases could benefit from 
> > > > multi-domain information exposure using ALTO. However, it also 
> > > > places requirements that the current ALTO design do not satisfy [1].
> > >
> > > There is definitively room for extensions. We have so many 
> > > architectural options (side note: what is now just the "ALTO 
> > > protocol" has been called in RFC 5693 and 6708 the "ALTO client 
> > > protocol", in anticipation that there might be an ALTO server-to-server protocol in the future).
> > >
> > >
> > > First area of questions: Data model.  If every ALTO server defines 
> > > the network map on its own and uses the default 
> > > smaller-value-is-better routingcost metric, which does not have a 
> > > well-defined base unit, it will be hard to impossible to merge information from different servers.
> > > Deprecate them?  Replace with what? Or go for an architectural 
> > > option (see below) that does not require information merging on 
> > > the ALTO protocol side.
> >
> > This is an interesting problem that i would like to go into further. 
> > But then, i have a very narrow Agenda on our use case, so maybe i am 
> > overshooting here.

Thanks for pointing out this problem. Personally, I think simply using the same data model of the ALTO client protocol is not good enough for the server-to-server information exchange. But we may not need to fully deprecate them. But we do need some extensions. e.g., include the well-defined base unit. And I still remember the "ingress point"
issue that Ingmar mentioned in the last IETF meeting. This information should also be considered.

> > > Second area of questions: placement of entities and protocols 
> > > between them. This is also not completely new, see Sec. 2.2.3 of 
> > > RFC 7971
> > >
> > > Ideas:
> > >
> > > discover several ALTO servers from different domains, ask each of 
> > > them, and then aggregate the information on the client-side.
> > >
> > > 2. Mesh-style ALTO server-to-server communication.  Do we need a 
> > > new s2s protocol?  Is a small extension to the ALTO protocol sufficient?
> > >
> > > 3. Hierarchical structure with backend information sources and 
> > > frontend servers towards the ALTO clients. Aggregation of the 
> > > information from different backends takes place in the frontends. 
> > > How to ensure consistency? Do we need a new protocol for backend 
> > > to frontend communication? Is a small extension to the ALTO protocol sufficient?
> > >
> > >
> > >  From an academic perspective, I'd love to investigate and specify 
> > > each of these options and many more.  But from an IETF 
> > > perspective: who is willing to deploy anything on a scale that 
> > > requires and justifies the effort for standardization?  If there 
> > > is one: what is their preference which way to go?  Is your group planning a deployment?
> >
> > We (BENOCS) have had that on our ToDo list for about a year now, but 
> > we have not come around to implementing it so far. The deployment 
> > would encase about

Very impressive. I'm fully from academia. So I would like to see the real requirements and issues coming from real industrial deployment.
But one debate is whether we use a hierarchical structure like DNS, or a collaborative structure like BGP, or a delegated structure like the master-slave cluster?

> > I have a fairly good plan on how to do it - although we were not 
> > going to use ALTO for cross domain ALTO (we call it cascading 
> > internally), but rather implement something ourselves that fits our 
> > use case of Hypergiant<->end-user mapping (we do not do P2P).
>
> standardization and are understood well enough so that we can put them 
> on the deliverables list for our rechartering discussion?
>
> > I would be open to discussing this in a broader sense to standardize 
> > the approach.
>
> We definitively should have a venue in the context of IETF or IRTF for 
> discussing this, but is is already ripe for standardization?

I agree. I cannot say it is already ripe. But I believe there are already many people interested in this topic. But to understand its feasibility, we need more discussions.

Ingmar, Sebastian, and Danny, many thanks for your great discussions.
As WG chairs suggest, would some of you like to give a single slide presentation to discuss this potential rechartering item during the IETF 108 ALTO session?

Thanks,
Jensen

_______________________________________________
alto mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/alto

---

From: Sebastian Kiesel <[email-masked]>
Date: Tue, 21 Jul 2020 20:35:32 +0200

Hi Jensen,

On Fri, Jul 17, 2020 at 07:08:13PM -0400, Jensen Zhang wrote:
> "domain" that I used above, right?

probably yes. maybe we should not think of BGP ASes or so when we try
to figure out what a domain in the context of ALTO could be.

What we have so far: The operator of a "local network" may populate his
DHCP and DNS servers with information to direct the clients in his
network to an ALTO server of his choice [RFC 7286].  He may also
configure his DNS servers in a way that ALTO clients outside his
network, that want to optimize traffic that originates or terminates in
his network, can discover an ALTO server of his choice [RFC 8686].
Using this ALTO server (most likely run by said network operator) the
operator may publish his the ALTO information (preferences or policies,
as you like), from * his point of view *.

Preferences or policies may be different, even conflicting, see my
example above. So the main challenge might not be just a common data
model and getting all data everywhere, but finding an "single truth"
that is merged from conflicting points of view...

> But the problem is how to make the

If we really talk about the entire Internet, I'm not only wondering
about technical feasibility, in the sense of protocols and algorithms,
but also: Who would be willing to run the infrastructure that collects
the data, computes and disseminates that single truth?  Would they also
accept the legal liability if overlay applications following that advice
cause a traffic meltdown in some part of the world?

So, how to move forward?  Can we find a realistic use case, that someone
actually wants to implement?  Maybe not for the whole Internet, but
with some domains, having different (maybe even conflicting) policies?
What additional standardization would be needed to support it?

> presentation to discuss this potential rechartering item during the

My feeling is that this is a topic for further research, but not for
being put on the deliverables list in next week's re-chartering
discussion.  If someone else has a clearer view what is already ripe for
standardization, I'll be more than happy to comment and contribute.

Thanks
Sebastian