Subject: Re: [alto] FW: New Version Notification

From: Bill Roome <[email-masked]>
Date: Mon, 12 Mar 2012 10:53:10 -0400

As for Ben's first point -- do we need incremental update -- I think it
depends completely on the number of PIDs:

 25 PIDs => 625 pairwise costs => approx 32 kbytes of json

These days that's a trivial; many web sites have dozens of jpgs that size.
But if we scale up,

 2,500 PIDS => 6,250,000 costs => approx 190 megs of json

That's significant. Even zipped, that's still significant.

So here's my take: if we're content to have a protocol that's limited to a
small number of PIDs, full-update is fine. But if we want the protocol to
scale up to 100s or 1,000s of PIDs, we need some mechanism to "send just
the changes".

BTW, if there are only (say) 25 PIDs, I don't think we even need a
conditional "get if changed" mechanism. That is, once the client opens a
TCP connection to the server, does it really matter whether the server
sends a 250 byte "no changed" response or a 32kb "full map" response?

As for Ben's second point -- push vs poll -- as far as I know, there are
only two types of "push" mechanisms.

In the first, we keep the TCP connection open and the server periodically
sends updates down the stream. However, there's a limit on how many
connections a server can keep open, if only because port numbers are
16-bits. So that limits the number of clients. Of course, you could argue
that only a few clients will need that push mechanism, but still it's a
potential problem. And it does leave the ALTO server vulnerable to DOS
attacks.

In the second, we close the initial connection and the server sends update
messages to the client via some other path, probably via UDP. That scales
up, but it complicates the client, and fails if the client is behind a
firewall.

So I'm reluctant to use a push mechanism. I think simple polling is
better. If we do use "push", I suggest the second approach -- push the
updates via UDP. Yes, that complicates the client and requires a
cooperative firewall, and will discourage "home network/personal computer"
clients from using the push mechanism. But I'd argue that's an advantage
-- home/pc clients probably shouldn't keep a full cost map anyway, and
that will discourage them from trying.

	- Bill Roome

On 03/08/2012 15:00, "[email-masked]" <[email-masked]> wrote:

>On 23 Feb 2012, at 05:48, sreekanth madhavan wrote:
>
>>A notification mechanism doesn't help you with that problem though. What
>you
>>need is a mechanism to provide incremental updates (i.e. only send the
>>things that have changed), just being notified there is a change doesn't
>>help with caching.
>>[SK]
>>The notification message sends the changed map information for which the
>>clients have already subscribed. We have mentioned this point in the
>>draft(Introduction section). The mechanism was not clearly explained. We
>can
>>add this to the draft.
>
>I think there are two distinct things here that we should be careful not
>to
>join together unnecessarily.
>
>The first is whether there is a need for a mechanism to provide
>incremental
>updates for efficiency. I have use cases for such a mechanism so would
>support its development.
>
>The second is whether there is a need for the ALTO server to push a
>notification that ALTO map data has changed to ALTO clients or whether
>polling for updates by clients (using HTTP Etags etc for efficiency) is
>sufficient. I think this will rest more on the requirements of clients,
>for
>example how fast after an change has occurred must the client require the
>new ALTO map data, along with how many clients are likely to be accessing
>a
>single ALTO server as the number of ALTO clients could significantly
>differ
>between a P2P and CDN use case, for example. I am not convinced that a
>pub/sub or push mechanism for notifications is always required and for
>several use cases reuse of HTTP polling of, for example, a "change feed"
>could result in clients efficiently learning of changed ALTO data in a
>reasonable timeframe.
>
>Ben
>