Subject: Loss patterns, Re: minutes for IPPM WG, 13 July 1999

From: Darryl Veitch <[email-masked]>
Date: Tue, 10 Aug 1999 17:58:31 +1000 (EST)

Hello all,

In a recent mail from Will Leland on the meeting in Oslo, the following comment 
was made:

&&&&&&&&&&&&&&&&&&&&&&&&
"  There was also a discussion about the fact that a burst loss
is considered terminated by a single good packet arrival.
A burst followed by a single good packet and another burst
would seem to be better characterized as one long burst.
Will noted that what is being measured could provide the
raw data for constructing a "less sensitive" metric.
Would some sort of "errored period" be a meaningful thing
to define?  "
&&&&&&&&&&&&&&&&&&&&&&&&

Two students and I did some work last year looking at characterising 
the loss process in detail.
One of the issues to clarify was precisely this one of, 'ok losses tend to occur
in small groups, but what about patterns such as  111011 ? 
The results we obtained were quite surprising. We found that the structure of losses
in UDP test packet streams (send out either as Poisson or with constant separation)
was extremely 'independent', in fact looking like an alternating renewal process
in most cases we observed (~100 experiments over 15 routes in 8 countries).

That is, take two random variables  L (Loss) and A (Arrival). 
 To a good approximation the loss process consisted in simply
taking sample values alternately from L and A, in an independent fashion.
This offers the prospect of a simple description of the loss process consisting
of a discrete probability distribution for L (with most of the mass on 1, 2 or 3
losses), and an exponential distribution for A.  
With this underlying model the probability of obtaining patterns 
such as 111011, or any others, is easily calculated.

I know it is late days as far as the RFC is concerned, but I think that such a 
description could be of use.   I will be taking up this work in more detail in
coming months.    Sue Moon presented similar finding at Infocom this year.

The paper appeared in Globecom 98 and can be downloaded from  
 http://www.serc.rmit.edu.au/~darryl     (paper at the bottom)

Darryl Veitch

+-------------------------------+------------------------------------------+
|      Dr. Darryl Veitch 	| Email:   [email-masked]         |
|      Senior Research Fellow   |                                          |
|      S.E.R.C.                 | Telephone:                               |
|      110 Victoria St          |        Direct-   +61 3 9925 4014         |
|      Carlton 3053             |     Inquiries-   +61 3 9925 4013         |
|      GPO Box 2476V            |                                          |
|      Melbourne                | Fax:             +61 3 9925 4094         |
|      Victoria 3001            |                                          |
|      Australia                | Web: http://www.serc.rmit.edu.au/~darryl |
+-------------------------------+------------------------------------------+

---

From: Andy Scherrer <[email-masked]>
Date: Tue, 10 Aug 1999 08:34:54 -0500 (CDT)

> Two students and I did some work last year looking at characterising the
> loss process in detail.  One of the issues to clarify was precisely this
> one of, 'ok losses tend to occur in small groups, but what about
> patterns such as 111011 ?  The results we obtained were quite
> surprising. We found that the structure of losses in UDP test packet
> streams (send out either as Poisson or with constant separation)  was
> extremely 'independent', in fact looking like an alternating renewal
> process in most cases we observed (~100 experiments over 15 routes in 8
> countries). 
> 
>  To a good approximation the loss process consisted in simply taking
> sample values alternately from L and A, in an independent fashion. 

But in what proportions?  Of course, that's a question you ask of the
data, but in a sequence whose beginning and end are difficult to define,
the time scale becomes an issue. And so while loss of one packet may not
be dependent on loss of any other packet, there is dependency on time
scale, since it will probably matter if you're looking at such proportions
in 3 sec bursts vs. 30 sec or 3 min bursts.

> This offers the prospect of a simple description of the loss process
> consisting of a discrete probability distribution for L (with most of
> the mass on 1, 2 or 3 losses), and an exponential distribution for A. 

But note that using a simple univariate exponential distribution for A
implies a Poisson distribution. This group has discussed problems with
Poisson modelling of these data. That is not to say it shouldn't perhaps
be discussed more, perhaps with some thought to heirarchical modeling of
the Poisson parameter (lambda) -- that would be one approach for
addressing the time issue I mentioned above. 

> With this underlying model the probability of obtaining patterns such as
> 111011, or any others, is easily calculated. 

Well, I submit that there are actually a suite of underlying models to
consider. A binary outcome such as your example can result from one of
many distributions, each of which thinks of the "question" in a different
way. 

For example, if you think of it as Binomial problem (a series of
Bernoulli[0-1] trials), you have a fixed, known number of trials (N), 
and model the proportion out of N that are 1 (or 0).

However, another way of looking at the _same_ data could be "How many
trials (packets) do I see before I observe a 1"? Doing this repeatedly
might give a set of data consisting of sequence lengths.

 ...which is different from "How many trials (packets) do I observe before
I can count some number C of 1s?" This can also give data consisting of
sequence lengths, but counted differently. 

> I know it is late days as far as the RFC is concerned, but I think that
> such a description could be of use.  I will be taking up this work in
> more detail in coming months.  Sue Moon presented similar finding at
> Infocom this year. 
> 

Thanks for the pointer - I'll take a look. 

Andy Scherrer
Statistician
Matrix Information and Directory Services, Inc.
1106 Clayton Lane, Ste. 501W, Austin, TX 78723
http://www.mids.org/
+1-512-451-7602