CURRENT_MEETING_REPORT_

Reported by Jim McQuaid/Wandel & Goltermann Technologies

Minutes of the Benchmarking Methodology Working Group (BMWG)



Network Element Benchmarking

The first item of discussion was the status of the Internet-Draft on
Network Element benchmarking, draft-bmwg-methodology-00.txt.  Since this
draft is relatively mature, there was not a lot of discussion.  The
group agreed to move forward towards seeing this issued as an
Informational RFC. There will be a final call on the list shortly.

Two points were raised, however, with reference to the methodologies
described in this draft.  In both cases it was agreed that another
document should be generated which references the present draft, rather
than hold up the present draft and expand it.

The first point concerns test of devices which switch, with a focus on
call setup overhead in a true switched environment.  The discussion
raised questions about whether this is management overhead or analogous
to routing.  Kevin Dubray volunteered to offer a draft about testing
issues in this area.

The second point concerned testing of congestion and congestion behavior
in a network switch.  The group felt that many of the issues were
covered, at least in a general way, in the present draft.  However,
there are some particular cases of many to one which merit further
attention.  Merike Kaeo volunteered to draft a document about testing
issues in this area.



Benchmarking Methodologies for Overall Network Performance

Jim McQuaid spent a few minutes giving an overview of the second
Internet-Draft recently posted, draft-bmwg-overallperf-00.txt,
describing methods for gross delay testing in a working network using
time synchronization.  One concept put forward was the idea of using
unsynchronized send and receive processes to measure the variance of
delay, e.g., by strobing out a test packet every two seconds.  Another
discussion point centered on ``instrumented traceroute'' e.g., isolating
each hop in the cloud for potential discrete delay measurement.  After
some discussion and comment on the possible issues and value of such
testing generally, Jim agreed to continue this discussion on the list
with the goal of forwarding an edited draft at the next IETF.



Benchmarking For End Systems

Rick Jones then presented a fairly extensive set of thoughts on the
subject of benchmarking for end systems.  His slides follow these
minutes.


Testing Routing Convergence

Some issues around testing routing convergence were raised and
discussed, but no further action is to be taken at this time.