[Docs] [txt|pdf] [Tracker] [WG] [Email] [Nits]

Versions: 00

Network Working Group                                      J. H. Dunn
INTERNET-DRAFT                                             C. E. Martin
Expires: February, 2001                                    ANC, Inc.

                                                           July, 2000
                     Framework for Router Benchmarking
                  <draft-ietf-bmwg-rtr-framework-00.txt>

Status of this Memo

   This  document  is an Internet-Draft and is in full conformance with all
   provisions of  Section  10  of  RFC2026.   Internet-Drafts  are  working
   documents  of the Internet Engineering Task Force (IETF), its areas, and
   its working groups.  Note that other groups may also distribute  working
   documents  as Internet-Drafts. Internet-Drafts are draft documents valid
   for a maximum of six months and may be updated, replaced,  or  obsoleted
   by  other  documents  at any time.  It is inappropriate to use Internet-
   Drafts as reference material or to cite them  other  than  as  "work  in
   progress."  To  view  the entire list of current Internet-Drafts, please
   check the "1id-abstracts.txt" listing contained in  the  Internet-Drafts
   Shadow  Directories  on  ftp.is.co.za  (Africa), ftp.nordu.net (Northern
   Europe), ftp.nis.garr.it (Southern Europe), munnari.oz.au (Pacific Rim),
   ftp.ietf.org (US East Coast), or ftp.isi.edu (US West Coast).

     The list of current Internet-Drafts can be accessed at
     http://www.ietf.org/ietf/1id-abstracts.txt

     The list of Internet-Draft Shadow Directories can be accessed at
     http://www.ietf.org/shadow.html.




Copyright Notice

   Copyright (C) The Internet Society (1999).  All Rights Reserved.

Abstract

   This  memo  discusses and proposes a framework for the development of IP
   performance benchmarking methodologies in the case of systems under test
   (SUT) running IETF routing protocols.  The intent of this document is to
   facilitate the use of existing metrics developed by the BMWG  and  other
   working   groups.   This  will  be  accomplished  by  specifying  router
   configuration and state parameters and characterizing their effect on IP
   packet forwarding in terms of these existing metrics.  The terms defined
   in this memo will be used in addition to terms  defined  in  RFCs  1242,
   2285,  and  2544  and  2761.  This memo is a product of the Benchmarking
   Methodology Working Group (BMWG) of the Internet Engineering Task  Force
   (IETF).




Dunn & Martin                                                      [Page 1]


INTERNET-DRAFT       Framework for Router Benchmarking            July 2000


I. Background.

1. Introduction.

   The  purpose  of  this  document  is  to  define a general framework for
   particular methodologies to be developed by the Benchmarking Methodology
   Working  Group  (BMWG)  of  the Operational Requirements Area. This memo
   extends  existing  work,  specifically  RFC  2330  "Framework   for   IP
   Performance Metrics", May 1998.  The goal of this effort is to produce a
   set  of  metrics  and  methodologies,  based  on  existing  metrics,  to
   characterize  the  effects  of  a router's configuration and state on IP
   packet forwarding performance  as  defined  in  RFC  2544  "Benchmarking
   Methodology  for  Network Interconnect Devices". Metrics will be defined
   in accordance with the framework specified  in  RFC  1242  "Benchmarking
   Terminology  for  Network  Interconnect  Devices". Methodologies will be
   defined in accordance with the framework specified in  RFC  2544.   This
   effort  will  focus  on  the  effects of the following configuration and
   states parameter sets on IP packet forwarding performance:

   1.  Static  configuration  parameters,  e.g.,  route  cache  vs.   total
   available memory size

   2.      Dynamic      configuration      parameters,      e.g.,      BGP4
   MinRouteAdvertisementInterval

   3. Static states, e.g., response to BGP4 NLRI updates

   4. Dynamics states, e.g., response  to  conflicting  BGP4  NLRI  updates
   (route flapping)

   Metrics  will be defined which characterize both the impact on IP packet
   forwarding performance and router response to  route  updates  based  on
   these states.  Packet forwarding performance assessment will be based on
   metrics described in RFCs 1242, 2285 and 2761.  The assessment of router
   response  will  be  based  on the values of MIB objects described in RFC
   1850 and the upcoming BGP MIB document.  This  document  will  reference
   Internet  vocabulary,  clearly  describing  Internet  components such as
   routers, routing protocols, and router  MIB  element  definitions.   Any
   additional router related vocabulary necessary to develop router metrics
   will be defined in this document. Measurement uncertainties  and  errors
   will be described, including how they relate to the analytical framework
   shared by many aspects of the Internet engineering discipline.



Dunn & Martin                                                      [Page 2]


INTERNET-DRAFT       Framework for Router Benchmarking            July 2000


2. Existing Definitions.


   RFC 1242 "Benchmarking Terminology for Network Interconnect Devices" and
   RFC  2330  "Framework  for  IP  Performance Metrics" should be consulted
   before attempting to make use of this document.  RFC 2544  "Benchmarking
   Methodology  for Network Interconnect Devices" contains discussions of a
   number of terms relevant to the benchmarking of  switching  devices  and
   should  be  consulted.   RFC  2285  "Benchmarking  Terminology  for  LAN
   Switching Devices" contains a number  of  terms  pertaining  to  traffic
   distributions  and datagram interarrival.  RFC 2761 "Terminology for ATM
   Benchmarking" contains a number terms pertaining to  traffic  management
   [TM4.0,  TM4.1].  RFC  1812  "Requirements  for  IP  Version  4 Routers"
   contains an excellent glossary with which it is assumed that the  reader
   is  familiar.   For  the sake of clarity and continuity, this RFC adopts
   the template for definitions set out in Section 2 of RFC 1242.

3. Requirements

   In this document, the words that are used to define the significance  of
   each  particular  requirement are capitalized. These words are: * "MUST"
   This word, or the words "REQUIRED" and "SHALL" mean that the item is  an
   absolute  requirement of the specification.  * "SHOULD" This word or the
   adjective "RECOMMENDED" means that there  may  exist  valid  reasons  in
   particular  circumstances to ignore this item, but the full implications
   should be understood and the case carefully weighed  before  choosing  a
   different  course.   * "MAY" This word or the adjective "OPTIONAL" means
   that this item is truly optional.  One vendor may choose to include  the
   item because a particular marketplace requires it or because it enhances
   the product, for example; another vendor may omit the same item.

   An implementation is not compliant if it fails to satisfy one or more of
   the   MUST   requirements   for   the   protocols   it  implements.   An
   implementation  that  satisfies  all  the  MUST  and  all   the   SHOULD
   requirements   for   its   protocols  is  said  to  be  "unconditionally
   compliant"; one that satisfies all the MUST requirements but not all the
   SHOULD  requirements  for  its  protocols  is  said to be "conditionally
   compliant".

4. Criteria for Router Metrics

   RFC 2330 "Framework for IP Performance Metrics"  contains  a  number  of



Dunn & Martin                                                      [Page 3]


INTERNET-DRAFT       Framework for Router Benchmarking            July 2000


   relevant   criteria   for  performance  metrics.   In  addition,  router
   performance metrics should be independent  of  router  architecture  and
   routing  protocol.   As with IP performance metrics, router metrics will
   be dependent on the parameter sets mentioned  in  section  I.1  of  this
   document.   Since  all  metrics  must  produce  reproducible  and  self-
   consistent results, router metrics are only meaningful in the case where
   these parameter sets are stable enough for measurements to be performed.
   As a result, auxiliary observations may be required to determine whether
   the router is in a stable state.

II. Definitions

   The  definitions  presented  in  this section have been divided into two
   groups.  The first group is formal definitions, which  are  required  in
   the  definitions  of  the  performance  metrics  but  are not themselves
   strictly metrics.  These definitions are subsumed from other  work  done
   in  other  working  groups  both  inside and outside the IETF.  They are
   provided as a courtesy to the reader.
1. Formal Definitions

1.1. Definition Format (from RFC 1242)

Term to be defined.

Definition: The specific definition for the term.

Discussion: A brief  discussion  of  the  term,  its  application  and  any
restrictions on measurement procedures.

Specification:  The  working  group  and  document  in  which the terms are
specified and are listed in the references section.

1.2. Generic Definitions.

1.2.1. Forwarding Decision

Definition: The process by which a router determines, based on  information
in the FIB, what the disposition of a packet will be.

Discussion:   The forwarding decision is made by the router's forwarder and
is used by the forwarder to correctly forward the packet.




Dunn & Martin                                                      [Page 4]


INTERNET-DRAFT       Framework for Router Benchmarking            July 2000


Specification: N/A

1.2.2. Routing Information Entry

Definition: A single  entry  in  the  FIB  that  provides  all  information
required for the forwarder to make a forwarding decision.

Discussion:  A routing information entry may provide forwarding information
mapping a single network, a class of networks, a specific data  link  layer
type or other datagram attribute to a unique egress port on the router.

Specification: N/A

1.2.2. Router Port

Definition:  A  single point of datagram entry and exit, characterized by a
single data link layer connection to other network equipment, access  speed
and, perhaps, access controls.

Discussion:   A  router  port  is  the  primary  source of datagrams to the
forwarder and destination of datagrams from the forwarder.

Specification: N/A

1.3. Static Configuration Parameters.

Definition:  Static Configuration Parameters  are  those  attributes  of  a
router,  which are not altered by information gathered during the operation
of a routing protocol.

1.3.1. Forwarding Information Base Cache

Definition: A dedicated portion of the FIB, which contains the most current
or most often accessed routing information.

Discussion:   The  cache  provides  a  method for rapidly accessing routing
information without resorting to a full scale search of the FIB.

Specification: N/A

1.3.2. Forwarding Information Base Cache Size




Dunn & Martin                                                      [Page 5]


INTERNET-DRAFT       Framework for Router Benchmarking            July 2000


Definition: The number of routing information entries allocated to the  FIB
cache.

Discussion:   The size of the cache allocation significantly effects router
performance.  If the cache allocation is not large  enough,  a  full  scale
search  of  the  FIB is required in order to make forwarding decisions.  If
the cache allocation is a significant percentage of the FIB size, the cache
search differs very little from a full scale FIB search.

1.3.3. Forwarding Information Base Size

Definition:  The  maximum number of routing information entries the FIB can
contain.

Discussion:   This  parameter  impacts  the  maximum  number  of  reachable
networks supported by the router; however, the relationship between the two
parameter is not always linear.


Specification: N/A

1.4. Dynamic Configuration Parameters.

Definition:  Dynamic Configuration Parameters are  those  attributes  of  a
router, which are altered by information gathered during the operation of a
routing protocol.

1.3.2. Number of Reachable Networks

Definition: The current number of networks that the routing information  in
the FIB allows the router to reach.

Discussion:   The  number  of  reachable  networks  will  depend  on static
parameters, such as the maximum number of routing entries in the  FIB,  and
dynamic parmeters, such as whether the FIB accurately reflects the state of
the network and route aggregation.

Specification: N/A

3. Security Considerations.

   As this document is  solely  for  providing  terminology  and  describes



Dunn & Martin                                                      [Page 6]


INTERNET-DRAFT       Framework for Router Benchmarking            July 2000


   neither  a  protocol  nor  an  implementation,  there  are  no  security
   considerations associated with this document.


4. Notices

Internet Engineering Task Force

   The IETF takes no position  regarding  the  validity  or  scope  of  any
   intellectual  property or other rights that might be claimed to  pertain
   to the implementation  or  use  of  the  technology  described  in  this
   document  or  the extent to which any license under such rights might or
   might not be available; neither does it represent that it has  made  any
   effort to identify any such rights.  Information on the IETFs procedures
   with  respect  to  rights  in  standards-track   and   standards-related
   documentation  can  be found in BCP-11.  Copies of claims of rights made
   available for publication and any assurances  of  licenses  to  be  made
   available,  or the result of an attempt made to obtain a general license
   or permission for the use of such proprietary rights by implementors  or
   users  of  this specification can be obtained from the IETF Secretariat.
   The IETF invites any interested party to  bring  to  its  attention  any
   copyrights, patents or patent applications, or other proprietary rights,
   which may cover  technology  that  may  be  required  to  practice  this
   standard.    Please  address  the  information  to  the  IETF  Executive
   Director.

5. Disclaimer

   Copyright (C) The Internet Society (1999). All Rights Reserved.

   This document and translations of it may  be  copied  and  furnished  to
   others,  and derivative works that comment on or otherwise explain it or
   assist in its implementation may  be  prepared,  copied,  published  and
   distributed,  in  whole  or  in  part,  without restriction of any kind,
   provided that the above copyright notice and this paragraph are included
   on  all such copies and derivative works.  However, this document itself
   may not be modified in any way, such as by removing the copyright notice
   or  references  to the Internet Society or other Internet organizations,
   except as needed for the purpose of  developing  Internet  standards  in
   which  case  the  procedures  for  copyrights  defined  in  the Internet
   Standards process must be followed, or as required to translate it  into
   languages other than English.  The limited permissions granted above are



Dunn & Martin                                                      [Page 7]


INTERNET-DRAFT       Framework for Router Benchmarking            July 2000


   perpetual and will not  be  revoked  by  the  Internet  Society  or  its
   successors  or  assigns.  This  document  and  the information contained
   herein is provided on an "AS IS" basis and THE INTERNET SOCIETY AND  THE
   INTERNET  ENGINEERING  TASK  FORCE  DISCLAIMS ALL WARRANTIES, EXPRESS OR
   IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE  OF  THE
   INFORMATION   HEREIN  WILL  NOT  INFRINGE  ANY  RIGHTS  OR  ANY  IMPLIED
   WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE.

6. References

   [IPv4-Router]  Baker, F., "Requirements for IP Version 4  Routers",  RFC
   1812, June 1995

7. Editors Addresses

   Jeffrey Dunn
   Advanced Network Consultants, Inc.
   4214 Crest Place, Ellicott City, MD 21043 USA
   Phone: +1 (410) 750-1700, E-mail: Jeffrey.Dunn@worldnet.att.net

   Cynthia Martin
   Advanced Network Consultants, Inc.
   4214 Crest Place, Ellicott City, MD 21043 USA
   Phone: +1 (410) 750-1700, E-mail: Cynthia.E.Martin@worldnet.att.net





















Dunn & Martin                                                      [Page 8]


Html markup produced by rfcmarkup 1.123, available from https://tools.ietf.org/tools/rfcmarkup/