[Docs] [txt|pdf] [Tracker] [WG] [Email] [Diff1] [Diff2] [Nits]

Versions: 00 01 02 03 04 05 06 07 08 09 10 11 12 13

   Network Working Group
   INTERNET-DRAFT
   Expires in: January 2005
                                                Scott Poretsky
                                                Quarry Technologies

                                                Shankar Rao
                                                Qwest Communications

                                                July 2004



             Terminology for Accelerated Stress Benchmarking

              <draft-ietf-bmwg-acc-bench-term-03.txt>



   Status of this Memo

   This document is an Internet-Draft and is in full conformance with
   all provisions of Section 10 of RFC2026.

   Internet-Drafts are working documents of the Internet Engineering
   Task Force  (IETF), its areas, and its working groups.  Note that
   other groups may also distribute working documents as Internet-
   Drafts.

   Internet-Drafts are draft documents valid for a maximum of six
   months and may be updated, replaced, or obsoleted by other
   documents at any time.  It is inappropriate to use Internet-Drafts
   as reference material or to cite them other than as "work in
   progress."

   The list of current Internet-Drafts can be accessed at
   http://www.ietf.org/ietf/1id-abstracts.txt

   The list of Internet-Draft Shadow Directories can be accessed at
   http://www.ietf.org/shadow.html.


   ABSTRACT
   This document provides the Terminology for performing Stress
   Benchmarking of networking devices.  The three phases of the Stress
   Test: Startup, Instability and Recovery are defined along with the
   benchmarks and configuration terms associated with the each phase.
   Also defined are the Benchmark Planes fundamental to stress testing
   configuration, setup and measurement.  The terminology is to be
   used with the companion framework and methodology documents.

   Table of Contents
     1. Introduction ............................................... 3
     2. Existing definitions ....................................... 3
     3. Term definitions............................................ 3

Poretsky and Rao                                                        [Page 1]


INTERNET-DRAFT Terminology for Accelerated Stress Benchmarking  July 2004

      3.1 General Terms............................................. 3
        3.1.1 Benchmark Planes...................................... 3
        3.1.2 Configuration Sets.................................... 4
        3.1.3 Startup Conditions.................................... 4
        3.1.4 Instability Conditions................................ 5
        3.1.5 Aggregate Forwarding Rate............................. 6
        3.1.6 Controlled Session Loss............................... 6
        3.1.7 Uncontrolled Session Loss............................. 6
      3.2 Benchmark Planes.......................................... 7
        3.2.1 Control Plane......................................... 7
        3.2.2 Data Plane............................................ 7
        3.2.3 Management Plane...................................... 8
        3.2.4 Security Plane........................................ 8
      3.3 Startup................................................... 9
        3.3.1 Startup Phase......................................... 9
        3.3.2 Benchmarks............................................10
          3.3.2.1 Stable Aggregate Forwarding Rate..................10
          3.3.2.2 Stable Session Count..............................11
        3.3.3 Control Plane.........................................11
          3.3.3.1 Control Plane Configuration Set...................11
          3.3.3.2 Control Plane Startup Conditions..................12
        3.3.4 Data Plane............................................12
          3.3.4.1 Data Plane Configuration Set......................12
          3.3.4.2 Traffic Profile...................................13
        3.3.5 Management Plane......................................13
          3.3.5.1 Management Plane Configuration Set................13
        3.3.6 Security Plane........................................14
          3.3.6.1 Security Plane Configuration Set..................14
          3.3.6.2 Security Plane Startup Conditions.................15
      3.4 Instability...............................................15
        3.4.1 Instability Phase.....................................15
        3.4.2 Benchmarks............................................16
          3.4.2.1 Unstable Aggregate Forwarding Rate................16
          3.4.2.2 Degraded Aggregate Forwarding Rate................16
          3.4.2.3 Average Degraded Aggregate Forwarding Rate........17
          3.4.2.4 Unstable Uncontrolled Sessions Lost...............17
        3.4.3 Instability Conditions................................18
          3.4.3.1 Control Plane Instability Conditions..............18
          3.4.3.2 Data Plane Instability Conditions.................18
          3.4.3.3 Management Plane Instability Conditions...........19
          3.4.3.4 Security Plane Instability Conditions.............19
      3.5 Recovery..................................................20
        3.5.1 Recovery Phase........................................20
        3.5.2 Benchmarks............................................20
          3.5.2.1 Recovered Aggregate Forwarding Rate...............20
          3.5.2.2 Recovery Time.....................................21
          3.5.2.3 Recovered Uncontrolled Sessions Lost..............21
     4. Security Considerations.....................................22
     5. References..................................................22
     6. Author's Address............................................22
     7. Full Copyright Statement....................................23
     Appendix 1 - White Box Benchmarks..............................23

Poretsky and Rao                                                        [Page 2]


INTERNET-DRAFT Terminology for Accelerated Stress Benchmarking  July 2004

   1. Introduction

   Routers in an operational network are simultaneously configured with
   multiple protocols and security policies while forwarding traffic and
   being managed.  To accurately benchmark a router for deployment it is
   necessary to test that router in operational conditions by
   simultaneously configuring and scaling network protocols and security
   policies, forwarding traffic, and managing the device.  It is helpful
   to accelerate these network operational conditions so that the
   router under test can be benchmarked with faster test duration.
   Testing a router in accelerated network conditions is known as
   Accelerated Stress Testing.

   This document provides the Terminology for performing Stress
   Benchmarking of networking devices.  The three phases of the Stress
   Test: Startup, Instability and Recovery are defined along with the
   benchmark and configuration terms associated with the each phase.
   Benchmarks for stress testing are defined using the Aggregate
   Forwarding Rate and control plane Session Count during each phase
   of the test.  Also defined are the Benchmark Planes fundamental to
   stress testing configuration, setup and measurement.  These are
   the Control Plane, Data Plane, Management Plane and Security Plane
   For each plane, the Configuration Set, Startup Conditions, and
   Instability Conditions are defined.  White Box benchmarks are
   provided in Appendix 1 for additional DUT behavior measurements.
   The terminology is to be used with the companion methodology
   document [6].

   2.  Existing definitions

   RFC 1242 "Benchmarking Terminology for Network Interconnect
   Devices" and RFC 2285 "Benchmarking Terminology for LAN Switching
   Devices" should be consulted before attempting to make use of this
   document.

   For the sake of clarity and continuity this RFC adopts the template
   for definitions set out in Section 2 of RFC 1242.  Definitions are
   indexed and grouped together in sections for ease of reference.

   The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
   "SHOULD", "SHOULD NOT", "RECOMMENDED",  "MAY", and "OPTIONAL" in
   this document are to be interpreted as described in RFC 2119.

   3. Term definitions
   3.1 General Terms
   3.1.1 Benchmark Planes

      Definition:
        The features, conditions, and behavior for the Accelerated Stress
        Benchmarking.



Poretsky and Rao                                                        [Page 3]


INTERNET-DRAFT Terminology for Accelerated Stress Benchmarking  July 2004

      Discussion:
        There are four Benchmark Planes: Control Plane, Data Plane,
        Management Plane, and Security Plane as shown in Figure 1. The
        Benchmark Planes define the Configuration, Startup Conditions,
        Instability Conditions, and Failure Conditions used for the test.

         ___________             ___________
        |  Control  |           | Management|
        |   Plane   |___     ___|   Plane   |
        |           |   |   |   |           |
         -----------    |   |    -----------
                       \/  \/                  ___________
                      ___________             | Security  |
                     |           |<-----------|   Plane   |
                     |    DUT    |            |           |
                |--->|           |<---|        -----------
                |     -----------     |
                |                     |
                |     ___________     |
                |    |   Data    |    |
                |--->|   Plane   |<---|
                     |           |
                      -----------

        Figure 1.  Router Accelerated Stress Benchmarking Planes

      Measurement units:
        N/A

      Issues:
        None

      See Also:
        Control Plane
        Data Plane
        Management Plane
        Security Plane

    3.1.2 Configuration Sets

      Definition:
        The features and scaling limits used during the Accelerated Stress
        Benchmarking.

      Discussion:
        There are four Configuration Sets: Control Plane Configuration Set,
        Data Plane Configuration Set, Management Plane Configuration Set,
        and Security Plane Configuration Set.

      Measurement units:
        N/A


Poretsky and Rao                                                        [Page 4]


INTERNET-DRAFT Terminology for Accelerated Stress Benchmarking  July 2004

      Issues:
        None

      See Also:
        Control Plane Configuration Set
        Data Plane Configuration Set
        Management Plane Configuration Set
        Security Plane Configuration Set

   3.1.3 Startup Conditions

      Definition:
        Test conditions that occur at the start of the Accelerated
        Life Benchmark to establish conditions for the remainder of
        the test.

      Discussion:
        Startup Conditions may cause stress on the DUT and produce
        failure.  Startup Conditions are defined for the Control
        Plane and Security Plane.

      Measurement units:
        N/A

      Issues:
        None

      See Also:
        Control Plane Startup Conditions
        Data Plane Startup Conditions
        Management Plane Startup Conditions
        Security Plane Startup Conditions

   3.1.4 Instability Conditions

      Definition:
        Test conditions that occur during the Accelerated Stress
        Benchmark to produce instability and stress the DUT.

      Discussion:
        Instability Conditions are applied to the DUT after the
        Startup Conditions have completed.  Instability Conditions
        occur for the Control Plane, Data Plane, Management Plane,
        and Security Plane.

      Measurement units:
        N/A

      Issues:
        None



Poretsky and Rao                                                        [Page 5]


INTERNET-DRAFT Terminology for Accelerated Stress Benchmarking  July 2004

     See Also:
        Control Plane Instability Conditions
        Data Plane Instability Conditions
        Management Plane Instability Conditions
        Security Plane Instability Conditions

   3.1.5 Aggregate Forwarding Rate

     Definition:
        Sum of forwarding rates for all interfaces on the
        DUT during the Startup Phase.

      Discussion:
        Each interface of the DUT forwards traffic at some
        measured rate.  The Aggregate Forwarding Rate is the
        sum of forwarding rates for all interfaces on the DUT.

      Measurement units:
        pps

      Issues:
        None

      See Also:
        Startup Phase

   3.1.6 Controlled Session Loss

     Definition:
        Control Plane sessions that are intentionally brought
        down during the Stress test.

      Discussion:
        The test equipment is able to control protocol
        session state with the DUT.

      Measurement units:
        None

      Issues:
        None

      See Also:
        Uncontrolled Session Loss


   3.1.7 Uncontrolled Session Loss

     Definition:
        Control Plane sessions that are in the down state
        but were not intentionally brought down during the
        Stress test.

Poretsky and Rao                                                        [Page 6]


INTERNET-DRAFT Terminology for Accelerated Stress Benchmarking  July 2004

      Discussion:
        The test equipment is able to control protocol
        session state with the DUT.  The test equipment
        is also to monitor for sessions lost with the
        DUT which the test equipment itself did not
        intentionally bring down.

      Measurement units:
        N/A

      Issues:
        None

      See Also:
        Controlled Session Loss

   3.2 Benchmark Planes

   3.2.1 Control Plane
      Definition:
        The Description of the control protocols enabled for
        the Accelerated Stress Benchmarking.

      Discussion:
        The Control Plane defines the Configuration, Startup
        Conditions, Instability Conditions, and Failure
        Conditions of the control protocols used for the test.
        Control Plane protocols may include routing protocols,
        multicast protocols, and MPLS protocols.  These can be
        enabled or disabled for a benchmark test.

      Measurement units:
        N/A

      Issues:
        None

      See Also:
        Benchmark Planes
        Control Plane Configuration Set
        Control Plane Startup Conditions
        Control Plane Instability Conditions
        Control Plane Failure Conditions

  3.2.2 Data Plane

      Definition:
        The data traffic profile used for the Accelerated Stress
        Benchmarking.




Poretsky and Rao                                                        [Page 7]


INTERNET-DRAFT Terminology for Accelerated Stress Benchmarking  July 2004

      Discussion:
        The Data Plane defines the Configuration, Startup
        Conditions, Instability Conditions, and Failure
        Conditions of the data traffic used for the test.
        The Data Plane includes the traffic and interface
        profile.

      Measurement Units:
        N/A

      See Also:
        Benchmark Planes
        Data Plane Configuration Set
        Data Plane Startup Conditions
        Data Plane Instability Conditions
        Data Plane Failure Conditions

   3.2.3 Management Plane

      Definition:
        The Management features and tools used for the
        Accelerated Stress Benchmarking.

      Discussion:
        A key component of the Accelerated Stress Benchmarking is the
        Management Plane to assess manageability of the router
        under stress.  The Management Plane defines the Configuration,
        Startup Conditions, Instability Conditions, and Failure
        Conditions of the management protocols and features used for
        the test.  The Management Plane includes SNMP, Logging/Debug,
        and Statistics Collection.

      Measurement units:
        N/A

      Issues:
        None

      See Also:
        Benchmark Planes
        Management Plane Configuration Set
        Management Plane Startup Conditions
        Management Plane Instability Conditions
        Management Plane Failure Conditions


   3.2.4 Security Plane

      Definition:
        The Security features used during the Accelerated Stress
        Benchmarking.


Poretsky and Rao                                                        [Page 8]


INTERNET-DRAFT Terminology for Accelerated Stress Benchmarking  July 2004

      Discussion:
        The Control Plane defines the Configuration, Startup
        Conditions, Instability Conditions, and Failure
        Conditions of the security features and protocols used for
        the test.  The Security Plane includes the ACLs, Firewall,
        Secure Protocols, and User Login.

      Measurement units:
        N/A

      Issues:
        None

      See Also:
        Benchmark Planes
        Security Plane Configuration Set
        Security Plane Startup Conditions
        Security Plane Instability Conditions
        Security Plane Failure Conditions

   3.3 Startup

   3.3.1 Startup Phase

     Definition
        The portion of the benchmarking test in which the
        Startup Conditions are generated with the DUT.  This
        begins with the attempt to establish the first session
        and ends when the last Control Plane session is
        established.

     Discussion:
        The Startup Phase is the first Phase of the
        benchmarking test preceding the Instability Phase
        and Recovery Phase.  It is specified by the
        Configuration Sets and Startup Conditions for
        each Benchmark Plane.  The Startup Phase ends
        and Instability Phase may begin when the
        Configuration Sets are achieved with the DUT.

     Measurement Units:
        None

     Issues:
        The 'last control plane session is established' may not
        be a sufficient indicator that steady-state is achieved
        and Instability Conditions can be applied to begin the
        Instability Phase.





Poretsky and Rao                                                        [Page 9]


INTERNET-DRAFT Terminology for Accelerated Stress Benchmarking  July 2004

     See Also:
        Benchmark Plane
        Configuration Sets
        Startup Conditions
        Instability Phase
        Recovery Phase

   3.3.2 Benchmarks

   3.3.2.1 Stable Aggregate Forwarding Rate

     Definition:
        Average rate of traffic forwarded by the DUT during the
        Startup Phase.

      Discussion:
        Stable Aggregate Forwarding Rate is the calculated
        average the Aggregate Forwarding Rates measured during
        the Startup Phase.  It is recommended that the
        Aggregate Forwarding Rate is measured at one-second
        intervals until the Startup Phase ends.

      Measurement units:
        pps

      Issues:
        The act of the DUT establishing the Startup Conditions
        could influence the forwarding rate in certain
        implementations so that this "baseline" for the
        remainder of the test is lowered.  The alternative is
        to change the definition of Startup Aggregate
        Forwarding Rate so that it measured during the Startup
        Phase, but after Startup Conditions are achieved.
        The disadvantage of this definition would be that it
        loses measurement of any impact establishing Startup
        Conditions would have on forwarding rate. When
        comparing the Startup Aggregate Forwarding Rate benchmark
        of two devices it is preferred to know the impact
        establishing Startup Conditions has on Forwarding Rate.
        The definition was therefore selected so that Stable
        Aggregate Forwarding Rate is calculated from measurement
        samples throughout the entire Startup Phase.

      See Also:
        Instability Conditions
        Aggregate Forwarding Rate
        Stable Aggregate Forwarding Rate






Poretsky and Rao                                                        [Page 10]


INTERNET-DRAFT Terminology for Accelerated Stress Benchmarking  July 2004

   3.3.2.2 Stable Session Count

     Definition:
        Total number of control plane sessions/adjacencies
        established and maintained by the DUT prior to
        Instability Conditions being initiated.

      Discussion:
        This measurement should be made after the Control
        Plane Startup Conditions are applied to the DUT.

      Measurement units:
        sessions

      Issues:
        None

      See Also:
        Instability Conditions

   3.3.3 Control Plane

   3.3.3.1 Control Plane Configuration Set
      Definition:
        The routing protocols and scaling values used for the Accelerated
        Life Benchmarking.

      Discussion:
        Control Plane Configuration Set is shown in Figure 2 and specifies
        the Routing Protocols, Multicast, and MPLS configuration.  Specific
        protocols can be enabled or disabled for a benchmark test.

      Measurement units:
        N/A

      Issues:
        None

         ____________            ____________            ____________
        |  Routing   |          |  Multicast |          |    MPLS    |
        |  Protocols |___       |  Protocols |        __|  Protocols |
        |            |   |      |            |        | |            |
         ------------    |       ------------         |  ------------
                         |              |             |
                         |              |             |
                         |             \/             |
                         |         ___________        |
                         |        |           |       |
                         |------->|    DUT    |<------|
                                ``|           |
                                   -----------
        Figure 2.  Control Plane Configuration Module

Poretsky and Rao                                                        [Page 11]


INTERNET-DRAFT Terminology for Accelerated Stress Benchmarking  July 2004

      See Also:
        Data Plane Configuration Set
      Management Configuration Set
      Security Configuration Set

   3.3.3.2 Control Plane Startup Conditions

      Definition:
        Control Plane conditions that occur at the start
        of the Accelerated Stress Benchmarking to establish conditions
        for the remainder of the test.

      Discussion:
        Startup Conditions may cause stress on the DUT and produce
        failure.  Startup Conditions for the Control Plane include
        session establishment rate, number of sessions established
        and number of routes learned.

      Measurement units:
        N/A

      Issues:
        None

      See Also:
        Startup Conditions
        Security Plane Startup Conditions
        Control Plane Configuration Set

   3.3.4 Data Plane

   3.3.4.1 Data Plane Configuration Set

      Definition:
        The data traffic profile enabled for the Accelerated Stress
        Benchmarking.

      Discussion:
        Data Plane Configuration Set includes the Traffic Profile and
        interfaces used for the Accelerated Stress Benchmarking.

      Measurement Units:
        N/A

      Issues:
        None

      See Also:
        Traffic Profile




Poretsky and Rao                                                        [Page 12]


INTERNET-DRAFT Terminology for Accelerated Stress Benchmarking  July 2004

 3.3.4.2 Traffic Profile
       Definition
        The characteristics of the Offered Load to the DUT used for
        the Accelerated Stress Benchmarking.

      Discussion
        The Traffic Profile specifies the number of packet size(s), packet
        rate per interface, number of flows, and encapsulation used for the
        offered load to the DUT.

      Measurement Units:
      Traffic Profile is reported as follows:

        Parameter                       Units
        ---------                       ------
        Packet Size(s)                  bytes
        Packet Rate(interface)          array of packets per second
        Number of Flows                 number
        Encapsulation(flow)             array of encapsulation type

      Issues:
        None

      See Also:
        Data Plane Configuration Set

  3.3.5 Management Plane
  3.3.5.1 Management Plane Configuration Set

      Definition:
        The router management features enabled for the
        Accelerated Stress Test.

      Discussion:
        A key component of the Accelerated Stress Test is the Management
        Configuration Set to assess manageability of the router under
        stress.  The Management Configuration Set defines the management
        configuration of the DUT.  Features that are part of the
        Management Configuration Set include SNMP, Logging/Debug, and
        Statistics Collection, and services such as FTP, as shown in
        Figure 3.

      Measurement units:
      N/A

      Issues:
        None

      See Also:
        Control Plane Configuration Set
        Data Plane Configuration Set
      Security Plane Configuration Set

Poretsky and Rao                                                        [Page 13]


INTERNET-DRAFT Terminology for Accelerated Stress Benchmarking  July 2004
                          ____________        ____________
                         |            |      |  Logging/  |
                         |    SNMP    |    __|   Debug    |
                         |            |   |  |            |
                          ------------    |   ------------
                                |         |
                                |         |
                                \/        |
                          ___________     |
                         |           |    |
                         |    DUT    |<---|
                         |           |
                          -----------
                              |
                              |
                              \/
                        ___________
                       |   Packet  |
                       | Statistics|
                       | Collector |
                       |           |
                        -----------

                Figure 3.  Management Plane Configuration Set


   3.3.6 Security Plane

   3.3.6.1 Security Plane Configuration Set

      Definition:
        Security features and scaling enabled for the Accelerated Stress
        Test.

      Discussion:
        The Security Plane Configuration Set includes the configuration
        and scaling of ACLs, Firewall, IPsec, and User Access, as shown
        in Figure 4.

         ____________      ____________         ____________
        |            |    |   Secure   |       |    User    |
        |ACL/Firewall|    |  Protocol  |     __|   Access   |
        |            |    |            |    |  |            |
         ------------      ------------     |   ------------
             |                  |           |
             |                  |           |
             |                  \/          |
             |         ___________          |
             |        |           |         |
             |------->|    DUT    |<--------|
                      |           |
                       -----------
                Figure 4.  Security Configuration Module

Poretsky and Rao                                                        [Page 14]


INTERNET-DRAFT Terminology for Accelerated Stress Benchmarking  July 2004

      Measurement units:
        N/A

      Issues:
        None

      See Also:
        ACL Configuration Set
        Secure Protocol Configuration Set
        Password Login Configuration Set

   3.3.6.2 Security Plane Startup Conditions

      Definition:
        Security Plane conditions that occur at the start
        of the Accelerated Stress Benchmarking to establish conditions
        for the remainder of the test.

      Discussion:
        Startup Conditions may cause stress on the DUT and produce
        failure.  Startup Conditions for the Security Plane include
        session establishment rate, number of sessions established
        and number of policies learned, and number of user access
        sessions opened.

      Measurement units:
        N/A

      Issues:
        None

      See Also:
        Startup Conditions
        Data Plane Startup Conditions
        Management Plane Startup Conditions
        Security Plane Startup Conditions

   3.4 Instability

   3.4.1 Instability Phase

     Definition:
        The portion of the benchmarking test in which the
        Instability Conditions are offered to the DUT.

     Discussion:
        The Instability Phase is the middle Phase of
        of the benchmarking test following the Startup
        Phase and preceding the Recovery Phase.

     Measurement Units:
        None

Poretsky and Rao                                                        [Page 15]


INTERNET-DRAFT Terminology for Accelerated Stress Benchmarking  July 2004

     Issues:
        None

     See Also:
        Instability Conditions
        Startup Phase
        Recovery Phase

   3.4.2 Benchmarks

   3.4.2.1 Unstable Aggregate Forwarding Rate

     Definition:
        Rate of taffic forwarded by the DUT during the
        Instability Phase.

      Discussion:
        Unstable Aggregated Forwarding Rate is an instantaneous
        measurement of the Aggregate Forwarding Rate during the
        Instability Phase.  It is recommended that the
        Unstable Aggregate Forwarding Rate is measured at
        one-second intervals.

      Measurement units:
        pps

      Issues:
        None

      See Also:
        Instability Conditions
        Aggregate Forwarding Rate

   3.4.2.2 Degraded Aggregate Forwarding Rate

      Definition:
        The reduction in Aggregate Forwarding Rate during the
        Instability Phase.

      Discussion:
        The Degraded Aggregate Forwarding Rate is calculated
        for each measurement of the Unstable Aggregate
        Forwarding Rate.  The Degraded Aggregate Forwarding
        Rate is calculated by subtracting each measurement
        of the Unstable Aggregate Forwarding Rate from the
        Stable Aggregate Forwarding Rate, such that

        Degraded Forwarding Rate =
        Stable Aggregate Forwarding Rate -
                Unstable Aggregate Forwarding Rate

        Ideally, the Degraded Aggregate Forwarding Rate is zero.

Poretsky and Rao                                                        [Page 16]


INTERNET-DRAFT Terminology for Accelerated Stress Benchmarking  July 2004

      Measurement Units:
        pps

      Issues:
        None

      See Also:
        Instability Phase
        Unstable Aggregate Forwarding Rate

   3.4.2.3 Average Degraded Aggregate Forwarding Rate

      Definition
        DUT Benchmark that is the calculated average of the
        obtained Degraded Forwarding Rates.

      Discussion:

      Measurement Units:
        pps

      Issues:
        None

      See Also:
        Degraded Aggregate Forwarding Rate

   3.4.2.4 Unstable Uncontrolled Sessions Lost

     Definition:
        Control Plane sessions that are in the down state
        but were not intentionally brought down during the
        Instability Phase.

      Discussion:
        The test equipment is able to control protocol
        session state with the DUT.  The test equipment
        is also to monitor for sessions lost with the
        DUT which the test equipment itself did not
        intentionally bring down.

      Measurement units:
        sessions

      Issues:
        None

      See Also:
        Controlled Session Loss
        Uncontrolled Session Loss



Poretsky and Rao                                                        [Page 17]


INTERNET-DRAFT Terminology for Accelerated Stress Benchmarking  July 2004

   3.4.3 Instability Conditions

   3.4.3.1 Control Plane Instability Conditions

      Definition:
        Control Plane conditions that occur during the Accelerated Stress
        Benchmark to produce instability and stress the DUT.

      Discussion:
        Control Plane Instability Conditions are experienced by the DUT
        after the Startup Conditions have completed.  Control Plane
        Instability Conditions experienced by the DUT include session
        loss, route withdrawal, and route cost changes.

      Measurement units:
        N/A

      Issues:
        None

      See Also:
        Instability Conditions
        Data Plane Instability Conditions
        Management Plane Instability Conditions
        Security Plane Instability Conditions

   3.4.3.2 Data Plane Instability Conditions

     Definition:
        Data Plane conditions that occur during the Accelerated Stress
        Benchmark to produce instability and stress the DUT.

      Discussion:
        Data Plane Instability Conditions are experienced by the DUT
        after the Startup Conditions have completed.  Data Plane
        Instability Conditions experienced by the DUT include interface
        shutdown, link loss, and overloaded links.

      Measurement units:
        N/A

      Issues:
        None

      See Also:
        Instability Conditions
        Control Plane Instability Conditions
        Management Plane Instability Conditions
        Security Plane Instability Conditions




Poretsky and Rao                                                        [Page 18]


INTERNET-DRAFT Terminology for Accelerated Stress Benchmarking  July 2004

   3.4.3.3 Management Plane Instability Conditions

     Definition:
        Management Plane conditions that occur during the Accelerated
        Life Benchmark to produce instability and stress the DUT.

      Discussion:
        Management Plane Instability Conditions are experienced by the DUT
        after the Startup Conditions have completed.  Management Plane
        Instability Conditions experienced by the DUT include repeated
        FTP of large files.

      Measurement units:
        N/A

      Issues:
        None

      See Also:
        Instability Conditions
        Control Plane Instability Conditions
        Data Plane Instability Conditions
        Security Plane Instability Conditions

   3.4.3.4 Security Plane Instability Conditions

     Definition:
        Security Plane conditions that occur during the Accelerated
        Life Benchmark to produce instability and stress the DUT.

      Discussion:
        Security Plane Instability Conditions are experienced by the DUT
        after the Startup Conditions have completed.  Security Plane
        Instability Conditions experienced by the DUT include session
        loss and uninitiated policy changes.

      Measurement units:
        N/A

      Issues:
        None

      See Also:
        Instability Conditions
        Control Plane Instability Conditions
        Data Plane Instability Conditions
        Management Plane Instability Conditions






Poretsky and Rao                                                        [Page 19]


INTERNET-DRAFT Terminology for Accelerated Stress Benchmarking  July 2004

   3.5 Recovery

   3.5.1 Recovery Phase

     Definition:
        The portion of the benchmarking test in which the
        Startup Conditions are generated with the DUT, but
        the Instability Conditions are no longer offered to
        the DUT.

     Discussion:
        The Recovery Phase is the final Phase of the
        benchmarking test following the Startup Phase and
        Instability Phase. Startup Conditions must not be
        Restarted.

     Measurement Units:
        None

     Issues:
        None

     See Also:
        Startup Conditions
        Startup Phase
        Instability Conditions
        Instability Phase

   3.5.2 Benchmarks

   3.5.2.1 Recovered Aggregate Forwarding Rate

      Definition
        Rate of traffic forwarded by the DUT during the Recovery
        Phase.

      Discussion:
        Recovered Aggregate Forwarding Rate is an instantaneous
        measurement of the Aggregate Forwarding Rate during the
        Recovery Phase.  It is recommended that the Recovered
        Aggregate Forwarding Rate is measured at one-second
        intervals.  Ideally, each measurement of the Recovered
        Aggregate Forwarding Rate equals the Stable Aggregate
        Forwarding Rate because the Instability Conditions
        do not exist in both the Startup and Recovery Phases.

      Measurement Units:
        pps

      Issues:
        None


Poretsky and Rao                                                        [Page 20]


INTERNET-DRAFT Terminology for Accelerated Stress Benchmarking  July 2004

      See Also:
        Aggregate Forwarding Rate
        Recovery Phase
        Recovered Aggregate Forwarding Rate
        Startup Phase
        Stable Aggregate Forwarding Rate

   3.5.2.2 Recovery Time

      Definition
        The amount of time for the Recovered Aggregate Forwarding
        Rate to become equal to the Stable Aggregate Forwarding Rate.

      Discussion
        Recovery Time is measured beginning at the instant the
        Instability Phase ends until the Recovered Aggregate
        Forwarding Rate equals the Stable Aggregate Forwarding
        Rate for a minimum of 180 consecutive seconds.

      Measurement Units:
        seconds

      Issues:
        None

      See Also:
        Recovered Aggregate Forwarding  Rate
        Stable Aggregate Forwarding Rate

   3.5.2.3 Recovered Uncontrolled Control Plane Sessions Lost

     Definition:
        Control Plane sessions that are in the down state
        but were not intentionally brought down during the
        Recovery Phase.

      Discussion:
        The test equipment is able to control protocol
        session state with the DUT.  The test equipment
        is also to monitor for sessions lost with the
        DUT which the test equipment itself did not
        intentionally bring down.

      Measurement units:
        sessions

      Issues:
        None

      See Also:
        Controlled Session Loss
        Uncontrolled Session Loss

Poretsky and Rao                                                        [Page 21]


INTERNET-DRAFT Terminology for Accelerated Stress Benchmarking  July 2004

   4. Security Considerations
        Documents of this type do not directly effect the security of
        the Internet or of corporate networks as long as benchmarking
        is not performed on devices or systems connected to operating
        networks.


   5. References

      [1]   Bradner, S., Editor, "Benchmarking Terminology for Network
            Interconnection Devices", RFC 1242, July 1991.

      [2]   Mandeville, R., "Benchmarking Terminology for LAN Switching
            Devices", RFC 2285, June 1998.

      [3]   Bradner, S. and McQuaid, J., "Benchmarking Methodology for
            Network Interconnect Devices", RFC 2544, March 1999.

      [4]   "Core Router Evaluation for Higher Availability", Scott
            Poretsky, NANOG 25, June 8, 2002, Toronto, CA.

      [5]   "Router Stress Testing to Validate Readiness for Network
            Deployment", Scott Poretsky, IEEE CQR 2003.

      [6]   Poretsky, S. and Rao, S., "Methodology for Accelerated
            Stress Benchmarking", draft-ietf-bmwg-acc-bench-meth-00,
            work in progress, July 2004.

   6. Author's Address

        Scott Poretsky
        Quarry Technologies
        8 New England Executive Park
        Burlington, MA 01803
        USA
        Phone: + 1 781 395 5090
        EMail: sporetsky@quarrytech.com

        Shankar Rao
        Qwest Communications
        Denver, CO
        USA
        Phone: + 1 303 437 6643
        Email: shankar.rao@qwest.com





Poretsky and Rao                                                        [Page 22]


INTERNET-DRAFT Terminology for Accelerated Stress Benchmarking  July 2004

7.  Full Copyright Statement

        Copyright (C) The Internet Society (1998).  All Rights
        Reserved.

        This document and translations of it may be copied and
        furnished to others, and derivative works that comment on or
        otherwise explain it or assist in its implementation may be
        prepared, copied, published and distributed, in whole or in
        part, without restriction of any kind, provided that the above
        copyright notice and this paragraph are included on all such
        copies and derivative works.  However, this document itself may
        not be modified in any way, such as by removing the copyright
        notice or references to the Internet Society or other Internet
        organizations, except as needed for the purpose of developing
        Internet standards in which case the procedures for copyrights
        defined in the Internet Standards process must be followed, or
        as required to translate it into languages other than English.

        The limited permissions granted above are perpetual and will
        not be revoked by the Internet Society or its successors or
        assigns.  This document and the information contained herein is
        provided on an "AS IS" basis and THE INTERNET SOCIETY AND THE
        INTERNET ENGINEERING TASK FORCE DISCLAIMS ALL WARRANTIES,
        EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY
        THAT THE USE OF THE INFORMATION HEREIN WILL NOT INFRINGE ANY
        RIGHTS OR ANY IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS
        FOR A PARTICULAR PURPOSE.




   Appendix 1.  White Box Benchmarking Terminology

   Minimum Available Memory

      Definition:
        Minimum DUT Available Memory during the duration of the
        Accelerated Stress Test.

      Discussion:
        It is necessary to monitor DUT memory to measure this
        benchmark.

      Measurement units:
        bytes

      Issues:
        None

      See Also:
        Maximum CPU Utilization

Poretsky and Rao                                                        [Page 23]


INTERNET-DRAFT Terminology for Accelerated Stress Benchmarking  July 2004

   Maximum CPU Utilization

      Definition:
        Maximum DUT CPU utilization during the duration of the
        Accelerated Stress Test.

      Discussion:
        It is necessary to monitor DUT CPU Utilization to measure
        this benchmark.

      Measurement units:
        %

      Issues:
        None

      See Also:
        Minimum Available Memory














Poretsky and Rao                                                        [Page 24]


Html markup produced by rfcmarkup 1.129c, available from https://tools.ietf.org/tools/rfcmarkup/