[Docs] [txt|pdf] [Tracker] [WG] [Email] [Diff1] [Diff2] [Nits]

Versions: (draft-hamilton-bmwg-ca-bench-meth) 00 01 02 03 04

Internet Engineering Task Force                              M. Hamilton
Internet-Draft                                                      Ixia
Intended status: Informational                                  S. Banks
Expires: August 5, 2013                                Aerohive Networks
                                                                Feb 2013


       Benchmarking Methodology for Content-Aware Network Devices
                    draft-ietf-bmwg-ca-bench-meth-04

Abstract

   This document defines a set of test scenarios and metrics that can be
   used to benchmark content-aware network devices.  The scenarios in
   the following document are intended to more accurately predict the
   performance of these devices when subjected to dynamic traffic
   patterns.  This document will operate within the constraints of the
   Benchmarking Working Group charter, namely black box characterization
   in a laboratory environment.

Status of this Memo

   This Internet-Draft is submitted in full conformance with the
   provisions of BCP 78 and BCP 79.

   Internet-Drafts are working documents of the Internet Engineering
   Task Force (IETF).  Note that other groups may also distribute
   working documents as Internet-Drafts.  The list of current Internet-
   Drafts is at http://datatracker.ietf.org/drafts/current/.

   Internet-Drafts are draft documents valid for a maximum of six months
   and may be updated, replaced, or obsoleted by other documents at any
   time.  It is inappropriate to use Internet-Drafts as reference
   material or to cite them other than as "work in progress."

   This Internet-Draft will expire on August 5, 2013.

Copyright Notice

   Copyright (c) 2013 IETF Trust and the persons identified as the
   document authors.  All rights reserved.

   This document is subject to BCP 78 and the IETF Trust's Legal
   Provisions Relating to IETF Documents
   (http://trustee.ietf.org/license-info) in effect on the date of
   publication of this document.  Please review these documents
   carefully, as they describe your rights and restrictions with respect
   to this document.  Code Components extracted from this document must



Hamilton & Banks         Expires August 5, 2013                 [Page 1]


Internet-Draft    Methodology for Content-Aware Devices         Feb 2013


   include Simplified BSD License text as described in Section 4.e of
   the Trust Legal Provisions and are provided without warranty as
   described in the Simplified BSD License.


Table of Contents

   1.  Introduction . . . . . . . . . . . . . . . . . . . . . . . . .  4
     1.1.  Requirements Language  . . . . . . . . . . . . . . . . . .  5
   2.  Scope  . . . . . . . . . . . . . . . . . . . . . . . . . . . .  5
   3.  Test Setup . . . . . . . . . . . . . . . . . . . . . . . . . .  5
     3.1.  Test Considerations  . . . . . . . . . . . . . . . . . . .  6
     3.2.  Clients and Servers  . . . . . . . . . . . . . . . . . . .  6
     3.3.  Traffic Generation Requirements  . . . . . . . . . . . . .  6
     3.4.  Discussion of Network Limitations  . . . . . . . . . . . .  6
     3.5.  Framework for Traffic Specification  . . . . . . . . . . .  8
     3.6.  Multiple Client/Server Testing . . . . . . . . . . . . . .  8
     3.7.  Device Configuration Considerations  . . . . . . . . . . .  8
       3.7.1.  Network Addressing . . . . . . . . . . . . . . . . . .  9
       3.7.2.  Network Address Translation  . . . . . . . . . . . . .  9
       3.7.3.  TCP Stack Considerations . . . . . . . . . . . . . . .  9
       3.7.4.  Other Considerations . . . . . . . . . . . . . . . . .  9
   4.  Benchmarking Tests . . . . . . . . . . . . . . . . . . . . . .  9
     4.1.  Maximum Application Session Establishment Rate . . . . . . 10
       4.1.1.  Objective  . . . . . . . . . . . . . . . . . . . . . . 10
       4.1.2.  Setup Parameters . . . . . . . . . . . . . . . . . . . 10
       4.1.3.  Procedure  . . . . . . . . . . . . . . . . . . . . . . 10
       4.1.4.  Measurement  . . . . . . . . . . . . . . . . . . . . . 10
         4.1.4.1.  Maximum Application Flow Rate  . . . . . . . . . . 10
         4.1.4.2.  Application Flow Duration  . . . . . . . . . . . . 11
         4.1.4.3.  Application Efficiency . . . . . . . . . . . . . . 11
         4.1.4.4.  Application Flow Latency . . . . . . . . . . . . . 11
     4.2.  Application Throughput . . . . . . . . . . . . . . . . . . 11
       4.2.1.  Objective  . . . . . . . . . . . . . . . . . . . . . . 11
       4.2.2.  Setup Parameters . . . . . . . . . . . . . . . . . . . 11
       4.2.3.  Procedure  . . . . . . . . . . . . . . . . . . . . . . 12
       4.2.4.  Measurement  . . . . . . . . . . . . . . . . . . . . . 12
         4.2.4.1.  Maximum Throughput . . . . . . . . . . . . . . . . 12
         4.2.4.2.  Maximum Application Flow Rate  . . . . . . . . . . 12
         4.2.4.3.  Application Flow Duration  . . . . . . . . . . . . 12
         4.2.4.4.  Application Efficiency . . . . . . . . . . . . . . 12
         4.2.4.5.  Packet Loss  . . . . . . . . . . . . . . . . . . . 12
         4.2.4.6.  Application Flow Latency . . . . . . . . . . . . . 12
     4.3.  Malformed Traffic Handling . . . . . . . . . . . . . . . . 13
       4.3.1.  Objective  . . . . . . . . . . . . . . . . . . . . . . 13
       4.3.2.  Setup Parameters . . . . . . . . . . . . . . . . . . . 13
       4.3.3.  Procedure  . . . . . . . . . . . . . . . . . . . . . . 13
       4.3.4.  Measurement  . . . . . . . . . . . . . . . . . . . . . 13



Hamilton & Banks         Expires August 5, 2013                 [Page 2]


Internet-Draft    Methodology for Content-Aware Devices         Feb 2013


   5.  IANA Considerations  . . . . . . . . . . . . . . . . . . . . . 13
   6.  Security Considerations  . . . . . . . . . . . . . . . . . . . 13
   7.  References . . . . . . . . . . . . . . . . . . . . . . . . . . 14
     7.1.  Normative References . . . . . . . . . . . . . . . . . . . 14
     7.2.  Informative References . . . . . . . . . . . . . . . . . . 15
     7.3.  URL References . . . . . . . . . . . . . . . . . . . . . . 15
   Appendix A.  Example Traffic Mix . . . . . . . . . . . . . . . . . 15
   Appendix B.  Malformed Traffic Algorithm . . . . . . . . . . . . . 17
   Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 19










































Hamilton & Banks         Expires August 5, 2013                 [Page 3]


Internet-Draft    Methodology for Content-Aware Devices         Feb 2013


1.  Introduction

   Content-aware and deep packet inspection (DPI) device deployments
   have grown significantly in recent years.  No longer are devices
   simply using Ethernet and IP headers to make forwarding decisions.
   This class of device now uses application-specific data to make these
   decisions.  For example, a web-application firewall (WAF) may use
   search criteria upon the HTTP uniform resource indicator (URI)[1] to
   decide whether a HTTP GET method may traverse the network.  In the
   case of lawful/legal intercept technology, a device could use the
   phone number within the Session Description Protocol[14] to determine
   whether a voice-over-IP phone may be allowed to connect.  In addition
   to the development of entirely new classes of devices, devices that
   could historically be classified as 'stateless' or raw forwarding
   devices are now performing DPI functionality.  Devices such as core
   and edge routers are now being developed with DPI functionality to
   make more intelligent routing and forwarding decisions.

   The Benchmarking Working Group (BMWG) has historically produced
   Internet Drafts and Requests for Comment that are focused
   specifically on creating output metrics that are derived from a very
   specific and well-defined set of input parameters that are completely
   and unequivocally reproducible from test bed to test bed.  The end
   goal of such methodologies is to, in the words of the RFC 2544 [2],
   reduce "specsmanship" in the industry and hold vendors accountable
   for performance claims.

   The end goal of this methodology is to generate performance metrics
   in a lab environment that will closely relate to actual observed
   performance on production networks.  By utilizing dynamic traffic
   patterns relevant to modern networks, this methodology should be able
   to closely tie laboratory and production metrics.  It should be
   further noted than any metrics acquired from production networks
   SHOULD be captured according to the policies and procedures of the
   IPPM or PMOL working groups.

   An explicit non-goal of this document is to replace existing
   methodology/terminology pairs such as RFC 2544 [2]/RFC 1242 [3] or
   RFC 3511 [4]/RFC 2647 [5].  The explicit goal of this document is to
   create a methodology more suited for modern devices while
   complementing the data acquired using existing BMWG methodologies.
   This document does not assume completely repeatable input stimulus.
   The nature of application-driven networks is such that a single
   dropped packet inherently changes the input stimulus from a network
   perspective.  While application flows will be specified in great
   detail, it simply is not practical to require totally repeatable
   input stimulus.




Hamilton & Banks         Expires August 5, 2013                 [Page 4]


Internet-Draft    Methodology for Content-Aware Devices         Feb 2013


1.1.  Requirements Language

   The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
   "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
   document are to be interpreted as described in RFC 2119 [6].


2.  Scope

   Content-aware devices take many forms, shapes and architectures.
   These devices are advanced network interconnect devices that inspect
   deep into the application payload of network data packets to do
   classification.  They may be as simple as a firewall that uses
   application data inspection for rule set enforcement, or they may
   have advanced functionality such as performing protocol decoding and
   validation, anti-virus, anti-spam and even application exploit
   filtering.  The document will universally call these devices
   middleboxes, as defined by RFC 3234 [7].

   This document is strictly focused on examining performance and
   robustness across a focused set of metrics: throughput(min/max/avg/
   sample std dev), transaction rates(successful/failed), application
   response times, concurrent flows, and unidirectional packet latency.
   None of the metrics captured through this methodology are specific to
   a device and the results are DUT implementation independent.
   Functional testing of the DUT is outside the scope of this
   methodology.

   Devices such as firewalls, intrusion detection and prevention
   devices, wireless LAN controllers, application delivery controllers,
   deep packet inspection devices, wide-area network(WAN) optimization
   devices, and unified threat management systems generally fall into
   the content-aware category.  While this list may become obsolete,
   these are a subset of devices that fall under this scope of testing.


3.  Test Setup

   This document will be applicable to most test configurations and will
   not be confined to a discussion on specific test configurations.
   Since each DUT/SUT will have their own unique configuration, users
   SHOULD configure their device with the same parameters that would be
   used in the actual deployment of the device or a typical deployment,
   if the actual deployment is unknown.  A summary of the DUT
   configuration MUST be published with the final benchmarking results.
   In order to improve repeatability, the published configuration
   information SHOULD include command-line scripts used to configure the
   DUT, if any, and SHOULD also include any configuration information



Hamilton & Banks         Expires August 5, 2013                 [Page 5]


Internet-Draft    Methodology for Content-Aware Devices         Feb 2013


   for the test equipment used."

3.1.  Test Considerations

3.2.  Clients and Servers

   Content-aware device testing SHOULD involve multiple clients and
   multiple servers.  As with RFC 3511 [4], this methodology will use
   the terms virtual clients/servers because both the client and server
   will be represented by the tester and not actual clients/servers.
   Similarly defined in RFC 3511 [4], a data source may emulate multiple
   clients and/or servers within the context of the same test scenario.
   The test report SHOULD indicate the number of virtual clients/servers
   used during the test.  IANA has reserved address ranges for
   laboratory characterization.  These are defined for IPv4 and IPv6 by
   RFC 2544 Appendix C [2] and RFC 5180 Section 5.2 [8] respectively and
   SHOULD be consulted prior to testing.

3.3.  Traffic Generation Requirements

   The explicit purposes of content-aware devices vary widely, but these
   devices use information deeper inside the application flow to make
   decisions and classify traffic.  This methodology will utilize
   traffic flows that resemble real application traffic without
   utilizing captures from live production networks.  Application Flows,
   as defined in Section 1.1 RFC 2724 [9] are able to be well-defined
   without simply referring to a network capture.  An example traffic
   template is defined and listed in Appendix A of this document.  A
   user of this methodology is free to utilize the example mix as
   provided in the appendix.  If a user of this methodology understands
   the traffic patterns in their production network, that user MAY use
   the template provided in Appendix A to describe a traffic mix
   appropriate for their environment.  In all cases, users MUST report
   the traffic mix used in the test, and SHOULD report this using a
   template similar to that in Appendix A.

   The test tool SHOULD be able to create application flows between
   every client and server, regardless of direction.  The tester SHOULD
   be able to open TCP connections on multiple destination ports and
   SHOULD be able to direct UDP traffic to multiple destination ports.

3.4.  Discussion of Network Limitations

   Prior to executing the methodology as outlined in the following
   sections, it is imperative to understand the implications of
   utilizing representative application flows for the traffic content of
   the benchmarking effort.  One interesting aspect of utilizing
   application flows is that each flow is inherently different from



Hamilton & Banks         Expires August 5, 2013                 [Page 6]


Internet-Draft    Methodology for Content-Aware Devices         Feb 2013


   every other application flow.  The content of each flow will vary
   from application to application, and in most cases, even varies
   within the same type of application flow.  The following description
   of the methodology will individually benchmark every individual type
   and subset of application flow, prior to performing similar tests
   with a traffic mix as specified either by the example mix in
   Appendix A, or as defined by the user of this methodology.

   The purpose of this process is to ensure that any performance
   implications that are discovered during the mixed testing aren't due
   to the inherent physical network limitations.  As an example of this
   phenomena, it is useful to examine a network device inserted into a
   single path, as illustrated in the following diagram.

                                +----------+
                     +---+  1gE |   DUT/   | 1gE  +---+
                     |C/S|------|   SUT    |------|C/S|
                     +---+      +----------+      +---+

                      Simple Inline DUT Configuration

                    Figure 1: Simple Middle-box Example

   For the purpose of this discussion, let's take a hypothetical
   application flow that utilizes UDP for the transport layer.  Assume
   that the sample transaction we will be using to model this particular
   flow requires 10 UDP datagrams to complete the transaction.  For
   simplicity, each datagram within the flow is exactly 64 bytes,
   including associated Ethernet, IP, and UDP overhead.  With any
   network device,there are always three metrics which interact with
   each other: number of concurrent application flows, number of
   application flows per second, and layer-7 throughput.

   Our example test bed is a single-path device connected by 1 gigabit
   Ethernet links.  The purpose of this benchmark effort is to quantify
   the number of application flows per second that may be processed
   through our device under test.  Let's assume that the result from our
   scenario is that the DUT is able to process 10,000 application flows
   per second.  The question is whether that ceiling is the actual
   ceiling of the device, or if it is actually being limited by one of
   the other metrics.  If we do the appropriate math, 10000 flows per
   second, with each flow at 640 total bytes means that we are achieving
   an aggregate bitrate of roughly 49 Mbps.  This is dramatically less
   than the 1 gigabit physical link we are using.  We can conclude that
   10,000 flows per second is in fact the performance limit of the
   device.

   If we change the example slightly and increase the size of each



Hamilton & Banks         Expires August 5, 2013                 [Page 7]


Internet-Draft    Methodology for Content-Aware Devices         Feb 2013


   datagram to 1312 bytes, then it becomes necessary to recompute the
   load.  Assuming the same observed DUT limitation of 10,000 flows per
   second, it must be ensured that this is an artifact of the DUT, and
   not of physical limitations.  For each flow, we'll require 104,960
   bits. 10,000 flows per second implies a throughput of roughly 1 Gbps.
   At this point, we cannot definitively answer whether the DUT is
   actually limited to 10,000 flows per second.  If we are able to
   modify the scenario, and utilize 10 Gigabit interfaces, then perhaps
   the flow per second ceiling will be reached at a higher number than
   10,000.

   This example illustrates why a user of this methodology SHOULD
   benchmark each application variant individually to ensure that the
   cause of a measured limit is fully understood

3.5.  Framework for Traffic Specification

   The following table SHOULD be specified for each application flow
   variant.

   o  Data Exchanged By Flow, Bits

   o  Offered Percentage of Total Flows

   o  Transport Protocol(s)

   o  Destination Port(s)

3.6.  Multiple Client/Server Testing

   In actual network deployments, connections are being established
   between multiple clients and multiple servers simultaneously.  Device
   vendors have been known to optimize the operation of their devices
   for easily defined patterns.  The connection sequence ordering
   scenarios a device will see on a network will likely be much less
   deterministic.  In fact, many application flows have multiple layer 4
   connections within a single flow, with client and server reversing
   roles.  Flow initiation SHOULD be in a pseudo-random manner across
   ingress ports.

3.7.  Device Configuration Considerations

   The configuration of the DUT may have an effect on the observed
   results of the following methodology.  A comprehensive, but certainly
   not exhaustive, list of potential considerations is listed below.






Hamilton & Banks         Expires August 5, 2013                 [Page 8]


Internet-Draft    Methodology for Content-Aware Devices         Feb 2013


3.7.1.  Network Addressing

   The IANA has issued a range of IP addresses to the BMWG for purposes
   of benchmarking.  Please refer to RFC 2544 [2] and RFC 5180 [8] for
   more details.  If more IPv4 addresses are required than the RFC 2544
   allotment provides, then allocations from the private address space
   as defined in RFC 1918 [10] may be used.

3.7.2.  Network Address Translation

   Many content-aware devices are capable of performing Network Address
   Translation (NAT)[5].  If the final deployment of the DUT will have
   this functionality enabled, then the DUT SHOULD also have it enabled
   during the execution of this methodology.  It MAY be beneficial to
   perform the test series in both modes in order to determine the
   performance differential when using NAT.  The test report SHOULD
   indicate whether NAT was enabled during the testing process.

3.7.3.  TCP Stack Considerations

   The IETF has historically provided guidance and information on TCP
   stack considerations.  This methodology is strictly focused on
   performance metrics at layers above 4, thus does not specifically
   define any TCP stack configuration parameters of either the tester or
   the DUTs.  The TCP configuration of the tester MUST remain constant
   across all DUTs in order to ensure comparable results.  While the
   following list of references is not exhaustive, each document
   contains a relevant discussion on TCP stack considerations.

   The general IETF TCP roadmap is defined in RFC 4614 [11] and
   congestion control algorithms are discussed in Section 2 of RFC 3148
   [12] with even more detailed references.  TCP receive and congestion
   window sizes are discussed in detail in RFC 6349 [13].

3.7.4.  Other Considerations

   Various content-aware devices will have widely varying feature sets.
   In the interest of representative test results, the DUT features that
   will likely be enabled in the final deployment SHOULD be used.  This
   methodology is not intended to advise on which features should be
   enabled, but to suggest using actual deployment configurations.


4.  Benchmarking Tests

   Each of the following benchmark scenarios SHOULD be run with each of
   the single application flow templates.  Upon completion of all
   iterations, the mixed test SHOULD be completed, subject to the



Hamilton & Banks         Expires August 5, 2013                 [Page 9]


Internet-Draft    Methodology for Content-Aware Devices         Feb 2013


   traffic mix as defined by the user.

4.1.  Maximum Application Session Establishment Rate

4.1.1.  Objective

   To determine the maximum rate through which a device is able to
   establish and complete application flows as defined by
   draft-ietf-bmwg-ca-bench-term-00.

4.1.2.  Setup Parameters

   The following parameters SHOULD be used and reported for all tests:

   For each application protocol in use during the test run, the table
   provided in Section 3.5 SHOULD be published.

4.1.3.  Procedure

   The test SHOULD generate application network traffic that meets the
   conditions of Section 3.3.  The traffic pattern SHOULD begin with an
   application flow rate of 10% of expected maximum.  The test SHOULD be
   configured to increase the attempt rate in units of 10% up through
   110% of expected maximum.  In the case where expected maximum is
   limited by physical link rate as discovered through Appendix A, the
   maximum rate will attempted will be 100% of expected maximum, or
   "wire-speed performance".  The duration of each loading phase SHOULD
   be at least 30 seconds.  This test MAY be repeated, each subsequent
   iteration beginning at 5% of expected maximum and increasing session
   establishment rate to 110% of the maximum observed from the previous
   test run.

   This procedure MAY be repeated any reasonable number of times with
   the results being averaged together.

4.1.4.  Measurement

   The following metrics MAY be determined from this test, and SHOULD be
   observed for each application protocol within the traffic mix:

4.1.4.1.  Maximum Application Flow Rate

   The test tool SHOULD report the maximum rate at which application
   flows were completed, as defined by RFC 2647 [5], Section 3.7.  This
   rate SHOULD be reported individually for each application protocol
   present within the traffic mix.





Hamilton & Banks         Expires August 5, 2013                [Page 10]


Internet-Draft    Methodology for Content-Aware Devices         Feb 2013


4.1.4.2.  Application Flow Duration

   The test tool SHOULD report the minimum, maximum and average
   application duration, as defined by RFC 2647 [5], Section 3.9.  This
   duration SHOULD be reported individually for each application
   protocol present within the traffic mix.

4.1.4.3.  Application Efficiency

   The test tool SHOULD report the application efficiency, similarly
   defined for TCP by RFC 6349 [13].


                          Transmitted Bytes - Retransmitted Bytes
      App Efficiency % =  ---------------------------------------  X 100
                                     Transmitted Bytes

           Figure 2: Application Efficiency Percent Calculation

   Note than calculation less than 100% does not necessarily imply
   noticeably degraded performance since certain applications utilize
   algorithms to maintain a quality user experience in the face of data
   loss.

4.1.4.4.  Application Flow Latency

   The test tool SHOULD report the minimum, maximum and average amount
   of time an application flow member takes to traverse the DUT, as
   defined by RFC 1242 [3], Section 3.8.  This value SHOULD be reported
   individually for each application protocol present within the traffic
   mix.

4.2.  Application Throughput

4.2.1.  Objective

   To determine the maximum rate through which a device is able to
   forward bits when using application flows as defined in the previous
   sections.

4.2.2.  Setup Parameters

   The same parameter reporting procedure as described in Section 4.1.2
   SHOULD be used for all tests.







Hamilton & Banks         Expires August 5, 2013                [Page 11]


Internet-Draft    Methodology for Content-Aware Devices         Feb 2013


4.2.3.  Procedure

   This test will attempt to send application flows through the device
   at a flow rate of 30% of the maximum, as observed in Section 4.1.
   This procedure MAY be repeated with the results from each iteration
   averaged together.

4.2.4.  Measurement

   The following metrics MAY be determined from this test, and SHOULD be
   observed for each application protocol within the traffic mix:

4.2.4.1.  Maximum Throughput

   The test tool SHOULD report the minimum, maximum and average
   application throughput.

4.2.4.2.  Maximum Application Flow Rate

   The test tool SHOULD report the maximum rate at which application
   flows were completed, as defined by RFC 2647 [5], Section 3.7.  This
   rate SHOULD be reported individually for each application protocol
   present within the traffic mix.

4.2.4.3.  Application Flow Duration

   The test tool SHOULD report the minimum, maximum and average
   application duration, as defined by RFC 2647 [5], Section 3.9.  This
   duration SHOULD be reported individually for each application
   protocol present within the traffic mix.

4.2.4.4.  Application Efficiency

   The test tool SHOULD report the application efficiency as defined in
   Section 4.1.4.3.

4.2.4.5.  Packet Loss

   The test tool SHOULD report the number of packets lost or dropped
   from source to destination.

4.2.4.6.  Application Flow Latency

   The test tool SHOULD report the minimum, maximum and average amount
   of time an application flow member takes to traverse the DUT, as
   defined by RFC 1242 [3], Section 3.13.  This value SHOULD be reported
   individually for each application protocol present within the traffic
   mix.



Hamilton & Banks         Expires August 5, 2013                [Page 12]


Internet-Draft    Methodology for Content-Aware Devices         Feb 2013


4.3.  Malformed Traffic Handling

4.3.1.  Objective

   To determine the effects on performance and stability that malformed
   traffic may have on the DUT.

4.3.2.  Setup Parameters

   The same parameters SHOULD be used for Transport-Layer and
   Application Layer Parameters previously specified in Section 4.1.2
   and Section 4.2.2.

4.3.3.  Procedure

   This test will utilize the procedures specified previously in
   Section 4.1.3 and Section 4.2.3.  When performing the procedures
   listed previously, the tester should generate malformed traffic at
   all protocol layers.  This is commonly known as fuzzed traffic.
   Fuzzing techniques generally modify portions of packets, including
   checksum errors, invalid protocol options, and improper protocol
   conformance.

   The process by which the tester SHOULD generate the malformed traffic
   is outlined in detail in Appendix B.

4.3.4.  Measurement

   For each protocol present in the traffic mix, the metrics specified
   by Section 4.1.4 and Section 4.2.4 MAY be determined.  This data may
   be used to ascertain the effects of fuzzed traffic on the DUT.


5.  IANA Considerations

   This memo includes no request to IANA.

   All drafts are required to have an IANA considerations section (see
   the update of RFC 2434 [15] for a guide).  If the draft does not
   require IANA to do anything, the section contains an explicit
   statement that this is the case (as above).  If there are no
   requirements for IANA, the section will be removed during conversion
   into an RFC by the RFC Editor.


6.  Security Considerations

   Benchmarking activities as described in this memo are limited to



Hamilton & Banks         Expires August 5, 2013                [Page 13]


Internet-Draft    Methodology for Content-Aware Devices         Feb 2013


   technology characterization using controlled stimuli in a laboratory
   environment, with dedicated address space and the other constraints
   RFC 2544 [2].

   The benchmarking network topology will be an independent test setup
   and MUST NOT be connected to devices that may forward the test
   traffic into a production network, or mis-route traffic to the test
   management network


7.  References

7.1.  Normative References

   [1]   Berners-Lee, T., Fielding, R., and L. Masinter, "Uniform
         Resource Identifier (URI): Generic Syntax", STD 66, RFC 3986,
         January 2005.

   [2]   Bradner, S. and J. McQuaid, "Benchmarking Methodology for
         Network Interconnect Devices", RFC 2544, March 1999.

   [3]   Bradner, S., "Benchmarking terminology for network
         interconnection devices", RFC 1242, July 1991.

   [4]   Hickman, B., Newman, D., Tadjudin, S., and T. Martin,
         "Benchmarking Methodology for Firewall Performance", RFC 3511,
         April 2003.

   [5]   Newman, D., "Benchmarking Terminology for Firewall
         Performance", RFC 2647, August 1999.

   [6]   Bradner, S., "Key words for use in RFCs to Indicate Requirement
         Levels", BCP 14, RFC 2119, March 1997.

   [7]   Carpenter, B. and S. Brim, "Middleboxes: Taxonomy and Issues",
         RFC 3234, February 2002.

   [8]   Popoviciu, C., Hamza, A., Van de Velde, G., and D. Dugatkin,
         "IPv6 Benchmarking Methodology for Network Interconnect
         Devices", RFC 5180, May 2008.

   [9]   Handelman, S., Stibler, S., Brownlee, N., and G. Ruth, "RTFM:
         New Attributes for Traffic Flow Measurement", RFC 2724,
         October 1999.

   [10]  Rekhter, Y., Moskowitz, R., Karrenberg, D., Groot, G., and E.
         Lear, "Address Allocation for Private Internets", BCP 5,
         RFC 1918, February 1996.



Hamilton & Banks         Expires August 5, 2013                [Page 14]


Internet-Draft    Methodology for Content-Aware Devices         Feb 2013


   [11]  Duke, M., Braden, R., Eddy, W., and E. Blanton, "A Roadmap for
         Transmission Control Protocol (TCP) Specification Documents",
         RFC 4614, September 2006.

   [12]  Mathis, M. and M. Allman, "A Framework for Defining Empirical
         Bulk Transfer Capacity Metrics", RFC 3148, July 2001.

   [13]  Constantine, B., Forget, G., Geib, R., and R. Schrage,
         "Framework for TCP Throughput Testing", RFC 6349, August 2011.

7.2.  Informative References

   [14]  Handley, M., Jacobson, V., and C. Perkins, "SDP: Session
         Description Protocol", RFC 4566, July 2006.

   [15]  Narten, T. and H. Alvestrand, "Guidelines for Writing an IANA
         Considerations Section in RFCs", BCP 26, RFC 5226, May 2008.

7.3.  URL References

   [16]  Sandvine Corporation, "http://www.sandvine.com/general/
         document.download.asp?docID=58&sourceID=0", 2012.


Appendix A.  Example Traffic Mix

   This appendix shows an example case of a protocol mix that may be
   used with this methodology.  This mix closely represents the research
   published by Sandvine [16] in their biannual report for the first
   half of 2012 on North American fixed access service provider
   networks.

      +------------+------------------+--------------------+--------+
      |  Direction | Application Flow |       Options      |  Value |
      +------------+------------------+--------------------+--------+
      |  Upstream  |    BitTorrent    |                    |        |
      |            |                  | Avg Flow Size (L7) | 512 MB |
      |            |                  |   Flow Percentage  |  44.4% |
      |            |       HTTP       |                    |        |
      |            |                  | Avg Flow Size (L7) | 128 kB |
      |            |                  |   Flow Percentage  |  7.3%  |
      |            |       Skype      |                    |        |
      |            |                  | Avg Flow Size (L7) |  8 MB  |
      |            |                  |   Flow Percentage  |  4.9%  |
      |            |      SSL/TLS     |                    |        |
      |            |                  | Avg Flow Size (L7) | 128 kB |
      |            |                  |   Flow Percentage  |  3.2%  |
      |            |      Netflix     |                    |        |



Hamilton & Banks         Expires August 5, 2013                [Page 15]


Internet-Draft    Methodology for Content-Aware Devices         Feb 2013


      |            |                  | Avg Flow Size (L7) | 500 kB |
      |            |                  |   Flow Percentage  |  3.1%  |
      |            |     PPStream     |                    |        |
      |            |                  | Avg Flow Size (L7) | 500 MB |
      |            |                  |   Flow Percentage  |  2.2%  |
      |            |      YouTube     |                    |        |
      |            |                  | Avg Flow Size (L7) |  4 MB  |
      |            |                  |   Flow Percentage  |  1.9%  |
      |            |     Facebook     |                    |        |
      |            |                  | Avg Flow Size (L7) |  2 MB  |
      |            |                  |   Flow Percentage  |  1.9%  |
      |            |      Teredo      |                    |        |
      |            |                  | Avg Flow Size (L7) | 500 MB |
      |            |                  |   Flow Percentage  |  1.2%  |
      |            |  Apple iMessage  |                    |        |
      |            |                  | Avg Flow Size (L7) |  40 kB |
      |            |                  |   Flow Percentage  |  1.1%  |
      |            |     Bulk TCP     |                    |        |
      |            |                  | Avg Flow Size (L7) | 128 kB |
      |            |                  |   Flow Percentage  |  28.8% |
      | Downstream |      Netflix     |                    |        |
      |            |                  | Avg Flow Size (L7) | 512 MB |
      |            |                  |   Flow Percentage  |  32.9% |
      |            |      YouTube     |                    |        |
      |            |                  | Avg Flow Size (L7) |  5 MB  |
      |            |                  |   Flow Percentage  |  13.8% |
      |            |       HTTP       |                    |        |
      |            |                  | Avg Flow Size (L7) |  1 MB  |
      |            |                  |   Flow Percentage  |  12.1% |
      |            |    BitTorrent    |                    |        |
      |            |                  | Avg Flow Size (L7) | 500 MB |
      |            |                  |   Flow Percentage  |  6.3%  |
      |            |      iTunes      |                    |        |
      |            |                  | Avg Flow Size (L7) |  32 MB |
      |            |                  |   Flow Percentage  |  3.8%  |
      |            |    Flash Video   |                    |        |
      |            |                  | Avg Flow Size (L7) | 100 MB |
      |            |                  |   Flow Percentage  |  2.6%  |
      |            |       MPEG       |                    |        |
      |            |                  | Avg Flow Size (L7) | 100 MB |
      |            |                  |   Flow Percentage  |  2.0%  |
      |            |       RTMP       |                    |        |
      |            |                  | Avg Flow Size (L7) |  50 MB |
      |            |                  |   Flow Percentage  |  2.0%  |
      |            |       Hulu       |                    |        |
      |            |                  | Avg Flow Size (L7) | 300 MB |
      |            |                  |   Flow Percentage  |  1.8%  |
      |            |      SSL/TLS     |                    |        |



Hamilton & Banks         Expires August 5, 2013                [Page 16]


Internet-Draft    Methodology for Content-Aware Devices         Feb 2013


      |            |                  | Avg Flow Size (L7) | 256 kB |
      |            |                  |   Flow Percentage  |  1.6%  |
      |            |     Bulk TCP     |                    |        |
      |            |                  | Avg Flow Size (L7) | 500 kB |
      |            |                  |   Flow Percentage  |  21.1% |
      +------------+------------------+--------------------+--------+

                     Table 1: Example Traffic Pattern


Appendix B.  Malformed Traffic Algorithm

   Each application flow will be broken into multiple transport
   segments, IP packets, and Ethernet frames.  The malformed traffic
   algorithm looks very similar to the IP Stack Integrity Checker
   project at http://isic.sourceforge.net.

   The algorithm is very simple and starts by defining each of the
   fields within the TCP/IP stack that will be malformed during
   transmission.  The following table illustrates the Ethernet, IPv4,
   IPv6, TCP, and UDP fields which are able to be malformed by the
   algorithm.  The first column lists the protocol, the second column
   shows the actual header field name, with the third column showing the
   percentage of packets that should have the field modified by the
   malformation algorithm.


























Hamilton & Banks         Expires August 5, 2013                [Page 17]


Internet-Draft    Methodology for Content-Aware Devices         Feb 2013


         +--------------+--------------------------+-------------+
         |   Protocol   |       Header Field       | Malformed % |
         +--------------+--------------------------+-------------+
         | Total Frames |                          |      1%     |
         |   Ethernet   |                          |             |
         |              |      Destination MAC     |      0%     |
         |              |        Source MAC        |      1%     |
         |              |         Ethertype        |      1%     |
         |              |            CRC           |      1%     |
         | IP Version 4 |                          |             |
         |              |          Version         |      1%     |
         |              |            IHL           |      1%     |
         |              |      Type of Service     |      1%     |
         |              |       Total Length       |      1%     |
         |              |      Identification      |      1%     |
         |              |           Flags          |      1%     |
         |              |      Fragment Offset     |      1%     |
         |              |       Time to Live       |      1%     |
         |              |         Protocol         |      1%     |
         |              |      Header Checksum     |      1%     |
         |              |      Source Address      |      1%     |
         |              |    Destination Address   |      1%     |
         |              |          Options         |      1%     |
         |              |          Padding         |      1%     |
         |      UDP     |                          |             |
         |              |        Source Port       |      1%     |
         |              |     Destination Port     |      1%     |
         |              |          Length          |      1%     |
         |              |         Checksum         |      1%     |
         |      TCP     |                          |             |
         |              |        Source Port       |      1%     |
         |              |     Destination Port     |      1%     |
         |              |      Sequence Number     |      1%     |
         |              |  Acknowledgement Number  |      1%     |
         |              |        Data Offset       |      1%     |
         |              |      Reserved(3 bit)     |      1%     |
         |              |       Flags(9 bit)       |      1%     |
         |              |        Window Size       |      1%     |
         |              |         Checksum         |      1%     |
         |              |      Urgent Pointer      |      1%     |
         |              | Options(Variable Length) |      1%     |
         +--------------+--------------------------+-------------+

                     Table 2: Malformed Header Values

   This algorithm is to be used across the regular application flows
   used throughout the rest of the methodology.  As each frame is
   emitted from the test tool, a pseudo-random number generator will



Hamilton & Banks         Expires August 5, 2013                [Page 18]


Internet-Draft    Methodology for Content-Aware Devices         Feb 2013


   indicate whether the frame is to be malformed by creating a number
   between 0 and 100.  If the number is less than the percentage defined
   in the table, then that frame will be malformed.  If the frame is to
   be malformed, then each of the headers in the table present within
   the frame will follow the same process.  If it is determined that a
   header field should be malformed, the same pseudo-random number
   generator will be used to create a random number for the specified
   header field.


Authors' Addresses

   Mike Hamilton
   Ixia
   Austin, TX  78730
   US

   Phone: +1 512 636 2303
   Email: mhamilton@ixiacom.com


   Sarah Banks
   Aerohive Networks
   San Jose, CA  95134
   US

   Email: sbanks@aerohive.com
























Hamilton & Banks         Expires August 5, 2013                [Page 19]


Html markup produced by rfcmarkup 1.123, available from https://tools.ietf.org/tools/rfcmarkup/