Network Working Group                                    Brooks Hickman
Internet-Draft                                   Spirent Communications
Expiration Date: April November 2002                             David Newman
                                                           Network Test
                                                        Saldju Tadjudin
                                                 Spirent Communications
                                                           Terry Martin
                                                          M2networx INC
                                                           October 2001
                                                               May 2002

          Benchmarking Methodology for Firewall Performance
              <draft-ietf-bmwg-firewall-03.txt>
              <draft-ietf-bmwg-firewall-04.txt>

Status of this Memo

   This document is an Internet-Draft and is in full conformance with
   all provisions of Section 10 of RFC2026.

   Internet-Drafts are working documents of the Internet Engineering
   Task Force (IETF), its areas, and its working groups.  Note that
   other groups may also distribute working documents as Internet-
   Drafts.

   Internet-Drafts are draft documents valid for a maximum of six
   months and may be updated, replaced, or obsoleted by other documents
   at any time.  It is inappropriate to use Internet-Drafts as
   reference material or to cite them other than as "work in progress."

   The list of current Internet-Drafts can be accessed at
   http://www.ietf.org/ietf/1id-abstracts.txt

   The list of Internet-Draft Shadow Directories can be accessed at
   http://www.ietf.org/shadow.html.

Table of Contents

   1. Introduction . . . . . . . . . . . . . . . . . . . . . . . .  2
   2. Requirements . . . . . . . . . . . . . . . . . . . . . . . .  2
   3. Scope  . . . . . . . . . . . . . . . . . . . . . . . . . . .  2
   4. Test setup . . . . . . . . . . . . . . . . . . . . . . . . .  2
     4.1 Test Considerations   . . . . . . . . . . . . . . . . . .  3
     4.2 Virtual Client/Servers  . . . . . . . . . . . . . . . . .  3
     4.3 Test Traffic Requirements . . . . . . . . . . . . . . . .  4
     4.4 DUT/SUT Traffic Flows . . . . . . . . . . . . . . . . . .  4
     4.5 Multiple Client/Server Testing  . . . . . . . . . . . . .  5
     4.6 NAT(Network Address Translation)  . . . . . . . . . . . .  5
     4.7 Rule Sets . . . . . . . . . . . . . . . . . . . . . . . .  5
     4.8 Web Caching . . . . . . . . . . . . . . . . . . . . . . .  5
     4.9 Authentication  . . . . . . . . . . . . . . . . . . . . .  6
   5. Benchmarking Tests . . . . . . . . . . . . . . . . . . . . .  6
     5.1 Concurrent Connection Capacity IP throughput . . . . . . . . . . . . .  6
     5.2 Maximum Connection Setup Rate . . . . . . . . .  6
     5.2 Concurrent TCP Connection Capacity  . . . . .  7
     5.3 Connection Establishment Time . . . . . .  7
     5.3 Maximum TCP Connection Establishment Rate . . . . . . . .  9 10
     5.4 Maximum TCP Connection Teardown Time  . . . . . . Tear Down Rate . . . . . . . . . . 11 12
     5.5 Denial Of Service Handling  . . . . . . . . . . . . . . . 13
     5.6 HTTP Transfer Rate  . . . . . . . . . . . . . . . . . . . 14
     5.7 HTTP Concurrent Transaction Capacity  . . . . . . . . . . 17
     5.8 HTTP Transaction Rate . . 14
     5.7 IP Fragmentation Handling . . . . . . . . . . . . . . . . 16
     5.8 18
     5.9 Illegal Traffic Handling  . . . . . . . . . . . . . . . . 18
     5.9 Latency 19
     5.10 IP Fragmentation Handling  . . . . . . . . . . . . . . . 20
     5.11 Latency  . . . . . . . . . . . 19 . . . . . . . . . . . . . 22
   Appendices  . . . . . . . . . . . . . . . . . . . . . . . . . . 22 25
     A. HyperText Transfer Protocol(HTTP)  . . . . . . . . . . . . 22 25
     B. Connection Establishment Time Measurements . . . . . . . . 25
     C. Connection Tear Down Time Measurements . . . . . . . . . . 26
     C. References . . . . . . . . . . . . . . . . . . . . . . . . 23 26

1. Introduction

   This document provides methodologies for the performance
   benchmarking of firewalls. It provides methodologies in four areas:
   forwarding, connection, latency and filtering. In addition to
   defining the tests, this document also describes specific formats
   for reporting the results of the tests.

   A previous document, "Benchmarking Terminology for Firewall
   Performance" [1], defines many of the terms that are used in this
   document. The terminology document SHOULD be consulted before
   attempting to make use of this document.

2. Requirements

   The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
   "SHOULD", "SHOULD NOT", "RECOMMENDED",  "MAY", and "OPTIONAL" in
   this document are to be interpreted as described in RFC 2119.

3. Scope

   Firewalls can provide a single point of defense between networks.
   Usually, a firewall protects private networks from the public or
   shared networks to which it is connected. A firewall can be as
   simple as a device that filters different packets or as complex
   as a group of devices that combine packet filtering and
   application-level proxy or network translation services. This RFC
   will focus on developing benchmark testing of DUT/SUTs, wherever
   possible, independent of their implementation.

4. Test Setup

   Test configurations defined in this document will be confined to
   dual-homed and tri-homed as shown in figure 1 and figure 2
   respectively.

   Firewalls employing dual-homed configurations connect two networks.
   One interface of the firewall is attached to the unprotected
   network, typically the public network(Internet). The other interface
   is connected to the protected network, typically the internal LAN.

   In the case of dual-homed configurations, servers which are made
   accessible to the public(Unprotected) network are attached to the
   private(Protected) network.

      +----------+                                       +----------+
      |          |    |       +----------+        |      |          |
      | Servers/ |----|       |          |        |------| Servers/ |
      | Clients  |    |       |          |        |      | Clients  |
      |          |    |-------|  DUT/SUT |--------|      |          |
      +----------+    |       |          |        |      +----------+
                      |       +----------+        |
           Protected  |                           | Unprotected
            Network                                   Network
                           Figure 1(Dual-Homed)

   Tri-homed[1] configurations employ a third segment called a DMZ.
   Demilitarized Zone(DMZ). With tri-homed configurations, servers
   accessible to the public network are attached to the DMZ. Tri-Homed
   configurations offer additional security by separating server server(s)
   accessible to the public network from internal hosts.

      +----------+                                       +----------+
      |          |    |       +----------+        |      |          |
      | Clients  |----|       |          |        |------| Servers/ |
      |          |    |       |          |        |      | Clients  |
      +----------+    |-------|  DUT/SUT |--------|      |          |
                      |       |          |        |      +----------+
                      |       +----------+        |
            Protected |            |              | Unprotected
             Network               |                   Network
                                   |
                                   |
                           -----------------
                                       |    DMZ
                                       |
                                       |
                                +-----------+
                                |           |
                                | Servers   |
                                |           |
                                +-----------+

                             Figure 2(Tri-Homed)

4.1 Test Considerations

4.2 Virtual Clients/Servers

   Since firewall testing may involve data sources which emulate
   multiple users or hosts, the methodology uses the terms virtual
   clients/servers. For these firewall tests, virtual clients/servers
   specify application layer entities which may not be associated with
   a unique physical interface. For example, four virtual clients may
   originate from the same data source[1]. The test report SHOULD
   indicate the number of virtual clients and virtual servers
   participating in the test on a per interface(See 4.0) basis. test.

   Testers MUST synchronize all data sources participating in a test.

4.3 Test Traffic Requirements

   While the function of a firewall is to enforce access control
   policies, the criteria by which those policies are defined vary
   depending on the implementation. Firewalls may use network layer,
   transport layer or, in many cases, application-layer criteria to
   make access-control decisions. Therefore, the test equipment used to
   benchmark the DUT/SUT performance MUST consist of real clients and
   servers generating legitimate layer seven conversations.

   For the purposes of benchmarking firewall performance, performance this document
   references HTTP 1.1
   will be referenced in this document, or higher as the application layer entity,
   although the methodologies may be used as a template for
   benchmarking with other applications. Since testing may involve
   proxy based DUT/SUTs, HTTP version considerations are discussed in
   appendix A.

4.4 DUT/SUT Traffic Flows

   Since the number of interfaces are not fixed, the traffic flows will
   be dependent upon the configuration used in benchmarking the
   DUT/SUT. Note that the term "traffic flows" is associated with
   client-to- server
   client-to-server requests.

   For Dual-Homed configurations, there are two unique traffic flows:

      Client	   Server
      ------         ------
      Protected   -> Unprotected
      Unprotected -> Protected

   For Tri-Homed configurations, there are three unique traffic flows:

      Client	   Server
      ------         ------
      Protected ->   Unprotected
      Protected ->   DMZ
      Unprotected -> DMZ

4.5 Multiple Client/Server Testing

   One or more clients may target multiple servers for a given
   application. Each virtual client MUST initiate requests(Connection,
   object transfers, etc.) connections in a
   round-robin fashion. For example, if the test consisted of six
   virtual clients targeting three servers, the pattern would be as
   follows:

      Client          Target Server(In order of request)
      #1              1     2     3     1...
	#2              2     3     1     2...
	#3              3     1     2     3...
	#4              1     2     3     1...
	#5              2     3     1     2...
	#6              3     1     2     3...

4.6 NAT(Network Network Address Translation) Translation(NAT)

   Many firewalls implement network address translation(NAT), a
   function which translates internal host IP addresses attached to
   the protected network to a virtual IP address for communicating
   across the unprotected network(Internet). This involves additional
   processing on the part of the DUT/SUT and may impact performance.
   Therefore, tests SHOULD be ran with NAT disabled and NAT enabled
   to determine the performance differentials. The test report SHOULD MUST
   indicate whether NAT was enabled or disabled.

4.7 Rule Sets

   Rule sets[1] are a collection of access control policies that
   determines
   determine which packets the DUT/SUT will forward and which it will
   reject[1]. The Since criteria by which these access control policies may
   be defined will vary depending on the capabilities of the DUT/SUT.
   The scope of this document DUT/SUT,
   the following is limited to how the providing guidelines for configuring
   rule sets should
   be applied when testing the DUT/SUT.

   The firewall monitors the incoming traffic and checks to make sure
   that benchmarking the traffic meets one performance of the defined rules before allowing it
   to be forwarded. DUT/SUT.

   It is RECOMMENDED that a rule be entered for each host(Virtual
   client). Although many firewalls permit groups of IP
   addresses to be defined for a given rule, tests In addition, testing SHOULD be performed
   with large using different
   size rule sets, which are more stressful sets to determine its impact on the performance of the
   DUT/SUT.

   The DUT/SUT SHOULD Rule sets MUST be configured to denies access to all traffic
   which was not previously defined in the rule set.

4.7 Web a manner, such that, rules
   associated with actual test traffic are configured at the end of the
   rule set and not the beginning.

   The DUT/SUT SHOULD be configured to deny access to all traffic
   which was not previously defined in the rule set. The test report
   SHOULD include the DUT/SUT configured rule set(s).

4.7 Web Caching

   Some firewalls include caching agents in order to reduce network load. When
   making a request through a caching agent, the caching agent attempts
   to service the response from its internal memory. The cache itself
   saves responses it receives, such as responses for HTTP GET
   requests. The report Testing SHOULD indicate whether be performed with any caching
   was enabled or disabled agents on the DUT/SUT.
   DUT/SUT disabled.

4.8 Authentication

   Access control may involve authentication processes such as user,
   client or session authentication. Authentication is usually
   performed by devices external to the firewall itself, such as an
   authentication servers server(s) and may add to the latency of the system.
   Any authentication processes MUST be included as part of connection
   setup process.

5. Benchmarking Tests

5.1 Concurrent Connection Capacity IP Throughput

5.1.1 Objective

   To determine the maximum number throughput of TCP concurrent connections through
   or with network-layer data transversing
   the DUT/SUT, as defined in RFC2647[1]. RFC1242[1]. Note that while RFC1242
   uses the term frames, which is associated with the link layer, the
   procedure uses the term packets, since it is referencing the
   network layer. This test will employ
   a step algorithm is intended to obtain baseline the maximum number ability of concurrent TCP
   connections that
   the DUT/SUT can maintain. to forward packets at the network layer.

5.1.2 Setup Parameters

   The following parameters MUST be defined for all tests.

   Connection Attempt Rate defined:

      Packet size - The rate, expressed Number of bytes in connections per
   second, at which new TCP connection requests are attempted. The
   rate SHOULD be set lower than maximum rate at which the DUT/SUT can
   accept connection requests.

   Connection Step Count - Defines the number of additional TCP
   connections attempted for each iteration IP packet, exclusive of the step search
   algorithm.

   Object Size any
      link layer header or checksums.

      Test Duration - Defines the number Duration of bytes to be transferred the test, expressed in
   response to a HTTP 1.1 GET request . It is RECOMMENDED to use
   1-byte object sizes for this test. seconds.

5.1.3 Procedure

   Each virtual client

   The tester will attempt to establish TCP connections offer client/server traffic to its
   target server(s), using either the target server's DUT/SUT,
   consisting of unicast IP address or NAT
   proxy address, at a fixed rate in a round robin fashion. Each
   iteration will involve packets. The tester MUST offer the virtual clients attempting to establish packets
   at a
   fixed number of additional TCP connections. This search algorithm
   will be repeated until either:

      - One or more of the additional connection attempts fail to
        complete.
      - One or more of the previously established connections fail. constant rate. The test MUST include application layer data transfers in order to
   validate the TCP connections since, in the case MAY consist of proxy based
   DUT/SUTs, either bi-directional or
   unidirectional traffic, with the tester does not own both sides client offering a unicast stream of
   packets to the connection.
   After all the addition connections have been attempted server for each
   iteration of the test, the virtual client(s) will request an
   object from its target server(s) using latter.

   The test MAY employ an HTTP 1.1 GET request on iterative search algorithm. Each iteration
   will involve the additional connections as well as all previously established
   connections. Both tester varying the client request and server response MUST exclude intended load until the connection-token close maximum
   rate, at which no packet loss occurs, is found. Since backpressure
   mechanisms may be employed, resulting in the connection header(See Appendix A).

5.1.4 Measurements

   Maximum concurrent connections - Total number of TCP connections
   open for intended load and
   offered load being different, the last successful iteration test SHOULD be performed in either
   a packet based or time based manner as described in RFC2889[7]. As
   with RFC1242, the search
   algorithm.

5.1.5 Reporting Format

   5.1.5.1 Application-Layer Reporting: term packet is used in place of frame. The test report MUST note the use
   duration of HTTP 1.1 client and server
   and the object size.

   5.1.5.2 Transport-Layer Reporting:

   The test report portion of each trial MUST note be at least 30
   seconds.

   When comparing DUT/SUTs with different MTUs, it is RECOMMENDED to
   limit the connection attempt maximum IP size tested to the maximum MTU supported by all
   of the DUT/SUTs.

5.1.4 Measurement

5.1.4.1 Network Layer

   Throughput - Maximum offered load, expressed in either bits per
   second or packets per second, at which no packet loss is detected.

   Forwarding Rate - Forwarding rate, connection
   step count expressed in either bits per
   second or packets per second, the device is observed to
   successfully forward to the correct destination interface in
   response to a specified offered load.

5.1.4 Reporting Format

   The test report MUST note the packet size(s), test duration,
   throughput and maximum concurrent connections measured.

   5.1.5.3 Log Files forwarding rate. If the test involved offering
   packets which target more than one segment(Protected, Unprotected
   or DMZ), the report MUST identify the results as an aggregate
   throughput measurement.

   The throughput results SHOULD be reported in the format of a table
   with a row for each of the tested packet sizes.  There SHOULD be
   columns for the packet size, the intended load, the offered load,
   resultant throughput and forwarding rate for each test.

   A log file MAY be generated which includes the TCP connection
   attempt rate, connection step count, object size packet size, test
   duration and for each iteration:

      - Step Iteration
	- Pass/Fail Status. Status
      - Total TCP connections established. packets offered
      - Number of previously established TCP connections that failed. Total packets forwarded
      - Number of the additional TCP connections that failed to
        complete. Intended load
      - Offered load(If applicable)
      - Forwarding rate

5.2 Maximum Concurrent TCP Connection Setup Rate Capacity

5.2.1 Objective

   To determine the maximum number of concurrent TCP connection setup rate connections
   supported through or with the DUT/SUT, as defined by in RFC2647[1]. This test will employ a
   search algorithm to obtain the maximum rate at which TCP connections
   can be established through or with the DUT/SUT.

   5.2.2 Setup Parameters

   The following parameters MUST be defined.

   Initial defined for all tests:

   5.2.2.1 Transport-Layer Setup Parameters

   Connection Attempt Rate - The aggregate rate, expressed in
   connections per second, at which the initial new TCP connection requests are
   attempted.

   Number of Connections - Defines the number of TCP connections that
   must be established. The number MUST rate SHOULD be between the number of
   participating virtual clients and set at or lower than the maximum number supported by
   the DUT/SUT.

   Object Size - Defines
   rate at which the number of bytes to be transferred in
   response to a HTTP 1.1 GET request . It is RECOMMENDED to use
   1-byte object sizes for this test. DUT/SUT can accept connection requests.

   Age Time - The time, expressed in seconds, the DUT/SUT will keep a
   connection in it's state its connection table after receiving a TCP FIN or RST
   packet.

   5.2.2.2 Transport-Layer Setup Parameters

   Validation Method - HTTP 1.1 or higher MUST be used for this test.

   Object Size - Defines the number of bytes, excluding any bytes
   associated with the HTTP header, to be transferred in response to an
   HTTP 1.1 or higher GET request.

5.2.3 Procedure

   An iterative search algorithm will MAY be used to determine the maximum
   connection rate. This test iterates
   number of concurrent TCP connections supported through different connection rates or with a fixed the
   DUT/SUT.

   For each iteration, the aggregate number of concurrent TCP
   connections attempted by the virtual clients to
   their associated server(s).

   Each iteration client(s) will use the same connection establishment and
   connection validation algorithms defined in the concurrent capacity
   test(See section 5.1).

   Between each iteration be varied. The
   destination address will be that of the test, the tester must close all
   connections completed for server or that of the previous iteration. In addition,
   it is RECOMMENDED to abort all unsuccessful connections attempted. NAT
   proxy. The tester aggregate rate will wait for the period of time, specified be defined by age time,
   before continuing to the next iteration.

5.2.4 Measurements

   Highest connection rate - Highest attempt
   rate, and will be attempted in connections per second,
   for which a round-robin fashion(See 4.5).

   To validate all TCP connections completed successfully.

5.2.5 Reporting Format

   5.1.5.1 Application-Layer Reporting:

   The test report MUST note connections, the use of virtual client(s) MUST request an
   object using an HTTP 1.1 client and server
   and the object size.

   5.2.5.2 Transport-Layer Reporting: or higher GET request. The test report requests MUST note the number of connections, age time
   and highest connection rate measured.

   5.2.5.3 Log Files

   A log file MAY be generated which includes the number of TCP
   connections attempt, age time, object size and
   for
   initiated on each iteration:

      - Step Iteration
      - Pass/Fail Status.
      - Attempted Connection Establishment Rate
      - Total TCP connections established.
      - Number connection after all of the TCP connections that failed to complete.

5.3 Connection Establishment Time

5.3.1 Objective

  To determine the connection establishment times[1] through or with
  the DUT/SUT.

  A connection for a client/server application is not atomic, in that
  it not only involves transactions at have
   been established.

   When testing proxy-based DUT/SUTs, the application layer, but
  involves first establishing a connection virtual client(s) MUST
   request two objects using one HTTP 1.1 or more underlying
  connection oriented protocols(TCP, ATM, etc). Therefore, it higher GET requests. The first
   GET request is
  encouraged to make separate measurements required for each connection oriented
  protocol required time establishment
   measurements as specified in order to perform the application layer
  transaction.

5.3.2 Setup Parameters appendix B. The following parameters second request is used
   for validation as previously mentioned. When comparing proxy and
   non-proxy based DUT/SUTs, the test MUST be defined.

   Connection Attempt Rate - The rate, expressed performed in connections per
   second, at which new TCP connection requests are attempted. It the same
   manner.

   Between each iteration, it is RECOMMENDED not to exceed the maximum connection rate found in
   section 5.2.

   Connection Attempt Step count - Defines that the number of additional tester issue a
   TCP RST referencing all connections attempted for each iteration of the step algorithm.

   Maximum Attempt Connection Count - Defines the maximum number previous
   iteration, regardless of
   TCP connections attempted in whether or not the test.

5.3.3 Procedure

   Each virtual client will connection attempt to establish TCP connections to its
   target server(s) at a fixed rate in a round robin fashion. Each
   iteration was
   successful. The tester will involve the virtual clients attempting wait for age time before continuing to establish
   a fixed
   the next iteration.

5.2.4 Measurements

   5.2.4.1 Application-Layer measurements

   Number of objects requested

   Number of objects returned

   5.2.4.2 Transport-Layer measurements

   Maximum concurrent connections - Total number of additional TCP connections until
   open for the maximum attempt
   connection count is reached.

   After each connection has been completed, last successful iteration performed in the virtual client(s) MUST
   request search
   algorithm.

   The following measurements SHOULD be performed on a 1-byte object from its target server(s) using an HTTP 1.1
   GET request. Both the client request and server response MUST exclude

                             [Page  9]
   the connection-token close per iteration
   basis:

   Minimum connection establishment time - Lowest TCP connection
   establishment time measured as defined in the appendix B.

   Maximum connection header(See Appendix A).

   Since testing may involve proxy based DUT/SUTs, which terminates the establishment time - Highest TCP connection, making a direct measurement connection
   establishment time measured as defined in appendix B.

   Average connection establishment time - The mean of the TCP all measurements
   of connection establishment times.

   Aggregate connection establishment time is not possible since - The total of all
   measurements of connection establishment times.

5.2.5 Reporting Format

   5.2.5.1 Application-Layer Reporting:

   The test report MUST note the protocol involves an
   odd object size, number of messages completed
   requests and number of completed responses.

   The intermediate results of the search algorithm MAY be reported
   in establishing a connection. Therefore, when
   testing table format with proxy based firewalls, the datagram following the final
   ACK on the three-way handshake will a column for each iteration. There SHOULD be used in determining
   rows for the
   connection setup time.

   The following shows the timeline for the TCP connection setup
   involving a proxy DUT/SUT number of requests attempted, number of requests
   completed, number of responses attempted and is referenced in number of responses
   completed. The table MAY be combined with the measurements
   section(5.3.4). Note transport-layer
   reporting, provided that the table identify this methodology may be applied when
   measuring other connection oriented protocols involving as an odd number application
   layer measurement.

   Version information:

   The test report MUST note the version of messages in establishing a connection.

      t0: Client sends a SYN.
      t1: Proxy sends a SYN/ACK.
      t2: Client sends HTTP client(s) and
   server(s).

5.2.5.2 Transport-Layer Reporting:

   The test report MUST note the final ACK.
      t3: Proxy establishes separate connection with server.
      t4: Client sends TCP datagram to server.
      *t5: Proxy sends ACK attempt rate, age time and
   maximum concurrent connections measured.

   The intermediate results of the datagram to client.

   * While t5 is not considered part of search algorithm MAY be reported
   in the TCP connection establishment,
     acknowledgement format of t4 must a table with a column for each iteration. There
   SHOULD be received rows for the total number of TCP connections attempted,
   total number of TCP connections completed, minimum TCP connection to be
     considered successful.

   When comparing firewalls with different architectures, such as proxy
   based and stateful packet filtering, the same method SHOULD be used
   when measuring
   establishment times.

5.3.4 Measurements

  For each iteration of the test, the tester MUST measure the minimum, time, maximum and average TCP connection establishment times. If the DUT/SUT
  is proxy based, the time,
   average connection establishment time is considered to be
  from and the time aggregate connection
   establishment time.

5.3 Maximum TCP Connection Establishment Rate

5.3.1 Objective

   To determine the first bit of maximum TCP connection establishment rate through
   or with the first SYN packet is transmitted by
  the client to the time the client transmits the first bit DUT/SUT, as defined by RFC2647[1].

5.3.2 Setup Parameters

   The following parameters MUST be defined for all tests:

   5.3.2.1 Transport-Layer Setup Parameters

   Number of Connections - Defines the first
  acknowledged aggregate number of TCP datagram(t4-t0
   connections that must be established.

   Age Time - The time, expressed in seconds, the above timeline). For non-proxy
  based DUT/SUTs , the establishment time shall be directly measured and
  is considered to DUT/SUT will keep a
   connection in it's state table after receiving a TCP FIN or RST
   packet.

   5.3.2.2 Transport-Layer Setup Parameters

   Validation Method - HTTP 1.1 or higher MUST be from the time used for this test.

   Object Size - Defines the first bit number of bytes, excluding any bytes
   associated with the first SYN packet
  is transmitted by the client HTTP header, to the time the last bit of the final ACK be transferred in response to an
   HTTP 1.1 or higher GET request.

5.3.3 Procedure Test

   An iterative search algorithm MAY be used to determine the three-way handshake is received by the target server.

  In addition, the tester SHOULD measure the minimum, maximum and
  average connection establishment times for all other underlying
   rate at which the DUT/SUT can accept TCP connection oriented protocols. requests.

   For purposes of benchmarking
  firewall performance, each iteration, the aggregate rate at which TCP connection establishment time
   requests are attempted by the virtual client(s) will be
  considered the interval between the transmission of the first bit
  of the first octet of the packet carrying the connection request
  to the DUT/SUT interface to receipt varied. The
   destination address will be that of the last bit server or that of the last
  octet NAT
   proxy. The aggregate number of the last packet connections, defined by number of the
   connections, will be attempted in a round-robin fashion(See 4.5).

   The same application-layer object transfers required for validation
   and establishment time measurements as described in the concurrent
   TCP connection setup traffic received
  on capacity test MUST be performed.

   Between each iteration, it is RECOMMENDED that the client or server, depending on whether tester issue a given connection
  requires an even or odd number
   TCP RST referencing all connections attempted for the previous
   iteration, regardless of messages, respectfully.

  Tester SHOULD measure whether or not the aggregate connection attempt was
   successful. The tester will wait for age time and before continuing to
   the total
  number next iteration.

5.3.4 Measurements

   5.3.4.1 Application-Layer measurements

   Number of objects requested

   Number of objects returned

   5.3.4.2 Transport-Layer measurements

   Highest connection rate - Highest rate, in connections completed per second,
   for all measured protocols which for each the search algorithm passed.

   The following measurements SHOULD performed on a per iteration
   basis:

   Minimum connection establishment time - Lowest TCP connection
   establishment time measured as defined in appendix B.

   Maximum connection establishment time - Highest TCP connection
   establishment time measured as defined in appendix B.

   Average connection establishment time - The mean of the test. all measurements
   of connection establishment times.

   Aggregate connection establishment time - The total of all
   measurements of connection establishment times.

5.3.5 Reporting Format

   5.3.5.1 Application-Layer Reporting:

   The test report MUST note the use object size(s), number of HTTP 1.1 client and server.

   5.3.5.2 Transport-Layer completed
   requests and Below Reporting: number of completed responses.

   The test report MUST note the TCP connection attempt rate, TCP
   connection attempt step count and maximum TCP connections attempted.

   For each connection oriented protocol the tester measured, the
   connection establishment time intermediate results SHOULD of the search algorithm MAY be reported
   in tabular form a table format with a row column for each iteration of the test. iteration. There SHOULD be
   rows for the number of requests and responses completed. The table
   MAY be combined with the transport-layer reporting, provided that
   the table identify this as an application layer measurement.

   Version information:

   The test report MUST note the version of HTTP client(s) and server(s).

   5.3.5.2 Transport-Layer Reporting:

   The test report MUST note the number of connections, age time and
   highest connection rate measured.

   The intermediate results of the search algorithm MAY be reported
   in the format of a table with a column for each iteration. There
   SHOULD be rows for the iteration count, connection attempt rate, total number of
   TCP connections attempted, total number of TCP connections
   completed, minimum TCP connection establishment time,
   average maximum TCP
   connection establishment time, maximum average connection establishment time, attempted connections completed time
   and the aggregate connection establishment time.

   The report MUST also identify the layer/protocol for which the
   measurements were made.

5.4 Maximum TCP Connection Teardown Time Tear Down Rate

5.4.1 Objective

   To determine the maximum TCP connection teardown time[1] tear down rate through or
   with the
   DUT/SUT. As with the connection establishment time, separate
   measurements SHOULD be taken for each connection oriented protocol
   involved in closing a connection. DUT/SUT, as defined by RFC2647[1].

5.4.2 Setup Parameters

   The following parameters MUST be defined. Each parameters is
   configured with the following considerations.

   Initial connections

   Number of Connections - Defines the number of TCP connections to
   initialize that
   the test with.

   Teardown tester will attempt rate to tear down.

   Age Time - The rate at which time, expressed in seconds, the tester DUT/SUT will attempt keep a
   connection in it's state table after receiving a TCP FIN or RST
   packet.

5.4.3 Procedure

   An iterative search algorithm MAY be used to determine the maximum
   TCP connection tear down rate. The test iterates through different
   TCP connection tear down rates with a fixed number of TCP
   connections.

5.4.3 Procedure

   The virtual clients client(s) will initialize the test by establishing TCP
   connections defined by initial number of connections. The test will use the
   same algorithm for establishing the TCP connections as described in
   the connection capacity test(Section 5.1) with the exception that
   no object transfers are required.

   The virtual clients virtual client(s)
   will then attempt to tear down all of TCP
   connections connections, at a rate
   defined by tear down attempt rate. The
   tester(Virtual Clients) MUST exclude any connections which do not
   properly close in its measurements. For example, connections in
   which benchmarking purposes, the DUT/SUT transmits a TCP RST in response to
   tester MUST use a TCP FIN
   packet or connections which do not acknowledge the FIN packet
   requesting when initiating the connection be closed. tear down.

   In the case of proxy based DUT/SUTs, the DUT/SUT will itself receive
   the final ACK when closing out it's side of in the TCP connection. three-way handshake when a connection is being
   torn down. For validation purposes, the virtual client(s) SHOULD MAY
   verify that the DUT/SUT received the final ACK in the connection tear
   down exchange for all connections by transmitting a TCP datagram(s) datagram
   referencing the previously town down connection(s). connection. A TCP RST should be
   received in response to the TCP datagram(s), if the ACK was received by the
   DUT/SUT. datagram.

5.4.4 Measurements

   The tester MUST measure the minimum, average and maximum TCP

   Highest connection tear down times.  The rate - Highest rate, in connections per
   second, for which all TCP connections were successfully torn down.

   Minimum connection tear down time will
   be considered the interval between the transmission of the first - Lowest TCP
   FIN packet transmitted by the tester requesting a connection tear down to receipt of the ACK packet on the same tester interface.

   The tester SHOULD measure the minimum, maximum and average
   time measured as defined in appendix C.

   Maximum connection tear down
   times for all other underlying connection oriented protocols. For
   purposes of benchmarking firewall performance, the time - Highest TCP connection tear down
   time will be considered the interval between the transmission of the
   first bit of the first octet of the packet carrying the measured as defined in appendix C.

   Average connection tear down
   request to the DUT/SUT interface to receipt of the last bit of the
   last octet time - The mean of the last packet all measurements of the
   connection tear down traffic
   headed in the opposite direction.

   The tester SHOULD measure the aggregate times.

   Aggregate connection tear down time and
   the - The total number of connections torn all measurements
   of connection tear down for each protocol measured. times.

5.4.5 Reporting Format

   The test report MUST note the initial connections , tear down attempt
   rate number of connections, age time and
   highest connection tear down step count.

   For each connection oriented protocol rate measured.

   The intermediate results of the tester measured, search algorithm SHOULD be reported
   in the
   report MUST note format of a table with a column for each iteration. There
   SHOULD be rows for the minimum, average and number of TCP tear downs attempted, number
   of TCP connection tear downs completed, minimum TCP connection tear
   down time, maximum TCP connection tear
   down. In addition, it SHOULD include the aggregate down time, average TCP
   connection tear down time and attempted tear downs completed. The report MUST
   identify the layer/protocol for which the measurements were made.

   Failure analysis:

   The test report SHOULD indicate the number of connections which failed the validation step. aggregate TCP connection tear down
   time.

5.5 Denial Of Service Handling

5.5.1 Objective

   To determine the effect of a denial of service attack on a DUT/SUTs DUT/SUT
   TCP connection establishment and/or forwarding rate. HTTP transfer rates. The Denial Of
   Service Handling denial
   of service handling test MUST be run after obtaining baseline
   Measurements
   measurements from sections 5.2 5.3 and/or 5.6.

   The TCP SYN flood attack exploits TCP's three-way handshake
   mechanism by having an attacking source host generate TCP SYN
   packets with random source addresses towards a victim host, thereby
   consuming that host's resources.

   Some firewalls employ mechanisms to guard against SYN attacks. If such
   mechanisms exist on the DUT/SUT, tests SHOULD be run with these
   mechanisms enabled to determine how well the DUT/SUT can maintain,
   under such attacks, the baseline connection rates and HTTP forwarding
   rates determined in section 5.2 and section 5.6, respectively.

5.5.2 Setup Parameters

   Use the same setup parameters as defined in section 5.2.2 or 5.6.2,
   depending on whether testing against the baseline TCP connection
   setup rate test or HTTP transfer rate test, respectfully.

   In addition, the following setup parameters MUST be defined.

   SYN Attack Rate attack rate - Defines the rate, Rate, expressed in packets per second second, at which
   the server(s) are or NAT proxy address is targeted with TCP SYN packets.

5.5.3 Procedure

   Use the same procedure as defined in section 5.2.3 5.3.3 or 5.6.3,
   Depending
   depending on whether testing against the baseline TCP connection setup
   establishment rate test or HTTP transfer rate test, respectfully. In
   addition, the tester will generate TCP SYN packets targeting the
   server(s) IP address or NAT proxy address at a rate defined by SYN
   attack rate.

   The tester originating the TCP SYN attack MUST be attached to the
   unprotected network. In addition, the tester MUST not respond to the
   SYN/ACK packets sent by target server or NAT proxy in response to
   the SYN packet.

5.5.4 Measurements

   Perform the same measurements as defined in section 5.2.4 or 5.6.4,
   depending on whether testing

   Some firewalls employ mechanisms to guard against the baseline SYN attacks. If
   such mechanisms exist on the DUT/SUT, tests SHOULD be run with these
   mechanisms enabled to determine how well the DUT/SUT can maintain,
   under such attacks, the baseline connection setup establishment rates and
   HTTP transfer rates determined in section 5.3 and section 5.6,
   respectively.

5.5.4 Measurements

   Perform the same measurements as defined in section 5.3.4 or 5.6.4,
   depending on whether testing against the baseline TCP connection
   establishment rate test or HTTP test, transfer rate, respectfully.

   In addition, the tester SHOULD track TCP SYN packets associated with
   the SYN attack which the DUT/SUT forwards on the protected or DMZ
   interface(s).

5.5.5 Reporting Format

   The test SHOULD use the same reporting format as described in
   section 5.2.5 5.3.5 or 5.6.5, depending on whether testing against the
   baseline throughput rates TCP connection establishment rate test or HTTP test, respectively. transfer rate,
   respectfully.

   In addition, the report MUST indicate a denial of service handling
   test, SYN attack rate, number TCP SYN attack packets transmitted
   and the number of TCP SYN attack packets received. forwarded by the DUT/SUT.
   The report SHOULD MUST indicate whether or not the DUT has any SYN attack
   mechanisms enabled.

5.6 HTTP Transfer Rate

5.6.1 Objective

   To determine the bit forwarding rate, as defined by RFC2647, transfer rate of the
   DUT/SUT when presented with HTTP traffic flows. requested object transversing
   the DUT/SUT.

5.6.2 Setup Parameters

   Connection type -

   The tester following parameters MUST use HTTP 1.1 be defined for HTTP measurements. all tests:

   5.6.2.1 Transport-Layer Setup Parameters

   Number of GET Requests connections - Defines the aggregate number of connections
   attempted. The number SHOULD be a multiple of the number of virtual
   clients participating in the test
   5.6.2.2 Application-Layer Setup Parameters

   Session type - The virtual clients/servers MUST use HTTP 1.1 or
   higher.

   GET requests attempted per connection.

   GET Request Rate connection - Defines the rate, in GET requests per second, at
   which number of HTTP 1.1 or
   higher GET requests are attempted on any given per connection.

   Object Size - Defines the number of bytes, excluding any bytes
   associated with the HTTP header, to be transferred in response to an
   HTTP 1.1 or higher GET request.

5.6.3 HTTP Procedure

   Each HTTP 1.1 virtual or higher client will attempt to establish each
   connection to its request one or more objects from
   an HTTP 1.1 target server(s), or higher server using either the target
   server's IP address one or NAT proxy address, in a round robin fashion.
   The tester will initiate more HTTP GET requests for each connection at a
   constant rate requests.
   The aggregate number of connections attempted, defined by GET request rate, regardless of the state number of the DUT/SUT.

   Baseline measurements SHOULD
   connections, MUST be performed using a single GET request
   per connection with a 1-byte object size. evenly divided among all of the participating
   virtual clients.

   If the tester makes virtual client(s) make multiple HTTP GET requests per
   connection, it MUST request the same object size for each GET
   request. Testers SHOULD run multiple Multiple iterations of this test SHOULD be ran using other
   different object sizes and/or multiple requests per connection.
   See appendix A when testing proxy based DUT/SUT regarding HTTP version
   considerations. sizes.

   5.6.4 Measurements

   Version information:

   The test report MUST note the use of an HTTP 1.1 client and server.

   Application Layer

   Bit Forwarding

   5.6.4.1 Application-Layer measurements

   Average Transfer Rate - The bit forwarding average transfer rate of the DUT/SUT MUST,
   at a minimum,
   MUST be measured at the application layer and shall be referenced to the requested object(s).
   The measurement will start on transmission of the first bit of the
   first requested object and end on transmission of the last bit of
   the last requested object. The aggregate bit forwarding average transfer rate, in bits per
   second, will be calculated using the following formula:

                           OBJECTS * OBJECTSIZE * 8
   FORWARDING
   TRANSFER RATE(bit/s) =  --------------------------
                                DURATION

   OBJECTS - Objects successfully transferred

   OBJECTSIZE - Object size in bytes

   DURATION - Aggregate transfer time based on aforementioned time
              references.

   5.6.4.2 Measurements at or below the Transport-Layer and Below

   Bit forwarding rate

   The tester SHOULD make goodput[1] measurements for layers connection-
   oriented protocols at or  below the transport layer SHOULD
   also be performed. Bit forwarding rate for these underlying
   layers/protocols MAY be measured in either layer. Goodput
   measurements MUST only reference the protocols payload, excluding
   any of the protocols header. In addition, the tester MUST exclude
   any bits per seconds associated with the connection establishment, connection
   tear down, security associations or UOTs
   per second. In both cases, connection maintenance.

   Since connection-oriented protocols require that data be
   acknowledged, the measurement offered load[6] will start on transmission vary over the duration of the first packet containing
   test. When performing forwarding rate measurements, the first HTTP GET request and end on
   transmission of tester
   should measure the last packet containing average forwarding rate over the last octet duration of the last
   requested object.
   test.

5.6.5 Reporting Format

   5.6.5.1 Application-Layer reporting

   The aggregate bit forwarding rate, in bits test report MUST note number of GET requests per second,
   will connection,
   and object size.

   The transfer rate results SHOULD be calculated using the following formula:

                                      TX - RETX
   FORWARDING RATE(bit/s or UOT/s) =  -----------
                                       DURATION
   TX - If measuring reported in units of bits per seconds, TOTAL is the total
        bits transmitted including header and optional data for a
        given protocol. If measuring in UOTs per seconds, total is the
        total number of UOTs transmitted. This excludes any bits or UOT
        that are associated with connection maintenance[1], such as TCP
        keep-alives.

   RETX - If measuring in units of bits per seconds, RETX is the total
          number of bits, including header and optional data for a given
          protocol, that was retransmitted. If measuring in units of UOTs
          per second, RETX is the number of UOTs retransmitted. This
          excludes any bits or UOTs that are associated with connection
          maintenance, such as TCP keep-alives.

   DURATION - Test duration based on aforementioned time references.

5.6.5 Reporting Format

   The test report MUST note the number of GET requests, GET request
   rate and object size.

   Application layer bit forwarding rate results SHOULD be reported in
   tabular form with a row for each tabular form with
   a row for each of the object sizes. There SHOULD be columns a column for the
   object size, the number of completed requests, the number of
   completed responses, and the bit forwarding transfer rate results for each test.

   When reporting bit forwarding measurements for layers below

   Failure analysis:

   The test report SHOULD indicate the
   application layer, such as TCP number and percentage of HTTP
   GET request or IP, the responses that failed to complete.

   Version information:

   The test report MUST note whether the measurements are in bit per second or UOTs per second version of HTTP client(s) and the
   object size transferred.
   server(s).

   5.6.5.2 Transport-Layer and below reporting

   The test report MUST note the aggregate number of connections. In
   addition, the report MUST identify the layer/protocol for which the
   measurement was made.

   The results SHOULD be in tabular form with a row column for each layer/protocol.
   iteration of the test. There should be columns for transmitted bits/UOTs, bits,
   retransmitted bits/UOTs bits and the measured
   forwarding rate. goodput.

   Failure analysis:

   The test report SHOULD indicate the number and percentage of HTTP GET
   request or responses
   connections that failed to complete.

5.7 IP Fragmentation HTTP Concurrent Transaction Capacity

5.7.1 Objective

   To determine

   Determine the performance impact when maximum number of concurrent or simultaneous HTTP
   transactions the DUT/SUT can support. This test is presented
   with IP fragmented[5] traffic. IP datagrams which have been
   fragmented, due to crossing a network that supports a smaller
   MTU(Maximum Transmission Unit) than the actual datagram, may
   require the firewall to perform re-assembly prior to the datagram
   being applied intended to
   find the rule set.

   While IP fragmentation is a common form of attack, either on the
   firewall itself or on internal hosts, this test will focus on
   determining how the additional processing associated with the
   re-assembly of the datagrams has on the goodput maximum number of the DUT/SUT. users that can simultaneously access
   web objects.

5.7.2 Setup Parameters

   The following parameters MUST be defined.

   Trial duration - Trial duration SHOULD be set for 30 seconds.

   5.7.2.1 Non-Fragmented Traffic Parameters

   Session

   GET request rate - Defines the The aggregate rate, expressed in sessions request per
   second, that the at which HTTP sessions 1.1 or higher GET requests are attempted.

   Requests per session offered by the
   virtual client(s).

   Session type - Defines The virtual clients/servers MUST use HTTP 1.1 or
   higher.

5.7.3 Procedure

   An iterative search algorithm MAY be used to determine the number of maximum
   HTTP GET requests per
   session.

   Object Size - Defines concurrent transaction capacity.

   For each iteration, the virtual client(s) will vary the number of bytes to be transferred in
   response to
   concurrent or simultaneous HTTP transactions - that is, on-going
   GET requests. The HTTP 1.1 or higher virtual client(s) will request
   one object, across each connection, from an HTTP 1.1 or higher
   server using one HTTP GET request.

   5.7.2.1 Fragmented Traffic Parameters

   Packet size, expressed as The aggregate rate at which the number of bytes in
   virtual client(s) will offer the IP/UDP packet,
   exclusive of link-layer headers and checksums.

   Fragmentation Length requests will be defined by GET
   request rate.

   The object size requested MUST be large enough, such that, the
   transaction - Defines that is, the length of request/response cycle -- will exist for
   the data portion duration of the
   IP datagram and MUST be multiple of 8. Testers SHOULD use the minimum
   value, but MAY use other sizes as well.

   Intended Load -  Intended load, expressed as percentage of media
   utilization.

5.7.3 Procedure

   Each HTTP 1.1 virtual client will attempt to establish sessions
   to its HTTP 1.1 target server(s), using either the target server's
   IP address or NAT proxy address, at a fixed rate in a round robin
   fashion. test. At the same time, a client attached to the unprotected side
   of the network will offer a unidirectional stream of unicast UDP/IP
   packets to a server connected to the protected side end of each iteration, the network.
   The tester
   MUST offer IP/UDP packets in a steady state.

   Baseline measurements SHOULD be performed with a deny rule(s) validate that
   filters the fragmented traffic. If the DUT/SUT has logging
   capability, the log SHOULD be checked to determine if it contains
   the correct information regarding the fragmented traffic.

   The test SHOULD be repeated with the DUT/SUT rule set changed to
   allow the fragmented traffic through. When running multiple
   iterations all transactions are still active. After all of
   the test, it is RECOMMENDED to vary transactions are checked, the fragment
   length while keeping all other parameters constant. transactions MAY be aborted.

5.7.4 Measurements

   Aggregate Goodput

   Maximum concurrent transactions - The aggregate bit forwarding rate Total number of the
   requested concurrent HTTP objects.(See section 5.6). Only objects which have
   successfully completed transferring within
   transactions active for the trial duration are
   to be included last successful iteration performed in
   the goodput measurement.

   Transmitted UDP/IP Packets - Number of UDP packets transmitted by
   client.

   Received UDP/IP Packets - Number of UDP/IP Packets received by
   server. search algorithm.

5.7.5 Reporting Format

   5.7.5.1 Application-Layer reporting

   The test report MUST note the test duration.

   The test report MUST note the packet size(s), offered load(s) GET request rate and
   IP fragmentation length of the UDP/IP traffic. It SHOULD also note
   whether the DUT/SUT egresses the offered UDP/IP traffic fragmented
   or not.

   The test report MUST note the object size(s), session rate and
   requests per session. maximum
   concurrent transactions measured.

   The intermediate results SHOULD of the search algorithm MAY be reported
   in the format of a table format with a
   row column for each of the fragmentation lengths. iteration. There SHOULD be columns
   rows for the fragmentation length, IP/UDP packets transmitted by client,
   IP/UDP packets received by server, HTTP object size, number of concurrent transactions attempted, GET
   request rate, number of aborted transactions and measured
   goodput.

5.8 Illegal Traffic Handling

5.8.1 Objective

   To determine number of
   transactions active at the behavior end of the DUT/SUT when presented with a
   combination test iteration.

   Version information:

   The test report MUST note the version of both legal HTTP client(s) and Illegal traffic flows. Note
   server(s).

5.8 HTTP Transaction Rate

5.8.1 Objective

   Determine the maximum HTTP transaction rate that
   Illegal traffic does not refer to an attack, but to traffic which
   has been explicitly defined by a rule(s) to drop. DUT/SUT can
   sustain.

5.8.2 Setup Parameters

   Connection type

   Session Type - The tester MUST use HTTP 1.1 or higher MUST be used for HTTP measurements.

   Number of GET Requests this test.

   Test Duration - Defines Time, expressed in seconds, for which the
   virtual client(s) will sustain the number of HTTP 1.1 GET
   requests attempted per connection. GET Request Rate - Defines request rate.
   It is RECOMMENDED that the rate, in GET requests per second, duration be at
   which HTTP GET least 30 seconds.

   Requests per connection - Number of object requests are attempted on any given per connection.

   Object Size - Defines the number of bytes, excluding any bytes
   associated with the HTTP header, to be transferred in response to an
   HTTP 1.1 or higher GET request.

   Illegal traffic percentage - Percentage of HTTP connections which
   have been explicitly defined in a rule(s) to drop.

5.8.3 Procedure

   Each HTTP 1.1 virtual client will attempt to establish sessions
   to its

   An iterative search algorithm MAY be used to determine the maximum
   transaction rate that the DUT/SUT can sustain.

   For each iteration, HTTP 1.1 target server(s), using either the target server's
   IP address or NAT proxy address, at a fixed higher virtual client(s) will
   vary the aggregate GET request rate in a round robin
   fashion. offered to HTTP 1.1 or higher
   server(s). The tester MUST present virtual client(s) will maintain the connection requests, both legal and
   illegal, in an evenly distributed manner. Many firewalls have offered request
   rate for the capability to filter on different traffic criteria( IP
   addresses, Port numbers, etc). Testers may run defined test duration.

   If the tester makes multiple HTTP GET requests per connection, it
   MUST request the same object size for each GET request rate.
   Multiple iterations of this test MAY be performed with the DUT/SUT configured to filter
   on objects of
   different traffic criteria. sizes.

5.8.4 Measurements

   Tester SHOULD perform the same bit forwarding measurements as defined
   in HTTP test(Section 5.6.4).

5.8.5 Reporting Format

   Tester SHOULD report

   Maximum Transaction Rate - The maximum rate at which all
   transactions -- that is all requests/responses cycles -- are
   completed.

   Transaction Time - The tester SHOULD be measure minimum, maximum and
   average transaction times. The transaction time will start when the same as specified in
   virtual client issues the HTTP
   test(Section 5.6.5).

   In addition, GET request and end when the report MUST note requesting
   virtual client receives the percentage last bit of illegal HTTP
   connections.

   Failure analysis

   Test the requested object.

5.8.5 Reporting Format

   The test report MUST note the number test duration, object size, requests
   per connection and percentage of illegal connections
   allowed by the DUT/SUT.

5.9 Latency

5.9.1 Objective

   To determine the latency measured maximum transaction rate.

   The intermediate results of network-layer or application-layer data
   traversing the DUT/SUT. RFC 1242 [3] defines latency.

5.9.2 Setup Parameters

   The following parameters MUST search algorithm MAY be defined:

   5.9.2.1 Network-layer Measurements

      Packet size, expressed as the number of bytes reported
   in a table format with a column for each iteration. There SHOULD be
   rows for the IP packet,
      exclusive GET request attempt rate, number of link-layer headers requests attempted,
   number and checksums.

      Intended load, expressed as percentage of media utilization.

      Offered load, expressed as requests completed, number of responses
   attempted, number and percentage of media utilization.

      Test duration, expressed in seconds.

      Test instruments responses completed, minimum
   transaction time, average transaction time and maximum transaction
   time.

   Version information:

   The test report MUST generate packets with unique timestamp signatures.

   5.9.2.2 Application-layer Measurements

      Object size, expressed as note the number version of bytes to be transferred across a
      connection in response to an HTTP GET request. Testers SHOULD use client(s) and
   server(s).

5.9 Illegal Traffic Handling

 5.9.1 Objective

   To determine the
      minimum object size supported by behavior of the media, DUT/SUT when presented with a
   combination of both legal and Illegal traffic flows. Note that
   Illegal traffic does not refer to an attack, but MAY traffic which
   has been explicitly defined by a rule(s) to drop.

5.9.2 Setup Parameters

   Setup parameters will use other object
      sizes the same parameters as well.

      Connection type. The tester specified in the
   HTTP transfer rate test. In addition, the following setup
   parameters MUST use one be defined:

   Illegal traffic percentage - Percentage of HTTP 1.1 connection for latency
      measurements.

      Number of objects requested.

      Number of objects transferred.

      Test duration, expressed or higher
   connections which have been explicitly defined in seconds.

      Test instruments MUST generate packets with unique timestamp signatures. a rule(s) to drop.

5.9.3 Network-layer procedure

   A client will offer a unidirectional stream of unicast packets to a server.
   The packets MUST use a connectionless protocol like IP or UDP/IP.

   The tester MUST offer packets in a steady state. As noted in the latency
   discussion in RFC 2544 [4], latency measurements MUST be taken at the
   throughput level -- that is, at the highest offered load with zero packet
   loss. Measurements taken at the throughput level are the only ones that can
   legitimately be termed latency.

   It is RECOMMENDED that implementers use offered loads not only at the
   throughput level, but also at load levels that are less than or greater
   than the throughput level. To avoid confusion with existing terminology,
   measurements from such tests MUST be labeled as delay rather than latency.
   If desired, the tester MAY use a step test in which offered loads increment
   or decrement through a range of load levels.

   The duration of the test portion of each trial MUST be at least 30 seconds.

5.9.4 Application layer procedure

   An Procedure

   Each HTTP 1.1 or higher client will request one or more objects from
   an HTTP 1.1 or higher server using one or more HTTP GET requests. If
   The aggregate number of connections attempted, defined by number of
   connections, MUST be evenly divided among all of the tester makes multiple HTTP GET
   requests, it participating
   virtual clients.

   The virtual client(s) MUST request offer the same-sized object each time. connection requests, both legal
   and illegal, in an evenly distributed manner. Many firewalls have
   the capability to filter on different traffic criteria( IP
   addresses, Port numbers, etc). Testers may run multiple
   iterations of this test with objects of different sizes.

Implementers MAY configure the tester DUT/SUT configured to run for a fixed duration. In this
   case, the tester MUST filter
   on different traffic criteria.

5.9.4 Measurements

   Tester SHOULD perform the same measurements as defined in HTTP
   test(Section 5.6.4). Unlike the HTTP transfer rate test, the
   tester MUST not include any bits which are associated with illegal
   traffic in its forwarding rate measurements.

5.9.5 Reporting Format

   Test report SHOULD be the same as specified in the HTTP
   test(Section 5.6.5).

   In addition, the report MUST note the percentage of illegal HTTP
   connections.

   Failure analysis:

   Test report MUST note the number and percentage of illegal
   connections that were allowed by the DUT/SUT.

5.10 IP Fragmentation Handling

5.10.1 Objective

   To determine the performance impact when the DUT/SUT is presented
   with IP fragmented[5] traffic. IP packets which have been
   fragmented, due to crossing a network that supports a smaller
   MTU(Maximum Transmission Unit) than the actual IP packet, may
   require the firewall to perform re-assembly prior to the rule set
   being applied.

   While IP fragmentation is a common form of attack, either on the
   firewall itself or on internal hosts, this test will focus on
   determining how the additional processing associated with the
   re-assembly of the packets have on the forwarding rate of the
   DUT/SUT. RFC 1858 addresses some fragmentation attacks that
   get around IP filtering processes used in routers and hosts.

5.10.2 Setup Parameters

   The following parameters MUST be defined.

   5.10.2.1 Non-Fragmented Traffic Parameters

   Setup parameters will be the same as defined in the HTTP transfer
   rate test(Sections 5.6.2.1 and 5.6.2.2).

5.10.2.2 Fragmented Traffic Parameters

   Packet size - Number of bytes in the IP/UDP packet, exclusive of
   link-layer headers and checksums, prior to fragmentation.

   MTU - Maximum transmission unit, expressed in bytes. For testing
   purposes, this MAY be configured to values smaller than the MTU
   supported by the link layer.

   Intended Load -  Intended load, expressed as percentage of media
   utilization.

5.10.3 Procedure

   Each HTTP 1.1 or higher client will request one or more objects from
   an HTTP 1.1 or higher server using one or more HTTP GET requests.
   The aggregate number of connections attempted, defined by number of
   connections, MUST be evenly divided among all of the participating
   virtual clients. If the virtual client(s) make multiple HTTP GET
   requests per connection, it MUST request the same object size for
   each GET request.

   A tester attached to the unprotected side of the network, will offer
   a unidirectional stream of unicast IP/UDP targeting a server
   attached to either the protected or DMZ. The tester MUST offer the
   unidirectional stream over the duration of the test.

   Baseline measurements SHOULD be performed with IP filtering deny
   rule(s) to filter fragmented traffic. If the DUT/SUT has logging
   capability, the log SHOULD be checked to determine if it contains
   the correct information regarding the fragmented traffic.

   The test SHOULD be repeated with the DUT/SUT rule set changed to
   allow the fragmented traffic through. When running multiple
   iterations of the test, it is RECOMMENDED to vary the MTU while
   keeping all other parameters constant.

   Then setup the DUT/SUT to the policy or rule set the manufacturer
   required to be defined to protect against fragmentation attacks and
   repeat the measurements outlined in the baseline procedures.

5.10.4 Measurements

   Tester SHOULD perform the same measurements as defined in HTTP
   test(Section 5.6.4).

   Transmitted UDP/IP Packets - Number of UDP packets transmitted by
   client.

   Received UDP/IP Packets - Number of UDP/IP Packets received by
   server.

5.10.5 Reporting Format

   5.10.1 Non-Fragmented Traffic

   The test report SHOULD be the same as described in section 5.6.5.
   Note that any forwarding rate measurements for the HTTP traffic
   excludes any bits associated with the fragmented traffic which
   may be forward by the DUT/SUT.

   5.10.2 Fragmented Traffic

   The test report MUST note the packet size, MTU size, intended load,
   number of UDP/IP packets transmitted and number of UDP/IP packets
   forwarded. The test report SHOULD also note whether or not the
   DUT/SUT forwarded the offered UDP/IP traffic fragmented.

5.11 Latency

5.11.1 Objective

   To determine the latency of network-layer or application-layer data
   traversing the DUT/SUT. RFC 1242 [3] defines latency.

5.11.2 Setup Parameters

   The following parameters MUST be defined:

   5.11.2.1 Network-layer Measurements

      Packet size, expressed as the number of bytes in the IP packet,
      exclusive of link-layer headers and checksums.

      Intended load, expressed as percentage of media utilization.

      Test duration, expressed in seconds.

      Test instruments MUST generate packets with unique timestamp
      signatures.

   5.11.2.2 Application-layer Measurements

      Object Size - Defines the number of bytes, excluding any bytes
      associated with the HTTP header, to be transferred in response to
      an HTTP 1.1 or higher GET request. Testers SHOULD use the minimum
      object size supported by the media, but MAY use other object
      sizes as well.

      Connection type. The tester MUST use one HTTP 1.1 or higher
      connection for latency measurements.

      Number of objects requested.

      Number of objects transferred.

      Test duration, expressed in seconds.

      Test instruments MUST generate packets with unique timestamp
      signatures.

5.11.3 Network-layer procedure

   A client will offer a unidirectional stream of unicast packets to a
   server. The packets MUST use a connectionless protocol like IP or
   UDP/IP.

   The tester MUST offer packets in a steady state. As noted in the
   latency discussion in RFC 2544 [4], latency measurements MUST be
   taken at the throughput level -- that is, at the highest offered
   load with zero packet loss. Measurements taken at the throughput
   level are the only ones that can legitimately be termed latency.

   It is RECOMMENDED that implementers use offered loads not only at
   the throughput level, but also at load levels that are less than
   or greater than the throughput level. To avoid confusion with
   existing terminology, measurements from such tests MUST be labeled
   as delay rather than latency.

   If desired, the tester MAY use a step test in which offered loads
   increment or decrement through a range of load levels.

   The duration of the test portion of each trial MUST be at least 30
   seconds.

5.11.4 Application layer procedure

   An HTTP 1.1 or higher client will request one or more objects from
   an HTTP or higher 1.1 server using one or more HTTP GET requests. If
   the tester makes multiple HTTP GET requests, it MUST request the
   same-sized object each time. Testers may run multiple iterations of
   this test with objects of different sizes.

   Implementers MAY configure the tester to run for a fixed duration.
   In this case, the tester MUST report the number of objects requested
   and returned for the duration of the test. For fixed-duration tests
   it is RECOMMENDED that the duration be at least 30 seconds.

5.11.5 Measurements

   Minimum delay - The smallest delay incurred by data traversing the
   DUT/SUT at the network layer or application layer, as appropriate.

   Maximum delay - The largest delay incurred by data traversing the
   DUT/SUT at the network layer or application layer, as appropriate.

   Average delay - The mean of all measurements of delay incurred by
   data traversing the DUT/SUT at the network layer or application
   layer, as appropriate.

   Delay distribution - A set of histograms of all delay measurements
   observed for data traversing the DUT/SUT at the network layer or
   application layer, as appropriate.

5.11.6 Network-layer reporting format

   The test report MUST note the packet size(s), offered load(s) and
   test duration used.

   The latency results SHOULD be reported in the format of a table with
   a row for each of the tested packet sizes.  There SHOULD be columns
   for the packet size, the intended rate, the offered rate, and the
   resultant latency or delay values for each test.

5.11.7 Application-layer reporting format

   The test report MUST note the object size(s) and number of requests
   and responses completed. If applicable, the report MUST note the
   test duration if a fixed duration was used.

   The latency results SHOULD be reported in the format of a table with
   a row for each of the object sizes. There SHOULD be columns for the
   object size, the number of completed requests, the number of objects requested
   completed responses, and returned the resultant latency or delay values for
   each test.

   Failure analysis:

   The test report SHOULD indicate the duration number and percentage of the test. For fixed-duration tests it is RECOMMENDED HTTP
   GET request or responses that failed to complete within the duration be at least 30 seconds.

5.9.5 Measurements

   Minimum delay - test
   duration.

Version information:

   The smallest delay incurred by data traversing the DUT/SUT
   at test report MUST note the network layer or application layer, as appropriate.

   Maximum delay - version of HTTP client and server.

APPENDIX A: HTTP(HyperText Transfer Protocol)

   The largest delay incurred most common versions of HTTP in use today are HTTP/1.0 and
   HTTP/1.1 with the main difference being in regard to persistent
   connections.  HTTP 1.0, by data traversing default, does not support persistent
   connections. A separate TCP connection is opened up for each
   GET request the DUT/SUT
   at client wants to initiate and closed after the network layer or application layer, as appropriate.

   Average delay - The mean of all measurements
   requested object transfer is completed. While some implementations
   HTTP/1.0 supports persistence through the use of delay incurred a keep-alive,
   there is no official specification for how the keep-alive operates.
   In addition, HTTP 1.0 proxies do support persistent connection as
   they do not recognize the connection header.

   HTTP/1.1, by data
   traversing default, does support persistent connection and
   is therefore the DUT/SUT at version that is referenced in this methodology.
   Proxy based DUT/SUTs may monitor the network layer or application layer, as
   appropriate.

   Delay distribution - A set of histograms TCP connection and after a
   timeout, close the connection if no activity is detected. The
   duration of all delay measurements observed
   for data traversing this timeout is not defined in the HTTP/1.1
   specification and will vary between DUT/SUTs. If the DUT/SUT at
   closes inactive connections, the network layer or application layer,
   as appropriate.

5.9.6 Network-layer reporting format

   The test report MUST note aging timer on the packet size(s), offered load(s) and test
   duration used.

   The latency results DUT SHOULD
   be reported in the format of a table with a row configured for each of a duration that exceeds the tested packet sizes.  There SHOULD test time.

   While this document cannot foresee future changes to HTTP
   and it impact on the methodologies defined herein, such
   changes should be columns accommodated for so that newer versions of
   HTTP may be used in benchmarking firewall performance.

APPENDIX B: Connection Establishment Time Measurements

  For purposes of benchmarking firewall performance, the
   packet size, connection
  establishment time will be considered the intended rate, interval between the offered rate, and
  transmission of the resultant latency
   or delay values for each test.

5.9.7 Application-layer reporting format

   The test report MUST note first bit of the object size(s) and number first octet of requests and
   responses completed. If applicable, the report MUST note packet
  carrying the test duration
   if a fixed duration was used.

   The latency results SHOULD be reported in connection request to the format DUT/SUT interface to
  receipt of a table with a row
   for each the last bit of the object sizes.  There SHOULD be columns for last octet of the object size, last packet of
  the connection setup traffic received on the client or server,
  depending on whether a given connection requires an even or odd
  number of messages, respectfully.

  Some connection oriented protocols, such as TCP, involve an odd
  number of completed requests, messages when establishing a connection. In the number case of completed responses, and
  proxy based DUT/SUTs, the
   resultant latency or delay values for each test.

   Failure analysis:

   The test report SHOULD indicate DUT/SUT will terminate the number and percentage of HTTP GET
   request or responses that failed connection,
  setting up a separate connection to complete within the test duration.

   Version information:

   The test report MUST note the use of an HTTP 1.1 client and server.

APPENDICES

APPENDIX A: HTTP(HyperText Transfer Protocol)

   The most common versions of HTTP Since, in use today are HTTP/1.0 and
   HTTP/1.1 with such
  cases, the main difference being in regard to persistent
   connections.  HTTP 1.0, by default, tester does not support persistent
   connections. A separate TCP connection is opened up for each
   GET request own both sides of the client wants to initiate and closed after connection,
  measurements will be made two different ways. While the
   requested object transfer is completed. Some implementations of
   HTTP/1.0 supports persistence by adding an additional header
   to following
  describes the request/response:

      Connection: Keep-Alive

   However, under HTTP 1.0, there is no official specification for
   how measurements with reference to TCP, the keep-alive operates. In addition, HTTP 1.0 proxies do
   support persistent methodology
  may be used with other connection as they do not recognize oriented protocols which involve
  an odd number of messages.

  For non-proxy based DUT/SUTs , the
   connection header.

   HTTP/1.1, by default, does support persistent connection establishment time shall be
  directly measured and is therefore considered to be from the version that time the first
  bit of the first SYN packet is referenced in this methodology.
   When HTTP/1.1 entities want transmitted by the underlying transport layer
   connection closed after a transaction has completed, client to the
   request/response will include a connection-token close
  time the last bit of the final ACK in the
   connection header:

      Connection: close three-way handshake is
  received by the target server.

  If no such connection-token the DUT/SUT is present, proxy based, the connection remains
   open after establishment time is
  considered to be from the transaction time the first bit of the first SYN packet
  is completed. In addition, proxy
   based DUT/SUTs may monitor transmitted by the client to the time the client transmits the first
  bit of the first acknowledged TCP connection and after datagram(t4-t0 in the following
  timeline).

      t0: Client sends a
   timeout, close SYN.
      t1: Proxy sends a SYN/ACK.
      t2: Client sends the final ACK.
      t3: Proxy establishes separate connection if no activity is detected. The
   duration with server.
      t4: Client sends TCP datagram to server.
      *t5: Proxy sends ACK of this timeout the datagram to client.

   * While t5 is not defined in considered part of the HTTP/1.1
   specification and TCP connection establishment,
     acknowledgement of t4 must be received for the connection to be
     considered successful.

APPENDIX C: Connection Tear Time Measurements

  The TCP connection tear down time will vary be considered the interval
  between DUT/SUTs. If the DUT/SUT
   closes inactive connections, transmission of the aging timer on first TCP FIN packet transmitted
  by the DUT SHOULD
   be configured for tester requesting a duration that exceeds the test time.

   While this document cannot foresee future changes connection tear down to HTTP
   and it impact receipt of the
  ACK packet on the methodologies defined herein, such
   changes should be accommodated for so that newer versions of
   HTTP may be used in benchmarking firewall performance. same tester interface.

Appendix B. D.  References

  [1] D. Newman, "Benchmarking Terminology for Firewall Devices", RFC 2647,
           August 1999.

  [2] R. Fielding, J. Gettys, J. Mogul, H Frystyk, L.Masinter, P. Leach,
      T. Berners-Lee , "Hypertext Transfer Protocol -- HTTP/1.1",
      RFC 2616 June 1999

  [3] S. Bradner, editor. "Benchmarking Terminology for Network
      Interconnection Devices," RFC 1242, July 1991.

  [4] S. Bradner, J. McQuaid, "Benchmarking Methodology for Network
      Interconnect Devices," RFC 2544, March 1999.

  [5] David C. Clark, "IP Datagram Reassembly Algorithm", RFC 815 ,
      July 1982.

  [6] Mandeville, R., "Benchmarking Terminology for LAN Switching
      Devices", RFC 2285, February 1998.

  [7] Mandeville, R., Perser,J., "Benchmarking Methodology for LAN
      Switching Devices", RFC 2889, August 2000.