draft-ietf-bmwg-firewall-03.txt   draft-ietf-bmwg-firewall-04.txt 
Network Working Group Brooks Hickman Network Working Group Brooks Hickman
Internet-Draft Spirent Communications Internet-Draft Spirent Communications
Expiration Date: April 2002 David Newman Expiration Date: November 2002 David Newman
Network Test Network Test
Saldju Tadjudin Saldju Tadjudin
Spirent Communications Spirent Communications
Terry Martin Terry Martin
M2networx INC M2networx INC
October 2001 May 2002
Benchmarking Methodology for Firewall Performance Benchmarking Methodology for Firewall Performance
<draft-ietf-bmwg-firewall-03.txt> <draft-ietf-bmwg-firewall-04.txt>
Status of this Memo Status of this Memo
This document is an Internet-Draft and is in full conformance with This document is an Internet-Draft and is in full conformance with
all provisions of Section 10 of RFC2026. all provisions of Section 10 of RFC2026.
Internet-Drafts are working documents of the Internet Engineering Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF), its areas, and its working groups. Note that Task Force (IETF), its areas, and its working groups. Note that
other groups may also distribute working documents as Internet- other groups may also distribute working documents as Internet-
Drafts. Drafts.
skipping to change at page 1, line 52 skipping to change at page 1, line 52
4.1 Test Considerations . . . . . . . . . . . . . . . . . . 3 4.1 Test Considerations . . . . . . . . . . . . . . . . . . 3
4.2 Virtual Client/Servers . . . . . . . . . . . . . . . . . 3 4.2 Virtual Client/Servers . . . . . . . . . . . . . . . . . 3
4.3 Test Traffic Requirements . . . . . . . . . . . . . . . . 4 4.3 Test Traffic Requirements . . . . . . . . . . . . . . . . 4
4.4 DUT/SUT Traffic Flows . . . . . . . . . . . . . . . . . . 4 4.4 DUT/SUT Traffic Flows . . . . . . . . . . . . . . . . . . 4
4.5 Multiple Client/Server Testing . . . . . . . . . . . . . 5 4.5 Multiple Client/Server Testing . . . . . . . . . . . . . 5
4.6 NAT(Network Address Translation) . . . . . . . . . . . . 5 4.6 NAT(Network Address Translation) . . . . . . . . . . . . 5
4.7 Rule Sets . . . . . . . . . . . . . . . . . . . . . . . . 5 4.7 Rule Sets . . . . . . . . . . . . . . . . . . . . . . . . 5
4.8 Web Caching . . . . . . . . . . . . . . . . . . . . . . . 5 4.8 Web Caching . . . . . . . . . . . . . . . . . . . . . . . 5
4.9 Authentication . . . . . . . . . . . . . . . . . . . . . 6 4.9 Authentication . . . . . . . . . . . . . . . . . . . . . 6
5. Benchmarking Tests . . . . . . . . . . . . . . . . . . . . . 6 5. Benchmarking Tests . . . . . . . . . . . . . . . . . . . . . 6
5.1 Concurrent Connection Capacity . . . . . . . . . . . . . 6 5.1 IP throughput . . . . . . . . . . . . . . . . . . . . . . 6
5.2 Maximum Connection Setup Rate . . . . . . . . . . . . . . 7 5.2 Concurrent TCP Connection Capacity . . . . . . . . . . . 7
5.3 Connection Establishment Time . . . . . . . . . . . . . . 9 5.3 Maximum TCP Connection Establishment Rate . . . . . . . . 10
5.4 Connection Teardown Time . . . . . . . . . . . . . . . . 11 5.4 Maximum TCP Connection Tear Down Rate . . . . . . . . . . 12
5.5 Denial Of Service Handling . . . . . . . . . . . . . . . 13 5.5 Denial Of Service Handling . . . . . . . . . . . . . . . 13
5.6 HTTP . . . . . . . . . . . . . . . . . . . . . . . . . . 14 5.6 HTTP Transfer Rate . . . . . . . . . . . . . . . . . . . 14
5.7 IP Fragmentation Handling . . . . . . . . . . . . . . . . 16 5.7 HTTP Concurrent Transaction Capacity . . . . . . . . . . 17
5.8 Illegal Traffic Handling . . . . . . . . . . . . . . . . 18 5.8 HTTP Transaction Rate . . . . . . . . . . . . . . . . . . 18
5.9 Latency . . . . . . . . . . . . . . . . . . . . . . . . . 19 5.9 Illegal Traffic Handling . . . . . . . . . . . . . . . . 19
Appendices . . . . . . . . . . . . . . . . . . . . . . . . . . 22 5.10 IP Fragmentation Handling . . . . . . . . . . . . . . . 20
A. HyperText Transfer Protocol(HTTP) . . . . . . . . . . . . 22 5.11 Latency . . . . . . . . . . . . . . . . . . . . . . . . 22
B. References . . . . . . . . . . . . . . . . . . . . . . . . 23 Appendices . . . . . . . . . . . . . . . . . . . . . . . . . . 25
A. HyperText Transfer Protocol(HTTP) . . . . . . . . . . . . 25
B. Connection Establishment Time Measurements . . . . . . . . 25
C. Connection Tear Down Time Measurements . . . . . . . . . . 26
C. References . . . . . . . . . . . . . . . . . . . . . . . . 26
1. Introduction 1. Introduction
This document provides methodologies for the performance This document provides methodologies for the performance
benchmarking of firewalls. It provides methodologies in four areas: benchmarking of firewalls. It provides methodologies in four areas:
forwarding, connection, latency and filtering. In addition to forwarding, connection, latency and filtering. In addition to
defining the tests, this document also describes specific formats defining the tests, this document also describes specific formats
for reporting the results of the tests. for reporting the results of the tests.
A previous document, "Benchmarking Terminology for Firewall A previous document, "Benchmarking Terminology for Firewall
skipping to change at page 3, line 20 skipping to change at page 3, line 20
| | | +----------+ | | | | | | +----------+ | | |
| Servers/ |----| | | |------| Servers/ | | Servers/ |----| | | |------| Servers/ |
| Clients | | | | | | Clients | | Clients | | | | | | Clients |
| | |-------| DUT/SUT |--------| | | | | |-------| DUT/SUT |--------| | |
+----------+ | | | | +----------+ +----------+ | | | | +----------+
| +----------+ | | +----------+ |
Protected | | Unprotected Protected | | Unprotected
Network Network Network Network
Figure 1(Dual-Homed) Figure 1(Dual-Homed)
Tri-homed[1] configurations employ a third segment called a DMZ. Tri-homed[1] configurations employ a third segment called a
With tri-homed configurations, servers accessible to the public Demilitarized Zone(DMZ). With tri-homed configurations, servers
network are attached to the DMZ. Tri-Homed configurations offer accessible to the public network are attached to the DMZ. Tri-Homed
additional security by separating server accessible to the public configurations offer additional security by separating server(s)
network from internal hosts. accessible to the public network from internal hosts.
+----------+ +----------+ +----------+ +----------+
| | | +----------+ | | | | | | +----------+ | | |
| Clients |----| | | |------| Servers/ | | Clients |----| | | |------| Servers/ |
| | | | | | | Clients | | | | | | | | Clients |
+----------+ |-------| DUT/SUT |--------| | | +----------+ |-------| DUT/SUT |--------| | |
| | | | +----------+ | | | | +----------+
| +----------+ | | +----------+ |
Protected | | | Unprotected Protected | | | Unprotected
Network | Network Network | Network
skipping to change at page 4, line 7 skipping to change at page 4, line 7
4.2 Virtual Clients/Servers 4.2 Virtual Clients/Servers
Since firewall testing may involve data sources which emulate Since firewall testing may involve data sources which emulate
multiple users or hosts, the methodology uses the terms virtual multiple users or hosts, the methodology uses the terms virtual
clients/servers. For these firewall tests, virtual clients/servers clients/servers. For these firewall tests, virtual clients/servers
specify application layer entities which may not be associated with specify application layer entities which may not be associated with
a unique physical interface. For example, four virtual clients may a unique physical interface. For example, four virtual clients may
originate from the same data source[1]. The test report SHOULD originate from the same data source[1]. The test report SHOULD
indicate the number of virtual clients and virtual servers indicate the number of virtual clients and virtual servers
participating in the test on a per interface(See 4.0) basis. participating in the test.
Testers MUST synchronize all data sources participating in a test. Testers MUST synchronize all data sources participating in a test.
4.3 Test Traffic Requirements 4.3 Test Traffic Requirements
While the function of a firewall is to enforce access control While the function of a firewall is to enforce access control
policies, the criteria by which those policies are defined vary policies, the criteria by which those policies are defined vary
depending on the implementation. Firewalls may use network layer, depending on the implementation. Firewalls may use network layer,
transport layer or, in many cases, application-layer criteria to transport layer or, in many cases, application-layer criteria to
make access-control decisions. Therefore, the test equipment used to make access-control decisions.
benchmark the DUT/SUT performance MUST consist of real clients and
servers generating legitimate layer seven conversations.
For the purposes of benchmarking firewall performance, HTTP 1.1 For the purposes of benchmarking firewall performance this document
will be referenced in this document, although the methodologies references HTTP 1.1 or higher as the application layer entity,
may be used as a template for benchmarking with other applications. although the methodologies may be used as a template for
Since testing may involve proxy based DUT/SUTs, HTTP version benchmarking with other applications. Since testing may involve
considerations are discussed in appendix A. proxy based DUT/SUTs, HTTP version considerations are discussed in
appendix A.
4.4 DUT/SUT Traffic Flows 4.4 DUT/SUT Traffic Flows
Since the number of interfaces are not fixed, the traffic flows will Since the number of interfaces are not fixed, the traffic flows will
be dependent upon the configuration used in benchmarking the be dependent upon the configuration used in benchmarking the
DUT/SUT. Note that the term "traffic flows" is associated with DUT/SUT. Note that the term "traffic flows" is associated with
client-to- server requests. client-to- server requests.
For Dual-Homed configurations, there are two unique traffic flows: For Dual-Homed configurations, there are two unique traffic flows:
skipping to change at page 5, line 8 skipping to change at page 4, line 51
Client Server Client Server
------ ------ ------ ------
Protected -> Unprotected Protected -> Unprotected
Protected -> DMZ Protected -> DMZ
Unprotected -> DMZ Unprotected -> DMZ
4.5 Multiple Client/Server Testing 4.5 Multiple Client/Server Testing
One or more clients may target multiple servers for a given One or more clients may target multiple servers for a given
application. Each virtual client MUST initiate requests(Connection, application. Each virtual client MUST initiate connections in a
object transfers, etc.) in a round-robin fashion. For example, if round-robin fashion. For example, if the test consisted of six
the test consisted of six virtual clients targeting three servers, virtual clients targeting three servers, the pattern would be as
the pattern would be as follows: follows:
Client Target Server(In order of request) Client Target Server(In order of request)
#1 1 2 3 1... #1 1 2 3 1...
#2 2 3 1 2... #2 2 3 1 2...
#3 3 1 2 3... #3 3 1 2 3...
#4 1 2 3 1... #4 1 2 3 1...
#5 2 3 1 2... #5 2 3 1 2...
#6 3 1 2 3... #6 3 1 2 3...
4.6 NAT(Network Address Translation) 4.6 Network Address Translation(NAT)
Many firewalls implement network address translation(NAT), a Many firewalls implement network address translation(NAT), a
function which translates internal host IP addresses attached to function which translates internal host IP addresses attached to
the protected network to a virtual IP address for communicating the protected network to a virtual IP address for communicating
across the unprotected network(Internet). This involves additional across the unprotected network(Internet). This involves additional
processing on the part of the DUT/SUT and may impact performance. processing on the part of the DUT/SUT and may impact performance.
Therefore, tests SHOULD be ran with NAT disabled and NAT enabled Therefore, tests SHOULD be ran with NAT disabled and NAT enabled
to determine the performance differentials. The test report SHOULD to determine the performance differentials. The test report MUST
indicate whether NAT was enabled or disabled. indicate whether NAT was enabled or disabled.
4.7 Rule Sets 4.7 Rule Sets
Rule sets[1] are a collection of access control policies that Rule sets[1] are a collection of access control policies that
determines which packets the DUT/SUT will forward and which it will determine which packets the DUT/SUT will forward and which it will
reject[1]. The criteria by which these access control policies may reject[1]. Since criteria by which these access control policies may
be defined will vary depending on the capabilities of the DUT/SUT. be defined will vary depending on the capabilities of the DUT/SUT,
The scope of this document is limited to how the rule sets should the following is limited to providing guidelines for configuring
be applied when testing the DUT/SUT. rule sets when benchmarking the performance of the DUT/SUT.
The firewall monitors the incoming traffic and checks to make sure It is RECOMMENDED that a rule be entered for each host(Virtual
that the traffic meets one of the defined rules before allowing it client). In addition, testing SHOULD be performed using different
to be forwarded. It is RECOMMENDED that a rule be entered for each size rule sets to determine its impact on the performance of the
host(Virtual client). Although many firewalls permit groups of IP DUT/SUT. Rule sets MUST be configured in a manner, such that, rules
addresses to be defined for a given rule, tests SHOULD be performed associated with actual test traffic are configured at the end of the
with large rule sets, which are more stressful to the DUT/SUT. rule set and not the beginning.
The DUT/SUT SHOULD be configured to denies access to all traffic The DUT/SUT SHOULD be configured to deny access to all traffic
which was not previously defined in the rule set. which was not previously defined in the rule set. The test report
SHOULD include the DUT/SUT configured rule set(s).
4.7 Web Caching 4.7 Web Caching
Some firewalls include caching agents in order to reduce network Some firewalls include caching agents to reduce network load. When
load. When making a request through a caching agent, the caching making a request through a caching agent, the caching agent attempts
agent attempts to service the response from its internal memory. to service the response from its internal memory. The cache itself
saves responses it receives, such as responses for HTTP GET
The cache itself saves responses it receives, such as responses requests. Testing SHOULD be performed with any caching agents on the
for HTTP GET requests. The report SHOULD indicate whether caching DUT/SUT disabled.
was enabled or disabled on the DUT/SUT.
4.8 Authentication 4.8 Authentication
Access control may involve authentication processes such as user, Access control may involve authentication processes such as user,
client or session authentication. Authentication is usually client or session authentication. Authentication is usually
performed by devices external to the firewall itself, such as an performed by devices external to the firewall itself, such as an
authentication servers and may add to the latency of the system. authentication server(s) and may add to the latency of the system.
Any authentication processes MUST be included as part of connection Any authentication processes MUST be included as part of connection
setup process. setup process.
5. Benchmarking Tests 5. Benchmarking Tests
5.1 Concurrent Connection Capacity 5.1 IP Throughput
5.1.1 Objective 5.1.1 Objective
To determine the maximum number of TCP concurrent connections through To determine the throughput of network-layer data transversing
or with the DUT/SUT, as defined in RFC2647[1]. This test will employ the DUT/SUT, as defined in RFC1242[1]. Note that while RFC1242
a step algorithm to obtain the maximum number of concurrent TCP uses the term frames, which is associated with the link layer, the
connections that the DUT/SUT can maintain. procedure uses the term packets, since it is referencing the
network layer. This test is intended to baseline the ability of
the DUT/SUT to forward packets at the network layer.
5.1.2 Setup Parameters 5.1.2 Setup Parameters
The following parameters MUST be defined for all tests. The following parameters MUST be defined:
Connection Attempt Rate - The rate, expressed in connections per
second, at which new TCP connection requests are attempted. The
rate SHOULD be set lower than maximum rate at which the DUT/SUT can
accept connection requests.
Connection Step Count - Defines the number of additional TCP Packet size - Number of bytes in the IP packet, exclusive of any
connections attempted for each iteration of the step search link layer header or checksums.
algorithm.
Object Size - Defines the number of bytes to be transferred in Test Duration - Duration of the test, expressed in seconds.
response to a HTTP 1.1 GET request . It is RECOMMENDED to use
1-byte object sizes for this test.
5.1.3 Procedure 5.1.3 Procedure
Each virtual client will attempt to establish TCP connections to its The tester will offer client/server traffic to the DUT/SUT,
target server(s), using either the target server's IP address or NAT consisting of unicast IP packets. The tester MUST offer the packets
proxy address, at a fixed rate in a round robin fashion. Each at a constant rate. The test MAY consist of either bi-directional or
iteration will involve the virtual clients attempting to establish a unidirectional traffic, with the client offering a unicast stream of
fixed number of additional TCP connections. This search algorithm packets to the server for the latter.
will be repeated until either:
- One or more of the additional connection attempts fail to
complete.
- One or more of the previously established connections fail.
The test MUST include application layer data transfers in order to The test MAY employ an iterative search algorithm. Each iteration
validate the TCP connections since, in the case of proxy based will involve the tester varying the intended load until the maximum
DUT/SUTs, the tester does not own both sides of the connection. rate, at which no packet loss occurs, is found. Since backpressure
After all the addition connections have been attempted for each mechanisms may be employed, resulting in the intended load and
iteration of the test, the virtual client(s) will request an offered load being different, the test SHOULD be performed in either
object from its target server(s) using an HTTP 1.1 GET request on a packet based or time based manner as described in RFC2889[7]. As
the additional connections as well as all previously established with RFC1242, the term packet is used in place of frame. The
connections. Both the client request and server response MUST exclude duration of the test portion of each trial MUST be at least 30
the connection-token close in the connection header(See Appendix A). seconds.
5.1.4 Measurements When comparing DUT/SUTs with different MTUs, it is RECOMMENDED to
limit the maximum IP size tested to the maximum MTU supported by all
of the DUT/SUTs.
Maximum concurrent connections - Total number of TCP connections 5.1.4 Measurement
open for the last successful iteration performed in the search
algorithm.
5.1.5 Reporting Format 5.1.4.1 Network Layer
5.1.5.1 Application-Layer Reporting: Throughput - Maximum offered load, expressed in either bits per
second or packets per second, at which no packet loss is detected.
The test report MUST note the use of HTTP 1.1 client and server Forwarding Rate - Forwarding rate, expressed in either bits per
and the object size. second or packets per second, the device is observed to
successfully forward to the correct destination interface in
response to a specified offered load.
5.1.5.2 Transport-Layer Reporting: 5.1.4 Reporting Format
The test report MUST note the connection attempt rate, connection The test report MUST note the packet size(s), test duration,
step count and maximum concurrent connections measured. throughput and forwarding rate. If the test involved offering
packets which target more than one segment(Protected, Unprotected
or DMZ), the report MUST identify the results as an aggregate
throughput measurement.
5.1.5.3 Log Files The throughput results SHOULD be reported in the format of a table
with a row for each of the tested packet sizes. There SHOULD be
columns for the packet size, the intended load, the offered load,
resultant throughput and forwarding rate for each test.
A log file MAY be generated which includes the TCP connection A log file MAY be generated which includes the packet size, test
attempt rate, connection step count, object size and for each duration and for each iteration:
iteration:
- Step Iteration - Step Iteration
- Pass/Fail Status. - Pass/Fail Status
- Total TCP connections established. - Total packets offered
- Number of previously established TCP connections that failed. - Total packets forwarded
- Number of the additional TCP connections that failed to - Intended load
complete. - Offered load(If applicable)
- Forwarding rate
5.2 Maximum Connection Setup Rate 5.2 Concurrent TCP Connection Capacity
5.2.1 Objective 5.2.1 Objective
To determine the maximum TCP connection setup rate through or with To determine the maximum number of concurrent TCP connections
the DUT/SUT, as defined by RFC2647[1]. This test will employ a supported through or with the DUT/SUT, as defined in RFC2647[1].
search algorithm to obtain the maximum rate at which TCP connections
can be established through or with the DUT/SUT.
5.2.2 Setup Parameters 5.2.2 Setup Parameters
The following parameters MUST be defined. The following parameters MUST be defined for all tests:
Initial Attempt Rate - The rate, expressed in connections per
second, at which the initial TCP connection requests are attempted.
Number of Connections - Defines the number of TCP connections that 5.2.2.1 Transport-Layer Setup Parameters
must be established. The number MUST be between the number of
participating virtual clients and the maximum number supported by
the DUT/SUT.
Object Size - Defines the number of bytes to be transferred in Connection Attempt Rate - The aggregate rate, expressed in
response to a HTTP 1.1 GET request . It is RECOMMENDED to use connections per second, at which new TCP connection requests are
1-byte object sizes for this test. attempted. The rate SHOULD be set at or lower than the maximum
rate at which the DUT/SUT can accept connection requests.
Age Time - The time, expressed in seconds, the DUT/SUT will keep a Age Time - The time, expressed in seconds, the DUT/SUT will keep a
connection in it's state table after receiving a TCP FIN or RST connection in its connection table after receiving a TCP FIN or RST
packet. packet.
5.2.2.2 Transport-Layer Setup Parameters
Validation Method - HTTP 1.1 or higher MUST be used for this test.
Object Size - Defines the number of bytes, excluding any bytes
associated with the HTTP header, to be transferred in response to an
HTTP 1.1 or higher GET request.
5.2.3 Procedure 5.2.3 Procedure
An iterative search algorithm will be used to determine the maximum An iterative search algorithm MAY be used to determine the maximum
connection rate. This test iterates through different connection rates number of concurrent TCP connections supported through or with the
with a fixed number of connections attempted by the virtual clients to DUT/SUT.
their associated server(s).
Each iteration will use the same connection establishment and For each iteration, the aggregate number of concurrent TCP
connection validation algorithms defined in the concurrent capacity connections attempted by the virtual client(s) will be varied. The
test(See section 5.1). destination address will be that of the server or that of the NAT
proxy. The aggregate rate will be defined by connection attempt
rate, and will be attempted in a round-robin fashion(See 4.5).
Between each iteration of the test, the tester must close all To validate all connections, the virtual client(s) MUST request an
connections completed for the previous iteration. In addition, object using an HTTP 1.1 or higher GET request. The requests MUST be
it is RECOMMENDED to abort all unsuccessful connections attempted. initiated on each connection after all of the TCP connections have
The tester will wait for the period of time, specified by age time, been established.
before continuing to the next iteration.
When testing proxy-based DUT/SUTs, the virtual client(s) MUST
request two objects using HTTP 1.1 or higher GET requests. The first
GET request is required for connection time establishment
measurements as specified in appendix B. The second request is used
for validation as previously mentioned. When comparing proxy and
non-proxy based DUT/SUTs, the test MUST be performed in the same
manner.
Between each iteration, it is RECOMMENDED that the tester issue a
TCP RST referencing all connections attempted for the previous
iteration, regardless of whether or not the connection attempt was
successful. The tester will wait for age time before continuing to
the next iteration.
5.2.4 Measurements 5.2.4 Measurements
Highest connection rate - Highest rate, in connections per second, 5.2.4.1 Application-Layer measurements
for which all TCP connections completed successfully.
Number of objects requested
Number of objects returned
5.2.4.2 Transport-Layer measurements
Maximum concurrent connections - Total number of TCP connections
open for the last successful iteration performed in the search
algorithm.
The following measurements SHOULD be performed on a per iteration
basis:
Minimum connection establishment time - Lowest TCP connection
establishment time measured as defined in appendix B.
Maximum connection establishment time - Highest TCP connection
establishment time measured as defined in appendix B.
Average connection establishment time - The mean of all measurements
of connection establishment times.
Aggregate connection establishment time - The total of all
measurements of connection establishment times.
5.2.5 Reporting Format 5.2.5 Reporting Format
5.1.5.1 Application-Layer Reporting: 5.2.5.1 Application-Layer Reporting:
The test report MUST note the use of HTTP 1.1 client and server The test report MUST note the object size, number of completed
and the object size. requests and number of completed responses.
5.2.5.2 Transport-Layer Reporting: The intermediate results of the search algorithm MAY be reported
in a table format with a column for each iteration. There SHOULD be
rows for the number of requests attempted, number of requests
completed, number of responses attempted and number of responses
completed. The table MAY be combined with the transport-layer
reporting, provided that the table identify this as an application
layer measurement.
The test report MUST note the number of connections, age time Version information:
and highest connection rate measured.
5.2.5.3 Log Files The test report MUST note the version of HTTP client(s) and
server(s).
A log file MAY be generated which includes the number of TCP 5.2.5.2 Transport-Layer Reporting:
connections attempt, age time, object size and
for each iteration:
- Step Iteration The test report MUST note the connection attempt rate, age time and
- Pass/Fail Status. maximum concurrent connections measured.
- Attempted Connection Establishment Rate
- Total TCP connections established.
- Number of TCP connections that failed to complete.
5.3 Connection Establishment Time The intermediate results of the search algorithm MAY be reported
in the format of a table with a column for each iteration. There
SHOULD be rows for the total number of TCP connections attempted,
total number of TCP connections completed, minimum TCP connection
establishment time, maximum TCP connection establishment time,
average connection establishment time and the aggregate connection
establishment time.
5.3.1 Objective 5.3 Maximum TCP Connection Establishment Rate
To determine the connection establishment times[1] through or with 5.3.1 Objective
the DUT/SUT.
A connection for a client/server application is not atomic, in that To determine the maximum TCP connection establishment rate through
it not only involves transactions at the application layer, but or with the DUT/SUT, as defined by RFC2647[1].
involves first establishing a connection using one or more underlying
connection oriented protocols(TCP, ATM, etc). Therefore, it is
encouraged to make separate measurements for each connection oriented
protocol required in order to perform the application layer
transaction.
5.3.2 Setup Parameters 5.3.2 Setup Parameters
The following parameters MUST be defined. The following parameters MUST be defined for all tests:
Connection Attempt Rate - The rate, expressed in connections per
second, at which new TCP connection requests are attempted. It is
RECOMMENDED not to exceed the maximum connection rate found in
section 5.2.
Connection Attempt Step count - Defines the number of additional 5.3.2.1 Transport-Layer Setup Parameters
TCP connections attempted for each iteration of the step algorithm.
Maximum Attempt Connection Count - Defines the maximum number of Number of Connections - Defines the aggregate number of TCP
TCP connections attempted in the test. connections that must be established.
5.3.3 Procedure Age Time - The time, expressed in seconds, the DUT/SUT will keep a
connection in it's state table after receiving a TCP FIN or RST
packet.
Each virtual client will attempt to establish TCP connections to its 5.3.2.2 Transport-Layer Setup Parameters
target server(s) at a fixed rate in a round robin fashion. Each
iteration will involve the virtual clients attempting to establish
a fixed number of additional connections until the maximum attempt
connection count is reached.
After each connection has been completed, the virtual client(s) MUST Validation Method - HTTP 1.1 or higher MUST be used for this test.
request a 1-byte object from its target server(s) using an HTTP 1.1
GET request. Both the client request and server response MUST exclude
[Page 9] Object Size - Defines the number of bytes, excluding any bytes
the connection-token close in the connection header(See Appendix A). associated with the HTTP header, to be transferred in response to an
HTTP 1.1 or higher GET request.
Since testing may involve proxy based DUT/SUTs, which terminates the 5.3.3 Procedure Test
TCP connection, making a direct measurement of the TCP connection
establishment time is not possible since the protocol involves an
odd number of messages in establishing a connection. Therefore, when
testing with proxy based firewalls, the datagram following the final
ACK on the three-way handshake will be used in determining the
connection setup time.
The following shows the timeline for the TCP connection setup An iterative search algorithm MAY be used to determine the maximum
involving a proxy DUT/SUT and is referenced in the measurements rate at which the DUT/SUT can accept TCP connection requests.
section(5.3.4). Note that this methodology may be applied when
measuring other connection oriented protocols involving an odd number
of messages in establishing a connection.
t0: Client sends a SYN. For each iteration, the aggregate rate at which TCP connection
t1: Proxy sends a SYN/ACK. requests are attempted by the virtual client(s) will be varied. The
t2: Client sends the final ACK. destination address will be that of the server or that of the NAT
t3: Proxy establishes separate connection with server. proxy. The aggregate number of connections, defined by number of
t4: Client sends TCP datagram to server. connections, will be attempted in a round-robin fashion(See 4.5).
*t5: Proxy sends ACK of the datagram to client.
* While t5 is not considered part of the TCP connection establishment, The same application-layer object transfers required for validation
acknowledgement of t4 must be received for the connection to be and establishment time measurements as described in the concurrent
considered successful. TCP connection capacity test MUST be performed.
When comparing firewalls with different architectures, such as proxy Between each iteration, it is RECOMMENDED that the tester issue a
based and stateful packet filtering, the same method SHOULD be used TCP RST referencing all connections attempted for the previous
when measuring establishment times. iteration, regardless of whether or not the connection attempt was
successful. The tester will wait for age time before continuing to
the next iteration.
5.3.4 Measurements 5.3.4 Measurements
For each iteration of the test, the tester MUST measure the minimum, 5.3.4.1 Application-Layer measurements
maximum and average TCP connection establishment times. If the DUT/SUT
is proxy based, the connection establishment time is considered to be
from the time the first bit of the first SYN packet is transmitted by
the client to the time the client transmits the first bit of the first
acknowledged TCP datagram(t4-t0 in the above timeline). For non-proxy
based DUT/SUTs , the establishment time shall be directly measured and
is considered to be from the time the first bit of the first SYN packet
is transmitted by the client to the time the last bit of the final ACK
in the three-way handshake is received by the target server.
In addition, the tester SHOULD measure the minimum, maximum and Number of objects requested
average connection establishment times for all other underlying
connection oriented protocols. For purposes of benchmarking
firewall performance, the connection establishment time will be
considered the interval between the transmission of the first bit
of the first octet of the packet carrying the connection request
to the DUT/SUT interface to receipt of the last bit of the last
octet of the last packet of the connection setup traffic received
on the client or server, depending on whether a given connection
requires an even or odd number of messages, respectfully.
Tester SHOULD measure the aggregate connection time and the total Number of objects returned
number of connections completed for all measured protocols for each
iteration of the test. 5.3.4.2 Transport-Layer measurements
Highest connection rate - Highest rate, in connections per second,
for which for the search algorithm passed.
The following measurements SHOULD performed on a per iteration
basis:
Minimum connection establishment time - Lowest TCP connection
establishment time measured as defined in appendix B.
Maximum connection establishment time - Highest TCP connection
establishment time measured as defined in appendix B.
Average connection establishment time - The mean of all measurements
of connection establishment times.
Aggregate connection establishment time - The total of all
measurements of connection establishment times.
5.3.5 Reporting Format 5.3.5 Reporting Format
5.3.5.1 Application-Layer Reporting: 5.3.5.1 Application-Layer Reporting:
The test report MUST note the use of HTTP 1.1 client and server. The test report MUST note object size(s), number of completed
requests and number of completed responses.
5.3.5.2 Transport-Layer and Below Reporting: The intermediate results of the search algorithm MAY be reported
in a table format with a column for each iteration. There SHOULD be
rows for the number of requests and responses completed. The table
MAY be combined with the transport-layer reporting, provided that
the table identify this as an application layer measurement.
The test report MUST note the TCP connection attempt rate, TCP Version information:
connection attempt step count and maximum TCP connections attempted.
For each connection oriented protocol the tester measured, the The test report MUST note the version of HTTP client(s) and server(s).
connection establishment time results SHOULD be in tabular form
with a row for each iteration of the test. There SHOULD be a column
for the iteration count, minimum connection establishment time,
average connection establishment time, maximum connection
establishment time, attempted connections completed and aggregate
connection time.
The report MUST also identify the layer/protocol for which the 5.3.5.2 Transport-Layer Reporting:
measurements were made.
5.4 Connection Teardown Time The test report MUST note the number of connections, age time and
highest connection rate measured.
The intermediate results of the search algorithm MAY be reported
in the format of a table with a column for each iteration. There
SHOULD be rows for the connection attempt rate, total number of
TCP connections attempted, total number of TCP connections
completed, minimum TCP connection establishment time, maximum TCP
connection establishment time, average connection establishment time
and the aggregate connection establishment time.
5.4 Maximum TCP Connection Tear Down Rate
5.4.1 Objective 5.4.1 Objective
To determine the connection teardown time[1] through or with the To determine the maximum TCP connection tear down rate through or
DUT/SUT. As with the connection establishment time, separate with the DUT/SUT, as defined by RFC2647[1].
measurements SHOULD be taken for each connection oriented protocol
involved in closing a connection.
5.4.2 Setup Parameters 5.4.2 Setup Parameters
The following parameters MUST be defined. Each parameters is Number of Connections - Defines the number of TCP connections that
configured with the following considerations. the tester will attempt to tear down.
Initial connections - Defines the number of TCP connections to
initialize the test with.
Teardown attempt rate - The rate at which the tester will attempt Age Time - The time, expressed in seconds, the DUT/SUT will keep a
to tear down TCP connections. connection in it's state table after receiving a TCP FIN or RST
packet.
5.4.3 Procedure 5.4.3 Procedure
The virtual clients will initialize the test by establishing TCP An iterative search algorithm MAY be used to determine the maximum
connections defined by initial connections. The test will use the TCP connection tear down rate. The test iterates through different
same algorithm for establishing the TCP connections as described in TCP connection tear down rates with a fixed number of TCP
the connection capacity test(Section 5.1) with the exception that connections.
no object transfers are required.
The virtual clients will then attempt to tear down all of TCP The virtual client(s) will initialize the test by establishing TCP
connections at a rate defined by tear down attempt rate. The connections defined by number of connections. The virtual client(s)
tester(Virtual Clients) MUST exclude any connections which do not will then attempt to tear down all of TCP connections, at a rate
properly close in its measurements. For example, connections in defined by tear down attempt rate. For benchmarking purposes, the
which the DUT/SUT transmits a TCP RST in response to a TCP FIN tester MUST use a TCP FIN when initiating the connection tear down.
packet or connections which do not acknowledge the FIN packet
requesting the connection be closed.
In the case of proxy based DUT/SUTs, the DUT/SUT will itself receive In the case of proxy based DUT/SUTs, the DUT/SUT will itself receive
the final ACK when closing out it's side of the TCP connection. For the final ACK in the three-way handshake when a connection is being
validation purposes, the virtual client(s) SHOULD verify that the torn down. For validation purposes, the virtual client(s) MAY
DUT/SUT received the final ACK in the connection tear down exchange verify that the DUT/SUT received the final ACK in the connection tear
for all connections by transmitting a TCP datagram(s) referencing down exchange for all connections by transmitting a TCP datagram
the previously town down connection(s). A TCP RST should be received referencing the previously town down connection. A TCP RST should be
in response to the TCP datagram(s), if the ACK was received by the received in response to the TCP datagram.
DUT/SUT.
5.4.4 Measurements 5.4.4 Measurements
The tester MUST measure the minimum, average and maximum TCP Highest connection tear down rate - Highest rate, in connections per
connection tear down times. The TCP connection tear down time will second, for which all TCP connections were successfully torn down.
be considered the interval between the transmission of the first TCP
FIN packet transmitted by the tester requesting a connection tear
down to receipt of the ACK packet on the same tester interface.
The tester SHOULD measure the minimum, maximum and average tear down Minimum connection tear down time - Lowest TCP connection tear down
times for all other underlying connection oriented protocols. For time measured as defined in appendix C.
purposes of benchmarking firewall performance, the connection tear down
time will be considered the interval between the transmission of the
first bit of the first octet of the packet carrying the tear down
request to the DUT/SUT interface to receipt of the last bit of the
last octet of the last packet of the connection tear down traffic
headed in the opposite direction.
The tester SHOULD measure the aggregate connection tear down time and Maximum connection tear down time - Highest TCP connection tear down
the total number of connections torn down for each protocol measured. time measured as defined in appendix C.
5.4.5 Reporting Format Average connection tear down time - The mean of all measurements of
connection tear down times.
The test report MUST note the initial connections , tear down attempt Aggregate connection tear down time - The total of all measurements
rate and tear down step count. of connection tear down times.
For each connection oriented protocol the tester measured, the 5.4.5 Reporting Format
report MUST note the minimum, average and maximum connection tear
down. In addition, it SHOULD include the aggregate connection tear
down time and attempted tear downs completed. The report MUST
identify the layer/protocol for which the measurements were made.
Failure analysis: The test report MUST note the number of connections, age time and
highest connection tear down rate measured.
The test report SHOULD indicate the number of connections which failed The intermediate results of the search algorithm SHOULD be reported
the validation step. in the format of a table with a column for each iteration. There
SHOULD be rows for the number of TCP tear downs attempted, number
of TCP connection tear downs completed, minimum TCP connection tear
down time, maximum TCP connection tear down time, average TCP
connection tear down time and the aggregate TCP connection tear down
time.
5.5 Denial Of Service Handling 5.5 Denial Of Service Handling
5.5.1 Objective 5.5.1 Objective
To determine the effect of a denial of service attack on a DUT/SUTs To determine the effect of a denial of service attack on a DUT/SUT
connection establishment and/or forwarding rate. The Denial Of TCP connection establishment and/or HTTP transfer rates. The denial
Service Handling test MUST be run after obtaining baseline of service handling test MUST be run after obtaining baseline
Measurements from sections 5.2 and/or 5.6. measurements from sections 5.3 and/or 5.6.
The TCP SYN flood attack exploits TCP's three-way handshake mechanism
by having an attacking source host generate TCP SYN packets with
random source addresses towards a victim host, thereby consuming that
host's resources.
Some firewalls employ mechanisms to guard against SYN attacks. If such The TCP SYN flood attack exploits TCP's three-way handshake
mechanisms exist on the DUT/SUT, tests SHOULD be run with these mechanism by having an attacking source host generate TCP SYN
mechanisms enabled to determine how well the DUT/SUT can maintain, packets with random source addresses towards a victim host, thereby
under such attacks, the baseline connection rates and HTTP forwarding consuming that host's resources.
rates determined in section 5.2 and section 5.6, respectively.
5.5.2 Setup Parameters 5.5.2 Setup Parameters
Use the same setup parameters as defined in section 5.2.2 or 5.6.2, Use the same setup parameters as defined in section 5.2.2 or 5.6.2,
depending on whether testing against the baseline connection setup depending on whether testing against the baseline TCP connection
rate test or HTTP test, respectfully. setup rate test or HTTP transfer rate test, respectfully.
In addition, the following setup parameters MUST be defined. In addition, the following setup parameters MUST be defined.
SYN Attack Rate - Defines the rate, in packets per second at which SYN attack rate - Rate, expressed in packets per second, at which
the server(s) are targeted with TCP SYN packets. the server(s) or NAT proxy address is targeted with TCP SYN packets.
5.5.3 Procedure 5.5.3 Procedure
Use the same procedure as defined in section 5.2.3 or 5.6.3, Use the same procedure as defined in section 5.3.3 or 5.6.3,
Depending on whether testing against the baseline connection setup depending on whether testing against the baseline TCP connection
rate test or HTTP test, respectfully. In addition, the tester will establishment rate or HTTP transfer rate test, respectfully. In
generate TCP SYN packets targeting the server(s) IP address or NAT addition, the tester will generate TCP SYN packets targeting the
proxy address at a rate defined by SYN attack rate. server(s) IP address or NAT proxy address at a rate defined by SYN
attack rate.
The tester originating the TCP SYN attack MUST be attached to the The tester originating the TCP SYN attack MUST be attached to the
unprotected network. In addition, the tester MUST not respond to the unprotected network. In addition, the tester MUST not respond to the
SYN/ACK packets sent by target server in response to the SYN packet. SYN/ACK packets sent by target server or NAT proxy in response to
the SYN packet.
Some firewalls employ mechanisms to guard against SYN attacks. If
such mechanisms exist on the DUT/SUT, tests SHOULD be run with these
mechanisms enabled to determine how well the DUT/SUT can maintain,
under such attacks, the baseline connection establishment rates and
HTTP transfer rates determined in section 5.3 and section 5.6,
respectively.
5.5.4 Measurements 5.5.4 Measurements
Perform the same measurements as defined in section 5.2.4 or 5.6.4, Perform the same measurements as defined in section 5.3.4 or 5.6.4,
depending on whether testing against the baseline connection setup depending on whether testing against the baseline TCP connection
rate test or HTTP test, respectfully. establishment rate test or HTTP transfer rate, respectfully.
In addition, the tester SHOULD track SYN packets associated with the In addition, the tester SHOULD track TCP SYN packets associated with
SYN attack which the DUT/SUT forwards on the protected or DMZ the SYN attack which the DUT/SUT forwards on the protected or DMZ
interface(s). interface(s).
5.5.5 Reporting Format 5.5.5 Reporting Format
The test SHOULD use the same reporting format as described in The test SHOULD use the same reporting format as described in
section 5.2.5 or 5.6.5, depending on whether testing against section 5.3.5 or 5.6.5, depending on whether testing against the
baseline throughput rates or HTTP test, respectively. baseline TCP connection establishment rate test or HTTP transfer rate,
respectfully.
In addition, the report MUST indicate a denial of service handling In addition, the report MUST indicate a denial of service handling
test, SYN attack rate, number SYN attack packets transmitted and test, SYN attack rate, number TCP SYN attack packets transmitted
number of SYN attack packets received. The report SHOULD indicate and the number of TCP SYN attack packets forwarded by the DUT/SUT.
whether or not the DUT has any SYN attack mechanisms enabled. The report MUST indicate whether or not the DUT has any SYN attack
mechanisms enabled.
5.6 HTTP 5.6 HTTP Transfer Rate
5.6.1 Objective 5.6.1 Objective
To determine the bit forwarding rate, as defined by RFC2647, of the To determine the transfer rate of HTTP requested object transversing
DUT/SUT when presented with HTTP traffic flows. the DUT/SUT.
5.6.2 Setup Parameters 5.6.2 Setup Parameters
Connection type - The tester MUST use HTTP 1.1 for HTTP measurements. The following parameters MUST be defined for all tests:
Number of GET Requests - Defines the number of HTTP 1.1 GET 5.6.2.1 Transport-Layer Setup Parameters
requests attempted per connection.
GET Request Rate - Defines the rate, in GET requests per second, at Number of connections - Defines the aggregate number of connections
which HTTP GET requests are attempted on any given connection. attempted. The number SHOULD be a multiple of the number of virtual
clients participating in the test
5.6.2.2 Application-Layer Setup Parameters
Object Size - Defines the number of bytes to be transferred in Session type - The virtual clients/servers MUST use HTTP 1.1 or
response to an HTTP GET request. higher.
5.6.3 HTTP Procedure GET requests per connection - Defines the number of HTTP 1.1 or
higher GET requests attempted per connection.
Each HTTP 1.1 virtual client will attempt to establish each Object Size - Defines the number of bytes, excluding any bytes
connection to its HTTP 1.1 target server(s), using either the target associated with the HTTP header, to be transferred in response to an
server's IP address or NAT proxy address, in a round robin fashion. HTTP 1.1 or higher GET request.
The tester will initiate GET requests for each connection at a
constant rate defined by GET request rate, regardless of the state
of the DUT/SUT.
Baseline measurements SHOULD be performed using a single GET request 5.6.3 Procedure
per connection with a 1-byte object size. If the tester makes multiple
HTTP GET requests per connection, it MUST request the same object size
for each GET request. Testers SHOULD run multiple iterations of this
test using other object sizes and/or multiple requests per connection.
See appendix A when testing proxy based DUT/SUT regarding HTTP version
considerations.
5.6.4 Measurements Each HTTP 1.1 or higher client will request one or more objects from
an HTTP 1.1 or higher server using one or more HTTP GET requests.
The aggregate number of connections attempted, defined by number of
connections, MUST be evenly divided among all of the participating
virtual clients.
Version information: If the virtual client(s) make multiple HTTP GET requests per
connection, it MUST request the same object size for each GET
request. Multiple iterations of this test SHOULD be ran using
different object sizes.
The test report MUST note the use of an HTTP 1.1 client and server. 5.6.4 Measurements
Application Layer 5.6.4.1 Application-Layer measurements
Bit Forwarding Rate - The bit forwarding rate of the DUT/SUT MUST, Average Transfer Rate - The average transfer rate of the DUT/SUT
at a minimum, be measured at the application layer and shall be MUST be measured and shall be referenced to the requested object(s).
referenced to the requested object(s). The measurement will The measurement will start on transmission of the first bit of the
start on transmission of the first bit of the first requested object first requested object and end on transmission of the last bit of
and end on transmission of the last bit of the last requested object. the last requested object. The average transfer rate, in bits per
The aggregate bit forwarding rate, in bits per second, will be second, will be calculated using the following formula:
calculated using the following formula:
OBJECTS * OBJECTSIZE * 8 OBJECTS * OBJECTSIZE * 8
FORWARDING RATE(bit/s) = -------------------------- TRANSFER RATE(bit/s) = --------------------------
DURATION DURATION
OBJECTS - Objects successfully transferred OBJECTS - Objects successfully transferred
OBJECTSIZE - Object size in bytes OBJECTSIZE - Object size in bytes
DURATION - Aggregate transfer time based on aforementioned time DURATION - Aggregate transfer time based on aforementioned time
references. references.
Transport-Layer and Below 5.6.4.2 Measurements at or below the Transport-Layer
Bit forwarding rate for layers at or below the transport layer SHOULD The tester SHOULD make goodput[1] measurements for connection-
also be performed. Bit forwarding rate for these underlying oriented protocols at or below the transport layer. Goodput
layers/protocols MAY be measured in either bits per seconds or UOTs measurements MUST only reference the protocols payload, excluding
per second. In both cases, the measurement will start on transmission any of the protocols header. In addition, the tester MUST exclude
of the first packet containing the first HTTP GET request and end on any bits associated with the connection establishment, connection
transmission of the last packet containing the last octet of the last tear down, security associations or connection maintenance.
requested object. The aggregate bit forwarding rate, in bits per second,
will be calculated using the following formula:
TX - RETX Since connection-oriented protocols require that data be
FORWARDING RATE(bit/s or UOT/s) = ----------- acknowledged, the offered load[6] will vary over the duration of the
DURATION test. When performing forwarding rate measurements, the tester
TX - If measuring in units of bits per seconds, TOTAL is the total should measure the average forwarding rate over the duration of the
bits transmitted including header and optional data for a test.
given protocol. If measuring in UOTs per seconds, total is the
total number of UOTs transmitted. This excludes any bits or UOT
that are associated with connection maintenance[1], such as TCP
keep-alives.
RETX - If measuring in units of bits per seconds, RETX is the total 5.6.5 Reporting Format
number of bits, including header and optional data for a given
protocol, that was retransmitted. If measuring in units of UOTs
per second, RETX is the number of UOTs retransmitted. This
excludes any bits or UOTs that are associated with connection
maintenance, such as TCP keep-alives.
DURATION - Test duration based on aforementioned time references. 5.6.5.1 Application-Layer reporting
5.6.5 Reporting Format The test report MUST note number of GET requests per connection,
and object size.
The test report MUST note the number of GET requests, GET request The transfer rate results SHOULD be reported in tabular form with
rate and object size. a row for each of the object sizes. There SHOULD be a column for the
object size, the number of completed requests, the number of
completed responses, and the transfer rate results for each test.
Application layer bit forwarding rate results SHOULD be reported in Failure analysis:
tabular form with a row for each of the object sizes. There SHOULD
be columns for the object size, the number of completed requests,
the number of completed responses, and the bit forwarding rate
results for each test.
When reporting bit forwarding measurements for layers below the The test report SHOULD indicate the number and percentage of HTTP
application layer, such as TCP or IP, the report MUST note whether GET request or responses that failed to complete.
the measurements are in bit per second or UOTs per second and the
object size transferred. The report SHOULD be in tabular form with Version information:
a row for each layer/protocol. There should be columns for
transmitted bits/UOTs, retransmitted bits/UOTs and the measured The test report MUST note the version of HTTP client(s) and
forwarding rate. server(s).
5.6.5.2 Transport-Layer and below reporting
The test report MUST note the aggregate number of connections. In
addition, the report MUST identify the layer/protocol for which the
measurement was made.
The results SHOULD be in tabular form with a column for each
iteration of the test. There should be columns for transmitted bits,
retransmitted bits and the measured goodput.
Failure analysis: Failure analysis:
The test report SHOULD indicate the number and percentage of HTTP GET The test report SHOULD indicate the number and percentage of
request or responses that failed to complete. connections that failed to complete.
5.7 IP Fragmentation 5.7 HTTP Concurrent Transaction Capacity
5.7.1 Objective 5.7.1 Objective
To determine the performance impact when the DUT/SUT is presented Determine the maximum number of concurrent or simultaneous HTTP
with IP fragmented[5] traffic. IP datagrams which have been transactions the DUT/SUT can support. This test is intended to
fragmented, due to crossing a network that supports a smaller find the maximum number of users that can simultaneously access
MTU(Maximum Transmission Unit) than the actual datagram, may web objects.
require the firewall to perform re-assembly prior to the datagram
being applied to the rule set.
While IP fragmentation is a common form of attack, either on the
firewall itself or on internal hosts, this test will focus on
determining how the additional processing associated with the
re-assembly of the datagrams has on the goodput of the DUT/SUT.
5.7.2 Setup Parameters 5.7.2 Setup Parameters
The following parameters MUST be defined. GET request rate - The aggregate rate, expressed in request per
second, at which HTTP 1.1 or higher GET requests are offered by the
virtual client(s).
Trial duration - Trial duration SHOULD be set for 30 seconds. Session type - The virtual clients/servers MUST use HTTP 1.1 or
higher.
5.7.2.1 Non-Fragmented Traffic Parameters 5.7.3 Procedure
Session rate - Defines the rate, in sessions per second, that the An iterative search algorithm MAY be used to determine the maximum
HTTP sessions are attempted. HTTP concurrent transaction capacity.
Requests per session - Defines the number of HTTP GET requests per For each iteration, the virtual client(s) will vary the number of
session. concurrent or simultaneous HTTP transactions - that is, on-going
GET requests. The HTTP 1.1 or higher virtual client(s) will request
one object, across each connection, from an HTTP 1.1 or higher
server using one HTTP GET request. The aggregate rate at which the
virtual client(s) will offer the requests will be defined by GET
request rate.
Object Size - Defines the number of bytes to be transferred in The object size requested MUST be large enough, such that, the
response to an HTTP GET request. transaction - that is, the request/response cycle -- will exist for
the duration of the test. At the end of each iteration, the tester
MUST validate that all transactions are still active. After all of
the transactions are checked, the transactions MAY be aborted.
5.7.2.1 Fragmented Traffic Parameters 5.7.4 Measurements
Packet size, expressed as the number of bytes in the IP/UDP packet, Maximum concurrent transactions - Total number of concurrent HTTP
exclusive of link-layer headers and checksums. transactions active for the last successful iteration performed in
the search algorithm.
Fragmentation Length - Defines the length of the data portion of the 5.7.5 Reporting Format
IP datagram and MUST be multiple of 8. Testers SHOULD use the minimum
value, but MAY use other sizes as well.
Intended Load - Intended load, expressed as percentage of media 5.7.5.1 Application-Layer reporting
utilization.
5.7.3 Procedure The test report MUST note the GET request rate and the maximum
concurrent transactions measured.
Each HTTP 1.1 virtual client will attempt to establish sessions The intermediate results of the search algorithm MAY be reported
to its HTTP 1.1 target server(s), using either the target server's in a table format with a column for each iteration. There SHOULD be
IP address or NAT proxy address, at a fixed rate in a round robin rows for the number of concurrent transactions attempted, GET
fashion. At the same time, a client attached to the unprotected side request rate, number of aborted transactions and number of
of the network will offer a unidirectional stream of unicast UDP/IP transactions active at the end of the test iteration.
packets to a server connected to the protected side of the network.
The tester MUST offer IP/UDP packets in a steady state.
Baseline measurements SHOULD be performed with a deny rule(s) that Version information:
filters the fragmented traffic. If the DUT/SUT has logging
capability, the log SHOULD be checked to determine if it contains
the correct information regarding the fragmented traffic.
The test SHOULD be repeated with the DUT/SUT rule set changed to The test report MUST note the version of HTTP client(s) and
allow the fragmented traffic through. When running multiple server(s).
iterations of the test, it is RECOMMENDED to vary the fragment
length while keeping all other parameters constant.
5.7.4 Measurements 5.8 HTTP Transaction Rate
Aggregate Goodput - The aggregate bit forwarding rate of the 5.8.1 Objective
requested HTTP objects.(See section 5.6). Only objects which have
successfully completed transferring within the trial duration are
to be included in the goodput measurement.
Transmitted UDP/IP Packets - Number of UDP packets transmitted by Determine the maximum HTTP transaction rate that a DUT/SUT can
client. sustain.
Received UDP/IP Packets - Number of UDP/IP Packets received by 5.8.2 Setup Parameters
server.
5.7.5 Reporting Format Session Type - HTTP 1.1 or higher MUST be used for this test.
The test report MUST note the test duration. Test Duration - Time, expressed in seconds, for which the
virtual client(s) will sustain the attempted GET request rate.
It is RECOMMENDED that the duration be at least 30 seconds.
The test report MUST note the packet size(s), offered load(s) and Requests per connection - Number of object requests per connection.
IP fragmentation length of the UDP/IP traffic. It SHOULD also note
whether the DUT/SUT egresses the offered UDP/IP traffic fragmented
or not.
The test report MUST note the object size(s), session rate and Object Size - Defines the number of bytes, excluding any bytes
requests per session. associated with the HTTP header, to be transferred in response to an
HTTP 1.1 or higher GET request.
The results SHOULD be reported in the format of a table with a 5.8.3 Procedure
row for each of the fragmentation lengths. There SHOULD be columns
for the fragmentation length, IP/UDP packets transmitted by client,
IP/UDP packets received by server, HTTP object size, and measured
goodput.
5.8 Illegal Traffic Handling An iterative search algorithm MAY be used to determine the maximum
transaction rate that the DUT/SUT can sustain.
5.8.1 Objective For each iteration, HTTP 1.1 or higher virtual client(s) will
vary the aggregate GET request rate offered to HTTP 1.1 or higher
server(s). The virtual client(s) will maintain the offered request
rate for the defined test duration.
To determine the behavior of the DUT/SUT when presented with a If the tester makes multiple HTTP GET requests per connection, it
combination of both legal and Illegal traffic flows. Note that MUST request the same object size for each GET request rate.
Illegal traffic does not refer to an attack, but to traffic which Multiple iterations of this test MAY be performed with objects of
has been explicitly defined by a rule(s) to drop. different sizes.
5.8.2 Setup Parameters 5.8.4 Measurements
Connection type - The tester MUST use HTTP 1.1 for HTTP measurements. Maximum Transaction Rate - The maximum rate at which all
transactions -- that is all requests/responses cycles -- are
completed.
Number of GET Requests - Defines the number of HTTP 1.1 GET Transaction Time - The tester SHOULD measure minimum, maximum and
requests attempted per connection. average transaction times. The transaction time will start when the
virtual client issues the GET request and end when the requesting
virtual client receives the last bit of the requested object.
GET Request Rate - Defines the rate, in GET requests per second, at 5.8.5 Reporting Format
which HTTP GET requests are attempted on any given connection.
Object Size - Defines the number of bytes to be transferred in The test report MUST note the test duration, object size, requests
response to an HTTP GET request. per connection and the measured maximum transaction rate.
Illegal traffic percentage - Percentage of HTTP connections which The intermediate results of the search algorithm MAY be reported
have been explicitly defined in a rule(s) to drop. in a table format with a column for each iteration. There SHOULD be
rows for the GET request attempt rate, number of requests attempted,
number and percentage of requests completed, number of responses
attempted, number and percentage of responses completed, minimum
transaction time, average transaction time and maximum transaction
time.
5.8.3 Procedure Version information:
Each HTTP 1.1 virtual client will attempt to establish sessions The test report MUST note the version of HTTP client(s) and
to its HTTP 1.1 target server(s), using either the target server's server(s).
IP address or NAT proxy address, at a fixed rate in a round robin
fashion.
The tester MUST present the connection requests, both legal and 5.9 Illegal Traffic Handling
illegal, in an evenly distributed manner. Many firewalls have
5.9.1 Objective
To determine the behavior of the DUT/SUT when presented with a
combination of both legal and Illegal traffic flows. Note that
Illegal traffic does not refer to an attack, but traffic which
has been explicitly defined by a rule(s) to drop.
5.9.2 Setup Parameters
Setup parameters will use the same parameters as specified in the
HTTP transfer rate test. In addition, the following setup
parameters MUST be defined:
Illegal traffic percentage - Percentage of HTTP 1.1 or higher
connections which have been explicitly defined in a rule(s) to drop.
5.9.3 Procedure
Each HTTP 1.1 or higher client will request one or more objects from
an HTTP 1.1 or higher server using one or more HTTP GET requests.
The aggregate number of connections attempted, defined by number of
connections, MUST be evenly divided among all of the participating
virtual clients.
The virtual client(s) MUST offer the connection requests, both legal
and illegal, in an evenly distributed manner. Many firewalls have
the capability to filter on different traffic criteria( IP the capability to filter on different traffic criteria( IP
addresses, Port numbers, etc). Testers may run multiple addresses, Port numbers, etc). Testers may run multiple
iterations of this test with the DUT/SUT configured to filter iterations of this test with the DUT/SUT configured to filter
on different traffic criteria. on different traffic criteria.
5.8.4 Measurements 5.9.4 Measurements
Tester SHOULD perform the same bit forwarding measurements as defined Tester SHOULD perform the same measurements as defined in HTTP
in HTTP test(Section 5.6.4). test(Section 5.6.4). Unlike the HTTP transfer rate test, the
tester MUST not include any bits which are associated with illegal
traffic in its forwarding rate measurements.
5.8.5 Reporting Format 5.9.5 Reporting Format
Tester SHOULD report SHOULD be the same as specified in the HTTP Test report SHOULD be the same as specified in the HTTP
test(Section 5.6.5). test(Section 5.6.5).
In addition, the report MUST note the percentage of illegal HTTP In addition, the report MUST note the percentage of illegal HTTP
connections. connections.
Failure analysis Failure analysis:
Test report MUST note the number and percentage of illegal connections Test report MUST note the number and percentage of illegal
allowed by the DUT/SUT. connections that were allowed by the DUT/SUT.
5.9 Latency 5.10 IP Fragmentation Handling
5.9.1 Objective 5.10.1 Objective
To determine the performance impact when the DUT/SUT is presented
with IP fragmented[5] traffic. IP packets which have been
fragmented, due to crossing a network that supports a smaller
MTU(Maximum Transmission Unit) than the actual IP packet, may
require the firewall to perform re-assembly prior to the rule set
being applied.
While IP fragmentation is a common form of attack, either on the
firewall itself or on internal hosts, this test will focus on
determining how the additional processing associated with the
re-assembly of the packets have on the forwarding rate of the
DUT/SUT. RFC 1858 addresses some fragmentation attacks that
get around IP filtering processes used in routers and hosts.
5.10.2 Setup Parameters
The following parameters MUST be defined.
5.10.2.1 Non-Fragmented Traffic Parameters
Setup parameters will be the same as defined in the HTTP transfer
rate test(Sections 5.6.2.1 and 5.6.2.2).
5.10.2.2 Fragmented Traffic Parameters
Packet size - Number of bytes in the IP/UDP packet, exclusive of
link-layer headers and checksums, prior to fragmentation.
MTU - Maximum transmission unit, expressed in bytes. For testing
purposes, this MAY be configured to values smaller than the MTU
supported by the link layer.
Intended Load - Intended load, expressed as percentage of media
utilization.
5.10.3 Procedure
Each HTTP 1.1 or higher client will request one or more objects from
an HTTP 1.1 or higher server using one or more HTTP GET requests.
The aggregate number of connections attempted, defined by number of
connections, MUST be evenly divided among all of the participating
virtual clients. If the virtual client(s) make multiple HTTP GET
requests per connection, it MUST request the same object size for
each GET request.
A tester attached to the unprotected side of the network, will offer
a unidirectional stream of unicast IP/UDP targeting a server
attached to either the protected or DMZ. The tester MUST offer the
unidirectional stream over the duration of the test.
Baseline measurements SHOULD be performed with IP filtering deny
rule(s) to filter fragmented traffic. If the DUT/SUT has logging
capability, the log SHOULD be checked to determine if it contains
the correct information regarding the fragmented traffic.
The test SHOULD be repeated with the DUT/SUT rule set changed to
allow the fragmented traffic through. When running multiple
iterations of the test, it is RECOMMENDED to vary the MTU while
keeping all other parameters constant.
Then setup the DUT/SUT to the policy or rule set the manufacturer
required to be defined to protect against fragmentation attacks and
repeat the measurements outlined in the baseline procedures.
5.10.4 Measurements
Tester SHOULD perform the same measurements as defined in HTTP
test(Section 5.6.4).
Transmitted UDP/IP Packets - Number of UDP packets transmitted by
client.
Received UDP/IP Packets - Number of UDP/IP Packets received by
server.
5.10.5 Reporting Format
5.10.1 Non-Fragmented Traffic
The test report SHOULD be the same as described in section 5.6.5.
Note that any forwarding rate measurements for the HTTP traffic
excludes any bits associated with the fragmented traffic which
may be forward by the DUT/SUT.
5.10.2 Fragmented Traffic
The test report MUST note the packet size, MTU size, intended load,
number of UDP/IP packets transmitted and number of UDP/IP packets
forwarded. The test report SHOULD also note whether or not the
DUT/SUT forwarded the offered UDP/IP traffic fragmented.
5.11 Latency
5.11.1 Objective
To determine the latency of network-layer or application-layer data To determine the latency of network-layer or application-layer data
traversing the DUT/SUT. RFC 1242 [3] defines latency. traversing the DUT/SUT. RFC 1242 [3] defines latency.
5.9.2 Setup Parameters 5.11.2 Setup Parameters
The following parameters MUST be defined: The following parameters MUST be defined:
5.9.2.1 Network-layer Measurements 5.11.2.1 Network-layer Measurements
Packet size, expressed as the number of bytes in the IP packet, Packet size, expressed as the number of bytes in the IP packet,
exclusive of link-layer headers and checksums. exclusive of link-layer headers and checksums.
Intended load, expressed as percentage of media utilization. Intended load, expressed as percentage of media utilization.
Offered load, expressed as percentage of media utilization.
Test duration, expressed in seconds. Test duration, expressed in seconds.
Test instruments MUST generate packets with unique timestamp signatures. Test instruments MUST generate packets with unique timestamp
signatures.
5.9.2.2 Application-layer Measurements 5.11.2.2 Application-layer Measurements
Object size, expressed as the number of bytes to be transferred across a Object Size - Defines the number of bytes, excluding any bytes
connection in response to an HTTP GET request. Testers SHOULD use the associated with the HTTP header, to be transferred in response to
minimum object size supported by the media, but MAY use other object an HTTP 1.1 or higher GET request. Testers SHOULD use the minimum
object size supported by the media, but MAY use other object
sizes as well. sizes as well.
Connection type. The tester MUST use one HTTP 1.1 connection for latency Connection type. The tester MUST use one HTTP 1.1 or higher
measurements. connection for latency measurements.
Number of objects requested. Number of objects requested.
Number of objects transferred. Number of objects transferred.
Test duration, expressed in seconds. Test duration, expressed in seconds.
Test instruments MUST generate packets with unique timestamp signatures. Test instruments MUST generate packets with unique timestamp
signatures.
5.9.3 Network-layer procedure 5.11.3 Network-layer procedure
A client will offer a unidirectional stream of unicast packets to a server. A client will offer a unidirectional stream of unicast packets to a
The packets MUST use a connectionless protocol like IP or UDP/IP. server. The packets MUST use a connectionless protocol like IP or
UDP/IP.
The tester MUST offer packets in a steady state. As noted in the latency The tester MUST offer packets in a steady state. As noted in the
discussion in RFC 2544 [4], latency measurements MUST be taken at the latency discussion in RFC 2544 [4], latency measurements MUST be
throughput level -- that is, at the highest offered load with zero packet taken at the throughput level -- that is, at the highest offered
loss. Measurements taken at the throughput level are the only ones that can load with zero packet loss. Measurements taken at the throughput
legitimately be termed latency. level are the only ones that can legitimately be termed latency.
It is RECOMMENDED that implementers use offered loads not only at the It is RECOMMENDED that implementers use offered loads not only at
throughput level, but also at load levels that are less than or greater the throughput level, but also at load levels that are less than
than the throughput level. To avoid confusion with existing terminology, or greater than the throughput level. To avoid confusion with
measurements from such tests MUST be labeled as delay rather than latency. existing terminology, measurements from such tests MUST be labeled
If desired, the tester MAY use a step test in which offered loads increment as delay rather than latency.
or decrement through a range of load levels.
The duration of the test portion of each trial MUST be at least 30 seconds. If desired, the tester MAY use a step test in which offered loads
increment or decrement through a range of load levels.
5.9.4 Application layer procedure The duration of the test portion of each trial MUST be at least 30
seconds.
An HTTP 1.1 client will request one or more objects from an HTTP 1.1 server 5.11.4 Application layer procedure
using one or more HTTP GET requests. If the tester makes multiple HTTP GET
requests, it MUST request the same-sized object each time. Testers may run
multiple iterations of this test with objects of different sizes.
Implementers MAY configure the tester to run for a fixed duration. In this An HTTP 1.1 or higher client will request one or more objects from
case, the tester MUST report the number of objects requested and returned an HTTP or higher 1.1 server using one or more HTTP GET requests. If
for the duration of the test. For fixed-duration tests it is RECOMMENDED the tester makes multiple HTTP GET requests, it MUST request the
that the duration be at least 30 seconds. same-sized object each time. Testers may run multiple iterations of
this test with objects of different sizes.
5.9.5 Measurements Implementers MAY configure the tester to run for a fixed duration.
In this case, the tester MUST report the number of objects requested
and returned for the duration of the test. For fixed-duration tests
it is RECOMMENDED that the duration be at least 30 seconds.
Minimum delay - The smallest delay incurred by data traversing the DUT/SUT 5.11.5 Measurements
at the network layer or application layer, as appropriate.
Maximum delay - The largest delay incurred by data traversing the DUT/SUT Minimum delay - The smallest delay incurred by data traversing the
at the network layer or application layer, as appropriate. DUT/SUT at the network layer or application layer, as appropriate.
Average delay - The mean of all measurements of delay incurred by data Maximum delay - The largest delay incurred by data traversing the
traversing the DUT/SUT at the network layer or application layer, as DUT/SUT at the network layer or application layer, as appropriate.
appropriate.
Delay distribution - A set of histograms of all delay measurements observed Average delay - The mean of all measurements of delay incurred by
for data traversing the DUT/SUT at the network layer or application layer, data traversing the DUT/SUT at the network layer or application
as appropriate. layer, as appropriate.
5.9.6 Network-layer reporting format Delay distribution - A set of histograms of all delay measurements
observed for data traversing the DUT/SUT at the network layer or
application layer, as appropriate.
The test report MUST note the packet size(s), offered load(s) and test 5.11.6 Network-layer reporting format
duration used.
The latency results SHOULD be reported in the format of a table with a row The test report MUST note the packet size(s), offered load(s) and
for each of the tested packet sizes. There SHOULD be columns for the test duration used.
packet size, the intended rate, the offered rate, and the resultant latency
or delay values for each test.
5.9.7 Application-layer reporting format The latency results SHOULD be reported in the format of a table with
a row for each of the tested packet sizes. There SHOULD be columns
for the packet size, the intended rate, the offered rate, and the
resultant latency or delay values for each test.
The test report MUST note the object size(s) and number of requests and 5.11.7 Application-layer reporting format
responses completed. If applicable, the report MUST note the test duration
if a fixed duration was used.
The latency results SHOULD be reported in the format of a table with a row The test report MUST note the object size(s) and number of requests
for each of the object sizes. There SHOULD be columns for the object size, and responses completed. If applicable, the report MUST note the
the number of completed requests, the number of completed responses, and the test duration if a fixed duration was used.
resultant latency or delay values for each test.
The latency results SHOULD be reported in the format of a table with
a row for each of the object sizes. There SHOULD be columns for the
object size, the number of completed requests, the number of
completed responses, and the resultant latency or delay values for
each test.
Failure analysis: Failure analysis:
The test report SHOULD indicate the number and percentage of HTTP GET The test report SHOULD indicate the number and percentage of HTTP
request or responses that failed to complete within the test duration. GET request or responses that failed to complete within the test
duration.
Version information: Version information:
The test report MUST note the use of an HTTP 1.1 client and server. The test report MUST note the version of HTTP client and server.
APPENDICES
APPENDIX A: HTTP(HyperText Transfer Protocol) APPENDIX A: HTTP(HyperText Transfer Protocol)
The most common versions of HTTP in use today are HTTP/1.0 and The most common versions of HTTP in use today are HTTP/1.0 and
HTTP/1.1 with the main difference being in regard to persistent HTTP/1.1 with the main difference being in regard to persistent
connections. HTTP 1.0, by default, does not support persistent connections. HTTP 1.0, by default, does not support persistent
connections. A separate TCP connection is opened up for each connections. A separate TCP connection is opened up for each
GET request the client wants to initiate and closed after the GET request the client wants to initiate and closed after the
requested object transfer is completed. Some implementations of requested object transfer is completed. While some implementations
HTTP/1.0 supports persistence by adding an additional header HTTP/1.0 supports persistence through the use of a keep-alive,
to the request/response: there is no official specification for how the keep-alive operates.
In addition, HTTP 1.0 proxies do support persistent connection as
Connection: Keep-Alive they do not recognize the connection header.
However, under HTTP 1.0, there is no official specification for
how the keep-alive operates. In addition, HTTP 1.0 proxies do
support persistent connection as they do not recognize the
connection header.
HTTP/1.1, by default, does support persistent connection and HTTP/1.1, by default, does support persistent connection and
is therefore the version that is referenced in this methodology. is therefore the version that is referenced in this methodology.
When HTTP/1.1 entities want the underlying transport layer Proxy based DUT/SUTs may monitor the TCP connection and after a
connection closed after a transaction has completed, the
request/response will include a connection-token close in the
connection header:
Connection: close
If no such connection-token is present, the connection remains
open after the transaction is completed. In addition, proxy
based DUT/SUTs may monitor the TCP connection and after a
timeout, close the connection if no activity is detected. The timeout, close the connection if no activity is detected. The
duration of this timeout is not defined in the HTTP/1.1 duration of this timeout is not defined in the HTTP/1.1
specification and will vary between DUT/SUTs. If the DUT/SUT specification and will vary between DUT/SUTs. If the DUT/SUT
closes inactive connections, the aging timer on the DUT SHOULD closes inactive connections, the aging timer on the DUT SHOULD
be configured for a duration that exceeds the test time. be configured for a duration that exceeds the test time.
While this document cannot foresee future changes to HTTP While this document cannot foresee future changes to HTTP
and it impact on the methodologies defined herein, such and it impact on the methodologies defined herein, such
changes should be accommodated for so that newer versions of changes should be accommodated for so that newer versions of
HTTP may be used in benchmarking firewall performance. HTTP may be used in benchmarking firewall performance.
Appendix B. References APPENDIX B: Connection Establishment Time Measurements
For purposes of benchmarking firewall performance, the connection
establishment time will be considered the interval between the
transmission of the first bit of the first octet of the packet
carrying the connection request to the DUT/SUT interface to
receipt of the last bit of the last octet of the last packet of
the connection setup traffic received on the client or server,
depending on whether a given connection requires an even or odd
number of messages, respectfully.
Some connection oriented protocols, such as TCP, involve an odd
number of messages when establishing a connection. In the case of
proxy based DUT/SUTs, the DUT/SUT will terminate the connection,
setting up a separate connection to the server. Since, in such
cases, the tester does not own both sides of the connection,
measurements will be made two different ways. While the following
describes the measurements with reference to TCP, the methodology
may be used with other connection oriented protocols which involve
an odd number of messages.
For non-proxy based DUT/SUTs , the establishment time shall be
directly measured and is considered to be from the time the first
bit of the first SYN packet is transmitted by the client to the
time the last bit of the final ACK in the three-way handshake is
received by the target server.
If the DUT/SUT is proxy based, the connection establishment time is
considered to be from the time the first bit of the first SYN packet
is transmitted by the client to the time the client transmits the first
bit of the first acknowledged TCP datagram(t4-t0 in the following
timeline).
t0: Client sends a SYN.
t1: Proxy sends a SYN/ACK.
t2: Client sends the final ACK.
t3: Proxy establishes separate connection with server.
t4: Client sends TCP datagram to server.
*t5: Proxy sends ACK of the datagram to client.
* While t5 is not considered part of the TCP connection establishment,
acknowledgement of t4 must be received for the connection to be
considered successful.
APPENDIX C: Connection Tear Time Measurements
The TCP connection tear down time will be considered the interval
between the transmission of the first TCP FIN packet transmitted
by the tester requesting a connection tear down to receipt of the
ACK packet on the same tester interface.
Appendix D. References
[1] D. Newman, "Benchmarking Terminology for Firewall Devices", RFC 2647, [1] D. Newman, "Benchmarking Terminology for Firewall Devices", RFC 2647,
August 1999. August 1999.
[2] R. Fielding, J. Gettys, J. Mogul, H Frystyk, L.Masinter, P. Leach, [2] R. Fielding, J. Gettys, J. Mogul, H Frystyk, L.Masinter, P. Leach,
T. Berners-Lee , "Hypertext Transfer Protocol -- HTTP/1.1", T. Berners-Lee , "Hypertext Transfer Protocol -- HTTP/1.1",
RFC 2616 June 1999 RFC 2616 June 1999
[3] S. Bradner, editor. "Benchmarking Terminology for Network [3] S. Bradner, editor. "Benchmarking Terminology for Network
Interconnection Devices," RFC 1242, July 1991. Interconnection Devices," RFC 1242, July 1991.
[4] S. Bradner, J. McQuaid, "Benchmarking Methodology for Network [4] S. Bradner, J. McQuaid, "Benchmarking Methodology for Network
Interconnect Devices," RFC 2544, March 1999. Interconnect Devices," RFC 2544, March 1999.
[5] David C. Clark, "IP Datagram Reassembly Algorithm", RFC 815 , [5] David C. Clark, "IP Datagram Reassembly Algorithm", RFC 815 ,
July 1982. July 1982.
[6] Mandeville, R., "Benchmarking Terminology for LAN Switching
Devices", RFC 2285, February 1998.
[7] Mandeville, R., Perser,J., "Benchmarking Methodology for LAN
Switching Devices", RFC 2889, August 2000.
 End of changes. 

This html diff was produced by rfcdiff 1.23, available from http://www.levkowetz.com/ietf/tools/rfcdiff/