draft-ietf-bmwg-secperf-07.txt   rfc2647.txt 
Network Working Group D. Newman
INTERNET-DRAFT Data Communications
Expires in November 1999 May 1999
Benchmarking Terminology for Firewall Performance
<draft-ietf-bmwg-secperf-07.txt>
Status of This Memo
This document is an Internet-Draft and is in full conformance with
all provisions of Section 10 of RFC2026.
Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF), its areas, and its working groups. Note that
other groups may also distribute working documents as Internet-
Drafts.
Internet-Drafts are draft documents valid for a maximum of six months
and may be updated, replaced, or obsoleted by other documents at any
time. It is inappropriate to use Internet-Drafts as reference
material or to cite them other than as "work in progress."
The list of current Internet-Drafts can be accessed at
http://www.ietf.org/ietf/1id-abstracts.txt
The list of Internet-Draft Shadow Directories can be accessed at
http://www.ietf.org/shadow.html.
1. Introduction......................................................2
2. Existing definitions..............................................3
3. Term definitions..................................................3
3.1 Allowed traffic..................................................3
3.2 Application proxy................................................4
3.3 Authentication...................................................4
3.4 Bit forwarding rate..............................................5
3.5 Circuit proxy....................................................5
3.6 Concurrent connections...........................................6
3.7 Connection.......................................................7
3.8 Connection establishment.........................................8
3.9 Connection establishment time....................................9
3.10 Connection maintenance..........................................9
Newman Page [1]
3.11 Conection overhead.............................................10
3.12 Connection teardown............................................10
3.13 Connection teardown time.......................................11
3.14 Data source....................................................11
3.15 Demilitarized zone.............................................12
3.16 Firewall.......................................................12
3.17 Goodput........................................................13
3.18 Homed..........................................................13
3.19 Illegal traffic................................................14
3.20 Logging........................................................14
3.21 Network address translation....................................15
3.22 Packet filtering...............................................15
3.23 Policy.........................................................16
3.24 Protected network..............................................16
3.25 Proxy..........................................................17
3.26 Rejected traffic...............................................17
3.27 Rule set.......................................................18 Network Working Group D. Newman
Request for Comments: 2647 Data Communications
3.28 Security association...........................................18 Category: Informational August 1999
3.29 Stateful packet filtering......................................19
3.30 Tri-homed......................................................19
3.31 Unit of transfer...............................................20 Benchmarking Terminology for Firewall Performance
3.32 Unprotected network............................................20 Status of this Memo
3.33 User...........................................................21 This memo provides information for the Internet community. It does
not specify an Internet standard of any kind. Distribution of this
memo is unlimited.
4. Security considerations..........................................21 Copyright Notice
5. References.......................................................22 Copyright (C) The Internet Society (1999). All Rights Reserved.
6. Acknowledgments..................................................22 Table of Contents
7. Contact information..............................................23 1. Introduction...................................................2
2. Existing definitions...........................................2
3. Term definitions...............................................3
3.1 Allowed traffic...............................................3
3.2 Application proxy.............................................3
3.3 Authentication................................................4
3.4 Bit forwarding rate...........................................5
3.5 Circuit proxy.................................................6
3.6 Concurrent connections........................................6
3.7 Connection....................................................7
3.8 Connection establishment......................................9
3.9 Connection establishment time.................................9
3.10 Connection maintenance......................................10
3.11 Conection overhead..........................................11
3.12 Connection teardown.........................................11
3.13 Connection teardown time....................................12
3.14 Data source.................................................12
3.15 Demilitarized zone..........................................13
3.16 Firewall....................................................13
3.17 Goodput.....................................................14
3.18 Homed.......................................................15
3.19 Illegal traffic.............................................15
3.20 Logging.....................................................16
3.21 Network address translation.................................16
3.22 Packet filtering............................................17
3.23 Policy......................................................17
3.24 Protected network...........................................18
3.25 Proxy.......................................................19
3.26 Rejected traffic............................................19
3.27 Rule set....................................................20
3.28 Security association........................................20
3.29 Stateful packet filtering...................................21
3.30 Tri-homed...................................................22
3.31 Unit of transfer............................................22
3.32 Unprotected network.........................................23
3.33 User........................................................23
4. Security considerations.......................................24
5. References....................................................25
6. Acknowledgments...............................................25
7. Contact Information...........................................25
8. Full Copyright Statement......................................26
Newman Page [2]
1. Introduction 1. Introduction
This document defines terms used in measuring the performance of This document defines terms used in measuring the performance of
firewalls. It extends the terminology already used for benchmarking firewalls. It extends the terminology already used for benchmarking
routers and switches with definitions specific to firewalls. routers and switches with definitions specific to firewalls.
Forwarding rate and connection-oriented measurements are the primary
metrics used in this document.
Why do we need firewall performance measurements? First, despite the Forwarding rate and connection-oriented measurements are the primary
rapid rise in firewall deployment, there is no standard method of metrics used in this document.
performance measurement. Second, implementations vary widely, making
it difficult to do direct performance comparisons. Finally, more and Why do we need firewall performance measurements? First, despite the
more organizations are deploying firewalls on internal networks rapid rise in firewall deployment, there is no standard method of
operating at relatively high speeds, while most firewall performance measurement. Second, implementations vary widely, making
implementations remain optimized for use over relatively low-speed it difficult to do direct performance comparisons. Finally, more and
wide-area connections. As a result, users are often unsure whether more organizations are deploying firewalls on internal networks
the products they buy will stand up to relatively heavy loads. operating at relatively high speeds, while most firewall
implementations remain optimized for use over relatively low-speed
wide-area connections. As a result, users are often unsure whether
the products they buy will stand up to relatively heavy loads.
2. Existing definitions 2. Existing definitions
This document uses the conceptual framework established in RFCs 1242 This document uses the conceptual framework established in RFCs 1242
and 2544 (for routers) and RFC 2285 (for switches). The router and and 2544 (for routers) and RFC 2285 (for switches). The router and
switch documents contain discussions of several terms relevant to switch documents contain discussions of several terms relevant to
benchmarking the performance of firewalls. Readers should consult the benchmarking the performance of firewalls. Readers should consult the
router and switch documents before making use of this document. router and switch documents before making use of this document.
This document uses the definition format described in RFC 1242, This document uses the definition format described in RFC 1242,
Section 2. The sections in each definition are: definition, Section 2. The sections in each definition are: definition,
discussion, measurement units (optional), issues (optional), and discussion, measurement units (optional), issues (optional), and
cross-references. cross-references.
3. Term definitions 3. Term definitions
3.1 Allowed traffic 3.1 Allowed traffic
Definition: Definition:
Packets forwarded as a result of the rule set of the device under Packets forwarded as a result of the rule set of the device under
test/system under test (DUT/SUT). test/system under test (DUT/SUT).
Discussion:
Firewalls typically are configured to forward only those packets
explicitly permitted in the rule set. Forwarded packets must be
included in calculating the bit forwarding rate or maximum bit
forwarding rate of the DUT/SUT. All other packets must not be
included in bit forwarding rate calculations.
This document assumes 1:1 correspondence of allowed traffic offered Discussion:
to the DUT/SUT and forwarded by the DUT/SUT. There are cases where Firewalls typically are configured to forward only those packets
the DUT/SUT may forward more traffic than it is offered; for example, explicitly permitted in the rule set. Forwarded packets must be
the DUT/SUT may act as a mail exploder or a multicast server. Any included in calculating the bit forwarding rate or maximum bit
forwarding rate of the DUT/SUT. All other packets must not be
included in bit forwarding rate calculations.
Newman Page [3] This document assumes 1:1 correspondence of allowed traffic offered
attempt to benchmark forwarding rates of such traffic must include a to the DUT/SUT and forwarded by the DUT/SUT. There are cases where
description of how much traffic the tester expects to be forwarded. the DUT/SUT may forward more traffic than it is offered; for
example, the DUT/SUT may act as a mail exploder or a multicast
server. Any attempt to benchmark forwarding rates of such traffic
must include a description of how much traffic the tester expects
to be forwarded.
Unit of measurement: Unit of measurement:
not applicable not applicable
Issues: Issues:
See also: See also:
policy policy
rule set rule set
3.2 Application proxy 3.2 Application proxy
Definition: Definition:
A proxy service that is set up and torn down in response to a client A proxy service that is set up and torn down in response to a
request, rather than existing on a static basis. client request, rather than existing on a static basis.
Discussion: Discussion:
Circuit proxies always forward packets containing a given port number Circuit proxies always forward packets containing a given port
if that port number is permitted by the rule set. Application number if that port number is permitted by the rule set.
proxies, in contrast, forward packets only once a connection has been Application proxies, in contrast, forward packets only once a
established using some known protocol. When the connection closes, a connection has been established using some known protocol. When the
firewall using applicaton proxies rejects individual packets, even if connection closes, a firewall using applicaton proxies rejects
they contain port numbers allowed by a rule set. individual packets, even if they contain port numbers allowed by a
rule set.
Unit of measurement: Unit of measurement:
not applicable not applicable
Issues: Issues:
circuit proxy circuit proxy
rule sets rule sets
See also: See also:
allowed traffic allowed traffic
circuit proxy circuit proxy
proxy proxy
rejected traffic rejected traffic
rule set rule set
3.3 Authentication 3.3 Authentication
Definition: Definition:
The process of verifying that a user requesting a network resource is The process of verifying that a user requesting a network resource
who he, she, or it claims to be, and vice versa. is who he, she, or it claims to be, and vice versa.
Discussion:
Trust is a critical concept in network security. Any network resource
(such as a file server or printer) typically requires authentication
before granting access.
Authentication takes many forms, including but not limited to IP Discussion:
addresses; TCP or UDP port numbers; passwords; external token Trust is a critical concept in network security. Any network
resource (such as a file server or printer) typically requires
authentication before granting access.
Newman Page [4] Authentication takes many forms, including but not limited to IP
authentication cards; and biometric identification such as signature, addresses; TCP or UDP port numbers; passwords; external token
speech, or retina recognition systems. authentication cards; and biometric identification such as
signature, speech, or retina recognition systems.
The entity being authenticated might be the client machine (for The entity being authenticated might be the client machine (for
example, by proving that a given IP source address really is that example, by proving that a given IP source address really is that
address, and not a rogue machine spoofing that address) or a user (by address, and not a rogue machine spoofing that address) or a user
proving that the user really is who he, she, or it claims to be). (by proving that the user really is who he, she, or it claims to
Servers might also authenticate themselves to clients. be). Servers might also authenticate themselves to clients.
Testers should be aware that in an increasingly mobile society, Testers should be aware that in an increasingly mobile society,
authentication based on machine-specific criteria such as an IP authentication based on machine-specific criteria such as an IP
address or port number is not equivalent to verifying that a given address or port number is not equivalent to verifying that a given
individual is making an access request. At this writing systems that individual is making an access request. At this writing systems
verify the identity of users are typically external to the firewall, that verify the identity of users are typically external to the
and may introduce additional latency to the overall SUT. firewall, and may introduce additional latency to the overall SUT.
Unit of measurement: Unit of measurement:
not applicable not applicable
Issues: Issues:
See also: See also:
user user
3.4 Bit forwarding rate 3.4 Bit forwarding rate
Definition: Definition:
The number of bits per second of allowed traffic a DUT/SUT can be The number of bits per second of allowed traffic a DUT/SUT can be
observed to transmit to the correct destination interface(s) in observed to transmit to the correct destination interface(s) in
response to a specified offered load. response to a specified offered load.
Discussion: Discussion:
This definition differs substantially from section 3.17 of RFC 1242 This definition differs substantially from section 3.17 of RFC 1242
and section 3.6.1 of RFC 2285. and section 3.6.1 of RFC 2285.
Unlike both RFCs 1242 and 2285, this definition introduces the notion Unlike both RFCs 1242 and 2285, this definition introduces the
of different classes of traffic: allowed, illegal, and rejected (see notion of different classes of traffic: allowed, illegal, and
definitions for each term). For benchmarking purposes, it is assumed rejected (see definitions for each term). For benchmarking
that bit forwarding rate measurements include only allowed traffic. purposes, it is assumed that bit forwarding rate measurements
include only allowed traffic.
Unlike RFC 1242, there is no reference to lost or retransmitted data. Unlike RFC 1242, there is no reference to lost or retransmitted
Forwarding rate is assumed to be a goodput measurement, in that only data. Forwarding rate is assumed to be a goodput measurement, in
data successfully forwarded to the destination interface is measured. that only data successfully forwarded to the destination interface
Bit forwarding rate must be measured in relation to the offered load. is measured. Bit forwarding rate must be measured in relation to
Bit forwarding rate MAY be measured with differed load levels, the offered load. Bit forwarding rate may be measured with
traffic orientation, and traffic distribution. differed load levels, traffic orientation, and traffic
distribution.
Unlike RFC 2285, this measurement counts bits per second rather than Unlike RFC 2285, this measurement counts bits per second rather
frames per second. Testers interested in frame (or frame-like) than frames per second. Testers interested in frame (or frame-like)
measurements should use units of transfer. measurements should use units of transfer.
Unit of measurement: Unit of measurement:
bits per second bits per second
Newman Page [5] Issues:
Issues: Allowed traffic vs. rejected traffic
Allowed traffic vs. rejected traffic
See also: See also:
allowed traffic allowed traffic
goodput goodput
illegal traffic illegal traffic
rejected traffic rejected traffic
unit of transfer unit of transfer
3.5 Circuit proxy 3.5 Circuit proxy
Definition: Definition:
A proxy service that statically defines which traffic will be A proxy service that statically defines which traffic will be
forwarded. forwarded.
Discussion: Discussion:
The key difference between application and circuit proxies is that The key difference between application and circuit proxies is that
the latter are static and thus will always set up a connection if the the latter are static and thus will always set up a connection if
DUT/SUT's rule set allows it. For example, if a firewall's rule set the DUT/SUT's rule set allows it. For example, if a firewall's rule
permits ftp connections, a circuit proxy will always forward traffic set permits ftp connections, a circuit proxy will always forward
on TCP port 20 (ftp-data) even if no control connection was first traffic on TCP port 20 (ftp-data) even if no control connection was
established on TCP port 21 (ftp-control). first established on TCP port 21 (ftp-control).
Unit of measurement: Unit of measurement:
not applicable not applicable
Issues: Issues:
application proxy application proxy
rule sets rule sets
See also: See also:
allowed traffic allowed traffic
application proxy application proxy
proxy proxy
rejected traffic rejected traffic
rule set rule set
3.6 Concurrent connections 3.6 Concurrent connections
Definition: Definition:
The aggregate number of simultaneous connections between hosts across The aggregate number of simultaneous connections between hosts
the DUT/SUT, or between hosts and the DUT/SUT. across the DUT/SUT, or between hosts and the DUT/SUT.
Discussion: Discussion:
The number of concurrent connections a firewall can support is just The number of concurrent connections a firewall can support is just
as important a metric for some users as maximum bit forwarding rate. as important a metric for some users as maximum bit forwarding
rate.
While "connection" describes only a state and not necessarily the While "connection" describes only a state and not necessarily the
transfer of data, concurrency assumes that all existing connections transfer of data, concurrency assumes that all existing connections
are in fact capable of transferring data. If a data cannot be sent are in fact capable of transferring data. If a data cannot be sent
over a connection, that connection should not be counted toward the over a connection, that connection should not be counted toward the
number of concurrent connections. number of concurrent connections.
Newman Page [6] Further, this definition assumes that the ability (or lack thereof)
Further, this definition assumes that the ability (or lack thereof) to transfer data on a given connection is solely the responsibility
to transfer data on a given connection is solely the responsibility of the DUT/SUT. For example, a TCP connection that a DUT/SUT has
of the DUT/SUT. For example, a TCP connection that a DUT/SUT has left left in a FIN_WAIT_2 state clearly should not be counted. But
in a FIN_WAIT_2 state clearly should not be counted. But another another connection that has temporarily stopped transferring data
connection that has temporarily stopped transferring data because because some external device has restricted the flow of data is not
some external device has restricted the flow of data is not necessarily defunct. The tester should take measures to isolate
necessarily defunct. The tester should take measures to isolate changes in connection state to those effected by the DUT/SUT.
changes in connection state to those effected by the DUT/SUT.
Unit of measurement: Unit of measurement:
Concurrent connections Concurrent connections
Maximum number of concurrent connections Maximum number of concurrent connections
Issues: Issues:
See also: See also:
connections connections
connection establishment time connection establishment time
connection overhead connection overhead
3.7 Connection 3.7 Connection
Definition: Definition:
A state in which two hosts, or a host and the DUT/SUT, agree to A state in which two hosts, or a host and the DUT/SUT, agree to
exchange data using a known protocol. exchange data using a known protocol.
Discussion: Discussion:
A connection is an abstraction describing an agreement between two A connection is an abstraction describing an agreement between two
nodes: One agrees to send data and the other agrees to receive it. nodes: One agrees to send data and the other agrees to receive it.
Connections might use TCP, but they don't have to. Other protocols Connections might use TCP, but they don't have to. Other protocols
such as ATM also might be used, either instead of or in addition to such as ATM also might be used, either instead of or in addition to
TCP connections. TCP connections.
What constitutes a connection depends on the application. For a What constitutes a connection depends on the application. For a
native ATM application, connections and virtual circuits may be native ATM application, connections and virtual circuits may be
synonymous. For TCP/IP applications on ATM networks (where multiple synonymous. For TCP/IP applications on ATM networks (where multiple
TCP connections may ride over a single ATM virtual circuit), the TCP connections may ride over a single ATM virtual circuit), the
number of TCP connections may be the most important consideration. number of TCP connections may be the most important consideration.
Additionally, in some cases firewalls may handle a mixture of native Additionally, in some cases firewalls may handle a mixture of
TCP and native ATM connections. In this situation, the wrappers native TCP and native ATM connections. In this situation, the
around user data will differ. The most meaningful metric describes wrappers around user data will differ. The most meaningful metric
what an end-user will see. describes what an end-user will see.
Data connections describe state, not data transfer. The existence of Data connections describe state, not data transfer. The existence
a connection does not imply that data travels on that connection at of a connection does not imply that data travels on that connection
any given time, although if data cannot be forwarded on a previously at any given time, although if data cannot be forwarded on a
established connection that connection should not be considered in previously established connection that connection should not be
any aggregrate connection count (see concurrent connections). considered in any aggregrate connection count (see concurrent
connections).
A firewall's architecture dictates where a connection terminates. In A firewall's architecture dictates where a connection terminates.
the case of application or circuit proxy firewalls, a connection In the case of application or circuit proxy firewalls, a connection
terminates at the DUT/SUT. But firewalls using packet filtering or terminates at the DUT/SUT. But firewalls using packet filtering or
stateful packet filtering designs act only as passthrough devices,
in that they reside between two connection endpoints. Regardless of
firewall architecture, the number of data connections is still
relevant, since all firewalls perform some form of connection
maintenance; at the very least, all check connection requests
against their rule sets.
Newman Page [7] Further, note that connection is not an atomic unit of measurement
stateful packet filtering designs act only as passthrough devices, in in that it does not describe the various steps involved in
that they reside between two connection endpoints. Regardless of connection setup, maintenance, and teardown. Testers may wish to
firewall architecture, the number of data connections is still take separate measurements of each of these components.
relevant, since all firewalls perform some form of connection
maintenance; at the very least, all check connection requests
against their rule sets.
Further, note that connection is not an atomic unit of measurement in When benchmarking firewall performance, it's important to identify
that it does not describe the various steps involved in connection the connection establishment and teardown procedures, as these must
setup, maintenance, and teardown. Testers may wish to take separate not be included when measuring steady-state forwarding rates.
measurements of each of these components. Further, forwarding rates must be measured only after any security
associations have been established.
When benchmarking firewall performance, it's important to identify Though it seems paradoxical, connectionless protocols such as UDP
the connection establishment and teardown procedures, as these must may also involve connections, at least for the purposes of firewall
NOT be included when measuring steady-state forwarding rates. performance measurement. For example, one host may send UDP packets
Further, forwarding rates must be measured only after any security to another across a firewall. If the destination host is listening
associations have been established. on the correct UDP port, it receives the UDP packets. For the
purposes of firewall performance measurement, this is considered a
connection.
Though it seems paradoxical, connectionless protocols such as UDP may Unit of measurement:
also involve connections, at least for the purposes of firewall concurrent connections
performance measurement. For example, one host may send UDP packets connection
to another across a firewall. If the destination host is listening on connection establishment time
the correct UDP port, it receives the UDP packets. For the purposes maximum number of concurrent connections
of firewall performance measurement, this is considered a connection. connection teardown time
Unit of measurement: Issues:
concurrent connections application proxy vs. stateful packet filtering
connection TCP/IP vs. ATM
connection establishment time
maximum number of concurrent connections
connection teardown time
Issues: connection-oriented vs. connectionless
application proxy vs. stateful packet filtering
TCP/IP vs. ATM
connection-oriented vs. connectionless
See also: See also:
data source data source
concurrent connections concurrent connections
connection establishment connection establishment
connection establishment time connection establishment time
connection teardown connection teardown
connection teardown time connection teardown time
3.8 Connection establishment 3.8 Connection establishment
Definition: Definition:
The data exchanged between hosts, or between a host and the DUT/SUT, The data exchanged between hosts, or between a host and the
to initiate a connection. DUT/SUT, to initiate a connection.
Discussion:
Connection-oriented protocols like TCP have a proscribed handshaking
procedure when launching a connection. When benchmarking firewall
Newman Page [8] Discussion:
performance, it is import to identify this handshaking procedure so Connection-oriented protocols like TCP have a proscribed
that it is not included in measurements of bit forwarding rate or handshaking procedure when launching a connection. When
UOTs per second. benchmarking firewall performance, it is import to identify this
handshaking procedure so that it is not included in measurements of
bit forwarding rate or UOTs per second.
Testers may also be interested in measurements of connection Testers may also be interested in measurements of connection
establishment time through or with a given DUT/SUT. establishment time through or with a given DUT/SUT.
Unit of measurement: Unit of measurement:
not applicable not applicable
See also: See also:
connection connection
connection establishement time connection establishement time
connection maintenance connection maintenance
connection teardown connection teardown
Issues: Issues:
not applicable not applicable
3.9 Connection establishment time 3.9 Connection establishment time
Definition: Definition:
The length of time needed for two hosts, or a host and the DUT/SUT, The length of time needed for two hosts, or a host and the DUT/SUT,
to agree to set up a connection using a known protocol. to agree to set up a connection using a known protocol.
Discussion: Discussion:
Each connection-oriented protocol has its own defined mechanisms for Each connection-oriented protocol has its own defined mechanisms
setting up a connection. For purposes of benchmarking firewall for setting up a connection. For purposes of benchmarking firewall
performance, this shall be the interval between receipt of the first performance, this shall be the interval between receipt of the
bit of the first octet of the packet carrying a connection first bit of the first octet of the packet carrying a connection
establishment request on a DUT/SUT interface until transmission of establishment request on a DUT/SUT interface until transmission of
the last bit of the last octet of the last packet of the connection the last bit of the last octet of the last packet of the connection
setup traffic headed in the opposite direction. setup traffic headed in the opposite direction.
This definition applies only to connection-oriented protocols such as This definition applies only to connection-oriented protocols such
TCP. For connectionless protocols such as UDP, the notion of as TCP. For connectionless protocols such as UDP, the notion of
connection establishment time is not meaningful. connection establishment time is not meaningful.
Unit of measurement: Unit of measurement:
Connection establishment time Connection establishment time
Issues: Issues:
See also: See also:
concurrent connections concurrent connections
connection connection
connection maintenance connection maintenance
3.10 Connection maintenance 3.10 Connection maintenance
Definition: Definition:
The data exchanged between hosts, or between a host and the DUT/SUT, The data exchanged between hosts, or between a host and the
to ensure a connection is kept alive. DUT/SUT, to ensure a connection is kept alive.
Newman Page [9] Discussion:
Discussion: Some implementations of TCP and other connection-oriented protocols
Some implementations of TCP and other connection-oriented protocols use "keep-alive" data to maintain a connection during periods where
use "keep-alive" data to maintain a connection during periods where no user data is exchanged.
no user data is exchanged.
When benchmarking firewall performance, it is useful to identfy When benchmarking firewall performance, it is useful to identfy
connection maintenance traffic as distinct from UOTs per second. connection maintenance traffic as distinct from UOTs per second.
Given that maintenance traffic may be characterized by short bursts Given that maintenance traffic may be characterized by short bursts
at periodical intervals, it may not be possible to describe a steady- at periodical intervals, it may not be possible to describe a
state forwarding rate for maintenance traffic. One possible approach steady-state forwarding rate for maintenance traffic. One possible
is to identify the quantity of maintenance traffic, in bytes or bits, approach is to identify the quantity of maintenance traffic, in
over a given interval, and divide through to derive a measurement of bytes or bits, over a given interval, and divide through to derive
maintenance traffic forwarding rate. a measurement of maintenance traffic forwarding rate.
Unit of measurement: Unit of measurement:
maintenance traffic maintenance traffic
forwarding rate forwarding rate
See also: See also:
connection connection
connection establishment time connection establishment time
connection teardown connection teardown
connection teardown time connection teardown time
Issues: Issues:
not applicable not applicable
3.11 Connection overhead 3.11 Connection overhead
Definition: Definition:
The degradation in bit forwarding rate, if any, observed as a result The degradation in bit forwarding rate, if any, observed as a
of the addition of one connection between two hosts through the result of the addition of one connection between two hosts through
DUT/SUT, or the addition of one connection from a host to the the DUT/SUT, or the addition of one connection from a host to the
DUT/SUT. DUT/SUT.
Discussion: Discussion:
The memory cost of connection establishment and maintenance is highly The memory cost of connection establishment and maintenance is
implementation-specific. This metric is intended to describe that highly implementation-specific. This metric is intended to describe
cost in a method visible outside the firewall. that cost in a method visible outside the firewall.
It may also be desirable to invert this metric to show the It may also be desirable to invert this metric to show the
performance improvement as a result of tearing down one connection. performance improvement as a result of tearing down one connection.
Unit of measurement: Unit of measurement:
bit forwarding rate bit forwarding rate
Issues: Issues:
3.12 Connection teardown 3.12 Connection teardown
Definition: Definition:
The data exchanged between hosts, or between a host and the DUT/SUT, The data exchanged between hosts, or between a host and the
to close a connection. DUT/SUT, to close a connection.
Newman Page [10] Discussion:
Discussion: Connection-oriented protocols like TCP follow a stated procedure
Connection-oriented protocols like TCP follow a stated procedure when when ending a connection. When benchmarking firewall performance,
ending a connection. When benchmarking firewall performance, it is it is important to identify the teardown procedure so that it is
important to identify the teardown procedure so that it is not not included in measurements of bit forwarding rate or UOTs per
included in measurements of bit forwarding rate or UOTs per second. second.
Testers may also be interested in measurements of connection teardown Testers may also be interested in measurements of connection
time through or with a given DUT/SUT. teardown time through or with a given DUT/SUT.
Unit of measurement: Unit of measurement:
not applicable not applicable
See also: See also:
connection teardown time connection teardown time
Issues: Issues:
not applicable not applicable
3.13 Connection teardown time 3.13 Connection teardown time
Definition: Definition:
The length of time needed for two hosts, or a host and the DUT/SUT, The length of time needed for two hosts, or a host and the DUT/SUT,
to agree to tear down a connection using a known protocol. to agree to tear down a connection using a known protocol.
Discussion: Discussion:
Each connection-oriented protocol has its own defined mechanisms for Each connection-oriented protocol has its own defined mechanisms
dropping a connection. For purposes of benchmarking firewall for dropping a connection. For purposes of benchmarking firewall
performance, this shall be the interval between receipt of the first performance, this shall be the interval between receipt of the
bit of the first octet of the packet carrying a connection teardown first bit of the first octet of the packet carrying a connection
request on a DUT/SUT interface until transmission of the last bit of teardown request on a DUT/SUT interface until transmission of the
the last octet of the last packet of the connection teardown traffic last bit of the last octet of the last packet of the connection
headed in the opposite direction. teardown traffic headed in the opposite direction.
This definition applies only to connection-oriented protocols such as This definition applies only to connection-oriented protocols such
TCP. For connectionless protocols such as UDP, the notion of as TCP. For connectionless protocols such as UDP, the notion of
connection teardown time is not meaningful. connection teardown time is not meaningful.
Unit of measurement: Unit of measurement:
Connection teardown time Connection teardown time
Issues: Issues:
See also: See also:
concurrent connections concurrent connections
connection connection
connection maintenance connection maintenance
3.14 Data source 3.14 Data source
Definition: Definition:
A host capable of generating traffic to the DUT/SUT. A host capable of generating traffic to the DUT/SUT.
Discussion:
Newman Page [11] Discussion:
One data source MAY emulate multiple users or hosts. In addition, one One data source may emulate multiple users or hosts. In addition,
data source MAY offer traffic to multiple network interfaces on the one data source may offer traffic to multiple network interfaces on
DUT/SUT. the DUT/SUT.
The term "data source" is deliberately independent of any number of The term "data source" is deliberately independent of any number of
users. It is useful to think of data sources simply as traffic users. It is useful to think of data sources simply as traffic
generators, without any correlation to any given number of users. generators, without any correlation to any given number of users.
Unit of measurement: Unit of measurement:
not applicable not applicable
Issues: Issues:
user user
See also: See also:
connection connection
user user
3.15 Demilitarized zone 3.15 Demilitarized zone
Definition: Definition:
A network segment or segments located between protected and A network segment or segments located between protected and
unprotected networks. unprotected networks.
Discussion: Discussion:
As an extra security measure, networks may be designed such that As an extra security measure, networks may be designed such that
protected and unprotected segments are never directly connected. protected and unprotected segments are never directly connected.
Instead, firewalls (and possibly public resources such as HTTP or FTP Instead, firewalls (and possibly public resources such as HTTP or
servers) reside on a so-called DMZ network. FTP servers) reside on a so-called DMZ network.
DMZ networks are sometimes called perimeter networks. DMZ networks are sometimes called perimeter networks.
Unit of measurement: Unit of measurement:
not applicable not applicable
Issues: Issues:
Homed Homed
See also: See also:
protected network protected network
unprotected network unprotected network
3.16 Firewall 3.16 Firewall
Definition: Definition:
A device or group of devices that enforces an access control policy A device or group of devices that enforces an access control policy
between networks. between networks.
Discussion:
While there are many different ways to accomplish it, all firewalls
do the same thing: control access between networks.
The most common configuration involves a firewall connecting two Discussion:
segments (one protected and one unprotected), but this is not the While there are many different ways to accomplish it, all firewalls
do the same thing: control access between networks.
Newman Page [12] The most common configuration involves a firewall connecting two
only possible configuration. Many firewalls support tri-homing, segments (one protected and one unprotected), but this is not the
allowing use of a DMZ network. It is possible for a firewall to only possible configuration. Many firewalls support tri-homing,
accommodate more than three interfaces, each attached to a different allowing use of a DMZ network. It is possible for a firewall to
network segment. accommodate more than three interfaces, each attached to a
different network segment.
The criteria by which access are controlled are not specified here. The criteria by which access are controlled are not specified here.
Typically this has been done using network- or transport-layer Typically this has been done using network- or transport-layer
criteria (such as IP subnet or TCP port number), but there is no criteria (such as IP subnet or TCP port number), but there is no
reason this must always be so. A growing number of firewalls are reason this must always be so. A growing number of firewalls are
controlling access at the application layer, using user controlling access at the application layer, using user
identification as the criterion. And firewalls for ATM networks may identification as the criterion. And firewalls for ATM networks may
control access based on data link-layer criteria. control access based on data link-layer criteria.
Unit of measurement: Unit of measurement:
not applicable not applicable
Issues: Issues:
See also: See also:
DMZ DMZ
tri-homed tri-homed
user user
3.17 Goodput 3.17 Goodput
Definition: Definition:
The number of bits per unit of time forwarded to the correct The number of bits per unit of time forwarded to the correct
destination interface of the DUT/SUT, minus any bits lost or destination interface of the DUT/SUT, minus any bits lost or
retransmitted. retransmitted.
Discussion: Discussion:
Firewalls are generally insensitive to packet loss in the network. As Firewalls are generally insensitive to packet loss in the network.
such, measurements of gross bit forwarding rates are not meaningful As such, measurements of gross bit forwarding rates are not
since (in the case of proxy-based and stateful packet filtering meaningful since (in the case of proxy-based and stateful packet
firewalls) a receiving endpoint directly attached to a DUT/SUT would filtering firewalls) a receiving endpoint directly attached to a
not receive any data dropped by the DUT/SUT. DUT/SUT would not receive any data dropped by the DUT/SUT.
The type of traffic lost or retransmitted is protocol-dependent. TCP The type of traffic lost or retransmitted is protocol-dependent.
and ATM, for example, request different types of retransmissions. TCP and ATM, for example, request different types of
Testers must observe retransmitted data for the protocol in use, and retransmissions. Testers must observe retransmitted data for the
subtract this quantity from measurements of gross bit forwarding protocol in use, and subtract this quantity from measurements of
rate. gross bit forwarding rate.
Unit of measurement: Unit of measurement:
bits per second bits per second
Issues: Issues:
allowed vs. rejected traffic allowed vs. rejected traffic
See also: See also:
allowed traffic allowed traffic
bit forwarding rate bit forwarding rate
rejected traffic rejected traffic
Newman Page [13]
3.18 Homed 3.18 Homed
Definition: Definition:
The number of logical interfaces a DUT/SUT contains. The number of logical interfaces a DUT/SUT contains.
Discussion:
Firewalls typically contain at least two logical interfaces. In Discussion:
network topologies where a DMZ is used, the firewall usually contains Firewalls typically contain at least two logical interfaces. In
at least three interfaces and is said to be tri-homed. Additional network topologies where a DMZ is used, the firewall usually
interfaces would make a firewall quad-homed, quint-homed, and so on. contains at least three interfaces and is said to be tri-homed.
Additional interfaces would make a firewall quad-homed, quint-
homed, and so on.
It is theoretically possible for a firewall to contain one physical It is theoretically possible for a firewall to contain one physical
interface and multiple logical interfaces. This configuration is interface and multiple logical interfaces. This configuration is
discouraged for testing purposes because of the difficulty in discouraged for testing purposes because of the difficulty in
verifying that no leakage occurs between protected and unprotected verifying that no leakage occurs between protected and unprotected
segments. segments.
Unit of measurement: Unit of measurement:
not applicable not applicable
Issues: Issues:
See also: See also:
tri-homed tri-homed
3.19 Illegal traffic 3.19 Illegal traffic
Definition: Definition:
Packets specified for rejection in the rule set of the DUT/SUT. Packets specified for rejection in the rule set of the DUT/SUT.
Discussion: Discussion:
A buggy or misconfigured firewall might forward packets even though A buggy or misconfigured firewall might forward packets even though
its rule set specifies that these packets be dropped. Illegal traffic its rule set specifies that these packets be dropped. Illegal
differs from rejected traffic in that it describes all traffic traffic differs from rejected traffic in that it describes all
specified for rejection by the rule set, while rejected traffic traffic specified for rejection by the rule set, while rejected
specifies only those packets actually dropped by the DUT/SUT. traffic specifies only those packets actually dropped by the
DUT/SUT.
Unit of measurement: Unit of measurement:
not applicable not applicable
Issues: Issues:
See also: See also:
accepted traffic accepted traffic
policy policy
rejected traffic rejected traffic
rule set rule set
3.20 Logging 3.20 Logging
Definition: Definition:
The recording of user requests made to the firewall. The recording of user requests made to the firewall.
Newman Page [14] Discussion:
Discussion: Firewalls typically log all requests they handle, both allowed and
Firewalls typically log all requests they handle, both allowed and rejected. For many firewall designs, logging requires a significant
rejected. For many firewall designs, logging requires a significant amount of processing overhead, especially when complex rule sets
amount of processing overhead, especially when complex rule sets are are in use.
in use.
The type and amount of data logged varies by implementation. Testers The type and amount of data logged varies by implementation.
may find it desirable to log equivalent data when comparing different Testers may find it desirable to log equivalent data when comparing
DUT/SUTs. different DUT/SUTs.
Some systems allow logging to take place on systems other than the Some systems allow logging to take place on systems other than the
DUT/SUT. DUT/SUT.
Unit of measurement: Unit of measurement:
not applicable not applicable
Issues: Issues:
rule sets rule sets
See also: See also:
allowed traffic allowed traffic
connection connection
rejected traffic rejected traffic
3.21 Network address translation 3.21 Network address translation
Definition: Definition:
A method of mapping one or more private, reserved IP addresses to one A method of mapping one or more private, reserved IP addresses to
or more public IP addresses. one or more public IP addresses.
Discussion: Discussion:
In the interest of conserving the IPv4 address space, RFC 1918 In the interest of conserving the IPv4 address space, RFC 1918
proposed the use of certain private (reserved) blocks of IP proposed the use of certain private (reserved) blocks of IP
addresses. Connections to public networks are made by use of a device addresses. Connections to public networks are made by use of a
that translates one or more RFC 1918 addresses to one or more public device that translates one or more RFC 1918 addresses to one or
addresses--a network address translator (NAT). more public addresses--a network address translator (NAT).
The use of private addressing also introduces a security benefit in The use of private addressing also introduces a security benefit in
that RFC 1918 addresses are not visible to hosts on the public that RFC 1918 addresses are not visible to hosts on the public
Internet. Internet.
Some NAT implementations are computationally intensive, and may Some NAT implementations are computationally intensive, and may
affect bit forwarding rate. affect bit forwarding rate.
Unit of measurement: Unit of measurement:
not applicable not applicable
Issues: Issues:
See also: See also:
3.22 Packet filtering 3.22 Packet filtering
Definition: Definition:
The process of controlling access by examining packets based on the
Newman Page [15] content of packet headers.
The process of controlling access by examining packets based on the
content of packet headers.
Discussion: Discussion:
Packet-filtering devices forward or deny packets based on information Packet-filtering devices forward or deny packets based on
in each packet's header, such as IP address or TCP port number. A information in each packet's header, such as IP address or TCP port
packet-filtering firewall uses a rule set to determine which traffic number. A packet-filtering firewall uses a rule set to determine
should be forwarded and which should be blocked. which traffic should be forwarded and which should be blocked.
Unit of measurement: Unit of measurement:
not applicable not applicable
Issues: Issues:
static vs. stateful packet filtering static vs. stateful packet filtering
See also: See also:
application proxy application proxy
circuit proxy circuit proxy
proxy proxy
rule set rule set
stateful packet filtering stateful packet filtering
3.23 Policy 3.23 Policy
Definition: Definition:
A document defining acceptable access to protected, DMZ, and A document defining acceptable access to protected, DMZ, and
unprotected networks. unprotected networks.
Discussion: Discussion:
Security policies generally do not spell out specific configurations Security policies generally do not spell out specific
for firewalls; rather, they set general guidelines for what is and is configurations for firewalls; rather, they set general guidelines
not acceptable network access. for what is and is not acceptable network access.
The actual mechanism for controlling access is usually the rule set The actual mechanism for controlling access is usually the rule set
implemented in the DUT/SUT. implemented in the DUT/SUT.
Unit of measurement: Unit of measurement:
not applicable not applicable
Issues: Issues:
See also: See also:
rule set rule set
3.24 Protected network 3.24 Protected network
Definition: Definition:
A network segment or segments to which access is controlled by the A network segment or segments to which access is controlled by the
DUT/SUT. DUT/SUT.
Discussion:
Firewalls are intended to prevent unauthorized access either to or
from the protected network. Depending on the configuration specified
by the policy and rule set, the DUT/SUT may allow hosts on the
Newman Page [16] Discussion:
protected segment to act as clients for servers on either the DMZ or Firewalls are intended to prevent unauthorized access either to or
the unprotected network, or both. from the protected network. Depending on the configuration
specified by the policy and rule set, the DUT/SUT may allow hosts
on the protected segment to act as clients for servers on either
the DMZ or the unprotected network, or both.
Protected networks are often called "internal networks." That term is Protected networks are often called "internal networks." That term
not used here because firewalls increasingly are deployed within an is not used here because firewalls increasingly are deployed within
organization, where all segments are by definition internal. an organization, where all segments are by definition internal.
Unit of measurement: Unit of measurement:
not applicable not applicable
Issues: Issues:
See also: See also:
demilitarized zone (DMZ) demilitarized zone (DMZ)
unprotected network unprotected network
policy policy
rule set rule set
unprotected network unprotected network
3.25 Proxy 3.25 Proxy
Definition: Definition:
A request for a connection made on behalf of a host. A request for a connection made on behalf of a host.
Discussion: Discussion:
Proxy-based firewalls do not allow direct connections between hosts. Proxy-based firewalls do not allow direct connections between
Instead, two connections are established: one between the client host hosts. Instead, two connections are established: one between the
and the DUT/SUT, and another between the DUT/SUT and server host. client host and the DUT/SUT, and another between the DUT/SUT and
server host.
As with packet-filtering firewalls, proxy-based devices use a rule As with packet-filtering firewalls, proxy-based devices use a rule
set to determine which traffic should be forwarded and which should set to determine which traffic should be forwarded and which should
be rejected. be rejected.
There are two types of proxies: application proxies and circuit There are two types of proxies: application proxies and circuit
proxies. proxies.
Unit of measurement: Unit of measurement:
not applicable not applicable
Issues: Issues:
application application
See also: See also:
application proxy application proxy
circuit proxy circuit proxy
packet filtering packet filtering
stateful packet filtering stateful packet filtering
3.26 Rejected traffic 3.26 Rejected traffic
Definition: Definition:
Packets dropped as a result of the rule set of the DUT/SUT. Packets dropped as a result of the rule set of the DUT/SUT.
Newman Page [17] Discussion:
Discussion: For purposes of benchmarking firewall performance, it is expected
For purposes of benchmarking firewall performance, it is expected that firewalls will reject all traffic not explicitly permitted in
that firewalls will reject all traffic not explicitly permitted in the rule set. Dropped packets must not be included in calculating
the rule set. Dropped packets must not be included in calculating the the bit forwarding rate or maximum bit forwarding rate of the
bit forwarding rate or maximum bit forwarding rate of the DUT/SUT. DUT/SUT.
Unit of measurement: Unit of measurement:
not applicable not applicable
Issues: Issues:
See also: See also:
allowed traffic allowed traffic
illegal traffic illegal traffic
policy policy
rule set rule set
3.27 Rule set 3.27 Rule set
Definition: Definition:
The collection of access control rules that determines which packets The collection of access control rules that determines which
the DUT/SUT will forward and which it will reject. packets the DUT/SUT will forward and which it will reject.
Discussion: Discussion:
Rule sets control access to and from the network interfaces of the Rule sets control access to and from the network interfaces of the
DUT/SUT. By definition, rule sets do not apply equally to all network
interfaces; otherwise there would be no need for the firewall. For
benchmarking purposes, a specific rule set is typically applied to
each network interface in the DUT/SUT.
The tester must describe the complete contents of the rule set of DUT/SUT. By definition, rule sets do not apply equally to all
each DUT/SUT. network interfaces; otherwise there would be no need for the
firewall. For benchmarking purposes, a specific rule set is
typically applied to each network interface in the DUT/SUT.
To ensure measurements reflect only traffic forwarded by the DUT/SUT, The tester must describe the complete contents of the rule set of
testers are encouraged to include a rule denying all access except each DUT/SUT.
for those packets allowed by the rule set.
Unit of measurement: To ensure measurements reflect only traffic forwarded by the
not applicable DUT/SUT, testers are encouraged to include a rule denying all
access except for those packets allowed by the rule set.
Issues: Unit of measurement:
not applicable
See also: Issues:
allowed traffic
demilitarized zone (DMZ)
illegal traffic
policy
protected network
rejected traffic
unprotected network
3.28 Security association See also:
allowed traffic
demilitarized zone (DMZ)
illegal traffic
policy
protected network
rejected traffic
unprotected network
Definition: 3.28 Security association
Newman Page [18] Definition:
The set of security information relating to a given network The set of security information relating to a given network
connection or set of connections. connection or set of connections.
Discussion: Discussion:
This definition covers the relationship between policy and This definition covers the relationship between policy and
connections. Security associations (SAs) are typically set up during connections. Security associations (SAs) are typically set up
connection establishment, and they may be reiterated or revoked during connection establishment, and they may be reiterated or
during a connection. revoked during a connection.
For purposes of benchmarking firewall performance, measurements of For purposes of benchmarking firewall performance, measurements of
bit forwarding rate or UOTs per second must be taken after all bit forwarding rate or UOTs per second must be taken after all
security associations have been established. security associations have been established.
Unit of measurement: Unit of measurement:
not applicable not applicable
See also: See also:
connection connection
connection establishment connection establishment
policy policy
rule set rule set
3.29 Stateful packet filtering 3.29 Stateful packet filtering
Definition: Definition:
The process of forwarding or rejecting traffic based on the contents The process of forwarding or rejecting traffic based on the
of a state table maintained by a firewall. contents of a state table maintained by a firewall.
Discussion: Discussion:
Packet filtering and proxy firewalls are essentially static, in that Packet filtering and proxy firewalls are essentially static, in
they always forward or reject packets based on the contents of the that they always forward or reject packets based on the contents of
rule set. the rule set.
In contrast, devices using stateful packet filtering will only In contrast, devices using stateful packet filtering will only
forward packets if they correspond with state information maintained forward packets if they correspond with state information
by the device about each connection. For example, a stateful packet maintained by the device about each connection. For example, a
filtering device will reject a packet on port 20 (ftp-data) if no stateful packet filtering device will reject a packet on port 20
connection has been established over the ftp control port (usually (ftp-data) if no connection has been established over the ftp
port 21). control port (usually port 21).
Unit of measurement: Unit of measurement:
not applicable not applicable
Issues: Issues:
See also: See also:
applicaton proxy applicaton proxy
packet filtering packet filtering
proxy proxy
3.30 Tri-homed 3.30 Tri-homed
Definition: Definition:
A firewall with three network interfaces. A firewall with three network interfaces.
Newman Page [19] Discussion:
Discussion: Tri-homed firewalls connect three network segments with different
Tri-homed firewalls connect three network segments with different network addresses. Typically, these would be protected, DMZ, and
network addresses. Typically, these would be protected, DMZ, and unprotected segments.
unprotected segments.
A tri-homed firewall may offer some security advantages over A tri-homed firewall may offer some security advantages over
firewalls with two interfaces. An attacker on an unprotected network firewalls with two interfaces. An attacker on an unprotected
may compromise hosts on the DMZ but still not reach any hosts on the network may compromise hosts on the DMZ but still not reach any
protected network. hosts on the protected network.
Unit of measurement: Unit of measurement:
not applicable not applicable
Issues: Issues:
Usually the differentiator between one segment and another is its IP Usually the differentiator between one segment and another is its
address. However, firewalls may connect different networks of other IP address. However, firewalls may connect different networks of
types, such as ATM or Netware segments. other types, such as ATM or Netware segments.
See also: See also:
homed homed
3.31 Unit of transfer 3.31 Unit of transfer
Definition: Definition:
A discrete collection of bytes comprising at least one header and A discrete collection of bytes comprising at least one header and
optional user data. optional user data.
Discussion: Discussion:
This metric is intended for use in describing steady-state forwarding This metric is intended for use in describing steady-state
rate of the DUT/SUT. forwarding rate of the DUT/SUT.
The unit of transfer (UOT) definition is deliberately left open to The unit of transfer (UOT) definition is deliberately left open to
interpretation, allowing the broadest possible application. Examples interpretation, allowing the broadest possible application.
of UOTs include TCP segments, IP packets, Ethernet frames, and ATM Examples of UOTs include TCP segments, IP packets, Ethernet frames,
cells. and ATM cells.
While the definition is deliberately broad, its interpretation must While the definition is deliberately broad, its interpretation must
not be. The tester must describe what type of UOT will be offered to not be. The tester must describe what type of UOT will be offered
the DUT/SUT, and must offer these UOTs at a consistent rate. Traffic to the DUT/SUT, and must offer these UOTs at a consistent rate.
measurement must begin after all connection establishment routines Traffic measurement must begin after all connection establishment
complete and before any connection completion routine begins. routines complete and before any connection completion routine
Further, measurements must begin after any security associations begins. Further, measurements must begin after any security
(SAs) are established and before any SA is revoked. associations (SAs) are established and before any SA is revoked.
Testers also must compare only like UOTs. It is not appropriate, for Testers also must compare only like UOTs. It is not appropriate,
example, to compare forwarding rates by offering 1,500-byte Ethernet for example, to compare forwarding rates by offering 1,500-byte
UOTs to one DUT/SUT and 53-byte ATM cells to another. Ethernet UOTs to one DUT/SUT and 53-byte ATM cells to another.
Unit of measurement: Unit of measurement:
Units of transfer Units of transfer
Units of transfer per second Units of transfer per second
Issues: Issues:
Newman Page [20] See also:
See also: bit forwarding rate
bit forwarding rate connection
connection
3.32 Unprotected network 3.32 Unprotected network
Definition: Definition:
A network segment or segments to which access is not controlled by A network segment or segments to which access is not controlled by
the DUT/SUT. the DUT/SUT.
Discussion: Discussion:
Firewalls are deployed between protected and unprotected segments. Firewalls are deployed between protected and unprotected segments.
The unprotected network is not protected by the DUT/SUT. The unprotected network is not protected by the DUT/SUT.
Note that a DUT/SUT's policy MAY specify hosts on an unprotected Note that a DUT/SUT's policy may specify hosts on an unprotected
network. For example, a user on a protected network may be permitted network. For example, a user on a protected network may be
to access an FTP server on an unprotected network. But the DUT/SUT permitted to access an FTP server on an unprotected network. But
cannot control access between hosts on the unprotected network. the DUT/SUT cannot control access between hosts on the unprotected
network.
Unit of measurement: Unit of measurement:
not applicable not applicable
Issues: Issues:
See also: See also:
demilitarized zone (DMZ) demilitarized zone (DMZ)
policy policy
protected network protected network
rule set rule set
3.33 User 3.33 User
Definition: Definition:
A person or process requesting access to resources protected by the A person or process requesting access to resources protected by the
DUT/SUT. DUT/SUT.
Discussion:
"User" is a problematic term in the context of firewall performance
testing, for several reasons. First, a user may in fact be a process
or processes requesting services through the DUT/SUT. Second,
different "user" requests may require radically different amounts of
DUT/SUT resources. Third, traffic profiles vary widely from one
organization to another, making it difficult to characterize the load
offered by a typical user.
For these reasons, testers should not attempt to measure DUT/SUT Discussion:
performance in terms of users supported. Instead, testers should "User" is a problematic term in the context of firewall performance
describe performance in terms of maximum bit forwarding rate and testing, for several reasons. First, a user may in fact be a
maximum number of connections sustained. Further, testers should use process or processes requesting services through the DUT/SUT.
the term "data source" rather than user to describe traffic Second, different "user" requests may require radically different
generator(s). amounts of DUT/SUT resources. Third, traffic profiles vary widely
from one organization to another, making it difficult to
characterize the load offered by a typical user.
Unit of measurement: For these reasons, testers should not attempt to measure DUT/SUT
performance in terms of users supported. Instead, testers should
describe performance in terms of maximum bit forwarding rate and
maximum number of connections sustained. Further, testers should
use the term "data source" rather than user to describe traffic
generator(s).
Newman Page [21] Unit of measurement:
not applicable not applicable
Issues: Issues:
See also: See also:
data source data source
4. Security considerations 4. Security Considerations
The primary goal of this memo is to describe terms used in The primary goal of this memo is to describe terms used in
benchmarking firewall performance. However, readers should be aware benchmarking firewall performance. However, readers should be aware
that there is some overlap between performance and security issues. that there is some overlap between performance and security issues.
Specifically, the optimal configuration for firewall performance may Specifically, the optimal configuration for firewall performance may
not be the most secure, and vice-versa. not be the most secure, and vice-versa.
Further, certain forms of attack may degrade performance. One common Further, certain forms of attack may degrade performance. One common
form of denial-of-service (DoS) attack bombards a firewall with so form of denial-of-service (DoS) attack bombards a firewall with so
much rejected traffic that it cannot forward allowed traffic. DoS much rejected traffic that it cannot forward allowed traffic. DoS
attacks do not always involve heavy loads; by definition, DoS attacks do not always involve heavy loads; by definition, DoS
describes any state in which a firewall is offered rejected traffic describes any state in which a firewall is offered rejected traffic
that prohibits it from forwarding some or all allowed traffic. Even a that prohibits it from forwarding some or all allowed traffic. Even a
small amount of traffic may significantly degrade firewall small amount of traffic may significantly degrade firewall
performance, performance, or stop the firewall altogether. Further, the safeguards
or stop the firewall altogether. Further, the safeguards in firewalls in firewalls to guard against such attacks may have a significant
to guard against such attacks may have a significant negative impact negative impact on performance.
on performance.
Since the library of attacks is constantly expanding, no attempt is Since the library of attacks is constantly expanding, no attempt is
made here to define specific attacks that may affect performance. made here to define specific attacks that may affect performance.
Nonetheless, any reasonable performance benchmark should take into Nonetheless, any reasonable performance benchmark should take into
consideration safeguards against such attacks. Specifically, the same consideration safeguards against such attacks. Specifically, the same
safeguards should be in place when comparing performance of different safeguards should be in place when comparing performance of different
firewall implementations. firewall implementations.
5. References 5. References
Bradner, S., editor. "Benchmarking Terminology for Network Bradner, S., Ed., "Benchmarking Terminology for Network
Interconnection Devices." RFC 1242. Interconnection Devices", RFC 1242, July 1991.
Bradner, S., and McQuaid, J. "Benchmarking Methodology for Network Bradner, S. and J. McQuaid, "Benchmarking Methodology for Network
Interconnect Devices." RFC 2544. Interconnect Devices", RFC 2544, March 1999.
Mandeville, R. "Benchmarking Terminology for LAN Switching Devices." Mandeville, R., "Benchmarking Terminology for LAN Switching Devices",
RFC 2285. RFC 2285, February 1998.
Rekhter, Y., et al. "Address Allocation for Private Internets." RFC Rekhter, Y., Moskowitz, B., Karrenberg, D., de Groot, G. and E. Lear,
1918. "Address Allocation for Private Internets", BCP 5, RFC 1918,
February 1996.
Newman Page [22]
6. Acknowledgments 6. Acknowledgments
The author wishes to thank the IETF Benchmarking Working Group for The author wishes to thank the IETF Benchmarking Working Group for
agreeing to review this document. Several other persons offered agreeing to review this document. Several other persons offered
valuable contributions and critiques during this project: Ted Doty valuable contributions and critiques during this project: Ted Doty
(Internet Security Systems), Kevin Dubray (Ironbridge Networks), (Internet Security Systems), Kevin Dubray (Ironbridge Networks),
Helen Holzbaur (NSTL), Dale Lancaster (Axent Technologies), Robert Helen Holzbaur, Dale Lancaster, Robert Mandeville, Brent Melson
Mandeville (European Network Laboratories), Brent Melson (NSTL), (NSTL), Steve Platt (NSTL), Marcus Ranum (Network Flight Recorder),
Steve Platt (NSTL), Marcus Ranum (Network Flight Recorder), Greg Greg Shannon, Christoph Schuba (Sun Microsystems), Rick Siebenaler,
Shannon (Ascend Communications), Christoph Schuba (Sun Microsystems), and Greg Smith (Check Point Software Technologies).
Rick Siebenaler (Cyberguard), and Greg Smith (Check Point Software
Technologies).
7. Contact information 7. Contact Information
David Newman David Newman
Data Communications magazine Data Communications magazine
3 Park Ave. 3 Park Ave.
31st Floor 31st Floor
New York, NY 10016 New York, NY 10016
USA USA
212-592-8256 voice
212-592-8265 fax
dnewman@data.com
Newman Page [23] Phone: 212-592-8256
Fax: 212-592-8265
EMail: dnewman@data.com
8. Full Copyright Statement
Copyright (C) The Internet Society (1999). All Rights Reserved.
This document and translations of it may be copied and furnished to
others, and derivative works that comment on or otherwise explain it
or assist in its implementation may be prepared, copied, published
and distributed, in whole or in part, without restriction of any
kind, provided that the above copyright notice and this paragraph are
included on all such copies and derivative works. However, this
document itself may not be modified in any way, such as by removing
the copyright notice or references to the Internet Society or other
Internet organizations, except as needed for the purpose of
developing Internet standards in which case the procedures for
copyrights defined in the Internet Standards process must be
followed, or as required to translate it into languages other than
English.
The limited permissions granted above are perpetual and will not be
revoked by the Internet Society or its successors or assigns.
This document and the information contained herein is provided on an
"AS IS" basis and THE INTERNET SOCIETY AND THE INTERNET ENGINEERING
TASK FORCE DISCLAIMS ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING
BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION
HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF
MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE.
Acknowledgement
Funding for the RFC Editor function is currently provided by the
Internet Society.
 End of changes. 242 change blocks. 
921 lines changed or deleted 852 lines changed or added

This html diff was produced by rfcdiff 1.41. The latest version is available from http://tools.ietf.org/tools/rfcdiff/