draft-ietf-bmwg-secperf-04.txt   draft-ietf-bmwg-secperf-05.txt 
Network Working Group D. Newman, editor Network Working Group D. Newman
INTERNET-DRAFT Data Communications INTERNET-DRAFT Data Communications
Expires in January 1999 July 1998 Expires in July 1999 January 1999
Benchmarking Terminology for Firewall Performance Benchmarking Terminology for Firewall Performance
<draft-ietf-bmwg-secperf-04.txt> <draft-ietf-bmwg-secperf-05.txt>
Status of This Memo Status of This Memo
This document is an Internet-Draft. Internet-Drafts are working This document is an Internet-Draft. Internet-Drafts are working
documents of the Internet Engineering Task Force (IETF), its areas, documents of the Internet Engineering Task Force (IETF), its areas,
and its working groups. Note that other groups may also distribute and its working groups. Note that other groups may also distribute
working documents as Internet-Drafts. working documents as Internet-Drafts.
Internet-Drafts are draft documents valid for a maximum of six months Internet-Drafts are draft documents valid for a maximum of six months
and may be updated, replaced, or obsoleted by other documents at any and may be updated, replaced, or obsoleted by other documents at any
time. It is inappropriate to use Internet- Drafts as reference time. It is inappropriate to use Internet- Drafts as reference
material or to cite them other than as "work in progress." material or to cite them other than as "work in progress."
To view the entire list of current Internet-Drafts, please check the To view the entire list of current Internet-Drafts, please check the
"1id-abstracts.txt" listing contained in the Internet-Drafts Shadow "1id-abstracts.txt" listing contained in the Internet-Drafts Shadow
Directories on ftp.is.co.za (Africa), ftp.nordu.net (Europe), Directories on ftp.is.co.za (Africa), ftp.nordu.net (Europe),
munnari.oz.au (Pacific Rim), ftp.ietf.org (US East Coast), or munnari.oz.au (Pacific Rim), ds.internic.net (US East Coast), or
ftp.isi.edu (US West Coast). ftp.isi.edu (US West Coast).
Table of Contents 1. Introduction ....................................................2
Introduction .......................................................2
2. Existing definitions ............................................3 2. Existing definitions ............................................3
3. Term definitions ................................................3 3. Term definitions ................................................3
3.1 Allowed traffic .............................................3 3.1 Allowed traffic ..............................................3
3.2 Application proxy ...........................................3 3.2 Application proxy.............................................4
3.3 Authentication ..............................................4 3.3 Authentication ...............................................4
3.4 Bit forwarding rate .........................................5 3.4 Bit forwarding rate ..........................................5
3.5 Circuit proxy ...............................................6 3.5 Circuit proxy ................................................5
3.6 Concurrent connections ......................................6 3.6 Concurrent connections .......................................6
3.7 Connection ..................................................7 3.7 Connection ...................................................7
3.8 Connection establishment rate ...............................8 3.8 Connection establishment .....................................8
3.9 Connection overhead .........................................9 3.9 Connection establishment time ................................9
3.10 Data source ................................................9 3.10 Connection maintenance ......................................9
3.11 Demilitarized zone (DMZ) ..................................10 3.11 Conection overhead .........................................10
3.12 Firewall ..................................................10 3.12 Connection teardown ........................................10
3.13 Goodput ...................................................11
3.14 Homed .....................................................12 3.13 Connection teardown time ...................................11
3.14 Data source ................................................11
3.15 Illegal traffic ...........................................12 3.15 Demilitarized zone .........................................12
3.16 Logging ...................................................13 3.16 Firewall ...................................................12
3.17 Network address translation (NAT) .........................14 3.17 Goodput ....................................................13
3.18 Packet filtering ..........................................14 3.18 Homed ......................................................13
3.19 Perimeter network .........................................15 3.19 Illegal traffic.............................................14
3.20 Policy ....................................................15 3.20 Logging ....................................................14
3.21 Protected network .........................................16 3.21 Network address translation ................................15
3.22 Proxy .....................................................17 3.22 Packet filtering ...........................................15
3.23 Rejected traffic ..........................................17 3.23 Policy .....................................................16
3.24 Rule set ..................................................18 3.24 Protected network ..........................................16
3.25 Stateful packet filtering .................................18 3.25 Proxy ......................................................17
3.26 Tri-homed .................................................19 3.26 Rejected traffic ...........................................17
3.27 Unprotected network .......................................19 3.27 Rule set ...................................................18
3.28 User ......................................................20 3.28 Security association .......................................18
4. Security considerations ........................................21 3.29 Stateful packet filtering ..................................19
5. References .....................................................21 3.30 Tri-homed ..................................................19
6. Acknowledgments ................................................22 3.31 Unit of transfer ...........................................20
7. Contact information ............................................22 3.32 Unprotected network ........................................20
1. Introduction 3.33 User .......................................................21
4. Security considerations ...........................................21
5. References ........................................................22
6. Acknowledgments ...................................................22
7. Contact information ...............................................23
1. Introduction
This document defines terms used in measuring the performance of This document defines terms used in measuring the performance of
firewalls. It extends the terminology already used for benchmarking firewalls. It extends the terminology already used for benchmarking
routers and switches and adds terminology specific to firewalls. The routers and switches with definitions specific to firewalls.
primary metrics used in this document are bit forwarding rate and Forwarding rate and connection-oriented measurements are the primary
connections.
There are several reasons why firewall performance measurements are
needed. First, despite the rapid rise in firewall deployment, there
is no standard means of performance measurement. Second,
implementations vary widely, making it difficult to do direct
performance comparisons. Finally, more and more organizations are
Newman Page [2] Newman Page [2]
deploying firewalls on internal networks operating at relatively high metrics used in this document.
speeds, while most firewall implementations remain optimized for use
over low-speed wide-area connections. As a result, users are often Why do we need firewall performance measurements? First, despite the
unsure whether the products they buy will stand up to relatively rapid rise in firewall deployment, there is no standard method of
heavy loads. performance measurement. Second, implementations vary widely, making
it difficult to do direct performance comparisons. Finally, more and
more organizations are deploying firewalls on internal networks
operating at relatively high speeds, while most firewall
implementations remain optimized for use over relatively low-speed
wide-area connections. As a result, users are often unsure whether
the products they buy will stand up to relatively heavy loads.
2. Existing definitions 2. Existing definitions
This document uses the conceptual framework established in RFCs 1242 This document uses the conceptual framework established in RFCs 1242
and 1944 (for routers) and RFC 2285 (for switches). The router and and 1944 (for routers) and RFC 2285 (for switches). The router and
switch documents contain discussions of several terms relevant to switch documents contain discussions of several terms relevant to
benchmarking the performance of firewalls. Readers should consult the benchmarking the performance of firewalls. Readers should consult the
router and switch documents before making use of this document. router and switch documents before making use of this document.
This document uses the definition format described in RFC 1242, This document uses the definition format described in RFC 1242,
skipping to change at line 137 skipping to change at line 143
This document uses the definition format described in RFC 1242, This document uses the definition format described in RFC 1242,
Section 2. The sections in each definition are: definition, Section 2. The sections in each definition are: definition,
discussion, measurement units (optional), issues (optional), and discussion, measurement units (optional), issues (optional), and
cross-references. cross-references.
3. Term definitions 3. Term definitions
3.1 Allowed traffic 3.1 Allowed traffic
Definition: Definition:
Packets forwarded as a result of the rule set of the device under Packets forwarded as a result of the rule set of the device under
test/system under test (DUT/SUT). test/system under test (DUT/SUT).
Discussion: Discussion:
Firewalls typically are configured to forward only those packets Firewalls typically are configured to forward only those packets
explicitly permitted in the rule set. Forwarded packets MUST be explicitly permitted in the rule set. Forwarded packets MUST be
included in calculating the bit forwarding rate or maximum bit included in calculating the bit forwarding rate or maximum bit
forwarding rate of the DUT/SUT. All other packets MUST NOT be forwarding rate of the DUT/SUT. All other packets MUST NOT be
included in bit forwarding rate calculations. included in bit forwarding rate calculations.
Measurement units: This document assumes 1:1 correspondence of allowed traffic offered
to the DUT/SUT and forwarded by the DUT/SUT. There are cases where
the DUT/SUT may forward more traffic than it is offered; for example,
the DUT/SUT may act as a mail exploder or a multicast server. Any
attempt to benchmark forwarding rates of such traffic must include a
description of how much traffic the tester expects to be forwarded.
Measurement units:
not applicable not applicable
Issues: Issues:
See also: See also:
policy
policy rule set Newman Page [3]
rule set
3.2 Application proxy 3.2 Application proxy
Definition: Definition:
A proxy service that is set up and torn down in response to a client
A type of proxy service that is application aware. As such these request, rather than existing on a static basis.
proxies can ensure that only specific types of application
commands and data pass through; authenticate the user creating the
connection; dynamically open and close auxiliary ports sometimes
Newman Page [3]
required by applications.
Discussion: Discussion:
Circuit proxies always forward packets containing a given port number
Application proxies are aware of the type of data and application if that port number is permitted by the rule set. Application
commands that are expected to be associated with a given TCP or proxies, in contrast, forward packets only once a connection has been
UDP port and ensures that only such traffic is passed through established using some known protocol. When the connection closes, a
those ports. For example, TCP port 21 should only allow FTP firewall using applicaton proxies rejects individual packets, even if
commands and responses through and not allow a non-FTP protocol to they contain port numbers allowed by a rule set.
be used for potentially malicious means. An application proxy also
can determine which dynamically allocated ports are required to
allow the service to work properly. In the case of FTP, it
requires that port 20 (ftp-data) be open for a period of time and
then closed after the transfer is complete.
Measurement units: Measurement units:
not applicable not applicable
Issues: Issues:
circuit proxy circuit proxy
rule sets rule sets
See also: See also:
allowed traffic allowed traffic
circuit proxy circuit proxy
proxy rejected proxy
traffic rule set rejected traffic
rule set
3.3 Authentication 3.3 Authentication
Definition: Definition:
The process of verifying that a user requesting a network resource is
who he, she, or it claims to be, and vice versa.
The process of verifying that a user requesting a network resource Discussion:
is who he, she, or it claims to be, and vice versa. Trust is a critical concept in network security. Any network resource
(such as a file server or printer) with restricted access MUST
Discussion: Trust is a critical concept in network security. Any require authentication before granting access.
network resource (such as a file server or printer) with restricted
access MUST require authentication before granting access.
Authentication takes many forms, including but not limited to IP Authentication takes many forms, including but not limited to IP
addresses; TCP or UDP port numbers; passwords; external token addresses; TCP or UDP port numbers; passwords; external token
authentication cards; and biometric identification such as signature, authentication cards; and biometric identification such as signature,
speech, or retina recognition systems. speech, or retina recognition systems.
The entity being authenticated MAY be the client machine (for The entity being authenticated MAY be the client machine (for
example, by proving that a given IP source address really is that example, by proving that a given IP source address really is that
address, and not a rogue machine spoofing that address) or a user (by address, and not a rogue machine spoofing that address) or a user (by
proving that the user really is who he, she, or it claims to be). proving that the user really is who he, she, or it claims to be).
Servers SHOULD also authenticate themselves to clients. Servers SHOULD also authenticate themselves to clients.
Testers should be aware that in an increasingly mobile society, Testers should be aware that in an increasingly mobile society,
authentication based on machine-specific criteria such as an IP
Newman Page [4] Newman Page [4]
authentication based on machine-specific criteria such as an IP
address or port number is not equivalent to verifying that a given address or port number is not equivalent to verifying that a given
individual is making an access request. At this writing systems that individual is making an access request. At this writing systems that
verify the identity of users are typically external to the firewall, verify the identity of users are typically external to the firewall,
and may introduce additional latency to the overall SUT. and may introduce additional latency to the overall SUT.
Measurement units: Measurement units:
not applicable not applicable
Issues: Issues:
See also: See also:
user user
3.4 Bit forwarding rate 3.4 Bit forwarding rate
Definition: Definition:
The number of bits per second of allowed traffic a DUT/SUT can be The number of bits per second of allowed traffic a DUT/SUT can be
observed to transmit to the correct destination interface(s) in observed to transmit to the correct destination interface(s) in
response to a specified offered load. response to a specified offered load.
Discussion: Discussion:
This definition differs substantially from section 3.17 of RFC 1242
and section 3.6.1 of RFC 2285.
This definition differs substantially from section 3.17 of RFC Unlike both RFCs 1242 and 2285, this definition introduces the notion
1242 and section 3.6.1 of RFC 2285. of different classes of traffic: allowed, illegal, and rejected (see
definitions for each term). Any bit forwarding rate measurement MUST
Unlike both RFCs 1242 and 2285, this definition introduces the include only allowed traffic.
notion of different classes of traffic: allowed, illegal, and
rejected (see definitions for each term). Any bit forwarding rate
measurement MUST include only allowed traffic.
Unlike RFC 1242, there is no reference to lost or retransmitted Unlike RFC 1242, there is no reference to lost or retransmitted data.
data. Forwarding rate is assumed to be a goodput measurement, in Forwarding rate is assumed to be a goodput measurement, in that only
that only data successfully forwarded to the destination interface data successfully forwarded to the destination interface is measured.
is measured. Bit forwarding rate MUST be measured in relation to Bit forwarding rate MUST be measured in relation to the offered load.
the offered load. Bit forwarding rate MAY be measured with Bit forwarding rate MAY be measured with differed load levels,
differed load levels, traffic orientation, and traffic traffic orientation, and traffic distribution.
distribution.
Unlike RFC 2285, this measurement counts bits per second rather Unlike RFC 2285, this measurement counts bits per second rather than
than frames per second. Per-frame metrics are not meaningful in frames per second. Testers interested in frame (or frame-like)
the context of a flow of application data between endpoints. measurements should use units of transfer.
Units of measurement: Units of measurement:
bits per second bits per second
Issues: Issues:
Allowed traffic vs. rejected traffic Allowed traffic vs. rejected traffic
Newman Page [5]
See also: See also:
allowed traffic allowed traffic
goodput goodput
illegal traffic illegal traffic
rejected traffic rejected traffic
unit of transfer
3.5 Circuit proxy 3.5 Circuit proxy
Newman Page [5]
Definition: Definition:
A proxy service that statically defines which traffic will be
A type of proxy service that copies bytes between authorized forwarded.
source and destinations for defined TCP ports. The data is copied
without any intelligent processing and the proxy cannot
dynamically open and close application specific auxiliary ports
that are sometimes required.
Discussion: Discussion:
The key difference between application and circuit proxies is that
The key distinction with circuit proxies is that they are static the latter are static and thus will always set up a connection if the
and thus will always set up a connection if the DUT/SUT's rule set DUT/SUT's rule set allows it. For example, if a firewall's rule set
allows it. For example, if a firewall's rule set permits ftp permits ftp connections, a circuit proxy will always forward traffic
connections, a circuit proxy will forward traffic on TCP port 20 on TCP port 20 (ftp-data) even if no control connection was first
(ftp default data) even if no control connection was first established on TCP port 21 (ftp-control).
established on TCP port 21 (ftp control).
Measurement units: Measurement units:
not applicable not applicable
Issues: Issues:
application proxy application proxy
rule sets rule sets
See also: See also:
allowed traffic allowed traffic
application proxy application proxy
proxy proxy
rejected traffic rejected traffic
rule set rule set
3.6 Concurrent connections 3.6 Concurrent connections
Definition: Definition:
The aggregate number of simultaneous connections between hosts across
The aggregate number of simultaneous connections between hosts the DUT/SUT, or between hosts and the DUT/SUT.
across the DUT/SUT, or between hosts and the DUT/SUT.
Discussion: Discussion:
The number of concurrent connections a firewall can support is just
The number of concurrent connections a firewall can support is as important a metric for some users as maximum bit forwarding rate.
just as important a metric for some users as maximum bit
Newman Page [6]
forwarding rate.
While "connection" describes only a state and not necessarily the While "connection" describes only a state and not necessarily the
transfer of data, concurrency assumes that all existing transfer of data, concurrency assumes that all existing connections
connections are in fact capable of transferring data. If a data are in fact capable of transferring data. If a data cannot be sent
cannot be sent over a connection, that connection should not be over a connection, that connection should not be counted toward the
counted toward the number of concurrent connections. number of concurrent connections.
Measurement units: Further, this definition assumes that the ability (or lack thereof)
to transfer data on a given connection is solely the responsibility
of the DUT/SUT. For example, a TCP connection that a DUT/SUT has left
in a FIN_WAIT_2 state clearly should not be counted. But another
connection that has temporarily stopped transferring data because
some external device has restricted the flow of data is not
necessarily defunct. The tester should take measures to isolate
changes in connection state to those effected by the DUT/SUT.
Measurement units:
Concurrent connections Concurrent connections
Newman Page [6]
Maximum number of concurrent connections Maximum number of concurrent connections
Issues: Issues:
See also: See also:
connections connections
connection establishment rate connection establishment time
connection overhead connection overhead
3.7 Connection 3.7 Connection
Definition: Definition:
A state in which two hosts, or a host and the DUT/SUT, agree to A state in which two hosts, or a host and the DUT/SUT, agree to
exchange data using a known protocol. exchange data using a known protocol.
Discussion: Discussion:
A connection is an abstraction describing an agreement between two A connection is an abstraction describing an agreement between two
nodes: One agrees to send data and the other agrees to receive it. nodes: One agrees to send data and the other agrees to receive it.
Connections may be TCP sessions, but they don't have to be. Other Connections MAY use TCP, but they don't have to. Other protocols such
connection-oriented protocols such as ATM also may be used, either as ATM also may be used, either instead of or in addition to TCP
instead of or in addition to TCP connections. connections.
What constitutes a connection depends on the application. For a What constitutes a connection depends on the application. For a
"native ATM" application like a video stream, connections and native ATM application, connections and virtual circuits may be
virtual circuits can be synonymous. For TCP/IP applications on ATM synonymous. For TCP/IP applications on ATM networks (where multiple
networks (where multiple TCP sessions may ride over a single ATM TCP connections may ride over a single ATM virtual circuit), the
virtual circuit), the number of TCP connections is probably the number of TCP connections may be the most important consideration.
most important consideration.
Additionally, in some cases firewalls may handle a mixture of
native TCP and native ATM connections. In this situation, the
wrappers around user data will differ. The most meaningful metric
describes what an end-user will see.
Data connections describe state, not data transfer. The existence Additionally, in some cases firewalls may handle a mixture of native
of a connection does not imply that data travels on that TCP and native ATM connections. In this situation, the wrappers
connection at any given time, although if data cannot be forwarded around user data will differ. The most meaningful metric describes
on a previously established connection that connection should not what an end-user will see.
be considered in any aggregrate connection count (see concurrent
Newman Page [7] Data connections describe state, not data transfer. The existence of
connections). a connection does not imply that data travels on that connection at
any
given time, although if data cannot be forwarded on a previously
established connection that connection should not be considered in
any aggregrate connection count (see concurrent connections).
A firewall's architecture dictates where a connection is A firewall's architecture dictates where a connection terminates. In
terminated. In the case of proxy-based systems, a connection the case of application or circuit proxy firewalls, a connection
terminates at the DUT/SUT. But firewalls using packet filtering or terminates at the DUT/SUT. But firewalls using packet filtering or
stateful packet filtering designs act only as passthrough devices, stateful packet filtering designs act only as passthrough devices, in
in that they reside between two connection endpoints. Regardless that they reside between two connection endpoints. Regardless of
of firewall architecture, the number of data connections is still firewall architecture, the number of data connections is still
relevant, since all firewalls perform some form of connection relevant, since all firewalls perform some form of connection
maintenance; at the very least, all check connection requests maintenance; at the very least, all check connection requests
against their rule sets. against their rule sets.
Though it seems paradoxical, connectionless protocols such as UDP Further, note that connection is not an atomic unit of measurement in
may also involve connections, at least for the purposes of that it does not describe the various steps involved in connection
firewall performance measurement. For example, one host may send setup, maintenance, and teardown. Testers may wish to take separate
UDP packets to another across a firewall. If the destination host
is listening on the correct UDP port, it receives the UDP packets.
For the purposes of firewall performance measurement, this is
considered a connection. Indeed, some firewall implementations
dynamically alter their rule sets to allow such connections.
Measurement units: Newman Page [7]
measurements of each of these components.
Connection establishment rate When benchmarking firewall performance, it's important to identify
Concurrent connections the connection establishment and teardown procedures, as these MUST
Maximum number of concurrent connections NOT be included when measuring steady-state forwarding rates.
Further, forwarding rates MUST be measured only after any security
associations have been established.
Issues: Though it seems paradoxical, connectionless protocols such as UDP may
also involve connections, at least for the purposes of firewall
performance measurement. For example, one host may send UDP packets
to another across a firewall. If the destination host is listening on
the correct UDP port, it receives the UDP packets. For the purposes
of firewall performance measurement, this is considered a connection.
Measurement units:
concurrent connections
connections
connection establishment time
maximum number of concurrent connections
connection teardown time
Issues:
proxy-based vs. stateful packet filtering proxy-based vs. stateful packet filtering
TCP/IP vs. ATM TCP/IP vs. ATM
connection-oriented vs. connectionless connection-oriented vs. connectionless
See also: See also:
data source data source
concurrent connections concurrent connections
connection establishment rate connection establishment
connection establishment time
connection teardown
connection teardown time
3.8 Connection establishment rate 3.8 Connection establishment
Definition: Definition:
The data exchanged between hosts, or between a host and the DUT/SUT,
The length of time needed for two hosts, or a host and the to initiate a connection.
DUT/SUT, to agree to set up a data exchange using a known
protocol.
Discussion: Discussion:
Connection-oriented protocols like TCP have a proscribed handshaking
procedure when launching a connection. When benchmarking firewall
performance, it is import to identify this handshaking procedure so
that it is not included in measurements of bit forwarding rate or
UOTs per second.
Each connection-oriented protocol has its own defined mechanisms Testers may also be interested in measurements of connection
for setting up a connection. For purposes of benchmarking firewall establishment time through or with a given DUT/SUT.
performance, this shall be the interval between receipt of the
first octet of the packet carrying a connection establishment Measurement units:
request on a DUT/SUT interface until transmission of the last none
Newman Page [8] Newman Page [8]
octet of the last packet of the connection setup traffic headed in See also:
the opposite direction. connection
connection establishement time
connection maintenance
connection teardown
This definition applies only to connection-oriented protocols such Issues:
as TCP. For connectionless protocols such as UDP, the notion of none
connection setup time is not meaningful.
Measurement units: 3.9 Connection establishment time
Connection establishment rate Definition:
The length of time needed for two hosts, or a host and the DUT/SUT,
to agree to set up a connection using a known protocol.
Discussion:
Each connection-oriented protocol has its own defined mechanisms for
setting up a connection. For purposes of benchmarking firewall
performance, this shall be the interval between receipt of the first
bit of the first octet of the packet carrying a connection
establishment request on a DUT/SUT interface until transmission of
the last bit of the last octet of the last packet of the connection
setup traffic headed in the opposite direction.
This definition applies only to connection-oriented protocols such as
TCP. For connectionless protocols such as UDP, the notion of
connection establishment time is not meaningful.
Metric
Connection establishment time
Issues: Issues:
See also: See also:
concurrent connections concurrent connections
connection connection overhead connection
connection overhead
3.9 Connection overhead 3.10 Connection maintenance
Definition: Definition:
The data exchanged between hosts, or between a host and the DUT/SUT,
to ensure a connection is kept alive.
The degradation in bit forwarding rate, if any, observed as a Discussion:
result of the addition of one connection between two hosts through Some implementations of TCP and other connection-oriented protocols
the DUT/SUT, or the addition of one connection from a host to the use "keep-alive" data to maintain a connection during periods where
no user data is exchanged.
When benchmarking firewall performance, it is useful to identfy
connection maintenance traffic as distinct from UOTs per second.
Given that maintenance traffic may be characterized by short bursts
at periodical intervals, it may not be possible to describe a steady-
state forwarding rate for maintenance traffic. One possible approach
Newman Page [9]
is to identify the quantity of maintenance traffic, in bytes or bits,
over a given interval, and divide through to derive a measurement of
maintenance traffic forwarding rate.
Measurement units:
maintenance traffic forwarding rate
See also:
connection
connection establishment time
connection teardown
connection teardown time
Issues:
none
3.11 Connection overhead
Definition:
The degradation in bit forwarding rate, if any, observed as a result
of the addition of one connection between two hosts through the
DUT/SUT, or the addition of one connection from a host to the
DUT/SUT. DUT/SUT.
Discussion: Discussion:
The memory cost of connection establishment and maintenance is highly
The memory cost of connection establishment and maintenance is implementation-specific. This metric is intended to describe that
highly implementation-specific. This metric is intended to cost in a method visible outside the firewall.
describe that cost in a method visible outside the firewall.
It may also be desirable to invert this metric to show the It may also be desirable to invert this metric to show the
performance improvement as a result of tearing down one performance improvement as a result of tearing down one connection.
connection.
Measurement units: Measurement units:
bit forwarding rate bit forwarding rate
Issues: Issues:
3.10 Data source 3.12 Connection teardown
Definition: Definition:
The data exchanged between hosts, or between a host and the DUT/SUT,
A station capable of generating traffic to the DUT/SUT. to close a connection.
Discussion: Discussion:
Connection-oriented protocols like TCP follow a stated procedure when
ending a connection. When benchmarking firewall performance, it is
important to identify the teardown procedure so that it is not
included in measurements of bit forwarding rate or UOTs per second.
One data source MAY emulate multiple users or stations. In Testers may also be interested in measurements of connection teardown
addition, one data source MAY offer traffic to multiple network time through or with a given DUT/SUT.
interfaces on the DUT/SUT.
Newman Page [9]
Measurement units: Measurement units:
none
not applicable Newman Page [10]
See also:
connection teardown time
Issues: Issues:
none
The term "data source" is deliberately independent of any number 3.13 Connection teardown time
of users. It is useful to think of data sources simply as traffic
generators, without any correlation to any given number of users.
See also: Definition:
The length of time needed for two hosts, or a host and the DUT/SUT,
to agree to tear down a connection using a known protocol.
Discussion:
Each connection-oriented protocol has its own defined mechanisms for
dropping a connection. For purposes of benchmarking firewall
performance, this shall be the interval between receipt of the first
bit of the first octet of the packet carrying a connection teardown
request on a DUT/SUT interface until transmission of the last bit of
the last octet of the last packet of the connection teardown traffic
headed in the opposite direction.
This definition applies only to connection-oriented protocols such as
TCP. For connectionless protocols such as UDP, the notion of
connection teardown time is not meaningful.
Metric
Connection teardown time
Issues:
See also:
concurrent connections
connection connection
connection overhead
3.11 Demilitarized zone (DMZ) 3.14 Data source
Definition: Definition:
A host capable of generating traffic to the DUT/SUT.
Discussion:
One data source MAY emulate multiple users or hosts. In addition, one
data source MAY offer traffic to multiple network interfaces on the
DUT/SUT.
The term "data source" is deliberately independent of any number of
users. It is useful to think of data sources simply as traffic
generators, without any correlation to any given number of users.
Measurement units:
not applicable
Issues:
Newman Page [11]
user
See also:
connection
user
3.15 Demilitarized zone
Definition:
A network segment or segments located between protected and A network segment or segments located between protected and
unprotected networks. DMZ networks are sometimes called perimeter unprotected networks.
networks.
Discussion: Discussion:
As an extra security measure, networks may be designed such that
As an extra security measure, networks are often designed such protected and unprotected segments are never directly connected.
that protected and unprotected segments are never directly Instead, firewalls (and possibly public resources such as WWW or FTP
connected. Instead, firewalls (and possibly public resources such servers) reside on a so-called DMZ network. To connect protected,
as WWW or FTP servers) often reside on the so-called DMZ network. DMZ, and unprotected networks with one device, the device MUST have
To connect protected, DMZ, and unprotected networks with one at least three network interfaces.
device, the device MUST have at least three network interfaces.
Multiple firewalls MAY bound the DMZ. In this case, the firewalls Multiple firewalls MAY bound the DMZ. In this case, the firewalls
connecting the protected network with the DMZ and the DMZ with the connecting the protected network with the DMZ and the DMZ with the
unprotected network MUST each have at least two network unprotected network MUST each have at least two network interfaces.
interfaces.
Measurement units: DMZ networks are sometimes called perimeter networks.
Measurement units:
not applicable not applicable
Issues: Issues:
Homed Homed
See also: See also:
unprotected network unprotected network
perimeter network
protected network protected network
3.12 Firewall 3.16 Firewall
Definition: Definition:
A device or group of devices that enforces an access control policy
Newman Page [10] between networks.
A device or group of devices that enforces an access control
policy between networks.
Discussion: Discussion:
While there are many different ways to accomplish it, all firewalls
While there are many different ways to accomplish it, all do the same thing: control access between networks.
firewalls do essentially the same thing: control access between
networks.
The most common configuration involves a firewall connecting two The most common configuration involves a firewall connecting two
segments (one protected and one unprotected), but this is not the segments (one protected and one unprotected), but this is not the
only possible configuration. Many firewalls support tri-homing, only possible configuration. Many firewalls support tri-homing,
allowing use of a DMZ network. It is possible for a firewall to allowing use of a DMZ network. It is possible for a firewall to
accommodate more than three interfaces, each attached to a accommodate more than three interfaces, each attached to a different
different network segment. network segment.
The criteria by which access is controlled is deliberately not The criteria by which access are controlled is deliberately not
Newman Page [12]
specified here. Typically this has been done using network- or specified here. Typically this has been done using network- or
transport-layer criteria (such as IP subnet or TCP port number), transport-layer criteria (such as IP subnet or TCP port number), but
but there is no reason this must always be so. A growing number of there is no reason this must always be so. A growing number of
firewalls are controlling access at the application layer, using firewalls are controlling access at the application layer, using user
user identification as the criterion. And firewalls for ATM identification as the criterion. And firewalls for ATM networks may
networks may control access based on data link-layer criteria. control access based on data link-layer criteria.
Measurement units: Measurement units:
not applicable not applicable
Issues: Issues:
See also: See also:
DMZ DMZ
tri-homed tri-homed
user user
3.13 Goodput 3.17 Goodput
Definition: Definition:
The number of bits per unit of time forwarded to the correct The number of bits per unit of time forwarded to the correct
destination interface of the DUT/SUT, minus any bits lost or destination interface of the DUT/SUT, minus any bits lost or
retransmitted. retransmitted.
Discussion: Discussion:
Firewalls are generally insensitive to packet loss in the network. As
such, measurements of gross bit forwarding rates are not meaningful
since (in the case of proxy-based and stateful packet filtering
firewalls) a receiving endpoint directly attached to a DUT/SUT would
not receive any data dropped by the DUT/SUT.
Firewalls are generally insensitive to packet loss in the network. The type of traffic lost or retransmitted is protocol-dependent. TCP
As such, measurements of gross bit forwarding rates are not and ATM, for example, request different types of retransmissions.
meaningful since (in the case of proxy-based and stateful packet Testers MUST observe retransmitted data for the protocol in use, and
filtering firewalls) a receiving endpoint directly attached to a subtract this quantity from measurements of gross bit forwarding
DUT/SUT would not receive any data dropped by the DUT/SUT. rate.
The type of traffic lost or retransmitted is protocol-dependent.
Newman Page [11]
TCP and ATM, for example, request different types of
retransmissions. Testers MUST observe retransmitted data for the
protocol in use, and subtract this quantity from measurements of
gross bit forwarding rate.
Measurement unit:
Unit of measurement:
bits per second bits per second
Issues: Issues:
allowed vs. rejected traffic allowed vs. rejected traffic
See also: See also:
allowed traffic allowed traffic
bit forwarding rate bit forwarding rate
rejected traffic rejected traffic
3.14 Homed 3.18 Homed
Definition: Definition:
The number of logical interfaces a DUT/SUT contains. The number of logical interfaces a DUT/SUT contains.
Discussion: Discussion:
Newman Page [13]
Firewalls MUST contain at least two logical interfaces. In network Firewalls MUST contain at least two logical interfaces. In network
topologies where a DMZ is used, the firewall contains at least topologies where a DMZ is used, the firewall contains at least three
three interfaces and is said to be tri-homed. Additional interfaces and is said to be tri-homed. Additional interfaces would
interfaces would make a firewall quad-homed, quint-homed, and so make a firewall quad-homed, quint-homed, and so on.
on.
It is theoretically possible for a firewall to contain one It is theoretically possible for a firewall to contain one physical
physical interface and multiple logical interfaces. This interface and multiple logical interfaces. This configuration is
configuration is strongly discouraged for testing purposes because strongly discouraged for testing purposes because of the difficulty
of the difficulty in verifying that no leakage occurs between in verifying that no leakage occurs between protected and unprotected
protected and unprotected segments. segments.
Measurement units: Measurement units:
not applicable not applicable
Issues: Issues:
See also: See also:
tri-homed tri-homed
3.15 Illegal traffic 3.19 Illegal traffic
Definition: Definition:
Packets specified for rejection in the rule set of the DUT/SUT. Packets specified for rejection in the rule set of the DUT/SUT.
Newman Page [12]
Discussion: Discussion:
A buggy or misconfigured firewall may forward packets even though its
A buggy or misconfigured firewall may forward packets even though rule set specifies that these packets be dropped. Illegal traffic
its rule set specifies that these packets be dropped. Illegal differs from rejected traffic in that it describes all traffic
traffic differs from rejected traffic in that it describes all specified for rejection by the rule set, while rejected traffic
traffic specified for rejection by the rule set, while rejected specifies only those packets actually dropped by the DUT/SUT.
traffic specifies only those packets actually dropped by the
DUT/SUT.
Measurement units: Measurement units:
not applicable not applicable
Issues: Issues:
See also: See also:
accepted traffic accepted traffic
policy policy
rejected traffic rejected traffic
rule set rule set
3.16 Logging 3.20 Logging
Definition: Definition:
The recording of user requests made to the firewall. The recording of user requests made to the firewall.
Discussion: Discussion:
Firewalls MUST log all requests they handle, both allowed and Firewalls MUST log all requests they handle, both allowed and
rejected. For many firewall designs, logging requires a rejected. For many firewall designs, logging requires a significant
significant amount of processing overhead, especially when complex amount of processing overhead, especially when complex rule sets are
rule sets are in use. in use.
The type and amount of data logged varies by implementation. The type and amount of data logged varies by implementation. Testers
Testers SHOULD attempt to log equivalent data when comparing
different DUT/SUTs. Newman Page [14]
SHOULD attempt to log equivalent data when comparing different
DUT/SUTs.
Logging MAY take place on systems other than the DUT/SUT. Logging MAY take place on systems other than the DUT/SUT.
Measurement units: Measurement units:
not applicable not applicable
Issues: Issues:
rule sets rule sets
See also: See also:
allowed traffic allowed traffic
connection connection
Newman Page [13]
rejected traffic rejected traffic
3.17 Network address translation (NAT) 3.21 Network address translation
Definition: Definition:
A method of mapping one or more private, reserved IP addresses to one
A function that maps the original IP source and/or destination or more public IP addresses.
addresses of packets arriving at a given interface of the firewall
to a different address or addresses.
Discussion: Discussion:
Firewalls use NAT to ensure that the IP addresses of a protected
network are not visible to systems and users on the Internet or
some other untrusted network. This is typically required since
many networks use RFC 1918 reserved addresses.
In the interest of conserving the IPv4 address space, RFC 1918 In the interest of conserving the IPv4 address space, RFC 1918
proposed the use of certain private (reserved) blocks of IP proposed the use of certain private (reserved) blocks of IP
addresses. Connections to public networks are made by use of a addresses. Connections to public networks are made by use of a device
device that translates one or more RFC 1918 addresses to one or that translates one or more RFC 1918 addresses to one or more public
more public addresses--a network address translator (NAT). addresses--a network address translator (NAT).
The use of private addressing also introduces a security benefit The use of private addressing also introduces a security benefit in
in that RFC 1918 addresses are not visible to hosts on the public that RFC 1918 addresses are not visible to hosts on the public
Internet. Internet.
Some NAT implementations are computationally intensive, and may Some NAT implementations are computationally intensive, and may
affect bit forwarding rate. affect bit forwarding rate.
There are two common methods for NAT: many to one (aggregation)
and one to one mapping. It should be noted that all proxy
firewalls always perform NAT as a function of their architecture,
while, by default, packet filtering firewalls do not. In general
all application and circuit proxy firewalls, by default, perform
NAT as a function of their architecture using the aggregation
method, while stateful packet filtering firewalls do not perform
NAT by default, but can be configured to do so.
Measurement units: Measurement units:
not applicable not applicable
Issues: Issues:
See also: See also:
3.18 Packet filtering 3.22 Packet filtering
Definition: Definition:
The process of controlling access by examining packets based on the
content of packet headers.
The process of controlling access by examining packets based on
packet header content.
Newman Page [14]
Discussion: Discussion:
Packet-filtering devices forward or deny packets based on information
in each packet's header, such as IP address or TCP port number. A
packet-filtering firewall uses a rule set to determine which traffic
should be forwarded and which should be blocked.
Packet-filtering firewalls forward or deny packets based on Newman Page [15]
information in each packet's header, such as IP address or TCP
port number. A packet- filtering firewall uses a rule set to
determine which traffic should be forwarded and which should be
blocked.
Measurement units: Measurement units:
not applicable not applicable
Issues: Issues:
static versus stateful packet filtering
static vs. stateful packet filtering
See also: See also:
application proxy application proxy
circuit proxy circuit proxy
proxy proxy
rule set rule set
stateful packet filtering stateful packet filtering
3.19 Perimeter network 3.23 Policy
Definition:
A network segment or segments located between protected and
unprotected networks. Perimeter networks are often called DMZ
networks.
Discussion:
See the definition of DMZ for a discussion.
Measurement units:
not applicable
Issues:
Tri-homed
See also:
demilitarized zone (DMZ)
unprotected network
protected network
3.20 Policy
Definition: Definition:
A document defining acceptable access to protected, DMZ, and A document defining acceptable access to protected, DMZ, and
Newman Page [15]
unprotected networks. unprotected networks.
Discussion: Discussion:
Security policies generally do not spell out specific configurations
for firewalls; rather, they set general guidelines for what is and is
not acceptable network access.
Security policies generally do not spell out specific The actual mechanism for controlling access is usually the rule set
configurations for firewalls; rather, they set general guidelines implemented in the DUT/SUT.
for what is and is not acceptable network access.
The actual mechanism for enforcing the access policies is usually
the rule set implemented in the DUT/SUT.
Measurement units: Measurement units:
not applicable not applicable
Issues: Issues:
See also: See also:
rule set rule set
3.21 Protected network 3.24 Protected network
Definition: Definition:
A network segment or segments to which access is controlled by the A network segment or segments to which access is controlled by the
DUT/SUT. DUT/SUT.
Discussion: Discussion:
Firewalls are intended to prevent unauthorized access either to or Firewalls are intended to prevent unauthorized access either to or
from the protected network. Depending on the configuration from the protected network. Depending on the configuration specified
specified by the policy and rule set, the DUT/SUT may allow by the policy and rule set, the DUT/SUT may allow hosts on the
stations on the protected segment to act as clients for servers on protected segment to act as clients for servers on either the DMZ or
either the DMZ or the unprotected network, or both. the unprotected network, or both.
Protected networks are often called "internal networks." That term Protected networks are often called "internal networks." That term is
is not used here because firewalls increasingly are deployed not used here because firewalls increasingly are deployed within an
within an organization, where all segments are by definition organization, where all segments are by definition internal.
internal. It is also possible for a firewall to protected
multiple, but different protected networks from each other.
Measurement units: Measurement units:
Newman Page [16]
not applicable not applicable
Issues: Issues:
See also: See also:
demilitarized zone (DMZ) demilitarized zone (DMZ)
unprotected network unprotected network
policy policy
rule set rule set
Newman Page [16]
unprotected network unprotected network
3.22 Proxy 3.25 Proxy
Definition: Definition:
A request for a connection made on behalf of a host. A request for a connection made on behalf of a host.
Discussion: Discussion:
Proxy-based firewalls do not allow direct connections between hosts.
Proxy-based firewalls do not allow direct connections between Instead, two connections are established: one between the client host
hosts. Instead, two connections are established: one between the and the DUT/SUT, and another between the DUT/SUT and server host.
client host and the DUT/SUT, and another between the DUT/SUT and
server host.
As with packet-filtering firewalls, proxy-based devices use a rule As with packet-filtering firewalls, proxy-based devices use a rule
set to determine which traffic should be forwarded and which set to determine which traffic should be forwarded and which should
should be rejected. be rejected.
There are two types of proxies: application proxies and circuit There are two types of proxies: application proxies and circuit
proxies. proxies.
Measurement units: Measurement units:
not applicable not applicable
Issues: Issues:
application application
See also: See also:
application proxy
application circuit proxy
proxy circuit
proxy
packet filtering packet filtering
stateful packet filtering stateful packet filtering
3.23 Rejected traffic 3.26 Rejected traffic
Definition: Definition:
Packets dropped as a result of the rule set of the DUT/SUT. Packets dropped as a result of the rule set of the DUT/SUT.
Discussion: Discussion:
Firewalls MUST reject any traffic not explicitly permitted in the Firewalls MUST reject any traffic not explicitly permitted in the
rule set. Dropped packets MUST NOT be included in calculating the rule set. Dropped packets MUST NOT be included in calculating the bit
bit forwarding rate or maximum bit forwarding rate of the DUT/SUT. forwarding rate or maximum bit forwarding rate of the DUT/SUT.
Measurement units: Measurement units:
not applicable not applicable
Newman Page [17]
Issues: Issues:
Newman Page [17]
See also: See also:
allowed traffic
illegal traffic
policy policy
rule set rule set
3.24 Rule set 3.27 Rule set
Definition: Definition:
The collection of access control rules that determines which packets
The collection of access control rules that determines which the DUT/SUT will forward and which it will reject.
packets the DUT/SUT will forward and which it will reject.
Discussion: Discussion:
Rule sets control access to and from the network interfaces of the Rule sets control access to and from the network interfaces of the
DUT/SUT. By definition, rule sets MUST NOT apply equally to all DUT/SUT. By definition, rule sets MUST NOT apply equally to all
network interfaces; otherwise there would be no need for the network interfaces; otherwise there would be no need for the
firewall. Therefore, a specific rule set MUST be applied to each firewall. Therefore, a specific rule set MUST be applied to each
network interface in the DUT/SUT. network interface in the DUT/SUT.
The order of rules within the rule set is critical. Firewalls The tester must describe the complete contents of the rule set of
generally scan rule sets in a "top down" fashion, which is to say each DUT/SUT.
that the device compares each packet received with each rule in
the rule set until it finds a rule that applies to the packet.
Once the device finds an applicable rule, it applies the actions
defined in that rule (such as forwarding or rejecting the packet)
and ignores all subsequent rules. For testing purposes, the rule
set MUST conclude with a rule denying all access.
Measurement units: To ensure that testers measure only traffic forwarded or rejected by
the DUT/SUT, each rule set MUST include a rule denying all access
except for those packets allowed by the rule set.
Measurement units:
not applicable not applicable
Issues: Issues:
See also: See also:
allowed traffic
demilitarized zone (DMZ) demilitarized zone (DMZ)
illegal traffic
policy policy
protected network protected network
rejected traffic rejected traffic
unprotected network unprotected network
3.25 Stateful packet filtering 3.28 Security association
Definition: Definition:
The set of security information relating to a given network
The process of forwarding or rejecting traffic based on the connection or set of connections.
contents of a state table maintained by a firewall.
Discussion: Discussion:
This definition, taken verbatim from RFC 1825, covers the
relationship between policy and connections. Security associations
(SAs) are typically set up during connection establishment, and they
may be reiterated or revoked during a connection.
For purposes of benchmarking firewall performance, measurements of
Newman Page [18] Newman Page [18]
Stateful packet filtering ensures that packets associated with an bit forwarding rate or UOTs per second MUST be taken after all
already established (and authorized) connection are allowed to be security associations have been established.
forwarded through the firewall. This differs from a simple packet
filter firewall that would allow any packets through regardless of Measurement units:
the state of that connection. For example, a stateful packet not applicable
See also:
connection
connection establishment
policy
rule set
3.29 Stateful packet filtering
Definition:
The process of forwarding or rejecting traffic based on the contents
of a state table maintained by a firewall.
Discussion:
Packet filtering and proxy firewalls are essentially static, in that
they always forward or reject packets based on the contents of the
rule set.
In contrast, devices using stateful packet filtering will only
forward packets if they correspond with state information maintained
by the device about each connection. For example, a stateful packet
filtering device will reject a packet on port 20 (ftp-data) if no filtering device will reject a packet on port 20 (ftp-data) if no
session has been established over the ftp control port (usually connection has been established over the ftp control port (usually
port 21). port 21).
Measurement units: Measurement units:
not applicable not applicable
Issues: Issues:
See also: See also:
applicaton proxy
application proxy
circuit proxy
packet filter packet filter
proxy proxy
3.26 Tri-homed 3.30 Tri-homed
Definition: Definition:
A firewall with three network interfaces. A firewall with three network interfaces.
Discussion: Discussion:
Tri-homed firewalls connect three network segments with different Tri-homed firewalls connect three network segments with different
network addresses. Typically, these would be protected, DMZ, and network addresses. Typically, these would be protected, DMZ, and
unprotected segments. unprotected segments.
A tri-homed firewall may offer some security advantages over A tri-homed firewall may offer some security advantages over
firewalls with two interfaces. An attacker on an unprotected firewalls with two interfaces. An attacker on an unprotected network
network may compromise hosts on the DMZ but still not reach any may compromise hosts on the DMZ but still not reach any hosts on the
hosts on the protected network. protected network.
Newman Page [19]
Measurement units: Measurement units:
not applicable not applicable
Issues: Issues:
Usually the differentiator between one segment and another is its IP
Usually the differentiator between one segment and another is its address. However, firewalls may connect different networks of other
IP address. However, firewalls may connect different networks of types, such as ATM or Netware segments.
other types, such as ATM or Netware segments.
See also: See also:
homed homed
3.27 Unprotected network 3.31 Unit of transfer
Newman Page [19]
Definition: Definition:
A discrete collection of bytes comprising at least one header and
optional user data.
Discussion:
This metric is intended for use in describing steady-state forwarding
rate of the DUT/SUT.
The unit of transfer (UOT) definition is deliberately left open to
interpretation, allowing the broadest possible application. Examples
of UOTs include TCP segments, IP packets, Ethernet frames, and ATM
cells.
While the definition is deliberately broad, its interpretation must
not be. The tester MUST describe what type of UOT will be offered to
the DUT/SUT, and MUST offer these UOTs at a consistent rate. Traffic
measurement MUST begin after all connection establishment routines
complete and before any connection completion routine begins.
Further, measurements MUST begin after any security associations
(SAs) are established and before any SA is revoked.
Testers also MUST compare only like UOTs. It is not appropriate, for
example, to compare forwarding rates by offering 1,500-byte Ethernet
UOTs to one DUT/SUT and 53-byte ATM cells to another.
Measurement units:
Units of transfer
Units of transfer per second
Issues:
See also:
Bit forwarding rate
connection
3.32 Unprotected network
Definition:
A network segment or segments to which access is not controlled by A network segment or segments to which access is not controlled by
the DUT/SUT. the DUT/SUT.
Newman Page [20]
Discussion: Discussion:
Firewalls are deployed between protected and unprotected segments. Firewalls are deployed between protected and unprotected segments.
The unprotected network is not protected by the DUT/SUT. The unprotected network is not protected by the DUT/SUT.
Note that a DUT/SUT's policy MAY specify hosts on an unprotected Note that a DUT/SUT's policy MAY specify hosts on an unprotected
network. For example, a user on a protected network may be network. For example, a user on a protected network may be permitted
permitted to access an FTP server on an unprotected network. But to access an FTP server on an unprotected network. But the DUT/SUT
the DUT/SUT cannot control access between hosts on the unprotected cannot control access between hosts on the unprotected network.
network.
Measurement units: Measurement units:
not applicable not applicable
Issues: Issues:
See also: See also:
demilitarized zone (DMZ) demilitarized zone (DMZ)
policy policy
protected network protected network
rule set rule set
3.28 User 3.33 User
Definition: Definition:
A person or process requesting access to resources protected by the
A person or process requesting access to resources protected by DUT/SUT.
the DUT/SUT.
Discussion: Discussion:
"User" is a problematic term in the context of firewall performance
testing, for several reasons. First, a user may in fact be a process
or processes requesting services through the DUT/SUT. Second,
different "user" requests may require radically different amounts of
DUT/SUT resources. Third, traffic profiles vary widely from one
organization to another, making it difficult to characterize the load
offered by a typical user.
"User" is a problematic term in the context of firewall For these reasons, testers SHOULD NOT attempt to measure DUT/SUT
performance testing, for several reasons. First, a user may in performance in terms of users supported. Instead, testers SHOULD
fact be a process or processes requesting services through the describe performance in terms of maximum bit forwarding rate and
DUT/SUT. Second, different "user" requests may require radically maximum number of connections sustained. Further, testers SHOULD use
different amounts of DUT/SUT resources. Third, traffic profiles the term "data source" rather than user to describe traffic
vary widely from one organization to another, making it difficult generator(s).
to characterize the load offered by a typical user.
For these reasons, it's probably not a good idea to measure
DUT/SUT performance in terms of users supported. The only
exception is in cases where traffic patterns are well understood
and constant--conditions that unfortunately don't exist in many
networks. Instead, it's preferable to describe performance in
terms of maximum bit forwarding rate and maximum number of
connections sustained. It's also preferable to use the term "data
Newman Page [20]
source" rather than "user" to describe the traffic generator(s) to
avoid any confusion with actual user data profiles.
Measurement units: Measurement units:
not applicable not applicable
Issues: Issues:
See also: See also:
data source data source
4. Security considerations 4. Security considerations
The primary goal of this memo is to describe terms used in The primary goal of this memo is to describe terms used in
Newman Page [21]
benchmarking firewall performance. However, readers should be aware benchmarking firewall performance. However, readers should be aware
that there is some overlap between performance and security issues. that there is some overlap between performance and security issues.
Specifically, the optimal configuration for firewall performance may Specifically, the optimal configuration for firewall performance may
not be the most secure, and vice-versa. not be the most secure, and vice-versa.
Further, certain forms of attack may degrade performance. One common Further, certain forms of attack may degrade performance. One common
form of denial-of-service (DoS) attack bombards a firewall with so form of denial-of-service (DoS) attack bombards a firewall with so
much rejected traffic that it cannot forward allowed traffic. DoS much rejected traffic that it cannot forward allowed traffic. DoS
attacks do not always involve heavy loads; DoS describes any state in attacks do not always involve heavy loads; by definition, DoS
which a firewall is offered rejected traffic that prohibits it from describes any state in which a firewall is offered rejected traffic
forwarding some or all allowed traffic. Even a small amount of that prohibits it from forwarding some or all allowed traffic. Even a
traffic-- such as the recent Teardrop2 attack involving a few packet small amount of traffic--such as the recent Teardrop2 attack
fragments-- may significantly degrade firewall performance, or stop involving a few packet fragments--may significantly degrade firewall
the firewall altogether. Further, the safeguards in firewalls to performance, or stop the firewall altogether. Further, the safeguards
guard against such attacks may have a significant negative impact on in firewalls to guard against such attacks may have have a
performance. significant negative impact on performance.
Since the library of attacks is constantly expanding, no attempt is Since the library of attacks is constantly expanding, no attempt is
made here to define specific attacks that may affect performance. made here to define specific attacks that may affect performance.
Nonetheless, any reasonable performance benchmark must take Nonetheless, any reasonable performance benchmark must take
safeguards against such attacks into consideration. Specifically, the safeguards against such attacks into consideration. Specifically, the
same safeguards must be in place when comparing performance of same safeguards must be in place when comparing performance of
different firewall implementations. different firewall implementations.
5. References 5. References
Atkinson, R. "Security Architecture for the Internet Protocol." RFC
1825.
Bradner, S., editor. "Benchmarking Terminology for Network Bradner, S., editor. "Benchmarking Terminology for Network
Interconnection Devices." RFC 1242. Interconnection Devices." RFC 1242.
Bradner, S., and McQuaid, J. "Benchmarking Methodology for Network Bradner, S., and McQuaid, J. "Benchmarking Methodology for Network
Interconnect Devices." RFC 1944. Interconnect Devices." RFC 1944.
Mandeville, R. "Benchmarking Terminology for LAN Switching Devices." Mandeville, R. "Benchmarking Terminology for LAN Switching Devices."
RFC 2285. RFC 2285.
Rekhter, Y., et al. "Address Allocation for Private Internets." RFC Rekhter, Y., et al. "Address Allocation for Private Internets." RFC
1918. 1918.
Newman Page [21]
6. Acknowledgments 6. Acknowledgments
The editor wishes to thank the IETF Benchmarking Working Group for The author wishes to thank the IETF Benchmarking Working Group for
agreeing to review this document. Many persons offered valuable agreeing to review this document. Several other persons offered
contributions and critiques during this project: Ted Doty (Internet valuable contributions and critiques during this project: Ted Doty
Security Systems), Kevin Dubray (Bay Networks), Helen Holzbaur (Internet Security Systems), Kevin Dubray (Ironbridge Networks),
(NSTL), Jim Hurd (NSTL), Dale Lancaster (Axent Technologies), Robert Helen Holzbaur (NSTL), Jim Hurd (NSTL), Dale Lancaster (Axent
Mandeville (European Network Laboratories), Brent Melson (NSTL), Technologies), Robert Mandeville (European Network Laboratories),
Steve Platt (NSTL), Marcus Ranum (Network Flight Recorder Inc.), Greg Brent Melson (NSTL), Steve Platt (NSTL), Marcus Ranum (Network Flight
Shannon (Ascend Communications), Christoph Schuba (Sun Microsystems), Recorder Inc.), Greg Shannon (Ascend Communications), Christoph
Rick Siebenaler (Cyberguard), and Greg Smith (Check Point Software Schuba (Sun Microsystems), Rick Siebenaler (Cyberguard), and Greg
Technologies). Smith (Check Point Software Technologies).
Newman Page [22]
7. Contact information 7. Contact information
David Newman David Newman
Data Communications magazine Data Communications magazine
1221 Avenue of the Americas, 41st floor 3 Park Ave.
New York, NY 10020 USA 31st Floor
212-512-6182 voice New York, NY 10016
212-512-6833 fax USA
212-592-8256 voice
212-592-8265 fax
dnewman@data.com dnewman@data.com
Newman Page [22] Newman Page [23]
 End of changes. 

This html diff was produced by rfcdiff 1.23, available from http://www.levkowetz.com/ietf/tools/rfcdiff/