draft-ietf-bmwg-secperf-01.txt   draft-ietf-bmwg-secperf-02.txt 
Network Working Group D. Newman Network Working Group D. Newman
INTERNET-DRAFT Data Communications INTERNET-DRAFT Data Communications
Expires in May 1998 H. Holzbaur, J. Hurd, and S. Platt Expires in September 1998 H. Holzbaur, J. Hurd, and S. Platt
National Software Testing Laboratories National Software Testing Laboratories
Benchmarking Terminology for Firewall Performance Benchmarking Terminology for Firewall Performance
<draft-ietf-bmwg-secperf-01.txt> <draft-ietf-bmwg-secperf-02.txt>
Status of This Memo Status of This Memo
This document is an Internet-Draft. Internet-Drafts are working This document is an Internet-Draft. Internet-Drafts are working
documents of the Internet Engineering Task Force (IETF), its areas, documents of the Internet Engineering Task Force (IETF), its areas,
and its working groups. Note that other groups may also distribute and its working groups. Note that other groups may also distribute
working documents as Internet-Drafts. working documents as Internet-Drafts.
Internet-Drafts are draft documents valid for a maximum of six months Internet-Drafts are draft documents valid for a maximum of six months
and may be updated, replaced, or obsoleted by other documents at any and may be updated, replaced, or obsoleted by other documents at any
time. It is inappropriate to use Internet- Drafts as reference time. It is inappropriate to use Internet- Drafts as reference
material or to cite them other than as "work in progress." material or to cite them other than as "work in progress."
To view the entire list of current Internet-Drafts, please check the To view the entire list of current Internet-Drafts, please check the
"1id-abstracts.txt" listing contained in the Internet-Drafts Shadow "1id-abstracts.txt" listing contained in the Internet-Drafts Shadow
Directories on ftp.is.co.za (Africa), ftp.nordu.net (Europe), Directories on ftp.is.co.za (Africa), ftp.nordu.net (Europe),
munnari.oz.au (Pacific Rim), ds.internic.net (US East Coast), or munnari.oz.au (Pacific Rim), ds.internic.net (US East Coast), or
ftp.isi.edu (US West Coast). ftp.isi.edu (US West Coast).
1. Introduction .......................................................2 1. Introduction ....................................................2
2. Existing definitions ...............................................2 2. Existing definitions ............................................3
3. Term definitions ...................................................2 3. Term definitions ................................................3
3.1 Allowed traffic ..............................................2 3.1 Allowed traffic ..............................................3
3.2 Authentication ...............................................3 3.2 Authentication ...............................................3
3.3 Data source ..................................................3 3.3 Connection ...................................................4
3.4 Data connection ..............................................4 3.4 Data source ..................................................5
3.5 Demilitarized zone (DMZ) .....................................4 3.5 Demilitarized zone (DMZ) .....................................5
3.6 Dual-homed ...................................................5 3.6 Dynamic proxy ................................................5
3.7 Dynamic proxy ................................................5 3.7 Firewall .....................................................6
3.8 External network .............................................6 3.8 Forwarding rate ..............................................6
3.9 Homed ........................................................6 3.9 Goodput ......................................................7
3.10 Packet filtering ............................................6 3.10 Homed .......................................................7
3.11 Perimeter network ...........................................7 3.11 Logging .....................................................8
3.12 Policy ......................................................7 3.12 Network address translation (NAT) ...........................8
3.13 Protected network ...........................................8 3.13 Packet filtering ............................................9
3.14 Proxy .......................................................8 3.14 Perimeter network ...........................................9
3.15 Rejected traffic ............................................9 3.15 Policy .....................................................10
3.16 Rule set ....................................................9 3.16 Protected network ..........................................10
3.17 Session .....................................................9 3.17 Proxy ......................................................11
3.18 Stateful inspection ........................................10 3.18 Rejected traffic ...........................................11
3.19 Tri-homed ..................................................11 3.19 Rule set ...................................................11
3.20 User .......................................................11 3.20 Session ....................................................12
4. Security considerations ..........................................11 3.21 Stateful packet filtering ..................................13
5. References ........................................................12 3.22 Tri-homed ..................................................13
6. Acknowledgments ...................................................12 3.23 Unprotected network ........................................14
7. Contact Information ...............................................12 3.24 User .......................................................14
4. Security considerations ...........................................15
5. References ........................................................15
6. Acknowledgments ...................................................15
7. Contact information ...............................................16
1. Introduction 1. Introduction
This document defines terms used in measuring the performance of This document defines terms used in measuring the performance of
firewalls. It extends the terminology already used for benchmarking firewalls. It extends the terminology already used for benchmarking
routers and switches and adds terminology specific to firewalls. The routers and switches and adds terminology specific to firewalls. The
primary metrics defined in this document are maximum forwarding rate primary metrics defined in this document are maximum forwarding rate
and maximum number of connections. and maximum number of connections.
Why are firewall performance measurements needed? First, despite the Why are firewall performance measurements needed? First, despite the
rapid rise in deployment of firewalls, there is no standard method rapid rise in firewall deployment, there is no standard means of
for benchmarking their performance. Second, implementations vary performance measurement. Second, implementations vary widely, making
widely, making it difficult to do direct performance comparisons. it difficult to do direct performance comparisons. Finally, more and
Finally, more and more organizations are deploying firewalls on more organizations are deploying firewalls on internal networks
internal networks operating at relatively high speeds, while most operating at relatively high speeds, while most firewall
firewall implementations remain optimized for use over low-speed implementations remain optimized for use over low-speed wide-area
wide-area connections. As a result, users are often unsure whether connections. As a result, users are often unsure whether the products
the products they buy will stand up to relatively heavy loads. they buy will stand up to relatively heavy loads.
We may also create additional terminology and methodology documents We may also create additional terminology and methodology documents
to define other types of network security products such as virtual to define other types of network security products such as virtual
private network (VPN) and encryption devices. This document, however, private network (VPN) and encryption devices. This document, however,
focuses solely on firewall terminology. focuses solely on firewall terminology.
2. Existing definitions 2. Existing definitions
This document uses the conceptual framework established in RFCs 1242 This document uses the conceptual framework established in RFCs 1242
and 1944 (for routers) and draft-ietf-bmwg-lanswitch-07.txt (for and 1944 (for routers) and RFC 2285 (for switches). The router and
switches). The router and switch documents contain discussions of switch documents contain discussions of several terms relevant to
several terms relevant to benchmarking the performance of firewalls. benchmarking the performance of firewalls. Readers should consult the
Readers should consult the router and switch documents before making router and switch documents before making use of this document.
use of this document.
This document uses the definition format described in RFC 1242, This document uses the definition format described in RFC 1242,
Section 2. The sections in each definition are: definition, Section 2. The sections in each definition are: definition,
discussion, measurement units (optional), issues (optional), and discussion, measurement units (optional), issues (optional), and
cross-references. cross-references.
3. Term definitions 3. Term definitions
3.1 Allowed traffic 3.1 Allowed traffic
skipping to change at line 106 skipping to change at line 110
Packets forwarded as a result of the rule set of the DUT/SUT. Packets forwarded as a result of the rule set of the DUT/SUT.
Discussion: Discussion:
Firewalls typically are configured to forward only those packets Firewalls typically are configured to forward only those packets
explicitly permitted in the rule set. Forwarded packets MUST be included explicitly permitted in the rule set. Forwarded packets MUST be included
in calculating the forwarding rate or maximum forwarding rate of the in calculating the forwarding rate or maximum forwarding rate of the
DUT/SUT. All other packets MUST NOT be included in forwarding rate DUT/SUT. All other packets MUST NOT be included in forwarding rate
calculations. calculations.
Measurement units: Measurement units:
Not applicable not applicable
Issues:
Newman et al. Page [2] Newman et al. Page [2]
Issues:
See also: See also:
policy (3.12) policy
rule set (3.15) rule set
3.2 Authentication 3.2 Authentication
Definition: Definition:
The process of verifying that a client user or machine requesting a The process of verifying that a user requesting a network resource is
network resource is who he, she, or it claims to be, and vice versa. who he, she, or it claims to be, and vice versa.
Discussion: Discussion:
Trust is a critical concept in network security. Any network resource Trust is a critical concept in network security. Any network resource
(such as a file server or printer) with restricted access MUST require (such as a file server or printer) with restricted access MUST require
authentication before granting access. authentication before granting access.
Authentication takes many forms, including but not limited to IP Authentication takes many forms, including but not limited to IP
addresses; TCP or UDP port numbers; passwords; external token addresses; TCP or UDP port numbers; passwords; external token
authentication cards; and biometric identification such as signature, authentication cards; and biometric identification such as signature,
speech, or retina recognition systems. speech, or retina recognition systems.
Authentication MAY work either by client machine (for example, by The entity being authenticated MAY be the client machine (for example,
proving that a given IP source address really is that address, and not a by proving that a given IP source address really is that address, and
rogue machine spoofing that address) or by user (by proving that the not a rogue machine spoofing that address) or a user (by proving that
user really is who he or she claims to be). Servers SHOULD also the user really is who he, she, or it claims to be). Servers SHOULD also
authenticate themselves to clients. authenticate themselves to clients.
Measurement units:
Not applicable
Issues:
Testers should be aware that in an increasingly mobile society, Testers should be aware that in an increasingly mobile society,
authentication based on machine-specific criteria such as an IP address authentication based on machine-specific criteria such as an IP address
or port number is not equivalent to verifying that a given individual is or port number is not equivalent to verifying that a given individual is
making an access request. At this writing systems that verify the making an access request. At this writing systems that verify the
identity of persons are typically external to the firewall, and may identity of users are typically external to the firewall, and may
introduce additional latency to the overall SUT. introduce additional latency to the overall SUT.
Measurement units:
not applicable
Issues:
See also: See also:
user (3.20) user
3.3 Data source 3.3 Connection
Definition: Definition:
A station capable of generating traffic to the DUT/SUT. A logical path established between two hosts, or between a host and the
Discussion:
One data source MAY emulate multiple users or stations. In addition, one
data source MAY offer traffic to multiple network interfaces on the
DUT/SUT. DUT/SUT.
Measurement units: Discussion:
Not applicable The number of concurrent connections a firewall can support is just as
important a metric for some users as maximum forwarding rate.
Issues: Connections MAY be TCP sessions, but they don't have to be. Users of
Newman et al. Page [3] Newman et al. Page [3]
The term "data source" is deliberately independent of any number of other connection-oriented protocols such as ATM may wish to use other
users. It is useful to think of data sources simply as traffic definitions of a connection, either instead of or in addition to TCP
generators, and not as a given number of users. connections.
What constitutes a connection depends on the application. For a "native
ATM" application like a video stream, connections and VCs can be
synonymous. For TCP/IP applications on ATM networks (where multiple TCP
sockets may ride over a single ATM virtual circuit), TCP sockets and
connections are synonymous.
Additionally, in some cases firewalls may handle a mixture of native TCP
and native ATM connections. In this situation, the wrappers around user
data will differ. The most meaningful metric describes what an end-user
will see.
Data connections describe state, not data transfer. The existence of a
connection does NOT imply that data travels on that connection at any
given time.
A firewall's architecture dictates where a connection is terminated. In
the case of proxy-based systems, a connection by definition terminates
at the DUT/SUT. But firewalls using packet filtering or stateful packet
filtering designs act only as passthrough devices, in that they reside
between two connection endpoints. Regardless of firewall architecture,
the number of data connections is still relevant, since all firewalls
perform some form of connection maintenance; at the very least, all
check connection requests against their rule sets.
Measurement units:
Maximum number of connections
Issues:
proxy-based vs. stateful packet filtering
TCP/IP vs. ATM
See also: See also:
data connection (3.4) data source
session
3.4 Data connection 3.4 Data source
Definition: Definition:
A logical link established between two hosts, or between a host and the A station capable of generating traffic to the DUT/SUT.
DUT/SUT.
Discussion: Discussion:
The number of concurrent data connections a firewall can field may be One data source MAY emulate multiple users or stations. In addition, one
just as important a metric for some users as the rate at which it can data source MAY offer traffic to multiple network interfaces on the
forward traffic. Data connections MAY be TCP sessions, but they don't DUT/SUT.
have to be. Users of other connection-oriented protocols such as ATM may
wish to measure these, either instead of or in addition to TCP
connections.
Measurement units: Measurement units:
Number of connections not applicable
Issues: Issues:
A firewall's architecture dictates where the connection is terminated. The term "data source" is deliberately independent of any number of
In the case of proxy-based systems, a connection by definition
terminates at the DUT/SUT. But firewalls using packet filtering or Newman et al. Page [4]
stateful inspection designs act only as passthrough devices, in that
they reside between two connection endpoints. Regardless of firewall users. It is useful to think of data sources simply as traffic
architecture, the number of data connections is still relevant, since generators, without any correlation to any given number of users.
all firewalls perform some form of connection maintenance; at the very
least, all check connection requests against their rule sets.
See also: See also:
data source (3.3) connection
3.5 Demilitarized zone (DMZ) 3.5 Demilitarized zone (DMZ)
Definition: Definition:
A network segment or segments located between protected and external A network segment or segments located between protected and unprotected
networks. DMZ networks are sometimes called perimeter networks. networks. DMZ networks are sometimes called perimeter networks.
Discussion: Discussion:
As an extra security measure, networks are often designed such that As an extra security measure, networks are often designed such that
protected and external segments are never directly connected. Instead, protected and unprotected segments are never directly connected.
firewalls (and possibly public resources such as WWW or FTP servers) Instead, firewalls (and possibly public resources such as WWW or FTP
often reside on the so-called DMZ network. To connect protected, DMZ, servers) often reside on the so-called DMZ network. To connect
and external networks with one device, the device MUST have at least protected, DMZ, and unprotected networks with one device, the device
three network interfaces. MUST have at least three network interfaces.
Multiple firewalls MAY bound the DMZ. In this case, the firewalls Multiple firewalls MAY bound the DMZ. In this case, the firewalls
connecting the protected network with the DMZ and the DMZ with the connecting the protected network with the DMZ and the DMZ with the
external network MUST each have at least two network interfaces. unprotected network MUST each have at least two network interfaces.
Newman et al. Page [4]
Measurement units: Measurement units:
Not applicable not applicable
Issues: Issues:
Dual-homed
Homed Homed
See also: See also:
external network (3.8) unprotected network
perimeter network (3.11) perimeter network
protected network (3.13) protected network
3.6 Dual-homed 3.6 Dynamic proxy
Definition: Definition:
A firewall with at least two network interfaces. A proxy service that is set up and torn down in response to a client
request, rather than existing on a static basis.
Discussion: Discussion:
Dual-homed firewalls connect two segments with different network Proxy services typically "listen" on a given TCP port number for client
addresses. requests. With static proxies, a firewall always forwards packets
containing a given TCP port number if that port number is permitted by
the rule set. Dynamic proxies, in contrast, forward TCP packets only
once an authenticated connection has been established. When the
connection closes, a firewall using dynamic proxies rejects individual
packets, even if they contain port numbers allowed by a rule set.
Measurement units: Measurement units:
Not applicable not applicable
Issues: Issues:
Typically the differentiator between one segment and another is its IP
address. However, firewalls may connect different networks of other Newman et al. Page [5]
types, such as ATM or Netware segments.
rule sets
See also: See also:
Homed (3.9) allowed traffic
Tri-homed (3.19) proxy
rejected traffic
rule set
3.7 Dynamic proxy 3.7 Firewall
Definition: Definition:
A proxy service that is set up and torn down in response to a client A device or group of devices that enforces an access control policy
request, rather than existing on a static basis. between networks.
Discussion: Discussion:
Proxy services (see section 3.14) typically "listen" on a given TCP port While there are many different ways to accomplish it, all firewalls do
number for client requests. With static proxies, a firewall always the same thing: control access between networks.
forwards packets containing a given TCP port number if that port number
is permitted by the rule set. Dynamic proxies, in contrast, forward TCP The most common configuration involves a firewall connecting two
packets only once an authenticated connection has been established. When segments (one protected and one unprotected), but this is not the only
the connection closes, a firewall using dynamic proxies rejects possible configuration. Many firewalls support tri-homing, allowing use
individual packets, even if they contain port numbers allowed by a rule of a DMZ network. It is possible for a firewall to accommodate more than
set. three interfaces, each attached to a different network segment.
The criteria by which access is controlled is deliberately not specified
here. Typically this has been done using network- or transport-layer
criteria (such as IP subnet or TCP port number), but there is no reason
this must always be so. A growing number of firewalls are controlling
access at the application layer, using user identification as the
criterion. And firewalls for ATM networks may control access based on
data link-layer criteria.
Measurement units: Measurement units:
Not applicable not applicable
Issues: Issues:
Newman et al. Page [5] See also:
DMZ
tri-homed
user
rule sets 3.8 Forwarding rate
Definition:
The number of bits per second that a firewall can be observed to
transmit successfully to the correct destination interface in response
to a specified offered load.
Discussion:
This definition differs substantially from section 3.17 of RFC 1242 and
section 3.6.1 of RFC 2285. Unlike RFC 1242, there is no reference to
lost or retransmitted data. Forwarding rate is assumed to be a goodput
measurement, in that only data successfully forwarded to the destination
Newman et al. Page [6]
interface is measured. Forwarding rate MUST be measured in relation to
the offered load. Forwarding rate MAY be measured with differed load
levels, traffic orientation, and traffic distribution.
Unlike RFC 2285, this measurement counts bits per second rather than
frames per second. Per-frame metrics are not meaningful in the context
of a flow of application data between endpoints.
Units of measurement:
bits per second
Issues:
Allowed traffic vs. rejected traffic
See also: See also:
allowed traffic (3.1) allowed traffic
proxy (3.14) goodput
rejected traffic (3.15) rejected traffic
rule set (3.16)
3.8 External network 3.9 Goodput
Definition: Definition:
The segment or segments not protected by the network security DUT/SUT. The number of bits per unit of time forwarded to the correct destination
interface of the DUT/SUT, minus any bits lost or retransmitted.
Discussion: Discussion:
Firewalls are deployed between protected and unprotected segments. The Firewalls are generally insensitive to packet loss in the network. As
external network is not protected by the DUT/SUT. such, measurements of gross forwarding rates are not meaningful since
(in the case of proxy-based and stateful packet filtering firewalls) a
receiving endpoint directly attached to a DUT/SUT would not receive any
data dropped by the DUT/SUT.
Measurement units: The type of traffic lost or retransmitted is protocol-dependent. TCP and
Not applicable ATM, for example, request different types of retransmissions. Testers
MUST observe retransmitted data for the protocol in use, and subtract
this quantity from measurements of gross forwarding rate.
Unit of measurement:
bits per second
Issues: Issues:
allowed vs. rejected traffic
See also: See also:
demilitarized zone (DMZ) (3.5) allowed traffic
protected network (3.13) forwarding rate
rejected traffic
3.9 Homed 3.10 Homed
Definition: Definition:
The number of logical interfaces a DUT/SUT contains. The number of logical interfaces a DUT/SUT contains.
Discussion: Discussion:
Firewalls MUST contain at least two interfaces, using a dual-homed
configuration. In network topologies where a DMZ is used, the firewall
contains at least three interfaces and is said to be tri-homed.
Additional interfaces would make a firewall quad-homed, quint-homed, and
so on.
Issues: Newman et al. Page [7]
Firewalls MUST contain at least two logical interfaces. In network
topologies where a DMZ is used, the firewall contains at least three
interfaces and is said to be tri-homed. Additional interfaces would make
a firewall quad-homed, quint-homed, and so on.
It is theoretically possible for a firewall to contain one physical It is theoretically possible for a firewall to contain one physical
interface and multiple logical interfaces. This configuration is interface and multiple logical interfaces. This configuration is
strongly discouraged for testing purposes because of the possibility of strongly discouraged for testing purposes because of the difficulty in
leakage between protected and unprotected segments. verifying that no leakage occurs between protected and unprotected
segments.
Measurement units:
not applicable
Issues:
See also: See also:
Dual-homed (3.6) tri-homed
Tri-homed (3.19)
3.10 Packet filtering 3.11 Logging
Definition:
The recording of user requests made to the firewall.
Discussion:
Firewalls SHOULD log all requests they handle, both allowed and
rejected. For many firewall designs, logging requires a significant
amount of processing overhead, especially when complex rule sets are in
use.
The type and amount of data logged varies by implementation. Testers
SHOULD attempt to log equivalent data when comparing different DUT/SUTs.
Logging MAY take place on systems other than the DUT/SUT.
Measurement units:
not applicable
Issues:
rule sets
See also:
allowed traffic
connection
rejected traffic
session
3.12 Network address translation (NAT)
Definition:
A method of mapping one or more private, reserved IP addresses to one or
more public IP addresses.
Discussion:
Newman et al. Page [8]
In the interest of conserving the IPv4 address space, RFC 1918 proposed
the use of certain private (reserved) blocks of IP addresses.
Connections to public networks are made by use of a device that
translates one or more RFC 1918 addresses to one or more public
addresses--a network address translator (NAT).
The use of private addressing also introduces a security benefit in that
RFC 1918 addresses are not visible to hosts on the public Internet.
Some NAT implementations are computationally intensive, and may affect
forwarding rate.
Measurement units:
not applicable
Issues:
See also:
3.13 Packet filtering
Definition: Definition:
The process of controlling access by examining packets based on packet The process of controlling access by examining packets based on packet
header content. header content.
Newman et al. Page [6]
Discussion: Discussion:
Packet-filtering devices forward or deny packets based on information in Packet-filtering devices forward or deny packets based on information in
each packet's header, such as IP address or TCP port number. A packet- each packet's header, such as IP address or TCP port number. A packet-
filtering firewall uses a rule set (see section 3.16) to determine which filtering firewall uses a rule set to determine which traffic should be
traffic should be forwarded and which should be blocked. forwarded and which should be blocked.
Measurement units: Measurement units:
Not applicable not applicable
Issues: Issues:
static versus stateful packet filtering
See also: See also:
dynamic proxy (3.7) dynamic proxy
proxy (3.14) proxy
rule set (3.16) rule set
stateful inspection (3.18) stateful packet filtering
3.11 Perimeter network 3.14 Perimeter network
Definition: Definition:
A network segment or segments located between protected and external A network segment or segments located between protected and unprotected
networks. Perimeter networks are often called DMZ networks. networks. Perimeter networks are often called DMZ networks.
Discussion: Discussion:
See the definition of DMZ for a discussion. See the definition of DMZ for a discussion.
Measurement units: Measurement units:
Not applicable not applicable
Newman et al. Page [9]
Issues: Issues:
Dual-homed
Tri-homed Tri-homed
See also: See also:
Demilitarized zone (DMZ) (3.5) demilitarized zone (DMZ)
external network (3.8) unprotected network
protected network (3.13) protected network
3.12 Policy 3.15 Policy
Definition: Definition:
A document defining acceptable use of protected, DMZ, and external A document defining acceptable access to protected, DMZ, and unprotected
networks. networks.
Discussion: Discussion:
Security policies generally do not spell out specific configurations for Security policies generally do not spell out specific configurations for
firewalls; rather, they set general guidelines for what it and is not firewalls; rather, they set general guidelines for what is and is not
acceptable network behavior. acceptable network access.
The actual mechanism for controlling access is usually the rule set (see The actual mechanism for controlling access is usually the rule set
section 3.16) implemented in the DUT/SUT. implemented in the DUT/SUT.
Measurement units: Measurement units:
Not applicable not applicable
Newman et al. Page [7]
Issues: Issues:
See also: See also:
Rule set (3.16) rule set
3.13 Protected network 3.16 Protected network
Definition: Definition:
A network segment or segments to which access is controlled by the A network segment or segments to which access is controlled by the
DUT/SUT. DUT/SUT.
Discussion: Discussion:
Firewalls are intended to prevent unauthorized access either to or from Firewalls are intended to prevent unauthorized access either to or from
the protected network. Depending on the configuration specified by the the protected network. Depending on the configuration specified by the
policy and rule set, the DUT/SUT may allow stations on the protected policy and rule set, the DUT/SUT may allow stations on the protected
segment to act as clients for servers on either the DMZ or the external segment to act as clients for servers on either the DMZ or the
network, or both. unprotected network, or both.
Protected networks are often called "internal networks." That term is Protected networks are often called "internal networks." That term is
not used here because firewalls increasingly are deployed within an not used here because firewalls increasingly are deployed within an
organization, where all segments are by definition internal. organization, where all segments are by definition internal.
Measurement units: Measurement units:
Not applicable not applicable
Issues: Issues:
See also: See also:
Demilitarized zone (DMZ) (3.5)
external network (3.8)
policy (3.12)
rule set (3.16)
3.14 Proxy Newman et al. Page [10]
demilitarized zone (DMZ)
unprotected network
policy
rule set
unprotected network
3.17 Proxy
Definition: Definition:
A request for a connection made on behalf of a host. A request for a connection made on behalf of a host.
Discussion: Discussion:
Proxy-based firewalls never allow direct connections between hosts. Proxy-based firewalls do not allow direct connections between hosts.
Instead, two connections are established: one between the client host Instead, two connections are established: one between the client host
and the DUT/SUT, and another between the DUT/SUT and server host. and the DUT/SUT, and another between the DUT/SUT and server host.
As with packet-filtering firewalls, proxy-based devices use a rule set As with packet-filtering firewalls, proxy-based devices use a rule set
to determine which traffic should be forwarded and which should be to determine which traffic should be forwarded and which should be
rejected. rejected.
Proxies are generally application-specific.
Measurement units: Measurement units:
Not applicable not applicable
Issues: Issues:
application
See also: See also:
dynamic proxy
packet filtering
stateful packet filtering
Newman et al. Page [8] 3.18 Rejected traffic
dynamic proxy (3.7)
packet filtering (3.10)
stateful inspection (3.18)
3.15 Rejected traffic
Definition: Definition:
Packets dropped as a result of the rule set of the DUT/SUT. Packets dropped as a result of the rule set of the DUT/SUT.
Discussion: Discussion:
Firewalls MUST reject any traffic not explicitly permitted in the rule Firewalls MUST reject any traffic not explicitly permitted in the rule
set. Dropped packets MUST NOT be included in calculating the forwarding set. Dropped packets MUST NOT be included in calculating the forwarding
rate or maximum forwarding rate of the DUT/SUT. rate or maximum forwarding rate of the DUT/SUT.
Measurement units: Measurement units:
Not applicable not applicable
Issues: Issues:
See also: See also:
policy (3.12) policy
rule set (3.16) rule set
3.16 Rule set 3.19 Rule set
Newman et al. Page [11]
Definition: Definition:
The collection of access control rules that determines which packets the The collection of access control rules that determines which packets the
DUT/SUT will forward and which it will reject. DUT/SUT will forward and which it will reject.
Discussion: Discussion:
Rule sets control access to and from the network interfaces of the Rule sets control access to and from the network interfaces of the
DUT/SUT. By definition, rule sets MUST NOT apply equally to all network DUT/SUT. By definition, rule sets MUST NOT apply equally to all network
interfaces; otherwise there would be no need for the firewall. interfaces; otherwise there would be no need for the firewall.
Therefore, a specific rule set MUST be applied to each network interface Therefore, a specific rule set MUST be applied to each network interface
skipping to change at line 497 skipping to change at line 646
The order of rules within the rule set is critical. Firewalls generally The order of rules within the rule set is critical. Firewalls generally
scan rule sets in a "top down" fashion, which is to say that the device scan rule sets in a "top down" fashion, which is to say that the device
compares each packet received with each rule in the rule set until it compares each packet received with each rule in the rule set until it
finds a rule that applies to the packet. Once the device finds an finds a rule that applies to the packet. Once the device finds an
applicable rule, it applies the actions defined in that rule (such as applicable rule, it applies the actions defined in that rule (such as
forwarding or rejecting the packet) and ignores all subsequent rules. forwarding or rejecting the packet) and ignores all subsequent rules.
For testing purposes, the rule set MUST conclude with a rule denying all For testing purposes, the rule set MUST conclude with a rule denying all
access. access.
Measurement units: Measurement units:
Not applicable not applicable
Issues: Issues:
See also: See also:
Demilitarized zone (DMZ) (3.5) demilitarized zone (DMZ)
external network (3.8) policy
policy (3.12) protected network
rejected traffic
Newman et al. Page [9] unprotected network
protected network (3.13)
rejected traffic (3.15)
3.17 Session 3.20 Session
Definition: Definition:
A logical connection established between two stations using a known Data flowing through a previously established connection established
protocol. between two stations using a known protocol.
Discussion: Discussion:
Because of the application-layer focus of many firewalls, sessions are a Because of the application-layer focus of many firewalls, sessions are a
more useful metric than the packet-based measurements used in more useful metric than the packet-based measurements used in
benchmarking routers and switches. Although firewall rule sets generally benchmarking routers and switches. Although firewall rule sets generally
work on a per-packet basis, it is ultimately sessions that a firewall work on a per-packet basis, it is ultimately sessions that a firewall
must handle. For example, the number of file transfer protocol (ftp) must handle. For example, the number of file transfer protocol (ftp)
sessions a DUT/SUT can handle concurrently is a more meaningful sessions a DUT/SUT can handle concurrently is a more meaningful
measurement in benchmarking performance than the number of ftp "open" measurement in benchmarking performance than the number of ftp "open"
packets it can reject. Further, a stateful inspection firewall will not packets it can reject. Further, a stateful packet filtering firewall
forward individual packets if those packets' headers conflict with state will not forward individual packets if those packets' headers conflict
information maintained by the firewall. with state information maintained by the firewall.
For purposes of this document, a session MUST be established using a For purposes of this document, a session MUST be established using a
known protocol such as TCP. A traffic pattern is not considered a known protocol such as TCP. A traffic pattern is not considered a
session until it successfully completes the establishment procedures session until it successfully completes the establishment procedures
defined by that protocol. defined by that protocol.
Newman et al. Page [12]
Also for purposes of this document, a session constitutes the logical Also for purposes of this document, a session constitutes the logical
connection between two end-stations and not the intermediate connections connection between two end-stations and not the intermediate connections
that proxy-based firewalls may use. that proxy-based firewalls may use.
Issues: Issues:
TCP/IP vs. ATM
See also: See also:
policy (3.12) connection
proxy (3.14) policy
rule set (3.16) proxy
stateful inspection (3.18) rule set
stateful packet filtering
3.18 Stateful inspection 3.21 Stateful packet filtering
Definition: Definition:
The process of forwarding or rejecting traffic based on the contents of The process of forwarding or rejecting traffic based on the contents of
a state table maintained by a firewall. a state table maintained by a firewall.
Discussion: Discussion:
Packet filtering and proxy firewalls are essentially static, in that Packet filtering and proxy firewalls are essentially static, in that
they always forward or reject packets based on the contents of the rule they always forward or reject packets based on the contents of the rule
set.
In contrast, devices using stateful inspection will only forward packets set.
if they correspond with state information maintained by the device about
each session. For example, a stateful inspection device will reject a
packet on port 20 (ftp-data) if no session has been established over the
ftp control port (usually port 21).
Newman et al. Page [10] In contrast, devices using stateful packet filtering will only forward
packets if they correspond with state information maintained by the
device about each session. For example, a stateful packet filtering
device will reject a packet on port 20 (ftp-data) if no session has been
established over the ftp control port (usually port 21).
Measurement units: Measurement units:
Not applicable not applicable
Issues: Issues:
See also: See also:
dynamic proxy (3.7) dynamic proxy
packet filter (3.10) packet filter
proxy (3.14) proxy
3.19 Tri-homed 3.22 Tri-homed
Definition: Definition:
A firewall with three network interfaces. A firewall with three network interfaces.
Discussion: Discussion:
Tri-homed firewalls connect three network segments with different Tri-homed firewalls connect three network segments with different
network addresses. Typically, these would be protected, DMZ, and network addresses. Typically, these would be protected, DMZ, and
external segments. unprotected segments.
A tri-homed firewall may offer some security advantages over firewalls
with two interfaces. An attacker on an unprotected network may
Newman et al. Page [13]
compromise hosts on the DMZ but still not reach any hosts on the
protected network.
Measurement units: Measurement units:
Not applicable not applicable
Issues: Issues:
Usually the differentiator between one segment and another is its IP Usually the differentiator between one segment and another is its IP
address. However, firewalls may connect different networks of other address. However, firewalls may connect different networks of other
types, such as ATM or Netware segments. types, such as ATM or Netware segments.
See also: See also:
Dual-homed (3.6) homed
Homed (3.9)
3.20 User 3.23 Unprotected network
Definition: Definition:
The person or machine requesting access to resources protected by the A network segment or segments to which access is not controlled by the
DUT/SUT.
Discussion:
Firewalls are deployed between protected and unprotected segments. The
unprotected network is not protected by the DUT/SUT.
Note that a DUT/SUT's policy MAY specify hosts on an unprotected
network. For example, a user on a protected network may be permitted to
access an FTP server on an unprotected network. But the DUT/SUT cannot
control access between hosts on the unprotected network.
Measurement units:
not applicable
Issues:
See also:
demilitarized zone (DMZ)
policy
protected network
rule set
3.24 User
Definition:
A person or process requesting access to resources protected by the
DUT/SUT. DUT/SUT.
Discussion: Discussion:
"User" is a problematic term in the context of firewall performance "User" is a problematic term in the context of firewall performance
testing, for several reasons. First, a user may in fact be a machine or testing, for several reasons. First, a user may in fact be a process or
machines requesting services through the DUT/SUT. Second, different processes requesting services through the DUT/SUT. Second, different
"user" requests may require radically different amounts of DUT/SUT "user" requests may require radically different amounts of DUT/SUT
resources. Third, traffic profiles vary widely from one organization to resources. Third, traffic profiles vary widely from one organization to
another, making it difficult to characterize the load offered by a another, making it difficult to characterize the load offered by a
typical users. typical user.
Newman et al. Page [14]
For these reasons, we prefer not to measure DUT/SUT performance in terms For these reasons, we prefer not to measure DUT/SUT performance in terms
of users supported. Instead, we describe performance in terms of maximum of users supported. Instead, we describe performance in terms of maximum
forwarding rate and maximum number of sessions sustained. Further, we forwarding rate and maximum number of sessions sustained. Further, we
use the term "data source" rather than user to describe the traffic use the term "data source" rather than user to describe the traffic
generator(s). generator(s).
Newman et al. Page [11]
Measurement units: Measurement units:
Not applicable not applicable
Issues: Issues:
See also: See also:
data source (3.3) data source
4. Security considerations 4. Security considerations
Security considerations are explicitly excluded from this memo. The
authors plan to address security and management concerns in a separate The primary goal of this memo is to describe terms used in measuring
proposal brought to the IETF's security directorate. firewall performance. However, readers should be aware that there is
some overlap between performance and security issues. Readers should be
aware that the optimal configuration for firewall performance may not be
the most secure, and vice-versa.
Further, certain forms of attack may degrade performance. One common
form of denial-of-service (DoS) attack bombards a firewall with so much
rejected traffic that it cannot forward allowed traffic. DoS attacks do
not always involve heavy loads; by definition, DoS describes any state
in which a firewall is offered rejected traffic that prohibits it from
forwarding some or all allowed traffic. Even a small amount of traffic--
such as the recent Teardrop2 attack involving a few packet fragments--
may significantly degrade firewall performance, or stop the firewall
altogether.
5. References 5. References
Bradner, S., editor. "Benchmarking Terminology for Network Bradner, S., editor. "Benchmarking Terminology for Network
Interconnection Devices." RFC 1242. Interconnection Devices." RFC 1242.
Bradner, S., and McQuaid, J. "Benchmarking Methodology for Network Bradner, S., and McQuaid, J. "Benchmarking Methodology for Network
Interconnect Devices." RFC 1944. Interconnect Devices." RFC 1944.
Mandeville, B. "Benchmarking Terminology for LAN Switching Devices." Mandeville, R. "Benchmarking Terminology for LAN Switching Devices." RFC
ftp://ietf.org/internet-drafts/draft-ietf-bmwg-lanswitch-07.txt 2285.
Newman, D., and Melson, B. "Can Firewalls Take the Heat?" Data Rekhter, Y., et al. "Address Allocation for Private Internets." RFC
Communications, November 21, 1995. 1918.
http://www.data.com/Lab_Tests/Firewalls.html
Newman, D., Holzbaur, H., and Bishop, K. "Firewalls: Don't Get Burned," 6. Acknowledgments
Data Communications, March 21, 1997.
http://www.data.com/lab_tests/firewalls97.html
Ranum, M. "Firewall Performance Measurement Techniques: A Scientific The authors wish to thank the IETF Benchmarking Working Group for
Approach." http://www.clark.net/pub/mjr/pubs/fwperf/intro.htm agreeing to review this document. Several other persons offered valuable
contributions and critiques during this project: Ted Doty (Internet
Security Systems), Shlomo Kramer (Check Point Software Technologies),
Robert Mandeville (European Network Laboratories), Brent Melson
Shannon, G. "Profile of Corporate Internet Application Traffic." Newman et al. Page [15]
http://www.milkyway.com/libr/prof.html
6. Acknowledgments (National Software Testing Laboratories), Marcus Ranum (Network Flight
The authors wish to thank the IETF Benchmarking Working Group for Recorder Inc.), Greg Shannon (Ascend Communications), Christoph Schuba
agreeing to review this document. Ted Doty (Internet Security Systems), (Sun Microsystems), Rick Siebenaler (Cyberguard), and Greg Smith (Check
Shlomo Kramer (Check Point Software Technologies), Bob Mandeville Point Software Technologies).
(European Network Laboratories), Brent Melson (National Software Testing
Laboratories), Marcus Ranum (Network Flight Recorder Inc.), Greg Shannon 7. Contact information
(Ascend Communications), Rick Siebenaler (Cyberguard), and Greg Smith
(Check Point Software Technologies) offered valuable contributions and
critiques during this project.
7. Contact Information
David Newman David Newman
Data Communications magazine Data Communications magazine
1221 Avenue of the Americas, 41st Floor 1221 Avenue of the Americas, 41st Floor
New York, NY 10020 New York, NY 10020
USA USA
Newman et al. Page [12]
212-512-6182 voice 212-512-6182 voice
212-512-6833 fax 212-512-6833 fax
dnewman@data.com dnewman@data.com
Helen Holzbaur Helen Holzbaur
National Software Testing Laboratories Inc. National Software Testing Laboratories Inc.
625 Ridge Pike 625 Ridge Pike
Conshohocken, PA 19428 Conshohocken, PA 19428
USA USA
helen@nstl.com helen@nstl.com
skipping to change at line 702 skipping to change at line 889
USA USA
jimh@nstl.com jimh@nstl.com
Steven Platt Steven Platt
National Software Testing Laboratories Inc. National Software Testing Laboratories Inc.
625 Ridge Pike 625 Ridge Pike
Conshohocken, PA 19428 Conshohocken, PA 19428
USA USA
steve@nstl.com steve@nstl.com
Newman et al. Page [13] Newman et al. Page [16]
 End of changes. 

This html diff was produced by rfcdiff 1.23, available from http://www.levkowetz.com/ietf/tools/rfcdiff/