Benchmarking
Network Working Group                                         D. Newman
INTERNET-DRAFT                                      Data Communications
Expires in January May 1998 	             H. Holzbaur, J. Hurd, and S. Platt
                                 National Software Testing Laboratories

           Benchmarking Terminology for Network Security Devices
                           <draft-ietf-bmwg-secperf-00.txt> Firewall Performance
                    <draft-ietf-bmwg-secperf-01.txt>

Status of This Memo

   This document is an Internet-Draft.  Internet-Drafts are working
   documents of the Internet Engineering Task Force (IETF), its areas,
   and its working groups.  Note that other groups may also distribute
   working documents as Internet-Drafts.

   Internet-Drafts are draft documents valid for a maximum of six months
   and may be updated, replaced, or obsoleted by other documents at any
   time.  It is inappropriate to use Internet- Drafts as reference
   material or to cite them other than as "work in progress."

   To view the entire list of current Internet-Drafts, please check the
   "1id-abstracts.txt" listing contained in the Internet-Drafts Shadow
   Directories on ftp.is.co.za (Africa), ftp.nordu.net (Europe),
   munnari.oz.au (Pacific Rim), ds.internic.net (US East Coast), or
   ftp.isi.edu (US West Coast).

1. Introduction .......................................................2
2. Existing definitions ...............................................2
3. Term definitions ...................................................2
     3.1 Allowed traffic ..............................................2
     3.2 Authentication ...............................................3
     3.3 Data source ..................................................3
     3.4 Data connection ..............................................4
     3.5 Demilitarized zone (DMZ) .....................................4
     3.6 Dual-homed ...................................................5
     3.7 Dynamic proxy ................................................5
     3.8 External network .............................................6
     3.9 Homed ........................................................6
     3.10 Packet filtering ............................................6
     3.11 Perimeter network ...........................................7
     3.12 Policy ......................................................7
     3.13 Protected network ...........................................8
     3.14 Proxy .......................................................8
     3.15 Rejected traffic ............................................9
     3.16 Rule set ....................................................9
     3.17 Session .....................................................9
     3.18 Stateful inspection ........................................10
     3.19 Tri-homed ..................................................11
     3.20 User .......................................................11
4.  Security considerations ..........................................11
5. References ........................................................12
6. Acknowledgments ...................................................12
7. Contact Information ...............................................12

1. Introduction
   This memo provides information for document defines terms used in measuring the Internet community.  This
         memo does not specify an Internet standard of any kind.
         Distribution performance of
   firewalls. It extends the terminology already used for benchmarking
   routers and switches and adds terminology specific to firewalls. The
   primary metrics defined in this memo is unlimited.

         1. Introduction
         Despite document are maximum forwarding rate
   and maximum number of connections.

   Why are firewall performance measurements needed? First, despite the
   rapid rise in deployment of network security devices
         such as firewalls and authentication/encryption products, firewalls, there is no standard method
   for evaluating the performance of these
         devices.

         The lack of a standard is troubling for two reasons. First,
         hardware and software benchmarking their performance. Second, implementations vary
   widely, making it difficult to do direct performance comparisons. Second, a growing
         number of
   Finally, more and more organizations are deploying these devices firewalls on
   internal networks that operate operating at relatively high data rates, speeds, while many
         network security devices are most
   firewall implementations remain optimized for use over relatively low-speed
   wide-area connections. As a result, users are often unsure whether
   the products they buy will stand up to the relatively heavy loads found on internal networks.

         This document defines terms used in measuring the performance of
         network security devices. It extends the loads.

   We may also create additional terminology already used
         for benchmarking routers and switches methodology documents
   to define other types of network security devices.
         The primary metrics defined in this document are maximum
         forwarding rate products such as virtual
   private network (VPN) and maximum number of connections.

         Depending encryption devices. This document, however,
   focuses solely on the outcome of discussions within the BMWG, we may
         also attempt to classify devices using various architectural
         considerations (proxy, packet filter) or offered load levels

         Newman et al                                             [Page 1]
         (high, medium, low) as criteria. Additionally, new metrics may
         need to be defined to evaluate application-level issues. firewall terminology.

2. Existing definitions
   This document uses the conceptual framework established in RFCs 1242
   and 1944 (for routers) and draft-ietf-bmwg-lanswitch-07.txt (for
   switches). The router and draft-ietf-bmwg-lanswitch-05.txt, which
         describes benchmarking of LAN switch performance. In addition to defining basic practices, these documents contain discussions of
   several terms relevant to benchmarking the performance of network security devices. firewalls.
   Readers should consult the router and switch documents before making
   use of this document.

   This document uses the definition format described in RFC 1242,
   Section 2. Readers should
         consult these documents before making use of this document. The sections in each definition are: definition,
   discussion, measurement units (optional), issues (optional), and
   cross-references.

3. Term definitions

3.1 Allowed traffic

Definition:
Packets forwarded as a result of the rule set of the DUT/SUT.

Discussion:
Firewalls typically are configured to forward only those packets
explicitly permitted in the rule set. Forwarded packets MUST be included
in calculating the forwarding rate or maximum forwarding rate of the
DUT/SUT. All other packets MUST NOT be included in forwarding rate
calculations.

Measurement units:
Not applicable

Issues:

Newman et al.                                                  Page [2]

See also:
policy (3.12)
rule set (3.15)

3.2 Authentication

Definition:
The process of verifying that a client user or machine requesting a
network resource is who he, she, or it claims to be, and vice versa.

Discussion:
Trust is a critical concept in network security. Obviously, any Any network resource
(such as a file server or printer) with restricted access MUST require
authentication before granting access.

Authentication takes many forms, including but not limited to IP
addresses; TCP or UDP port numbers; passwords; external token
authentication cards; and pattern matching based on human
         characteristics biometric identification such as signature,
speech, or retina patterns. recognition systems.

Authentication MAY work either by client machine (for example, by
proving that a given IP source address really is that address, and not a
rogue machine spoofing that address) or by user (by proving that the
user really is who he or she claims to be). Servers SHOULD also
authenticate themselves to clients.

Measurement units:
Not applicable

Issues:

         See also:
         forwarding rate (3.9)
         user (3.25)
         virtual client (3.26)

         3.2 Bidirectional traffic

         Definition:

         Newman et al                                             [Page 2]
         Packets presented
Testers should be aware that in an increasingly mobile society,
authentication based on machine-specific criteria such as an IP address
or port number is not equivalent to verifying that a DUT/SUT such given individual is
making an access request. At this writing systems that verify the network interfaces
identity of persons are typically external to the DUT/SUT both receive firewall, and transmit traffic.

         Discussion:
         Traffic patterns offered may
introduce additional latency to the DUT/SUT MUST be bidirectional or
         fully meshed. See forwarding rate (3.9) for a more complete
         discussion of issues with traffic patterns.

         Measurement units:
         Not applicable

         Issues:
         truncated binary exponential back-off algorithm overall SUT.

See Also:
         forwarding rate (3.9)
         fully meshed traffic (3.10)
         unidirectional traffic (3.24) also:
user (3.20)

3.3 Data source

Definition:
A station capable of generating traffic to the DUT/SUT.

Discussion:
One data source MAY emulate multiple users or stations. In addition, one
data source MAY offer traffic to multiple network interfaces on the
DUT/SUT. However, each virtual client MUST offer
         traffic to only one interface.

Measurement units:
Not applicable

Issues:

Newman et al.                                                  Page [3]

The term "data source" is deliberately independent of any number of
users. It is useful to think of data sources simply as traffic
generators, and not as a given number of users.

See also:
         user (3.25)
         virtual client (3.26)
data connection (3.4)

3.4 Demilitarized zone Data connection

Definition:
A logical link established between two hosts, or between a host and the
DUT/SUT.

Discussion:
The number of concurrent data connections a firewall can field may be
just as important a metric for some users as the rate at which it can
forward traffic. Data connections MAY be TCP sessions, but they don't
have to be. Users of other connection-oriented protocols such as ATM may
wish to measure these, either instead of or in addition to TCP
connections.

Measurement units:
Number of connections

Issues:
A firewall's architecture dictates where the connection is terminated.
In the case of proxy-based systems, a connection by definition
terminates at the DUT/SUT. But firewalls using packet filtering or
stateful inspection designs act only as passthrough devices, in that
they reside between two connection endpoints. Regardless of firewall
architecture, the number of data connections is still relevant, since
all firewalls perform some form of connection maintenance; at the very
least, all check connection requests against their rule sets.

See also:
data source (3.3)

3.5 Demilitarized zone (DMZ)

Definition:
A network segment or segments located between protected and external
networks. DMZ networks are sometimes called perimeter networks.

Discussion:
As an extra security measure, networks are often designed such that
protected and external segments are never directly connected. Instead, security devices
firewalls (and possibly other public resources such as WWW or FTP servers)
often reside in on the so-called DMZ network. To connect protected, DMZ,
and external networks with one device, the device MUST have at least
three network interfaces.

         Newman et al                                             [Page 3]

Multiple devices firewalls MAY constitute bound the DMZ, in which case DMZ. In this case, the devices
         connected firewalls
connecting the protected network with the DMZ and the DMZ with the
external network MUST each have at least two network interfaces.

Newman et al.                                                  Page [4]

Measurement units:
Not applicable

Issues:
Dual-homed
         Multihomed
Homed

See also:
external network (3.8)
perimeter network (3.15) (3.11)
protected network (3.17)

         3.5 Device under test (DUT)

         Definition:
         The network security device to which traffic is offered and
         response measured.

         Discussion:
         A single station, generally equipped with at least two network
         interfaces.

         Measurement units:
         Not applicable

         Issues:

         See also:
         system under test (SUT) (3.23) (3.13)

3.6 Dual-homed

Definition:
A station firewall with at least two network interfaces.

Discussion:
Dual-homed network security devices firewalls connect two segments with different network-layer network
addresses.

Measurement units:
Not applicable

Issues:
Typically the differentiator between one segment and another is its IP
address. However, firewalls may connect different networks of other
types, such as ATM or Netware segments.

See also:
         multihomed (3.12)
Homed (3.9)
Tri-homed (3.19)

3.7 Dynamic proxy

Definition:

         Newman et al                                             [Page 4]
A proxy service that is set up and torn down in response to a client
request, rather than existing on a static basis.

Discussion:
Proxy services (see section 3.18) 3.14) typically are configured to "listen" on a given TCP port
number for client requests. However, some
         devices set up With static proxies, a proxy service only when firewall always
forwards packets containing a client requests given TCP port number if that port number
is permitted by the
         service.

         Measurement units:
         Not applicable

         Issues: rule sets

         See also:
         proxy (3.18)
         rule sets (3.20)

         3.8 External network

         Definition:
         The segment or segments not protected by the network security
         DUT/SUT.

         Discussion:
         Network security devices are deployed between protected and
         unprotected segments. The external network is not protected by the
         DUT/SUT.

         Measurement units:
         Not applicable

         Issues:

         See also:
         demilitarized zone (DMZ) (3.4)
         protected network (3.17)

         3.9 Forwarding rate

         Definition: The number of bits per second a DUT/SUT can transmit
         to the correct destination network interface set. Dynamic proxies, in response to a
         specified offered load.

         Discussion:
         Network security devices are by definition session-oriented: They
         will contrast, forward TCP
packets only grant access to a desired resource once authentication
         occurs and a session has been established.

         Because application-layer sessions are always involved,
         unidirectional packet-per-second metrics are not meaningful in the
         context of testing network security devices. Instead, this

         Newman et al                                             [Page 5]
         definition MUST measure application-layer performance once a
         session an authenticated connection has been established.

         Forwarding rate refers to the number of bits per second observed
         on the output side of the network interface under test. Forwarding
         rate can be measured with different traffic orientations and
         distributions. When multiple network interfaces are measured,
         measurements MUST be observed from the interface with the highest
         forwarding rate.

         Measurement units:
         bits per second (bit/s)
         kilobits per second (kbit/s)
         Megabits per second (Mbit/s)

         Issues:
         truncated binary exponential back-off algorithm
         unidirectional vs. bidirectional

         See Also:
         authentication (3.1)
         maximum forwarding rate (3.11)
         offered load (3.13)
         unidirectional traffic (3.24)

         3.10 Fully meshed traffic

         Definition:
         Packets forwarded simultaneously among all of a designated number
         of network interfaces of a DUT/SUT such that each of
the
         interfaces under test will both forward packets to and receive
         packets from all of the other interfaces.

         Discussion:
         Fully meshed traffic is the most thorough method of exercising the
         transmitting and receiving capabilities of the DUT/SUT.

         Unlike past definitions for router or switch testing, it should be
         noted that fully meshed traffic in this context is not necessarily
         symmetrical. While all of connection closes, a designated group of network interfaces
         MUST simultaneously send and receive traffic, the type and amount
         of traffic offered MAY differ on each interface. For example, firewall using dynamic proxies rejects
individual packets, even if they contain port numbers allowed by a
         network security device may see more traffic from the protected
         network bound for the external network than the opposite (although
         the inverse could be true during an attack on the DUT/SUT). rule
set.

Measurement units:
Not applicable

Issues:
         Half duplex
         Full duplex

         See also:

Newman et al                                             [Page 6]
         bidirectional traffic (3.2)
         unidirectional traffic (3.24)

         3.11 Maximum forwarding rate

         Definition:
         The highest forwarding rate of a network security device taken
         from a set of iterative measurements.

         Discussion:
         Maximum forwarding rate may degrade before maximum load is
         offered.

         Unlike benchmarks for evaluating router and switch performance,
         this definition MUST involve measurement of application-layer
         performance rather than network-layer packet-per-second metrics.

         Measurement units:
         Megabits per second
         kbytes per second
         bytes per second

         Issues:
         full duplex vs. half duplex
         truncated binary exponential back-off algorithm al.                                                  Page [5]

rule sets

See also:
         bidirectional traffic (3.2)
         partially meshed
allowed traffic (3.24)
         unidirectional traffic

         3.12 Multihomed

         Definition:
         A network security device with more than two network interfaces.

         Discussion:
         Multihoming is a way to connect three or more networks-protected,
         DMZ, and external-with a single (3.1)
proxy (3.14)
rejected traffic (3.15)
rule set (3.16)

3.8 External network security device. However,
         this configuration is

Definition:
The segment or segments not mandatory if multiple protected by the network security
         devices DUT/SUT.

Discussion:
Firewalls are used. For example, one device could secure the
         connection between an external and DMZ network, while another
         could secure the connection deployed between a DMZ and protected network;
         the two stations collectively form the SUT.

         Because of the differences in traffic patterns between dual-homed
         and multihomed devices, direct performance comparisons should be
         avoided. However, it is acceptable to compare results between a
         dual-homed device and a DUT/SUT in which only two unprotected segments. The
external network
         interfaces are used. is not protected by the DUT/SUT.

Measurement units:
Not applicable

         Newman et al                                             [Page 7]

Issues:
         truncated binary exponential back-off algorithm

See also:
         bidirectional traffic (3.2)
         dual-homed (3.6)
         fully meshed traffic (3.10)

         3.13 Offered load
demilitarized zone (DMZ) (3.5)
protected network (3.13)

3.9 Homed

Definition:
The number of bits per second that an external source can transmit
         to logical interfaces a DUT/SUT for forwarding to contains.

Discussion:
Firewalls MUST contain at least two interfaces, using a specified dual-homed
configuration. In network interface or
         interfaces.

         Discussion:
         The load that an external source actually applies to topologies where a DUT/SUT may
         be lower than DMZ is used, the external source attempts firewall
contains at least three interfaces and is said to apply be tri-homed.
Additional interfaces would make a firewall quad-homed, quint-homed, and
so on.

Issues:
It is theoretically possible for a firewall to contain one physical
interface and multiple logical interfaces. This configuration is
strongly discouraged for testing purposes because of
         collisions on the wire. The transmission capabilities possibility of the
         external source SHOULD be verified without the DUT/SUT by
         transmitting unidirectional traffic.

         Measurement units:
         bits per second
         kilobits per second (kbit/s)
         Megabits per second (Mbit/s)

         Issues:
         truncated binary exponential back-off algorithm
leakage between protected and unprotected segments.

See also:
         forwarding rate (3.9)
         maximum forwarding rate (3.11)

         3.14
Dual-homed (3.6)
Tri-homed (3.19)

3.10 Packet filtering

Definition:
The process of controlling access by examining packets based on
         network-layer or transport-layer criteria. packet
header content.

Newman et al.                                                  Page [6]

Discussion:
Packet-filtering devices forward or deny packets based on information in
each packet's header. header, such as IP address or TCP port number. A packet-filtering network
         security device packet-
filtering firewall uses a rule set (see section 3.20) 3.16) to determine which
traffic should be forwarded and which should be blocked.
         Packet filtering may be used in a dual-homed or multihomed device.

Measurement units:
Not applicable

Issues:

See also:
dynamic proxy (3.7)

         Newman et al                                             [Page 8]
proxy (3.18) (3.14)
rule set (3.20) (3.16)
stateful inspection (3.22)

         3.15 (3.18)

3.11 Perimeter network

Definition:
A network segment or segments located between protected and external
networks. Perimeter networks are often called DMZ networks.

Discussion:
See the definition of DMZ (which see) for a discussion.

Measurement units:
Not applicable

Issues:
Dual-homed
         Multihomed
Tri-homed

See also:
Demilitarized zone (DMZ) (3.4) (3.5)
external network (3.8)
protected network (3.17)

         3.16 (3.13)

3.12 Policy

Definition:
A document defining acceptable use of protected, DMZ, and external
networks.

Discussion:
Security policies generally do not spell out specific configurations for network security devices;
firewalls; rather, they set general guidelines for what it and is not
acceptable network behavior.

The actual mechanism for controlling access is usually the rule set (see
section 3.20) 3.16) implemented in the DUT/SUT.

Measurement units:
Not applicable

Newman et al.                                                  Page [7]

Issues:

See also:
Rule set (3.20)

         3.17 (3.16)

3.13 Protected network

Definition:
A network segment or segments to which access is controlled by the
DUT/SUT.

         Newman et al                                             [Page 9]

Discussion:
         Network security devices
Firewalls are intended to prevent unauthorized access either to or from
the protected network. Depending on the configuration specified by the
policy and rule set, the DUT/SUT may allow stations on the protected
segment to act as clients for servers on either the DMZ or the external
network, or both.

Protected networks are often called "internal networks." That term is
not used here because network security devices firewalls increasingly are deployed within an
organization, where all segments are by definition internal.

Measurement units:
Not applicable

Issues:

See also:
Demilitarized zone (DMZ) (3.4) (3.5)
external network (3.8)
policy (3.16) (3.12)
rule set (3.20)

         3.18 (3.16)

3.14 Proxy

Definition:
         The process of requesting sessions with servers
A request for a connection made on behalf of
         clients. a host.

Discussion:
Proxy-based network security devices firewalls never involve allow direct connections between client and server. hosts.
Instead, two sessions connections are established: one between the client host
and the DUT/SUT, and another between the DUT/SUT and server. server host.

As with packet-filtering network security devices, firewalls, proxy-based devices use a rule set (which see)
to determine which traffic should be forwarded and which should be blocked.
rejected.

Measurement units:
Not applicable

Issues:

See also:

Newman et al.                                                  Page [8]

dynamic proxy (3.7)
packet filtering (3.14) (3.10)
stateful inspection (3.22)

         3.19 (3.18)

3.15 Rejected traffic

Definition:
Packets dropped as a result of the rule set of the DUT/SUT.

         Newman et al                                             [Page 10]

Discussion:
         Network security devices typically are configured to drop
Firewalls MUST reject any traffic not explicitly permitted in the rule set (which see).
set. Dropped packets MUST NOT be included in calculating the forwarding
rate or maximum forwarding rate of the DUT/SUT.

Measurement units:
Not applicable

Issues:

See also:
         forwarding rate (3.9)
         maximum forwarding rate (3.11)
policy (3.16) (3.12)
rule set (3.20)

         3.20 (3.16)

3.16 Rule set

Definition:
The collection of definitions access control rules that determines which packets the
DUT/SUT will forward and which it will reject.

Discussion:
Rule sets control access to and from the network interfaces of the
DUT/SUT. By definition, rule sets MUST NOT apply equally to all network
interfaces; otherwise there would be no need for the
         network security device. firewall.
Therefore, a specific rule set MUST be applied to each network device used interface
in the DUT/SUT.

The order of rules within the rule set is critical. Network
         security devices Firewalls generally
scan rule sets in a "top down" fashion, which is to say that the device
compares each packet received with each rule in the rule set until it
finds a rule that applies to the packet. Once the device finds an
applicable rule, it applies the actions defined in that rule (such as
forwarding or rejecting the packet) and ignores all subsequent rules.
For purposes of this
         document, testing purposes, the rule set MUST conclude with a rule denying all
         access except that which is permitted in the rule set.
access.

Measurement units:
Not applicable

Issues:

See also:
Demilitarized zone (DMZ) (3.4) (3.5)
external network (3.8)
policy (3.17) (3.12)

Newman et al.                                                  Page [9]

protected network (3.18) (3.13)
rejected traffic (3.19)

         3.21 (3.15)

3.17 Session

         Newman et al                                             [Page 11]

Definition:
A logical connection established between two stations using a known
protocol. For purposes of this document, a session MUST be
         conducted over either TCP (RFC 793) or UDP (RFC 768).

Discussion:
Because of the application-layer focus of many network security
         devices, firewalls, sessions are a
more useful metric than the packet-based measurements used in
benchmarking routers and switches. Although
         network security device firewall rule sets generally
work on a per-packet basis, it is ultimately sessions that a network security device firewall
must handle. For example, the number of file transfer protocol (ftp)
sessions a DUT/SUT can handle concurrently is a more meaningful
measurement in benchmarking performance than the number of ftp "open"
packets it can reject. Further, a stateful inspection device (which see) firewall will not
forward individual packets if those packets' headers conflict with state
information maintained in by the device's rule set. firewall.

For purposes of this document, a session MUST be established using a
known protocol. protocol such as TCP. A traffic pattern is not considered a
session until it successfully completes the establishment procedures
defined by that protocol.

Also for purposes of this document, a session constitutes the logical
connection between two end-stations and not the intermediate connections
that proxy-based network security devices firewalls may use.

Issues:

See also:
policy (3.16) (3.12)
proxy (3.18) (3.14)
rule set (3.20) (3.16)
stateful inspection (3.22)

         3.22 (3.18)

3.18 Stateful inspection

Definition:
The process of forwarding or rejecting traffic based on the contents of
a state table maintained by the network security
         device. a firewall.

Discussion:
Packet filtering and proxy devices firewalls are essentially static, in that
they always forward or reject traffic packets based on the contents of the rule
set. Devices

In contrast, devices using stateful inspection, in contrast, inspection will only forward traffic packets
if it corresponds they correspond with state information maintained by the device about
each session. For example, a stateful inspection device will reject a
packet on TCP port 21
         (ftp DATA) 20 (ftp-data) if no ftp session has been established.

         Newman et al                                             [Page 12]
         Measurement units:
         Not applicable

         Issues:

         See also:
         dynamic proxy (3.7)
         packet filter (3.14)
         proxy (3.18)

         3.23 System under test (SUT)

         Definition:
         The collective set of network security devices to which traffic is
         offered as a single entity and response measured.

         Discussion:
         A system under test may comprise multiple network security
         devices. A typical configuration involves two or more devices,
         with at least one located between the protected network and DMZ
         and at least one other located between been established over the DMZ and external
         network. Some devices may be active, such as firewalls or
         authentication products; other devices, such as systems for
         logging, may be passive.
ftp control port (usually port 21).

Newman et al.                                                  Page [10]

Measurement units:
Not applicable

Issues:

See also:
         demilitarized zone (DMZ) (3.4)
         device under test (3.5)
         external network (3.8)
         protected network (3.17)

         3.24 Unidirectional traffic
dynamic proxy (3.7)
packet filter (3.10)
proxy (3.14)

3.19 Tri-homed

Definition:
         Packets offered to the DUT/SUT such that the sending and receiving
A firewall with three network interface or interfaces are mutually exclusive. interfaces.

Discussion:
         This definition is included mainly for purposes of completeness;
         it is not particularly meaningful in the context of network
         security device performance. As noted in the discussion of
         forwarding rate (see section 3.9),
Tri-homed firewalls connect three network security devices almost
         invariably involve sessions segments with bidirectional traffic flow.

         However, unidirectional traffic is appropriate for evaluating the
         maximum forwarding rate of data sources (absent the DUT/SUT), different
network addresses. Typically, these would be protected, DMZ, and
         for evaluating the maximum forwarding rate of certain
         connectionless protocols.

         Newman et al                                             [Page 13]
external segments.

Measurement units:
Not applicable

Issues:
         half duplex vs. full duplex
Usually the differentiator between one segment and another is its IP
address. However, firewalls may connect different networks of other
types, such as ATM or Netware segments.

See also:
         bidirectional traffic (3.2)
         forwarding rate
Dual-homed (3.6)
Homed (3.9)
         maximum forwarding rate (3.11)

         3.25

3.20 User

Definition:
The person or machine requesting access to resources protected by the
DUT/SUT.

Discussion:
"User" is a problematic term in the context of security device firewall performance
testing, for several reasons. First, a user may in fact be a machine or
machines requesting services through the DUT/SUT. Second, different
"user" requests may require radically different amounts of DUT/SUT
resources. Third, traffic profiles vary widely from one organization to
another, making it difficult to characterize the load offered by a
typical users.

For these reasons, we prefer not to measure DUT/SUT performance in terms
of users supported. Instead, we describe performance in terms of maximum
forwarding rate and maximum number of sessions sustained.

         Measurement units:
         Not applicable

         Issues:

         See also:
         data source (3.3)
         virtual client (3.26)

         3.26 Virtual client

         Definition:
         A subset of a data source that represents one individual user.

         Discussion:
         In offering traffic to Further, we
use the DUT/SUT it may be useful for one data
         source to emulate multiple users, machines, or networks. For
         purposes of this document, each emulated term "data source" rather than user should be considered
         a virtual client.

         One data source MAY offer traffic from multiple virtual clients to
         multiple network interfaces on describe the DUT/SUT. However, each virtual
         client MUST offer traffic to just one network interface.
generator(s).

Newman et al                                             [Page 14] al.                                                  Page [11]

Measurement units:
Not applicable

Issues:

See also:
data source (3.3)
         user (3.25)

4.  Security considerations
Security considerations are explicitly excluded from this memo. The
authors plan to address security and management concerns in a separate
proposal brought to the IETF's security directorate.

5. References

Bradner, S., editor. "Benchmarking Terminology for Network
Interconnection Devices." RFC 1242.

Bradner, S., and McQuaid, J. "Benchmarking Methodology for Network
Interconnect Devices." RFC 1944.

Mandeville, B. "Benchmarking Terminology for LAN Switching Devices." ftp://ietf.org/internet-drafts/draft-ietf-bmwg-
         lanswitch-05.txt
ftp://ietf.org/internet-drafts/draft-ietf-bmwg-lanswitch-07.txt

Newman, D., and Melson, B. "Can Firewalls Take the Heat?" Data
Communications, November 21, 1995.
http://www.data.com/Lab_Tests/Firewalls.html

Newman, D., Holzbaur, H., and Bishop, K. "Firewalls: Don't Get Burned,"
Data Communications, March 21, 1997.
http://www.data.com/lab_tests/firewalls97.html

Ranum, M. "Firewall Performance Measurement Techniques: A Scientific
Approach." http://www.clark.net/pub/mjr/pubs/fwperf/intro.htm

Shannon, G. "Profile of Corporate Internet Application Traffic."
http://www.milkyway.com/libr/prof.html

6. Acknowledgments
The authors wish to thank the IETF Benchmarking Working Group for
agreeing to review this document. Ted Doty (Network (Internet Security Systems),
Shlomo Kramer (Check Point Software Technologies), Bob Mandeville
(European Network Laboratories), Brent Melson (National Software Testing
Laboratories), Marcus Ranum (Network Flight Recorder Inc.), Greg Shannon (Milkyway Networks),
(Ascend Communications), Rick Siebenaler (Cyberguard), and Greg Smith
(Check Point Software Technologies) offered valuable contributions and
critiques during this project.

7. Contact Information
David Newman
Data Communications magazine

         Newman et al                                             [Page 15]
1221 Avenue of the Americas, 41st Floor
New York, NY 10020
USA

Newman et al.                                                  Page [12]

212-512-6182 voice
212-512-6833 fax
dnewman@data.com

Helen Holzbaur
National Software Testing Laboratories Inc.
625 Ridge Pike
Conshohocken, PA 19428
USA
helen@nstl.com

Jim Hurd
National Software Testing Laboratories Inc.
625 Ridge Pike
Conshohocken, PA 19428
USA
jimh@nstl.com

Steven Platt, PhD. Platt
National Software Testing Laboratories Inc.
625 Ridge Pike
Conshohocken, PA 19428
USA
steve@nstl.com

Newman et al                                             [Page 16] al.                                                  Page [13]