draft-ietf-bmwg-ngfw-performance-11.txt   draft-ietf-bmwg-ngfw-performance-12.txt 
Benchmarking Methodology Working Group B. Balarajah Benchmarking Methodology Working Group B. Balarajah
Internet-Draft Internet-Draft
Obsoletes: 3511 (if approved) C. Rossenhoevel Obsoletes: 3511 (if approved) C. Rossenhoevel
Intended status: Informational EANTC AG Intended status: Informational EANTC AG
Expires: 23 April 2022 B. Monkman Expires: 20 May 2022 B. Monkman
NetSecOPEN NetSecOPEN
October 2021 November 2021
Benchmarking Methodology for Network Security Device Performance Benchmarking Methodology for Network Security Device Performance
draft-ietf-bmwg-ngfw-performance-11 draft-ietf-bmwg-ngfw-performance-12
Abstract Abstract
This document provides benchmarking terminology and methodology for This document provides benchmarking terminology and methodology for
next-generation network security devices including next-generation next-generation network security devices including next-generation
firewalls (NGFW), next-generation intrusion prevention systems firewalls (NGFW), next-generation intrusion prevention systems
(NGIPS), and unified threat management (UTM) implementations. This (NGIPS), and unified threat management (UTM) implementations. The
document aims to improve the applicability, reproducibility, and main areas covered in this document are test terminology, test
transparency of benchmarks and to align the test methodology with configuration parameters, and benchmarking methodology for NGFW and
today's increasingly complex layer 7 security centric network NGIPS. This document aims to improve the applicability,
application use cases. The main areas covered in this document are reproducibility, and transparency of benchmarks and to align the test
test terminology, test configuration parameters, and benchmarking methodology with today's increasingly complex layer 7 security
methodology for NGFW and NGIPS. centric network application use cases. As a result, this document
makes [RFC3511] obsolete.
Status of This Memo Status of This Memo
This Internet-Draft is submitted in full conformance with the This Internet-Draft is submitted in full conformance with the
provisions of BCP 78 and BCP 79. provisions of BCP 78 and BCP 79.
Internet-Drafts are working documents of the Internet Engineering Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF). Note that other groups may also distribute Task Force (IETF). Note that other groups may also distribute
working documents as Internet-Drafts. The list of current Internet- working documents as Internet-Drafts. The list of current Internet-
Drafts is at https://datatracker.ietf.org/drafts/current/. Drafts is at https://datatracker.ietf.org/drafts/current/.
Internet-Drafts are draft documents valid for a maximum of six months Internet-Drafts are draft documents valid for a maximum of six months
and may be updated, replaced, or obsoleted by other documents at any and may be updated, replaced, or obsoleted by other documents at any
time. It is inappropriate to use Internet-Drafts as reference time. It is inappropriate to use Internet-Drafts as reference
material or to cite them other than as "work in progress." material or to cite them other than as "work in progress."
This Internet-Draft will expire on 4 April 2022. This Internet-Draft will expire on 5 May 2022.
Copyright Notice Copyright Notice
Copyright (c) 2021 IETF Trust and the persons identified as the Copyright (c) 2021 IETF Trust and the persons identified as the
document authors. All rights reserved. document authors. All rights reserved.
This document is subject to BCP 78 and the IETF Trust's Legal This document is subject to BCP 78 and the IETF Trust's Legal
Provisions Relating to IETF Documents (https://trustee.ietf.org/ Provisions Relating to IETF Documents (https://trustee.ietf.org/
license-info) in effect on the date of publication of this document. license-info) in effect on the date of publication of this document.
Please review these documents carefully, as they describe your rights Please review these documents carefully, as they describe your rights
skipping to change at page 3, line 38 skipping to change at page 3, line 38
7.9.3. Test Parameters . . . . . . . . . . . . . . . . . . . 49 7.9.3. Test Parameters . . . . . . . . . . . . . . . . . . . 49
7.9.4. Test Procedures and Expected Results . . . . . . . . 51 7.9.4. Test Procedures and Expected Results . . . . . . . . 51
8. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 52 8. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 52
9. Security Considerations . . . . . . . . . . . . . . . . . . . 53 9. Security Considerations . . . . . . . . . . . . . . . . . . . 53
10. Contributors . . . . . . . . . . . . . . . . . . . . . . . . 53 10. Contributors . . . . . . . . . . . . . . . . . . . . . . . . 53
11. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 53 11. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 53
12. References . . . . . . . . . . . . . . . . . . . . . . . . . 53 12. References . . . . . . . . . . . . . . . . . . . . . . . . . 53
12.1. Normative References . . . . . . . . . . . . . . . . . . 53 12.1. Normative References . . . . . . . . . . . . . . . . . . 53
12.2. Informative References . . . . . . . . . . . . . . . . . 53 12.2. Informative References . . . . . . . . . . . . . . . . . 53
Appendix A. Test Methodology - Security Effectiveness Appendix A. Test Methodology - Security Effectiveness
Evaluation . . . . . . . . . . . . . . . . . . . . . . . 55 Evaluation . . . . . . . . . . . . . . . . . . . . . . . 54
A.1. Test Objective . . . . . . . . . . . . . . . . . . . . . 55 A.1. Test Objective . . . . . . . . . . . . . . . . . . . . . 55
A.2. Testbed Setup . . . . . . . . . . . . . . . . . . . . . . 55 A.2. Testbed Setup . . . . . . . . . . . . . . . . . . . . . . 55
A.3. Test Parameters . . . . . . . . . . . . . . . . . . . . . 55 A.3. Test Parameters . . . . . . . . . . . . . . . . . . . . . 55
A.3.1. DUT/SUT Configuration Parameters . . . . . . . . . . 55 A.3.1. DUT/SUT Configuration Parameters . . . . . . . . . . 55
A.3.2. Test Equipment Configuration Parameters . . . . . . . 55 A.3.2. Test Equipment Configuration Parameters . . . . . . . 55
A.4. Test Results Validation Criteria . . . . . . . . . . . . 56 A.4. Test Results Validation Criteria . . . . . . . . . . . . 56
A.5. Measurement . . . . . . . . . . . . . . . . . . . . . . . 56 A.5. Measurement . . . . . . . . . . . . . . . . . . . . . . . 56
A.6. Test Procedures and Expected Results . . . . . . . . . . 57 A.6. Test Procedures and Expected Results . . . . . . . . . . 57
A.6.1. Step 1: Background Traffic . . . . . . . . . . . . . 57 A.6.1. Step 1: Background Traffic . . . . . . . . . . . . . 57
A.6.2. Step 2: CVE Emulation . . . . . . . . . . . . . . . . 58 A.6.2. Step 2: CVE Emulation . . . . . . . . . . . . . . . . 58
skipping to change at page 4, line 18 skipping to change at page 4, line 18
terminology for firewalls initially ([RFC3511]). The requirements terminology for firewalls initially ([RFC3511]). The requirements
for network security element performance and effectiveness have for network security element performance and effectiveness have
increased tremendously since then. Security function implementations increased tremendously since then. Security function implementations
have evolved to more advanced areas and have diversified into have evolved to more advanced areas and have diversified into
intrusion detection and prevention, threat management, analysis of intrusion detection and prevention, threat management, analysis of
encrypted traffic, etc. In an industry of growing importance, well- encrypted traffic, etc. In an industry of growing importance, well-
defined, and reproducible key performance indicators (KPIs) are defined, and reproducible key performance indicators (KPIs) are
increasingly needed as they enable fair and reasonable comparison of increasingly needed as they enable fair and reasonable comparison of
network security functions. All these reasons have led to the network security functions. All these reasons have led to the
creation of a new next-generation network security device creation of a new next-generation network security device
benchmarking document and this document obsoletes [RFC3511]. benchmarking document, which makes [RFC3511] obsolete.
2. Requirements 2. Requirements
The keywords "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", The keywords "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
"SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and
"OPTIONAL" in this document are to be interpreted as described in BCP "OPTIONAL" in this document are to be interpreted as described in BCP
14 [RFC2119], [RFC8174] when, and only when, they appear in all 14 [RFC2119], [RFC8174] when, and only when, they appear in all
capitals, as shown here. capitals, as shown here.
3. Scope 3. Scope
skipping to change at page 16, line 40 skipping to change at page 16, line 40
as described in Section 8. If the test scenario requires more IP as described in Section 8. If the test scenario requires more IP
addresses or subnets than the IANA assigned, this document recommends addresses or subnets than the IANA assigned, this document recommends
using non routable Private IPv4 address ranges or Unique Local using non routable Private IPv4 address ranges or Unique Local
Address (ULA) IPv6 address ranges for the testing. Address (ULA) IPv6 address ranges for the testing.
4.3.2.3. HTTP / HTTPS Server Pool Endpoint Attributes 4.3.2.3. HTTP / HTTPS Server Pool Endpoint Attributes
The server pool for HTTP SHOULD listen on TCP port 80 and emulate the The server pool for HTTP SHOULD listen on TCP port 80 and emulate the
same HTTP version and settings chosen by the client (emulated web same HTTP version and settings chosen by the client (emulated web
browser). The Server MUST advertise server type in the Server browser). The Server MUST advertise server type in the Server
response header [RFC2616]. For HTTPS server, TLS 1.2 or higher MUST response header [RFC7230]. For HTTPS server, TLS 1.2 or higher MUST
be used with a maximum record size of 16 KByte and MUST NOT use be used with a maximum record size of 16 KByte and MUST NOT use
ticket resumption or session ID reuse. The server SHOULD listen on ticket resumption or session ID reuse. The server SHOULD listen on
TCP port 443. The server SHALL serve a certificate to the client. TCP port 443. The server SHALL serve a certificate to the client.
The HTTPS server MUST check host SNI information with the FQDN if SNI The HTTPS server MUST check host SNI information with the FQDN if SNI
is in use. Cipher suite and key size on the server side MUST be is in use. Cipher suite and key size on the server side MUST be
configured similar to the client side configuration described in configured similar to the client side configuration described in
Section 4.3.1.3. Section 4.3.1.3.
4.3.3. Traffic Flow Definition 4.3.3. Traffic Flow Definition
skipping to change at page 17, line 21 skipping to change at page 17, line 21
servers. When a client endpoint is needed, it will initialize and be servers. When a client endpoint is needed, it will initialize and be
given attributes such as a MAC and IP address. The behavior of the given attributes such as a MAC and IP address. The behavior of the
client is to sweep through the given server IP space, generating a client is to sweep through the given server IP space, generating a
recognizable service by the DUT. Sequential and pseudorandom sweep recognizable service by the DUT. Sequential and pseudorandom sweep
methods are acceptable. The method used MUST be stated in the final methods are acceptable. The method used MUST be stated in the final
report. Thus, a balanced mesh between client endpoints and server report. Thus, a balanced mesh between client endpoints and server
endpoints will be generated in a client IP and port to server IP and endpoints will be generated in a client IP and port to server IP and
port combination. Each client endpoint performs the same actions as port combination. Each client endpoint performs the same actions as
other endpoints, with the difference being the source IP of the other endpoints, with the difference being the source IP of the
client endpoint and the target server IP pool. The client MUST use client endpoint and the target server IP pool. The client MUST use
the server IP address or FQDN in the host header [RFC2616]. the server IP address or FQDN in the host header [RFC7230].
4.3.3.1. Description of Intra-Client Behavior 4.3.3.1. Description of Intra-Client Behavior
Client endpoints are independent of other clients that are Client endpoints are independent of other clients that are
concurrently executing. When a client endpoint initiates traffic, concurrently executing. When a client endpoint initiates traffic,
this section describes how the client steps through different this section describes how the client steps through different
services. Once the test is initialized, the client endpoints services. Once the test is initialized, the client endpoints
randomly hold (perform no operation) for a few milliseconds for randomly hold (perform no operation) for a few milliseconds for
better randomization of the start of client traffic. Each client better randomization of the start of client traffic. Each client
will either open a new TCP connection or connect to a TCP persistence will either open a new TCP connection or connect to a TCP persistence
skipping to change at page 52, line 36 skipping to change at page 52, line 36
phase. Follow step 3, if the measured value does not meet the target phase. Follow step 3, if the measured value does not meet the target
value or does not fulfill the test results validation criteria. value or does not fulfill the test results validation criteria.
7.9.4.3. Step 3: Test Iteration 7.9.4.3. Step 3: Test Iteration
Determine the achievable concurrent TCP connections within the test Determine the achievable concurrent TCP connections within the test
results validation criteria. results validation criteria.
8. IANA Considerations 8. IANA Considerations
This document makes no specific request of IANA.
The IANA has assigned IPv4 and IPv6 address blocks in [RFC6890] that The IANA has assigned IPv4 and IPv6 address blocks in [RFC6890] that
have been registered for special purposes. The IPv6 address block have been registered for special purposes. The IPv6 address block
2001:2::/48 has been allocated for the purpose of IPv6 Benchmarking 2001:2::/48 has been allocated for the purpose of IPv6 Benchmarking
[RFC5180] and the IPv4 address block 198.18.0.0/15 has been allocated [RFC5180] and the IPv4 address block 198.18.0.0/15 has been allocated
for the purpose of IPv4 Benchmarking [RFC2544]. This assignment was for the purpose of IPv4 Benchmarking [RFC2544]. This assignment was
made to minimize the chance of conflict in case a testing device were made to minimize the chance of conflict in case a testing device were
to be accidentally connected to part of the Internet. to be accidentally connected to part of the Internet.
9. Security Considerations 9. Security Considerations
The primary goal of this document is to provide benchmarking The primary goal of this document is to provide benchmarking
terminology and methodology for next-generation network security terminology and methodology for next-generation network security
devices. However, readers should be aware that there is some overlap devices for use in a laboratory isolated test environment. However,
between performance and security issues. Specifically, the optimal readers should be aware that there is some overlap between
performance and security issues. Specifically, the optimal
configuration for network security device performance may not be the configuration for network security device performance may not be the
most secure, and vice-versa. The cipher suites recommended in this most secure, and vice-versa. The cipher suites recommended in this
document are for test purpose only. The cipher suite recommendation document are for test purpose only. The cipher suite recommendation
for a real deployment is outside the scope of this document. for a real deployment is outside the scope of this document.
10. Contributors 10. Contributors
The following individuals contributed significantly to the creation The following individuals contributed significantly to the creation
of this document: of this document:
skipping to change at page 54, line 10 skipping to change at page 54, line 10
2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174, 2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174,
May 2017, <https://www.rfc-editor.org/info/rfc8174>. May 2017, <https://www.rfc-editor.org/info/rfc8174>.
12.2. Informative References 12.2. Informative References
[RFC2544] Bradner, S. and J. McQuaid, "Benchmarking Methodology for [RFC2544] Bradner, S. and J. McQuaid, "Benchmarking Methodology for
Network Interconnect Devices", RFC 2544, Network Interconnect Devices", RFC 2544,
DOI 10.17487/RFC2544, March 1999, DOI 10.17487/RFC2544, March 1999,
<https://www.rfc-editor.org/info/rfc2544>. <https://www.rfc-editor.org/info/rfc2544>.
[RFC2616] Fielding, R., Gettys, J., Mogul, J., Frystyk, H.,
Masinter, L., Leach, P., and T. Berners-Lee, "Hypertext
Transfer Protocol -- HTTP/1.1", RFC 2616,
DOI 10.17487/RFC2616, June 1999,
<https://www.rfc-editor.org/info/rfc2616>.
[RFC2647] Newman, D., "Benchmarking Terminology for Firewall [RFC2647] Newman, D., "Benchmarking Terminology for Firewall
Performance", RFC 2647, DOI 10.17487/RFC2647, August 1999, Performance", RFC 2647, DOI 10.17487/RFC2647, August 1999,
<https://www.rfc-editor.org/info/rfc2647>. <https://www.rfc-editor.org/info/rfc2647>.
[RFC3511] Hickman, B., Newman, D., Tadjudin, S., and T. Martin, [RFC3511] Hickman, B., Newman, D., Tadjudin, S., and T. Martin,
"Benchmarking Methodology for Firewall Performance", "Benchmarking Methodology for Firewall Performance",
RFC 3511, DOI 10.17487/RFC3511, April 2003, RFC 3511, DOI 10.17487/RFC3511, April 2003,
<https://www.rfc-editor.org/info/rfc3511>. <https://www.rfc-editor.org/info/rfc3511>.
[RFC5180] Popoviciu, C., Hamza, A., Van de Velde, G., and D. [RFC5180] Popoviciu, C., Hamza, A., Van de Velde, G., and D.
skipping to change at page 54, line 41 skipping to change at page 54, line 35
"Applicability Statement for RFC 2544: Use on Production "Applicability Statement for RFC 2544: Use on Production
Networks Considered Harmful", RFC 6815, Networks Considered Harmful", RFC 6815,
DOI 10.17487/RFC6815, November 2012, DOI 10.17487/RFC6815, November 2012,
<https://www.rfc-editor.org/info/rfc6815>. <https://www.rfc-editor.org/info/rfc6815>.
[RFC6890] Cotton, M., Vegoda, L., Bonica, R., Ed., and B. Haberman, [RFC6890] Cotton, M., Vegoda, L., Bonica, R., Ed., and B. Haberman,
"Special-Purpose IP Address Registries", BCP 153, "Special-Purpose IP Address Registries", BCP 153,
RFC 6890, DOI 10.17487/RFC6890, April 2013, RFC 6890, DOI 10.17487/RFC6890, April 2013,
<https://www.rfc-editor.org/info/rfc6890>. <https://www.rfc-editor.org/info/rfc6890>.
[RFC7230] Fielding, R., Ed. and J. Reschke, Ed., "Hypertext Transfer
Protocol (HTTP/1.1): Message Syntax and Routing",
RFC 7230, DOI 10.17487/RFC7230, June 2014,
<https://www.rfc-editor.org/info/rfc7230>.
[RFC8446] Rescorla, E., "The Transport Layer Security (TLS) Protocol [RFC8446] Rescorla, E., "The Transport Layer Security (TLS) Protocol
Version 1.3", RFC 8446, DOI 10.17487/RFC8446, August 2018, Version 1.3", RFC 8446, DOI 10.17487/RFC8446, August 2018,
<https://www.rfc-editor.org/info/rfc8446>. <https://www.rfc-editor.org/info/rfc8446>.
[RFC9000] Iyengar, J., Ed. and M. Thomson, Ed., "QUIC: A UDP-Based [RFC9000] Iyengar, J., Ed. and M. Thomson, Ed., "QUIC: A UDP-Based
Multiplexed and Secure Transport", RFC 9000, Multiplexed and Secure Transport", RFC 9000,
DOI 10.17487/RFC9000, May 2021, DOI 10.17487/RFC9000, May 2021,
<https://www.rfc-editor.org/info/rfc9000>. <https://www.rfc-editor.org/info/rfc9000>.
Appendix A. Test Methodology - Security Effectiveness Evaluation Appendix A. Test Methodology - Security Effectiveness Evaluation
 End of changes. 13 change blocks. 
23 lines changed or deleted 26 lines changed or added

This html diff was produced by rfcdiff 1.48. The latest version is available from http://tools.ietf.org/tools/rfcdiff/