draft-ietf-bmwg-virtual-net-03.txt   draft-ietf-bmwg-virtual-net-04.txt 
Network Working Group A. Morton Network Working Group A. Morton
Internet-Draft AT&T Labs Internet-Draft AT&T Labs
Intended status: Informational June 19, 2016 Intended status: Informational August 14, 2016
Expires: December 21, 2016 Expires: February 15, 2017
Considerations for Benchmarking Virtual Network Functions and Their Considerations for Benchmarking Virtual Network Functions and Their
Infrastructure Infrastructure
draft-ietf-bmwg-virtual-net-03 draft-ietf-bmwg-virtual-net-04
Abstract Abstract
Benchmarking Methodology Working Group has traditionally conducted Benchmarking Methodology Working Group has traditionally conducted
laboratory characterization of dedicated physical implementations of laboratory characterization of dedicated physical implementations of
internetworking functions. This memo investigates additional internetworking functions. This memo investigates additional
considerations when network functions are virtualized and performed considerations when network functions are virtualized and performed
in general purpose hardware. in general purpose hardware.
See the new version history section for updates. See the new version history section for updates.
skipping to change at page 1, line 43 skipping to change at page 1, line 43
Internet-Drafts are working documents of the Internet Engineering Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF). Note that other groups may also distribute Task Force (IETF). Note that other groups may also distribute
working documents as Internet-Drafts. The list of current Internet- working documents as Internet-Drafts. The list of current Internet-
Drafts is at http://datatracker.ietf.org/drafts/current/. Drafts is at http://datatracker.ietf.org/drafts/current/.
Internet-Drafts are draft documents valid for a maximum of six months Internet-Drafts are draft documents valid for a maximum of six months
and may be updated, replaced, or obsoleted by other documents at any and may be updated, replaced, or obsoleted by other documents at any
time. It is inappropriate to use Internet-Drafts as reference time. It is inappropriate to use Internet-Drafts as reference
material or to cite them other than as "work in progress." material or to cite them other than as "work in progress."
This Internet-Draft will expire on December 21, 2016. This Internet-Draft will expire on February 15, 2017.
Copyright Notice Copyright Notice
Copyright (c) 2016 IETF Trust and the persons identified as the Copyright (c) 2016 IETF Trust and the persons identified as the
document authors. All rights reserved. document authors. All rights reserved.
This document is subject to BCP 78 and the IETF Trust's Legal This document is subject to BCP 78 and the IETF Trust's Legal
Provisions Relating to IETF Documents Provisions Relating to IETF Documents
(http://trustee.ietf.org/license-info) in effect on the date of (http://trustee.ietf.org/license-info) in effect on the date of
publication of this document. Please review these documents publication of this document. Please review these documents
skipping to change at page 2, line 25 skipping to change at page 2, line 25
Table of Contents Table of Contents
1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 2 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 2
2. Scope . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 2. Scope . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
3. Considerations for Hardware and Testing . . . . . . . . . . . 4 3. Considerations for Hardware and Testing . . . . . . . . . . . 4
3.1. Hardware Components . . . . . . . . . . . . . . . . . . . 4 3.1. Hardware Components . . . . . . . . . . . . . . . . . . . 4
3.2. Configuration Parameters . . . . . . . . . . . . . . . . 5 3.2. Configuration Parameters . . . . . . . . . . . . . . . . 5
3.3. Testing Strategies . . . . . . . . . . . . . . . . . . . 6 3.3. Testing Strategies . . . . . . . . . . . . . . . . . . . 6
3.4. Attention to Shared Resources . . . . . . . . . . . . . . 7 3.4. Attention to Shared Resources . . . . . . . . . . . . . . 7
4. Benchmarking Considerations . . . . . . . . . . . . . . . . . 7 4. Benchmarking Considerations . . . . . . . . . . . . . . . . . 7
4.1. Comparison with Physical Network Functions . . . . . . . 7 4.1. Comparison with Physical Network Functions . . . . . . . 8
4.2. Continued Emphasis on Black-Box Benchmarks . . . . . . . 8 4.2. Continued Emphasis on Black-Box Benchmarks . . . . . . . 8
4.3. New Benchmarks and Related Metrics . . . . . . . . . . . 8 4.3. New Benchmarks and Related Metrics . . . . . . . . . . . 9
4.4. Assessment of Benchmark Coverage . . . . . . . . . . . . 9 4.4. Assessment of Benchmark Coverage . . . . . . . . . . . . 9
4.5. Power Consumption . . . . . . . . . . . . . . . . . . . . 12 4.5. Power Consumption . . . . . . . . . . . . . . . . . . . . 12
5. Security Considerations . . . . . . . . . . . . . . . . . . . 12 5. Security Considerations . . . . . . . . . . . . . . . . . . . 12
6. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 12 6. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 12
7. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 12 7. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 13
8. Version history . . . . . . . . . . . . . . . . . . . . . . . 13 8. Version history . . . . . . . . . . . . . . . . . . . . . . . 13
9. References . . . . . . . . . . . . . . . . . . . . . . . . . 13 9. References . . . . . . . . . . . . . . . . . . . . . . . . . 14
9.1. Normative References . . . . . . . . . . . . . . . . . . 13 9.1. Normative References . . . . . . . . . . . . . . . . . . 14
9.2. Informative References . . . . . . . . . . . . . . . . . 14 9.2. Informative References . . . . . . . . . . . . . . . . . 14
Author's Address . . . . . . . . . . . . . . . . . . . . . . . . 15 Author's Address . . . . . . . . . . . . . . . . . . . . . . . . 15
1. Introduction 1. Introduction
Benchmarking Methodology Working Group (BMWG) has traditionally Benchmarking Methodology Working Group (BMWG) has traditionally
conducted laboratory characterization of dedicated physical conducted laboratory characterization of dedicated physical
implementations of internetworking functions (or physical network implementations of internetworking functions (or physical network
functions, PNFs). The Black-box Benchmarks of Throughput, Latency, functions, PNFs). The Black-box Benchmarks of Throughput, Latency,
Forwarding Rates and others have served our industry for many years. Forwarding Rates and others have served our industry for many years.
skipping to change at page 4, line 44 skipping to change at page 4, line 44
3. Considerations for Hardware and Testing 3. Considerations for Hardware and Testing
This section lists the new considerations which must be addressed to This section lists the new considerations which must be addressed to
benchmark VNF(s) and their supporting infrastructure. The System benchmark VNF(s) and their supporting infrastructure. The System
Under Test (SUT) is composed of the hardware platform components, the Under Test (SUT) is composed of the hardware platform components, the
VNFs installed, and many other supporting systems. It is critical to VNFs installed, and many other supporting systems. It is critical to
document all aspects of the SUT to foster repeatability. document all aspects of the SUT to foster repeatability.
3.1. Hardware Components 3.1. Hardware Components
New Hardware devices will become part of the test set-up. New Hardware components will become part of the test set-up.
1. High volume server platforms (general-purpose, possibly with 1. High volume server platforms (general-purpose, possibly with
virtual technology enhancements). virtual technology enhancements).
2. Storage systems with large capacity, high speed, and high 2. Storage systems with large capacity, high speed, and high
reliability. reliability.
3. Network Interface ports specially designed for efficient service 3. Network Interface ports specially designed for efficient service
of many virtual NICs. of many virtual NICs.
4. High capacity Ethernet Switches. 4. High capacity Ethernet Switches.
The components above are subjects for development of specialized
benchmarks which are focused on the special demands of network
function deployment.
Labs conducting comparisons of different VNFs may be able to use the Labs conducting comparisons of different VNFs may be able to use the
same hardware platform over many studies, until the steady march of same hardware platform over many studies, until the steady march of
innovations overtakes their capabilities (as happens with the lab's innovations overtakes their capabilities (as happens with the lab's
traffic generation and testing devices today). traffic generation and testing devices today).
3.2. Configuration Parameters 3.2. Configuration Parameters
It will be necessary to configure and document the settings for the It will be necessary to configure and document the settings for the
entire general-purpose platform to ensure repeatability and foster entire general-purpose platform to ensure repeatability and foster
future comparisons, including but clearly not limited-to the future comparisons, including but clearly not limited-to the
skipping to change at page 5, line 48 skipping to change at page 6, line 4
o Virtual Machine (VM) o Virtual Machine (VM)
o Infrastructure Virtual Network (which interconnects Virtual o Infrastructure Virtual Network (which interconnects Virtual
Machines with physical network interfaces, or with each other Machines with physical network interfaces, or with each other
through virtual switches, for example) through virtual switches, for example)
and finally, the VNF itself, with items such as: and finally, the VNF itself, with items such as:
o specific function being implemented in VNF o specific function being implemented in VNF
o reserved resources for each function (e.g., CPU pinning and Non-
Uniform Memory Access, NUMA node assignment)
o reserved resources for each function (e.g., CPU pinning)
o number of VNFs (or sub-VNF components, each with its own VM) in o number of VNFs (or sub-VNF components, each with its own VM) in
the service function chain (see section 1.1 of [RFC7498] for a the service function chain (see section 1.1 of [RFC7498] for a
definition of service function chain) definition of service function chain)
o number of physical interfaces and links transited in the service o number of physical interfaces and links transited in the service
function chain function chain
In the physical device benchmarking context, most of the In the physical device benchmarking context, most of the
corresponding infrastructure configuration choices were determined by corresponding infrastructure configuration choices were determined by
the vendor. Although the platform itself is now one of the the vendor. Although the platform itself is now one of the
skipping to change at page 6, line 46 skipping to change at page 6, line 49
set of configuration and test parameters. set of configuration and test parameters.
4. The desire for elasticity and flexibility among network functions 4. The desire for elasticity and flexibility among network functions
will include tests where there is constant flux in the number of will include tests where there is constant flux in the number of
VM instances, the resources the VMs require, and the set-up/tear- VM instances, the resources the VMs require, and the set-up/tear-
down of network paths that support VM connectivity. Requests for down of network paths that support VM connectivity. Requests for
and instantiation of new VMs, along with Releases for VMs hosting and instantiation of new VMs, along with Releases for VMs hosting
VNFs that are no longer needed would be an normal operational VNFs that are no longer needed would be an normal operational
condition. In other words, benchmarking should include scenarios condition. In other words, benchmarking should include scenarios
with production life cycle management of VMs and their VNFs and with production life cycle management of VMs and their VNFs and
network connectivity in-progress, as well as static network connectivity in-progress, including VNF scaling up/down
configurations. operations, as well as static configurations.
5. All physical things can fail, and benchmarking efforts can also 5. All physical things can fail, and benchmarking efforts can also
examine recovery aided by the virtual architecture with different examine recovery aided by the virtual architecture with different
approaches to resiliency. approaches to resiliency.
6. The sheer number of test conditionas and configuration
combinations encourage increased efficiency, including automated
testing arrangements, combination sub-sampling through an
understanding of inter-relationships, and machine-readable test
results.
3.4. Attention to Shared Resources 3.4. Attention to Shared Resources
Since many components of the new NFV Infrastructure are virtual, test Since many components of the new NFV Infrastructure are virtual, test
set-up design must have prior knowledge of inter-actions/dependencies set-up design must have prior knowledge of inter-actions/dependencies
within the various resource domains in the System Under Test (SUT). within the various resource domains in the System Under Test (SUT).
For example, a virtual machine performing the role of a traditional For example, a virtual machine performing the role of a traditional
tester function such as generating and/or receiving traffic should tester function such as generating and/or receiving traffic should
avoid sharing any SUT resources with the Device Under Test DUT. avoid sharing any SUT resources with the Device Under Test DUT.
Otherwise, the results will have unexpected dependencies not Otherwise, the results will have unexpected dependencies not
encountered in physical device benchmarking. encountered in physical device benchmarking.
skipping to change at page 14, line 36 skipping to change at page 14, line 45
Service Function Chaining", RFC 7498, Service Function Chaining", RFC 7498,
DOI 10.17487/RFC7498, April 2015, DOI 10.17487/RFC7498, April 2015,
<http://www.rfc-editor.org/info/rfc7498>. <http://www.rfc-editor.org/info/rfc7498>.
9.2. Informative References 9.2. Informative References
[I-D.ietf-bmwg-sdn-controller-benchmark-meth] [I-D.ietf-bmwg-sdn-controller-benchmark-meth]
Vengainathan, B., Basil, A., Tassinari, M., Manral, V., Vengainathan, B., Basil, A., Tassinari, M., Manral, V.,
and S. Banks, "Benchmarking Methodology for SDN Controller and S. Banks, "Benchmarking Methodology for SDN Controller
Performance", draft-ietf-bmwg-sdn-controller-benchmark- Performance", draft-ietf-bmwg-sdn-controller-benchmark-
meth-01 (work in progress), March 2016. meth-02 (work in progress), July 2016.
[I-D.krishnan-nfvrg-policy-based-rm-nfviaas] [I-D.krishnan-nfvrg-policy-based-rm-nfviaas]
Krishnan, R., Figueira, N., Krishnaswamy, D., Lopez, D., Krishnan, R., Figueira, N., Krishnaswamy, D., Lopez, D.,
Wright, S., Hinrichs, T., Krishnaswamy, R., and A. Yerra, Wright, S., Hinrichs, T., Krishnaswamy, R., and A. Yerra,
"NFVIaaS Architectural Framework for Policy Based Resource "NFVIaaS Architectural Framework for Policy Based Resource
Placement and Scheduling", draft-krishnan-nfvrg-policy- Placement and Scheduling", draft-krishnan-nfvrg-policy-
based-rm-nfviaas-06 (work in progress), March 2016. based-rm-nfviaas-06 (work in progress), March 2016.
[I-D.vsperf-bmwg-vswitch-opnfv] [I-D.vsperf-bmwg-vswitch-opnfv]
Tahhan, M., O'Mahony, B., and A. Morton, "Benchmarking Tahhan, M., O'Mahony, B., and A. Morton, "Benchmarking
 End of changes. 14 change blocks. 
14 lines changed or deleted 25 lines changed or added

This html diff was produced by rfcdiff 1.45. The latest version is available from http://tools.ietf.org/tools/rfcdiff/