draft-ietf-bmwg-virtual-net-02.txt   draft-ietf-bmwg-virtual-net-03.txt 
Network Working Group A. Morton Network Working Group A. Morton
Internet-Draft AT&T Labs Internet-Draft AT&T Labs
Intended status: Informational March 21, 2016 Intended status: Informational June 19, 2016
Expires: September 22, 2016 Expires: December 21, 2016
Considerations for Benchmarking Virtual Network Functions and Their Considerations for Benchmarking Virtual Network Functions and Their
Infrastructure Infrastructure
draft-ietf-bmwg-virtual-net-02 draft-ietf-bmwg-virtual-net-03
Abstract Abstract
Benchmarking Methodology Working Group has traditionally conducted Benchmarking Methodology Working Group has traditionally conducted
laboratory characterization of dedicated physical implementations of laboratory characterization of dedicated physical implementations of
internetworking functions. This memo investigates additional internetworking functions. This memo investigates additional
considerations when network functions are virtualized and performed considerations when network functions are virtualized and performed
in general purpose hardware. in general purpose hardware.
See the new version history section for updates. See the new version history section for updates.
skipping to change at page 1, line 43 skipping to change at page 1, line 43
Internet-Drafts are working documents of the Internet Engineering Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF). Note that other groups may also distribute Task Force (IETF). Note that other groups may also distribute
working documents as Internet-Drafts. The list of current Internet- working documents as Internet-Drafts. The list of current Internet-
Drafts is at http://datatracker.ietf.org/drafts/current/. Drafts is at http://datatracker.ietf.org/drafts/current/.
Internet-Drafts are draft documents valid for a maximum of six months Internet-Drafts are draft documents valid for a maximum of six months
and may be updated, replaced, or obsoleted by other documents at any and may be updated, replaced, or obsoleted by other documents at any
time. It is inappropriate to use Internet-Drafts as reference time. It is inappropriate to use Internet-Drafts as reference
material or to cite them other than as "work in progress." material or to cite them other than as "work in progress."
This Internet-Draft will expire on September 22, 2016. This Internet-Draft will expire on December 21, 2016.
Copyright Notice Copyright Notice
Copyright (c) 2016 IETF Trust and the persons identified as the Copyright (c) 2016 IETF Trust and the persons identified as the
document authors. All rights reserved. document authors. All rights reserved.
This document is subject to BCP 78 and the IETF Trust's Legal This document is subject to BCP 78 and the IETF Trust's Legal
Provisions Relating to IETF Documents Provisions Relating to IETF Documents
(http://trustee.ietf.org/license-info) in effect on the date of (http://trustee.ietf.org/license-info) in effect on the date of
publication of this document. Please review these documents publication of this document. Please review these documents
skipping to change at page 2, line 36 skipping to change at page 2, line 36
4.2. Continued Emphasis on Black-Box Benchmarks . . . . . . . 8 4.2. Continued Emphasis on Black-Box Benchmarks . . . . . . . 8
4.3. New Benchmarks and Related Metrics . . . . . . . . . . . 8 4.3. New Benchmarks and Related Metrics . . . . . . . . . . . 8
4.4. Assessment of Benchmark Coverage . . . . . . . . . . . . 9 4.4. Assessment of Benchmark Coverage . . . . . . . . . . . . 9
4.5. Power Consumption . . . . . . . . . . . . . . . . . . . . 12 4.5. Power Consumption . . . . . . . . . . . . . . . . . . . . 12
5. Security Considerations . . . . . . . . . . . . . . . . . . . 12 5. Security Considerations . . . . . . . . . . . . . . . . . . . 12
6. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 12 6. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 12
7. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 12 7. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 12
8. Version history . . . . . . . . . . . . . . . . . . . . . . . 13 8. Version history . . . . . . . . . . . . . . . . . . . . . . . 13
9. References . . . . . . . . . . . . . . . . . . . . . . . . . 13 9. References . . . . . . . . . . . . . . . . . . . . . . . . . 13
9.1. Normative References . . . . . . . . . . . . . . . . . . 13 9.1. Normative References . . . . . . . . . . . . . . . . . . 13
9.2. Informative References . . . . . . . . . . . . . . . . . 15 9.2. Informative References . . . . . . . . . . . . . . . . . 14
Author's Address . . . . . . . . . . . . . . . . . . . . . . . . 16 Author's Address . . . . . . . . . . . . . . . . . . . . . . . . 15
1. Introduction 1. Introduction
Benchmarking Methodology Working Group (BMWG) has traditionally Benchmarking Methodology Working Group (BMWG) has traditionally
conducted laboratory characterization of dedicated physical conducted laboratory characterization of dedicated physical
implementations of internetworking functions (or physical network implementations of internetworking functions (or physical network
functions, PNFs). The Black-box Benchmarks of Throughput, Latency, functions, PNFs). The Black-box Benchmarks of Throughput, Latency,
Forwarding Rates and others have served our industry for many years. Forwarding Rates and others have served our industry for many years.
[RFC1242] and [RFC2544] are the cornerstones of the work. [RFC1242] and [RFC2544] are the cornerstones of the work.
skipping to change at page 4, line 30 skipping to change at page 4, line 30
A clearly related goal: the benchmarks for the capacity of a general- A clearly related goal: the benchmarks for the capacity of a general-
purpose platform to host a plurality of VNF instances should be purpose platform to host a plurality of VNF instances should be
investigated. Existing networking technology benchmarks will also be investigated. Existing networking technology benchmarks will also be
considered for adaptation to NFV and closely associated technologies. considered for adaptation to NFV and closely associated technologies.
A non-goal is any overlap with traditional computer benchmark A non-goal is any overlap with traditional computer benchmark
development and their specific metrics (SPECmark suites such as development and their specific metrics (SPECmark suites such as
SPECCPU). SPECCPU).
A colossal non-goal is any form of architecture development related A continued non-goal is any form of architecture development related
to NFV and associated technologies in BMWG, consistent with all to NFV and associated technologies in BMWG, consistent with all
chartered work since BMWG began in 1989. chartered work since BMWG began in 1989.
3. Considerations for Hardware and Testing 3. Considerations for Hardware and Testing
This section lists the new considerations which must be addressed to This section lists the new considerations which must be addressed to
benchmark VNF(s) and their supporting infrastructure. The System benchmark VNF(s) and their supporting infrastructure. The System
Under Test (SUT) is composed of the hardware platform components, the Under Test (SUT) is composed of the hardware platform components, the
VNFs installed, and many other supporting systems. It is critical to VNFs installed, and many other supporting systems. It is critical to
document all aspects of the SUT to foster repeatability. document all aspects of the SUT to foster repeatability.
skipping to change at page 14, line 10 skipping to change at page 14, line 10
[NFV.PER001] [NFV.PER001]
"Network Function Virtualization: Performance and "Network Function Virtualization: Performance and
Portability Best Practices", Group Specification ETSI GS Portability Best Practices", Group Specification ETSI GS
NFV-PER 001 V1.1.1 (2014-06), June 2014. NFV-PER 001 V1.1.1 (2014-06), June 2014.
[RFC2119] Bradner, S., "Key words for use in RFCs to Indicate [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate
Requirement Levels", BCP 14, RFC 2119, Requirement Levels", BCP 14, RFC 2119,
DOI 10.17487/RFC2119, March 1997, DOI 10.17487/RFC2119, March 1997,
<http://www.rfc-editor.org/info/rfc2119>. <http://www.rfc-editor.org/info/rfc2119>.
[RFC2330] Paxson, V., Almes, G., Mahdavi, J., and M. Mathis,
"Framework for IP Performance Metrics", RFC 2330,
DOI 10.17487/RFC2330, May 1998,
<http://www.rfc-editor.org/info/rfc2330>.
[RFC2544] Bradner, S. and J. McQuaid, "Benchmarking Methodology for [RFC2544] Bradner, S. and J. McQuaid, "Benchmarking Methodology for
Network Interconnect Devices", RFC 2544, Network Interconnect Devices", RFC 2544,
DOI 10.17487/RFC2544, March 1999, DOI 10.17487/RFC2544, March 1999,
<http://www.rfc-editor.org/info/rfc2544>. <http://www.rfc-editor.org/info/rfc2544>.
[RFC2679] Almes, G., Kalidindi, S., and M. Zekauskas, "A One-way
Delay Metric for IPPM", RFC 2679, DOI 10.17487/RFC2679,
September 1999, <http://www.rfc-editor.org/info/rfc2679>.
[RFC2680] Almes, G., Kalidindi, S., and M. Zekauskas, "A One-way
Packet Loss Metric for IPPM", RFC 2680,
DOI 10.17487/RFC2680, September 1999,
<http://www.rfc-editor.org/info/rfc2680>.
[RFC2681] Almes, G., Kalidindi, S., and M. Zekauskas, "A Round-trip
Delay Metric for IPPM", RFC 2681, DOI 10.17487/RFC2681,
September 1999, <http://www.rfc-editor.org/info/rfc2681>.
[RFC3393] Demichelis, C. and P. Chimento, "IP Packet Delay Variation
Metric for IP Performance Metrics (IPPM)", RFC 3393,
DOI 10.17487/RFC3393, November 2002,
<http://www.rfc-editor.org/info/rfc3393>.
[RFC3432] Raisanen, V., Grotefeld, G., and A. Morton, "Network
performance measurement with periodic streams", RFC 3432,
DOI 10.17487/RFC3432, November 2002,
<http://www.rfc-editor.org/info/rfc3432>.
[RFC4689] Poretsky, S., Perser, J., Erramilli, S., and S. Khurana, [RFC4689] Poretsky, S., Perser, J., Erramilli, S., and S. Khurana,
"Terminology for Benchmarking Network-layer Traffic "Terminology for Benchmarking Network-layer Traffic
Control Mechanisms", RFC 4689, DOI 10.17487/RFC4689, Control Mechanisms", RFC 4689, DOI 10.17487/RFC4689,
October 2006, <http://www.rfc-editor.org/info/rfc4689>. October 2006, <http://www.rfc-editor.org/info/rfc4689>.
[RFC4737] Morton, A., Ciavattone, L., Ramachandran, G., Shalunov, [RFC4737] Morton, A., Ciavattone, L., Ramachandran, G., Shalunov,
S., and J. Perser, "Packet Reordering Metrics", RFC 4737, S., and J. Perser, "Packet Reordering Metrics", RFC 4737,
DOI 10.17487/RFC4737, November 2006, DOI 10.17487/RFC4737, November 2006,
<http://www.rfc-editor.org/info/rfc4737>. <http://www.rfc-editor.org/info/rfc4737>.
[RFC5357] Hedayat, K., Krzanowski, R., Morton, A., Yum, K., and J.
Babiarz, "A Two-Way Active Measurement Protocol (TWAMP)",
RFC 5357, DOI 10.17487/RFC5357, October 2008,
<http://www.rfc-editor.org/info/rfc5357>.
[RFC5905] Mills, D., Martin, J., Ed., Burbank, J., and W. Kasch,
"Network Time Protocol Version 4: Protocol and Algorithms
Specification", RFC 5905, DOI 10.17487/RFC5905, June 2010,
<http://www.rfc-editor.org/info/rfc5905>.
[RFC7498] Quinn, P., Ed. and T. Nadeau, Ed., "Problem Statement for [RFC7498] Quinn, P., Ed. and T. Nadeau, Ed., "Problem Statement for
Service Function Chaining", RFC 7498, Service Function Chaining", RFC 7498,
DOI 10.17487/RFC7498, April 2015, DOI 10.17487/RFC7498, April 2015,
<http://www.rfc-editor.org/info/rfc7498>. <http://www.rfc-editor.org/info/rfc7498>.
9.2. Informative References 9.2. Informative References
[I-D.ietf-bmwg-sdn-controller-benchmark-meth] [I-D.ietf-bmwg-sdn-controller-benchmark-meth]
Vengainathan, B., Basil, A., Tassinari, M., Manral, V., Vengainathan, B., Basil, A., Tassinari, M., Manral, V.,
and S. Banks, "Benchmarking Methodology for SDN Controller and S. Banks, "Benchmarking Methodology for SDN Controller
skipping to change at page 15, line 38 skipping to change at page 14, line 48
[I-D.krishnan-nfvrg-policy-based-rm-nfviaas] [I-D.krishnan-nfvrg-policy-based-rm-nfviaas]
Krishnan, R., Figueira, N., Krishnaswamy, D., Lopez, D., Krishnan, R., Figueira, N., Krishnaswamy, D., Lopez, D.,
Wright, S., Hinrichs, T., Krishnaswamy, R., and A. Yerra, Wright, S., Hinrichs, T., Krishnaswamy, R., and A. Yerra,
"NFVIaaS Architectural Framework for Policy Based Resource "NFVIaaS Architectural Framework for Policy Based Resource
Placement and Scheduling", draft-krishnan-nfvrg-policy- Placement and Scheduling", draft-krishnan-nfvrg-policy-
based-rm-nfviaas-06 (work in progress), March 2016. based-rm-nfviaas-06 (work in progress), March 2016.
[I-D.vsperf-bmwg-vswitch-opnfv] [I-D.vsperf-bmwg-vswitch-opnfv]
Tahhan, M., O'Mahony, B., and A. Morton, "Benchmarking Tahhan, M., O'Mahony, B., and A. Morton, "Benchmarking
Virtual Switches in OPNFV", draft-vsperf-bmwg-vswitch- Virtual Switches in OPNFV", draft-vsperf-bmwg-vswitch-
opnfv-01 (work in progress), October 2015. opnfv-02 (work in progress), March 2016.
[IPMI2.0] "Intelligent Platform Management Interface, v2.0 with [IPMI2.0] "Intelligent Platform Management Interface, v2.0 with
latest Errata", latest Errata",
http://www.intel.com/content/www/us/en/servers/ipmi/ipmi- http://www.intel.com/content/www/us/en/servers/ipmi/ipmi-
intelligent-platform-mgt-interface-spec-2nd-gen-v2-0-spec- intelligent-platform-mgt-interface-spec-2nd-gen-v2-0-spec-
update.html, April 2015. update.html, April 2015.
[RFC1242] Bradner, S., "Benchmarking Terminology for Network [RFC1242] Bradner, S., "Benchmarking Terminology for Network
Interconnection Devices", RFC 1242, DOI 10.17487/RFC1242, Interconnection Devices", RFC 1242, DOI 10.17487/RFC1242,
July 1991, <http://www.rfc-editor.org/info/rfc1242>. July 1991, <http://www.rfc-editor.org/info/rfc1242>.
[RFC5481] Morton, A. and B. Claise, "Packet Delay Variation [RFC5481] Morton, A. and B. Claise, "Packet Delay Variation
Applicability Statement", RFC 5481, DOI 10.17487/RFC5481, Applicability Statement", RFC 5481, DOI 10.17487/RFC5481,
March 2009, <http://www.rfc-editor.org/info/rfc5481>. March 2009, <http://www.rfc-editor.org/info/rfc5481>.
[RFC6049] Morton, A. and E. Stephan, "Spatial Composition of [RFC6049] Morton, A. and E. Stephan, "Spatial Composition of
Metrics", RFC 6049, DOI 10.17487/RFC6049, January 2011, Metrics", RFC 6049, DOI 10.17487/RFC6049, January 2011,
<http://www.rfc-editor.org/info/rfc6049>. <http://www.rfc-editor.org/info/rfc6049>.
[RFC6248] Morton, A., "RFC 4148 and the IP Performance Metrics
(IPPM) Registry of Metrics Are Obsolete", RFC 6248,
DOI 10.17487/RFC6248, April 2011,
<http://www.rfc-editor.org/info/rfc6248>.
[RFC6390] Clark, A. and B. Claise, "Guidelines for Considering New
Performance Metric Development", BCP 170, RFC 6390,
DOI 10.17487/RFC6390, October 2011,
<http://www.rfc-editor.org/info/rfc6390>.
Author's Address Author's Address
Al Morton Al Morton
AT&T Labs AT&T Labs
200 Laurel Avenue South 200 Laurel Avenue South
Middletown,, NJ 07748 Middletown,, NJ 07748
USA USA
Phone: +1 732 420 1571 Phone: +1 732 420 1571
Fax: +1 732 368 1192 Fax: +1 732 368 1192
 End of changes. 10 change blocks. 
56 lines changed or deleted 8 lines changed or added

This html diff was produced by rfcdiff 1.45. The latest version is available from http://tools.ietf.org/tools/rfcdiff/