draft-ietf-bmwg-2544-as-02.txt   draft-ietf-bmwg-2544-as-03.txt 
Network Working Group S. Bradner Network Working Group S. Bradner
Internet-Draft Harvard University Internet-Draft Harvard University
Intended status: Informational K. Dubray Intended status: Informational K. Dubray
Expires: September 13, 2012 Juniper Networks Expires: October 28, 2012 Juniper Networks
J. McQuaid J. McQuaid
Turnip Video Turnip Video
A. Morton A. Morton
AT&T Labs AT&T Labs
March 12, 2012 April 26, 2012
RFC 2544 Applicability Statement: Use on Production Networks Considered RFC 2544 Applicability Statement: Use on Production Networks Considered
Harmful Harmful
draft-ietf-bmwg-2544-as-02 draft-ietf-bmwg-2544-as-03
Abstract Abstract
Benchmarking Methodology Working Group (BMWG) has been developing key Benchmarking Methodology Working Group (BMWG) has been developing key
performance metrics and laboratory test methods since 1990, and performance metrics and laboratory test methods since 1990, and
continues this work at present. Recent application of the methods continues this work at present. Recent application of the methods
beyond their intended scope is cause for concern. This memo beyond their intended scope is cause for concern. This memo
clarifies the scope of RFC 2544 and other benchmarking work for the clarifies the scope of RFC 2544 and other benchmarking work for the
IETF community. IETF community.
skipping to change at page 1, line 41 skipping to change at page 1, line 41
Internet-Drafts are working documents of the Internet Engineering Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF). Note that other groups may also distribute Task Force (IETF). Note that other groups may also distribute
working documents as Internet-Drafts. The list of current Internet- working documents as Internet-Drafts. The list of current Internet-
Drafts is at http://datatracker.ietf.org/drafts/current/. Drafts is at http://datatracker.ietf.org/drafts/current/.
Internet-Drafts are draft documents valid for a maximum of six months Internet-Drafts are draft documents valid for a maximum of six months
and may be updated, replaced, or obsoleted by other documents at any and may be updated, replaced, or obsoleted by other documents at any
time. It is inappropriate to use Internet-Drafts as reference time. It is inappropriate to use Internet-Drafts as reference
material or to cite them other than as "work in progress." material or to cite them other than as "work in progress."
This Internet-Draft will expire on September 13, 2012. This Internet-Draft will expire on October 28, 2012.
Copyright Notice Copyright Notice
Copyright (c) 2012 IETF Trust and the persons identified as the Copyright (c) 2012 IETF Trust and the persons identified as the
document authors. All rights reserved. document authors. All rights reserved.
This document is subject to BCP 78 and the IETF Trust's Legal This document is subject to BCP 78 and the IETF Trust's Legal
Provisions Relating to IETF Documents Provisions Relating to IETF Documents
(http://trustee.ietf.org/license-info) in effect on the date of (http://trustee.ietf.org/license-info) in effect on the date of
publication of this document. Please review these documents publication of this document. Please review these documents
skipping to change at page 3, line 14 skipping to change at page 3, line 14
1. Introduction 1. Introduction
This memo clarifies the scope of RFC 2544 [RFC2544], and other This memo clarifies the scope of RFC 2544 [RFC2544], and other
benchmarking work for the IETF community. benchmarking work for the IETF community.
Benchmarking Methodologies (beginning with [RFC2544]) have always Benchmarking Methodologies (beginning with [RFC2544]) have always
relied on test conditions that can only be produced and replicated relied on test conditions that can only be produced and replicated
reliably in the laboratory. Thus it was surprising to find that this reliably in the laboratory. Thus it was surprising to find that this
foundation methodology was being cited in several unintended foundation methodology was being cited in several unintended
applications, such as: specifications [Y.1731] and products performing applications such as:
1. Validation of telecommunication service configuration, such as 1. Validation of telecommunication service configuration, such as
the Committed Information Rate (CIR). the Committed Information Rate (CIR).
2. Validation of performance metrics in a telecommunication Service 2. Validation of performance metrics in a telecommunication Service
Level Agreement (SLA), such as frame loss and latency. Level Agreement (SLA), such as frame loss and latency.
3. As an integral part of telecommunication service activation 3. Telecommunication service activation testing, where traffic that
testing, where traffic that shares network resources with the shares network resources with the test might be adversely
test might be adversely affected. affected.
Above, we distinguish "telecommunication service" (where a network Above, we distinguish "telecommunication service" (where a network
service provider contracts with a customer to transfer information service provider contracts with a customer to transfer information
between specified interfaces at different geographic locations) from between specified interfaces at different geographic locations) from
the generic term "service". Also, we use the adjective "production" the generic term "service". Below, we use the adjective "production"
to refer to networks carrying live user traffic. [RFC2544] used the to refer to networks carrying live user traffic. [RFC2544] used the
term "real-world" to refer to production networks and to term "real-world" to refer to production networks and to
differentiate them from test networks. differentiate them from test networks.
Although RFC 2544 is held up as the standard reference for such Although RFC 2544 is held up as the standard reference for such
testing, we believe that the actual methods used vary from RFC 2544 testing, we believe that the actual methods used vary from RFC 2544
in significant ways. Since the only citation is to RFC 2544, the in significant ways. Since the only citation is to RFC 2544, the
modifications are opaque to the standards community and to users in modifications are opaque to the standards community and to users in
general (an undesirable situation). general (an undesirable situation).
skipping to change at page 7, line 27 skipping to change at page 7, line 27
solely on measurements observable external to the device under test/ solely on measurements observable external to the device under test/
system under test (DUT/SUT). system under test (DUT/SUT).
Special capabilities SHOULD NOT exist in the DUT/SUT specifically for Special capabilities SHOULD NOT exist in the DUT/SUT specifically for
benchmarking purposes. Any implications for network security arising benchmarking purposes. Any implications for network security arising
from the DUT/SUT SHOULD be identical in the lab and in production from the DUT/SUT SHOULD be identical in the lab and in production
networks. networks.
8. IANA Considerations 8. IANA Considerations
This memo makes no requests of IANA, and hopes that IANA will leave This memo makes no requests of IANA.
it alone as well.
9. Acknowledgements 9. Acknowledgements
Thanks to Matt Zekauskas, Bill Cerveny, Barry Constantine, Curtis Thanks to Matt Zekauskas, Bill Cerveny, Barry Constantine, Curtis
Villamizar, and David Newman for reading and suggesting improvements Villamizar, and David Newman for reading and suggesting improvements
to this memo. to this memo.
10. References 10. References
10.1. Normative References 10.1. Normative References
skipping to change at page 8, line 6 skipping to change at page 8, line 4
[RFC1242] Bradner, S., "Benchmarking terminology for network [RFC1242] Bradner, S., "Benchmarking terminology for network
interconnection devices", RFC 1242, July 1991. interconnection devices", RFC 1242, July 1991.
[RFC2119] Bradner, S., "Key words for use in RFCs to Indicate [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate
Requirement Levels", BCP 14, RFC 2119, March 1997. Requirement Levels", BCP 14, RFC 2119, March 1997.
[RFC2544] Bradner, S. and J. McQuaid, "Benchmarking Methodology for [RFC2544] Bradner, S. and J. McQuaid, "Benchmarking Methodology for
Network Interconnect Devices", RFC 2544, March 1999. Network Interconnect Devices", RFC 2544, March 1999.
[RFC5180] Popoviciu, C., Hamza, A., Van de Velde, G., and D. [RFC5180] Popoviciu, C., Hamza, A., Van de Velde, G., and D.
Dugatkin, "IPv6 Benchmarking Methodology for Network Dugatkin, "IPv6 Benchmarking Methodology for Network
Interconnect Devices", RFC 5180, May 2008. Interconnect Devices", RFC 5180, May 2008.
10.2. Informative References 10.2. Informative References
[Y.1731] ITU-T Recommendation Y.1540, "OAM functions and mechanisms
for Ethernet based networks", April 2011.
Authors' Addresses Authors' Addresses
Scott Bradner Scott Bradner
Harvard University Harvard University
29 Oxford St. 29 Oxford St.
Cambridge, MA 02138 Cambridge, MA 02138
USA USA
Phone: +1 617 495 3864 Phone: +1 617 495 3864
Fax: Fax:
 End of changes. 10 change blocks. 
11 lines changed or deleted 14 lines changed or added

This html diff was produced by rfcdiff 1.41. The latest version is available from http://tools.ietf.org/tools/rfcdiff/