draft-ietf-bmwg-acc-bench-term-05.txt   draft-ietf-bmwg-acc-bench-term-06.txt 
Network Working Group Network Working Group
INTERNET-DRAFT INTERNET-DRAFT
Expires in: October 2005 Expires in: January 2006
Scott Poretsky Scott Poretsky
Quarry Technologies Reef Point Systems
Shankar Rao Shankar Rao
Qwest Communications Qwest Communications
February 2005 July 2005
Terminology for Accelerated Stress Benchmarking Terminology for Accelerated Stress Benchmarking
<draft-ietf-bmwg-acc-bench-term-05.txt> <draft-ietf-bmwg-acc-bench-term-06.txt>
Intellectual Property Rights (IPR) statement: Intellectual Property Rights (IPR) statement:
By submitting this Internet-Draft, I certify that any applicable By submitting this Internet-Draft, each author represents that any
patent or other IPR claims of which I am aware have been disclosed, or applicable patent or other IPR claims of which he or she is aware
will be disclosed, and any of which I become aware will be disclosed, have been or will be disclosed, and any of which he or she becomes
in accordance with RFC 3668. aware will be disclosed, in accordance with Section 6 of BCP 79.
Status of this Memo Status of this Memo
This document is an Internet-Draft and is in full conformance with This document is an Internet-Draft and is in full conformance with
all provisions of Section 10 of RFC2026. all provisions of Section 10 of RFC2026.
Internet-Drafts are working documents of the Internet Engineering Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF), its areas, and its working groups. Note that Task Force (IETF), its areas, and its working groups. Note that
other groups may also distribute working documents as Internet- other groups may also distribute working documents as Internet-
Drafts. Drafts.
skipping to change at page 3, line 4 skipping to change at page 2, line 54
3.4.3.3 Management Plane Instability Conditions...........20 3.4.3.3 Management Plane Instability Conditions...........20
3.4.3.4 Security Plane Instability Conditions.............20 3.4.3.4 Security Plane Instability Conditions.............20
3.5 Recovery..................................................21 3.5 Recovery..................................................21
3.5.1 Recovery Phase........................................21 3.5.1 Recovery Phase........................................21
3.5.2 Benchmarks............................................21 3.5.2 Benchmarks............................................21
3.5.2.1 Recovered Aggregate Forwarding Rate...............21 3.5.2.1 Recovered Aggregate Forwarding Rate...............21
3.5.2.2 Recovered Latency.................................22 3.5.2.2 Recovered Latency.................................22
3.5.2.3 Recovery Time.....................................22 3.5.2.3 Recovery Time.....................................22
3.5.2.4 Recovered Uncontrolled Sessions Lost..............23 3.5.2.4 Recovered Uncontrolled Sessions Lost..............23
3.5.2.5 Variability Benchmarks............................23 3.5.2.5 Variability Benchmarks............................23
4. Security Considerations.....................................24 4. IANA Considerations.........................................24
5. Normative References........................................24 5. Security Considerations.....................................24
6. Informative References......................................24 6. References..................................................24
7. Author's Address............................................25 7. Author's Address............................................25
Appendix 1 - White Box Benchmarks..............................25 Appendix 1 - White Box Benchmarks..............................25
1. Introduction 1. Introduction
Routers in an operational network are simultaneously configured with Routers in an operational network are simultaneously configured with
multiple protocols and security policies while forwarding traffic and multiple protocols and security policies while forwarding traffic and
being managed. To accurately benchmark a router for deployment it is being managed. To accurately benchmark a router for deployment it is
necessary to test that router in operational conditions by necessary to test that router in operational conditions by
simultaneously configuring and scaling network protocols and security simultaneously configuring and scaling network protocols and security
skipping to change at page 3, line 52 skipping to change at page 3, line 51
Devices" and RFC 2285 "Benchmarking Terminology for LAN Switching Devices" and RFC 2285 "Benchmarking Terminology for LAN Switching
Devices" should be consulted before attempting to make use of this Devices" should be consulted before attempting to make use of this
document. For the sake of clarity and continuity this RFC adopts document. For the sake of clarity and continuity this RFC adopts
the template for definitions set out in Section 2 of RFC 1242. the template for definitions set out in Section 2 of RFC 1242.
Definitions are indexed and grouped together in sections for ease Definitions are indexed and grouped together in sections for ease
of reference. of reference.
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
"SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
document are to be interpreted as described in BCP 14, RFC 2119 document are to be interpreted as described in BCP 14, RFC 2119
[Br97]. RFC 2119 defines the use of these key words to help make the [5]. RFC 2119 defines the use of these key words to help make the
intent of standards track documents as clear as possible. While this intent of standards track documents as clear as possible. While this
document uses these keywords, this document is not a standards track document uses these keywords, this document is not a standards track
document. document.
Table 1. Phase Sequence and Benchmarks Table 1. Phase Sequence and Benchmarks
III. Recovery Phase II. Instability Phase I. Startup Phase III. Recovery Phase II. Instability Phase I. Startup Phase
<-----------------<---<-------------------<----<--------------< <-----------------<---<-------------------<----<--------------<
Remove Instability Achieve Configuration Apply Startup Remove Instability Achieve Configuration Apply Startup
Conditions Set Conditions Conditions Set Conditions
skipping to change at page 23, line 52 skipping to change at page 23, line 52
Controlled Session Loss Controlled Session Loss
Uncontrolled Session Loss Uncontrolled Session Loss
3.5.2.5 Variability Benchmarks 3.5.2.5 Variability Benchmarks
Definition: Definition:
The difference between the measured Benchmarks of the The difference between the measured Benchmarks of the
same DUT over multiple iterations. same DUT over multiple iterations.
Discussion: Discussion:
Ideally, the benchmarks measured should be the same for Ideally, the measured benchmarks should be the same for multiple
multiple iterations with the same DUT. Configuration iterations with the same DUT. Configuration Sets and Instability
Sets Instability conditions SHOULD be held constant for Conditions SHOULD be held constant for this benchmark. Whether the
this benchmark. Whether the DUT can exhibit such predictable DUT can exhibit such predictable and repeatable behavior is an
and repeatable behavior is an important benchmark in itself. important benchmark in itself.
Measurement units: Measurement units:
As applicable to each Benchmark. The results are to be As applicable to each Benchmark. The results are to be
presented in a table format for successive Iterations. presented in a table format for successive Iterations.
Ideally, the differences should be zero. Ideally, the differences should be zero.
Issues: Issues:
None None
See Also: See Also:
Startup Period Startup Period
Instability Period Instability Period
Recovery Period Recovery Period
4. Security Considerations 4. IANA Considerations
This document requires no IANA considerations.
5. Security Considerations
Documents of this type do not directly effect the security of Documents of this type do not directly effect the security of
the Internet or of corporate networks as long as benchmarking the Internet or of corporate networks as long as benchmarking
is not performed on devices or systems connected to operating is not performed on devices or systems connected to operating
networks. networks.
5. Normative References 6. References
6.1 Normative References
[1] Bradner, S., Editor, "Benchmarking Terminology for Network [1] Bradner, S., Editor, "Benchmarking Terminology for Network
Interconnection Devices", RFC 1242, October 1991. Interconnection Devices", RFC 1242, October 1991.
[2] Mandeville, R., "Benchmarking Terminology for LAN Switching [2] Mandeville, R., "Benchmarking Terminology for LAN Switching
Devices", RFC 2285, June 1998. Devices", RFC 2285, June 1998.
[3] Bradner, S. and McQuaid, J., "Benchmarking Methodology for [3] Bradner, S. and McQuaid, J., "Benchmarking Methodology for
Network Interconnect Devices", RFC 2544, March 1999. Network Interconnect Devices", RFC 2544, March 1999.
[4] Poretsky, S. and Rao, S., "Methodology for Accelerated [4] Poretsky, S. and Rao, S., "Methodology for Accelerated
Stress Benchmarking", draft-ietf-bmwg-acc-bench-meth-01, Stress Benchmarking", draft-ietf-bmwg-acc-bench-meth-03,
work in progress, February 2005. work in progress, July 2005.
6. Informative References [5] Bradner, S., "Key words for use in RFCs to Indicate Requirement
Levels", RFC 2119, March 1997.
6.2 Informative References
[RFC3871] RFC 3871 "Operational Security Requirements for Large [RFC3871] RFC 3871 "Operational Security Requirements for Large
Internet Service Provider (ISP) IP Network Infrastructure. Internet Service Provider (ISP) IP Network Infrastructure.
G. Jones, Ed.. IETF, September 2004. G. Jones, Ed.. IETF, September 2004.
[NANOG25] "Core Router Evaluation for Higher Availability", Scott [NANOG25] "Core Router Evaluation for Higher Availability", Scott
Poretsky, NANOG 25, June 8, 2002, Toronto, CA. Poretsky, NANOG 25, June 8, 2002, Toronto, CA.
[IEEECQR] "Router Stress Testing to Validate Readiness for Network [IEEECQR] "Router Stress Testing to Validate Readiness for Network
Deployment", Scott Poretsky, IEEE CQR 2003. Deployment", Scott Poretsky, IEEE CQR 2003.
[CONVMETH] Poretsky, S., "Benchmarking Methodology for IGP Data Plane
Route Convergence", draft-ietf-bmwg-igp-dataplane-conv-meth-05,
work in progress, February 2005.
[CONVTERM] Poretsky, S., "Benchmarking Terminology for IGP Data Plane
Route Convergence", draft-ietf-bmwg-igp-dataplane-conv-term-05,
work in progress, February 2005.
7. Author's Address 7. Author's Address
Scott Poretsky Reef Point Systems
Quarry Technologies
8 New England Executive Park 8 New England Executive Park
Burlington, MA 01803 Burlington, MA 01803
USA USA
Phone: + 1 781 395 5090 Phone: + 1 781 395 5090
EMail: sporetsky@quarrytech.com EMail: sporetsky@reefpoint.com
Shankar Rao Shankar Rao
1801 California Street
8th Floor
Qwest Communications Qwest Communications
Denver, CO Denver, CO 80202
USA USA
Phone: + 1 303 437 6643 Phone: + 1 303 437 6643
Email: shankar.rao@qwest.com Email: shankar.rao@qwest.com
Appendix 1. White Box Benchmarking Terminology Appendix 1. White Box Benchmarking Terminology
Minimum Available Memory Minimum Available Memory
Definition: Definition:
Minimum DUT Available Memory during the duration of the Minimum DUT Available Memory during the duration of the
Accelerated Stress Test. Accelerated Stress Test.
 End of changes. 

This html diff was produced by rfcdiff 1.25, available from http://www.levkowetz.com/ietf/tools/rfcdiff/