draft-ietf-bmwg-ospfconv-term-10.txt   rfc4062.txt 
Network Working Group Vishwas Manral Network Working Group V. Manral
Internet Draft Netplane Systems Request for Comments: 4062 SiNett Corp.
Russ White Category: Informational R. White
Cisco Systems Cisco Systems
Aman Shaikh A. Shaikh
Expiration Date: December 2004 University of California AT&T Labs (Research)
File Name: draft-ietf-bmwg-ospfconv-term-10.txt June 2004 April 2005
OSPF Benchmarking Terminology and Concepts OSPF Benchmarking Terminology and Concepts
draft-ietf-bmwg-ospfconv-term-10.txt
Status of this Memo
By submitting this Internet-Draft, I certify that any applicable
patent or other IPR claims of which I am aware have been disclosed,
and any of which I become aware will be disclosed, in accordance with
RFC 3668.
Internet Drafts are working documents of the Internet Engineering
Task Force (IETF), its Areas, and its Working Groups. Note that other
groups may also distribute working documents as Internet Drafts.
Internet Drafts are draft documents valid for a maximum of six Status of This Memo
months. Internet Drafts may be updated, replaced, or obsoleted by
other documents at any time. It is not appropriate to use Internet
Drafts as reference material or to cite them other than as a "working
draft" or "work in progress".
The list of current Internet-Drafts can be accessed at
http//www.ietf.org/ietf/1id-abstracts.txt
The list of Internet-Draft Shadow Directories can be accessed at This memo provides information for the Internet community. It does
http//www.ietf.org/shadow.html. not specify an Internet standard of any kind. Distribution of this
memo is unlimited.
Copyright Notice Copyright Notice
Copyright (C) The Internet Society (2004). All Rights Reserved. Copyright (C) The Internet Society (2005).
Abstract Abstract
This draft explains the terminology and concepts used in OSPF This document explains the terminology and concepts used in OSPF
benchmarking. While some of these terms may be defined elsewhere, and benchmarking. Although some of these terms may be defined elsewhere
we will refer the reader to those definitions in some cases, we also (and we will refer the reader to those definitions in some cases) we
include discussions concerning these terms as they relate include discussions concerning these terms, as they relate
specifically to the tasks involved in benchmarking the OSPF protocol. specifically to the tasks involved in benchmarking the OSPF protocol.
1. Specification of Requirements 1. Introduction
This document is a companion to [BENCHMARK], which describes basic
Open Shortest Path First [OSPF] testing methods. This document
explains terminology and concepts used in OSPF Testing Framework
Documents, such as [BENCHMARK].
2. Specification of Requirements
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
"SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
document are to be interpreted as described in RFC 2119 [RFC2119]. document are to be interpreted as described in RFC 2119 [RFC2119].
[RFC2119] keywords in this document are used to assure methodological [RFC2119] key words in this document are used to ensure
control, which is very important in the specification of benchmarks. methodological control, which is very important in the specification
This document does not specify a network related protocol. of benchmarks. This document does not specify a network-related
protocol.
2. Introduction
This draft is a companion to [BENCHMARK], which describes basic Open
Shortest Path First [OSPF] testing methods. This draft explains
terminology and concepts used in OSPF Testing Framework Drafts, such
as [BENCHMARK].
3. Common Definitions 3. Common Definitions
Definitions in this section are well known industry and benchmarking Definitions in this section are well-known industry and benchmarking
terms which may be defined elsewhere. terms that may be defined elsewhere.
o White Box (Internal) Measurements o White Box (Internal) Measurements
- Definition - Definition
White Box measurements are measurements reported and col- White box measurements are those reported and collected on
lected on the Device Under Test (DUT) itself. the Device Under Test (DUT) itself.
- Discussion - Discussion
These measurement rely on output and event recording, along These measurements rely on output and event recording,
with the clocking and time stamping available on the DUT along with the clocking and time stamping available on the
itself. Taking measurements on the DUT may impact the DUT itself. Taking measurements on the DUT may impact the
actual outcome of the test, since it can increase processor actual outcome of the test, since it can increase processor
loading, memory utilization, and timing factors. Some dev- loading, memory utilization, and timing factors. Some
ices may not have the required output readily available for devices may not have the required output readily available
taking internal measurements, as well. for taking internal measurements.
Note: White box measurements can be influenced by the Note: White box measurements can be influenced by the
vendor's implementation of the various timers and process- vendor's implementation of various timers and processing
ing models. Whenever possible, internal measurements should models. Whenever possible, internal measurements should be
be compared to external measurements to verify and validate compared to external measurements to verify and validate
them. them.
Because of the potential for variations in collection and Because of the potential for variations in collection and
presentation methods across different DUTs, white box meas- presentation methods across different DUTs, white box
urements MUST NOT be used as a basis of comparison in measurements MUST NOT be used as a basis for comparison in
benchmarks. This has been a guiding principal of Bench- benchmarks. This has been a guiding principle of the
marking Methodology Working Group. Benchmarking Methodology Working Group.
o Black Box (External) Measurements o Black Box (External) Measurements
- Definition - Definition
Black Box measurements infer the performance of the DUT Black box measurements infer the performance of the DUT
through observation of its communications with other dev- through observation of its communications with other
ices. devices.
- Discussion - Discussion
One example of a black box measurement is when a downstream One example of a black box measurement is when a downstream
device receives complete routing information from the DUT, device receives complete routing information from the DUT,
it can be inferred that the DUT has transmitted all the it can be inferred that the DUT has transmitted all the
routing information available. External measurements of routing information available. External measurements of
internal operations may suffer in that they include not internal operations may suffer in that they include not
just the protocol action times, but also propagation just the protocol action times, but also propagation
delays, queuing delays, and other such factors. delays, queuing delays, and other such factors.
For the purposes of [BENCHMARK], external techniques are For the purposes of [BENCHMARK], external techniques are
more readily applicable. more readily applicable.
o Multi-device Measurements o Multi-device Measurements
- Measurements assessing communications (usually in combina- - Measurements assessing communications (usually in
tion with internal operations) between two or more DUTs. combination with internal operations) between two or more
Multi-device measurements may be internal or external. DUTs. Multi-device measurements may be internal or
external.
4. Terms Defined Elsewhere 4. Terms Defined Elsewhere
Terms in this section are defined elsewhere, and included only to Terms in this section are defined elsewhere and are included only as
include a discussion of those terms in reference to [BENCHMARK]. they apply to [BENCHMARK].
o Point-to-Point links o Point-to-Point Links
- Definition - Definition
See [OSPF], Section 1.2. See [OSPF], Section 1.2.
- Discussion - Discussion
A point-to-point link can take lesser time to converge than A point-to-point link can take less time to converge than a
a broadcast link of the same speed because it does not have broadcast link of the same speed because it does not have
the overhead of DR election. Point-to-point links can be the overhead of DR election. Point-to-point links can be
either numbered or unnumbered. However in the context of either numbered or unnumbered. However, in the context of
[BENCHMARK] and [OSPF], the two can be regarded the same. [BENCHMARK] and [OSPF], the two can be regarded as the
same.
o Broadcast Link o Broadcast Link
- Definition - Definition
See [OSPF], Section 1.2. See [OSPF], Section 1.2.
- Discussion - Discussion
The adjacency formation time on a broadcast link can be The adjacency formation time on a broadcast link can be
more than that on a point-to-point link of the same speed, greater than that on a point-to-point link of the same
because DR election has to take place. All routers on a speed because DR election has to take place. All routers
broadcast network form adjacency with the DR and BDR. on a broadcast network form adjacency with the DR and BDR.
Asynchronous flooding also takes place thru the DR. In con- Asynchronous flooding also takes place through the DR. In
text of convergence, it may take more time for an LSA to be the context of convergence, it may take more time for an
flooded from one DR-other router to another DR-other LSA to be flooded from one DR-other router to another
router, because the LSA has to be first processed at the because the LSA first has to be processed at the DR.
DR.
o Shortest Path First Execution Time o Shortest Path First Execution Time
- Definition - Definition
The time taken by a router to complete the SPF process, as The time taken by a router to complete the SPF process, as
described in [OSPF]. described in [OSPF].
- Discussion - Discussion
This does not include the time taken by the router to give This does not include the time taken by the router to
routes to the forwarding engine. install routes in the forwarding engine.
Some implementations may force two intervals, the SPF hold Some implementations may force two intervals, the SPF hold
time and the SPF delay, between successive SPF calcula- time and the SPF delay, between successive SPF
tions. If an SPF hold time exists, it should be subtracted calculations. If an SPF hold time exists, it should be
from the total SPF execution time. If an SPF delay exists, subtracted from the total SPF execution time. If an SPF
it should be noted in the test results. delay exists, it should be noted in the test results.
- Measurement Units - Measurement Units
The SPF time is generally measured in milliseconds. The SPF time is generally measured in milliseconds.
o Hello Interval o Hello Interval
- Definition - Definition
See [OSPF], Section 7.1. See [OSPF], Section 7.1.
- Discussion - Discussion
The hello interval should be the same for all routers on a The hello interval must be the same for all routers on a
network. network.
Decreasing the hello interval can allow the router dead Decreasing the hello interval can allow the router dead
interval (below) to be reduced, thus reducing convergence interval (below) to be reduced, thus reducing convergence
times in those situations where the router dead interval times in those situations where the router dead interval's
timing out causes an OSPF process to notice an adjacency timing out causes an OSPF process to notice an adjacency
failure. Further discussion on small hello intervals is failure. Further discussion of small hello intervals is
given in [OSPF-SCALING]. given in [OSPF-SCALING].
o Router Dead interval o Router Dead Interval
- Definition - Definition
See [OSPF], Section 7.1. See [OSPF], Section 7.1.
- Discussion - Discussion
This is advertised in the router's Hello Packets in the Router- This is advertised in the router's Hello Packets in the
DeadInterval field. The router dead interval should be some mul- Router-DeadInterval field. The router dead interval should
tiple of the HelloInterval (say 4 times the hello interval), and be some multiple of the HelloInterval (perhaps 4 times the
must be the same for all routers attached to a common network. hello interval) and must be the same for all routers
attached to a common network.
5. Concepts 5. Concepts
5.1. The Meaning of Single Router Control Plane Convergence 5.1. The Meaning of Single Router Control Plane Convergence
A network is termed to be converged when all of the devices within A network is termed as converged when all the devices within the
the network have a loop free path to each possible destination. Since network have a loop-free path to each possible destination. However,
we are not testing network convergence, but performance for a partic- because we are not testing network convergence but testing
ular device within a network, however, this definition needs to be performance for a particular device within a network, this definition
narrowed somewhat to fit within a single device view. needs to be streamlined to fit within a single device view.
In this case, convergence will mean the point in time when the DUT In this case, convergence will mean the point in time when the DUT
has performed all actions needed to react to the change in topology has performed all actions needed in order to react to the change in
represented by the test condition; for instance, an OSPF device must the topology represented by the test condition. For instance, an
flood any new information it has received, rebuild its shortest path OSPF device must flood any new information it has received, rebuild
first (SPF) tree, and install any new paths or destinations in the its shortest path first (SPF) tree, and install any new paths or
local routing information base (RIB, or routing table). destinations in the local routing information base (RIB, or routing
table).
Note that the word convergence has two distinct meanings; the process Note that the word "convergence" has two distinct meanings: the
of a group of individuals meeting the same place, and the process of process of a group of individuals meeting at the same place, and the
a single individual meeting in the same place as an existing group. process of an individual coming to the same place as an existing
This work focuses on the second meaning of the word, so we consider group. This work focuses on the second meaning of the word, so we
the time required for a single device to adapt to a network change to consider the time required for a single device to adapt to a network
be Single Router Convergence. change to be Single Router Convergence.
This concept does not include the time required for the control plane This concept does not include the time required for the control plane
of the device to transfer the information required to forward packets of the device to transfer the information required to forward packets
to the data plane, nor the amount of time between the data plane to the data plane. It also does not include the amount of time
receiving that information and being able to actually forward between when the data plane receives that information and when it is
traffic. able to forward traffic.
5.2. Measuring Convergence 5.2. Measuring Convergence
Obviously, there are several elements to convergence, even under the Obviously, there are several elements to convergence, even under the
definition given above for a single device, including (but not lim- definition given above for a single device, including (but not
ited to): limited to) the following:
o The time it takes for the DUT to pass the information about a o The time it takes for the DUT to pass the information about a
network event on to its neighbors. network event on to its neighbors.
o The time it takes for the DUT to process information about a o The time it takes for the DUT to process information about a
network event and calculate a new Shortest Path Tree (SPT). network event and to calculate a new Shortest Path Tree (SPT).
o The time it takes for the DUT to make changes in its local rib o The time it takes for the DUT to make changes in its local RIB
reflecting the new shortest path tree. reflecting the new shortest path tree.
5.3. Types of Network Events 5.3. Types of Network Events
A network event is an event which causes a change in the network A network event is an event that causes a change in the network
topology. topology.
o Link or Neighbor Device Up o Link or Neighbor Device Up
The time needed for an OSPF implementation to recognize a new The time needed for an OSPF implementation to recognize a new
link coming up on the device, build any necessarily adjacencies, link coming up on the device, to build any necessary
synchronize its database, and perform all other needed actions adjacencies, to synchronize its database, and to perform all
to converge. other actions necessary to converge.
o Initialization o Initialization
The time needed for an OSPF implementation to be initialized, The time needed for an OSPF implementation to be initialized, to
recognize any links across which OSPF must run, build any needed recognize any links across which OSPF must run, to build any
adjacencies, synchronize its database, and perform other actions needed adjacencies, to synchronize its database, and to perform
needed to converge. other actions necessary to converge.
o Adjacency Down o Adjacency Down
The time needed for an OSPF implementation to recognize a link The time needed for an OSPF implementation to recognize a link
down/adjacency loss based on hello timers alone, propagate any down/adjacency loss based on hello timers alone, to propagate
information as necessary to its remaining adjacencies, and per- any information as necessary to its remaining adjacencies, and
form other actions needed to converge. to perform other actions necessary to converge.
o Link Down o Link Down
The time needed for an OSPF implementation to recognize a link The time needed for an OSPF implementation to recognize a link
down based on layer 2 provided information, propagate any infor- down based on layer 2-provided information, to propagate any
mation as needed to its remaining adjacencies, and perform other information as needed to its remaining adjacencies, and to
actions needed to converge. perform other actions necessary to converge.
6. IANA Considerations
This document requires no IANA considerations.
7. Security Considerations 6. Security Considerations
This document does not modify the underlying security considerations This document does not modify the underlying security considerations
in [OSPF]. in [OSPF].
8. Acknowledgements 7. Acknowledgements
The authors would like to thank Howard Berkowitz (hcb@clark.net), The authors would like to thank Howard Berkowitz (hcb@clark.net),
Kevin Dubray, (kdubray@juniper.net), Scott Poretsky Kevin Dubray (kdubray@juniper.net), Scott Poretsky
(sporetsky@avici.com), and Randy Bush (randy@psg.com) for their dis- (sporetsky@avici.com), and Randy Bush (randy@psg.com) for their
cussion, ideas, and support. discussion, ideas, and support.
9. Normative References 8. Normative References
[BENCHMARK] [BENCHMARK] Manral, V., White, R., and A. Shaikh, "Benchmarking
Manral, V., "Benchmarking Basic OSPF Single Router Control Plane Basic OSPF Single Router Control Plane Convergence",
Convergence", draft-bmwg-ospfconv-intraarea-10, May 2004. RFC 4061, April 2005.
[OSPF]Moy, J., "OSPF Version 2", RFC 2328, April 1998. [OSPF] Moy, J., "OSPF Version 2", STD 54, RFC 2328, April
1998.
[RFC2119] [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate
Bradner, S., "Key words for use in RFCs to Indicate Requirement Requirement Levels", BCP 14, RFC 2119, March 1997.
Levels", BCP 14, RFC 2119, March 1997
10. Informative References 9. Informative References
[OSPF-SCALING] [OSPF-SCALING] Choudhury, Gagan L., Editor, "Prioritized Treatment of
Choudhury, Gagan L., Editor, "Prioritized Treatment of Specific Specific OSPF Packets and Congestion Avoidance", Work
OSPF Packets and Congestion Avoidance", draft-ietf-ospf- in Progress, August 2003.
scalability-06.txt, August 2003.
11. Authors' Addresses Authors' Addresses
Vishwas Manral, Vishwas Manral,
Netplane Systems, SiNett Corp,
189 Prashasan Nagar, Ground Floor,
Road number 72, Embassy Icon Annexe,
Jubilee Hills, 2/1, Infantry Road,
Hyderabad. Bangalore, India
vmanral@netplane.com EMail: vishwas@sinett.com
Russ White Russ White
Cisco Systems, Inc. Cisco Systems, Inc.
7025 Kit Creek Rd. 7025 Kit Creek Rd.
Research Triangle Park, NC 27709 Research Triangle Park, NC 27709
riw@cisco.com EMail: riw@cisco.com
Aman Shaikh Aman Shaikh
University of California AT&T Labs (Research)
School of Engineering 180 Park Av, PO Box 971
1156 High Street Florham Park, NJ 07932
Santa Cruz, CA 95064
aman@soe.ucsc.edu EMail: ashaikh@research.att.com
Intellectual Property Statement Full Copyright Statement
The IETF takes no position regarding the validity or scope of any Intel- Copyright (C) The Internet Society (2005).
lectual Property Rights or other rights that might be claimed to pertain
to the implementation or use of the technology described in this docu-
ment or the extent to which any license under such rights might or might
not be available; nor does it represent that it has made any independent
effort to identify any such rights. Information on the procedures with
respect to rights in RFC documents can be found in BCP 78 and BCP 79.
Copies of IPR disclosures made to the IETF Secretariat and any This document is subject to the rights, licenses and restrictions
assurances of licenses to be made available, or the result of an attempt contained in BCP 78, and except as set forth therein, the authors
retain all their rights.
made to obtain a general license or permission for the use of such This document and the information contained herein are provided on an
proprietary rights by implementers or users of this specification can be "AS IS" basis and THE CONTRIBUTOR, THE ORGANIZATION HE/SHE REPRESENTS
obtained from the IETF on-line IPR repository at OR IS SPONSORED BY (IF ANY), THE INTERNET SOCIETY AND THE INTERNET
http://www.ietf.org/ipr. ENGINEERING TASK FORCE DISCLAIM ALL WARRANTIES, EXPRESS OR IMPLIED,
INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE
INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED
WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE.
The IETF invites any interested party to bring to its attention any Intellectual Property
copyrights, patents or patent applications, or other proprietary rights
that may cover technology that may be required to implement this stan-
dard. Please address the information to the IETF at ietf-ipr@ietf.org.
Disclaimer of Warranty The IETF takes no position regarding the validity or scope of any
Intellectual Property Rights or other rights that might be claimed to
pertain to the implementation or use of the technology described in
this document or the extent to which any license under such rights
might or might not be available; nor does it represent that it has
made any independent effort to identify any such rights. Information
on the procedures with respect to rights in RFC documents can be
found in BCP 78 and BCP 79.
This document and the information contained herein are provided on an Copies of IPR disclosures made to the IETF Secretariat and any
"AS IS" basis and THE CONTRIBUTOR, THE ORGANIZATION HE/SHE REPRESENTS OR assurances of licenses to be made available, or the result of an
IS SPONSORED BY (IF ANY), THE INTERNET SOCIETY AND THE INTERNET attempt made to obtain a general license or permission for the use of
ENGINEERING TASK FORCE DISCLAIM ALL WARRANTIES, EXPRESS OR IMPLIED, such proprietary rights by implementers or users of this
INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMA- specification can be obtained from the IETF on-line IPR repository at
TION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF http://www.ietf.org/ipr.
MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE.
Full Copyright Statement The IETF invites any interested party to bring to its attention any
copyrights, patents or patent applications, or other proprietary
rights that may cover technology that may be required to implement
this standard. Please address the information to the IETF at ietf-
ipr@ietf.org.
Copyright (C) The Internet Society (2004). This document is subject to Acknowledgement
the rights, licenses and restrictions contained in BCP 78, and except as
set forth therein, the authors retain all their rights. Funding for the RFC Editor function is currently provided by the
Internet Society.
 End of changes. 

This html diff was produced by rfcdiff 1.25, available from http://www.levkowetz.com/ietf/tools/rfcdiff/