draft-ietf-bmwg-igp-dataplane-conv-meth-12.txt   draft-ietf-bmwg-igp-dataplane-conv-meth-13.txt 
Network Working Group Network Working Group
INTERNET-DRAFT INTERNET-DRAFT
Expires in: August 2007 Expires in: January 2008
Intended Status: Informational Intended Status: Informational
Scott Poretsky Scott Poretsky
Reef Point Systems Reef Point Systems
Brent Imhoff Brent Imhoff
Juniper Networks Juniper Networks
February 2007
Benchmarking Methodology for Benchmarking Methodology for
IGP Data Plane Route Convergence IGP Data Plane Route Convergence
<draft-ietf-bmwg-igp-dataplane-conv-meth-12.txt> <draft-ietf-bmwg-igp-dataplane-conv-meth-13.txt>
Intellectual Property Rights (IPR) statement: Intellectual Property Rights (IPR) statement:
By submitting this Internet-Draft, each author represents that any By submitting this Internet-Draft, each author represents that any
applicable patent or other IPR claims of which he or she is aware applicable patent or other IPR claims of which he or she is aware
have been or will be disclosed, and any of which he or she becomes have been or will be disclosed, and any of which he or she becomes
aware will be disclosed, in accordance with Section 6 of BCP 79. aware will be disclosed, in accordance with Section 6 of BCP 79.
Status of this Memo Status of this Memo
Internet-Drafts are working documents of the Internet Engineering Internet-Drafts are working documents of the Internet Engineering
skipping to change at page 2, line 32 skipping to change at page 2, line 32
4.5 Convergence Due to Cost Change.............................11 4.5 Convergence Due to Cost Change.............................11
4.6 Convergence Due to ECMP Member Interface Failure...........11 4.6 Convergence Due to ECMP Member Interface Failure...........11
4.7 Convergence Due to Parallel Link Interface Failure.........12 4.7 Convergence Due to Parallel Link Interface Failure.........12
5. IANA Considerations.........................................13 5. IANA Considerations.........................................13
6. Security Considerations.....................................13 6. Security Considerations.....................................13
7. Acknowledgements............................................13 7. Acknowledgements............................................13
8. Normative References........................................13 8. Normative References........................................13
9. Author's Address............................................14 9. Author's Address............................................14
1. Introduction 1. Introduction
This draft describes the methodology for benchmarking IGP Route This document describes the methodology for benchmarking IGP Route
Convergence. The applicability of this testing is described in Convergence. The applicability of this testing is described in
[Po07a] and the new terminology that it introduces is defined in [Po07a] and the new terminology that it introduces is defined in
[Po07t]. Service Providers use IGP Convergence time as a key metric [Po07t]. Service Providers use IGP Convergence time as a key metric
of router design and architecture. Customers of Service Providers of router design and architecture. Customers of Service Providers
observe convergence time by packet loss, so IGP Route Convergence observe convergence time by packet loss, so IGP Route Convergence
is considered a Direct Measure of Quality (DMOQ). The test cases is considered a Direct Measure of Quality (DMOQ). The test cases
in this document are black-box tests that emulate the network in this document are black-box tests that emulate the network
events that cause route convergence, as described in [Po07a]. The events that cause route convergence, as described in [Po07a]. The
black-box test designs benchmark the data plane and account for black-box test designs benchmark the data plane and account for
all of the factors contributing to convergence time, as discussed all of the factors contributing to convergence time, as discussed
in [Po07a]. The methodology (and terminology) for benchmarking route in [Po07a]. The methodology (and terminology) for benchmarking route
convergence can be applied to any link-state IGP such as ISIS [Ca90] convergence can be applied to any link-state IGP such as ISIS [Ca90]
and OSPF [Mo98]. These methodologies apply to IPv4 and IPv6 traffic and OSPF [Mo98]. These methodologies apply to IPv4 and IPv6 traffic
as well as IPv4 and IPv6 IGPs. as well as IPv4 and IPv6 IGPs.
2. Existing definitions 2. Existing definitions
This document uses much of the terminology defined in [Po07t]. The
term "Throughput" is defined in RFC 2544 [Br99].
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
"SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
document are to be interpreted as described in BCP 14, RFC 2119 document are to be interpreted as described in BCP 14, RFC 2119
[Br97]. RFC 2119 defines the use of these key words to help make the [Br97]. RFC 2119 defines the use of these key words to help make the
intent of standards track documents as clear as possible. While this intent of standards track documents as clear as possible. While this
document uses these keywords, this document is not a standards track document uses these keywords, this document is not a standards track
document. The term Throughput is defined in RFC 2544 [Br99]. document.
IGP Data Plane Route Convergence IGP Data Plane Route Convergence
3. Test Setup 3. Test Setup
3.1 Test Topologies 3.1 Test Topologies
Figure 1 shows the test topology to measure IGP Route Convergence due Figure 1 shows the test topology to measure IGP Route Convergence due
to local Convergence Events such as SONET Link Failure, Layer 2 to local Convergence Events such as SONET Link Failure, Layer 2
Session Failure, IGP Adjacency Failure, Route Withdrawal, and route Session Failure, IGP Adjacency Failure, Route Withdrawal, and route
cost change. These test cases discussed in section 4 provide route cost change. These test cases discussed in section 4 provide route
convergence times that account for the Event Detection time, SPF convergence times that account for the Event Detection time, SPF
Processing time, and FIB Update time. These times are measured Processing time, and FIB Update time. These times are measured
by observing packet loss in the data plane. by observing packet loss in the data plane at the Tester.
--------- Ingress Interface --------- --------- Ingress Interface ---------
| |<--------------------------------| | | |<--------------------------------| |
| | | | | | | |
| | Preferred Egress Interface | | | | Preferred Egress Interface | |
| DUT |-------------------------------->| Tester| | DUT |-------------------------------->| Tester|
| | | | | | | |
| |~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~>| | | |~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~>| |
| | Next-Best Egress Interface | | | | Next-Best Egress Interface | |
--------- --------- --------- ---------
Figure 1. IGP Route Convergence Test Topology for Local Changes Figure 1. IGP Route Convergence Test Topology for Local Changes
Figure 2 shows the test topology to measure IGP Route Convergence Figure 2 shows the test topology to measure IGP Route Convergence
time due to remote changes in the network topology. These times are time due to remote changes in the network topology. These times are
measured by observing packet loss in the data plane. In this measured by observing packet loss in the data plane at the Tester.
topology the three routers are considered a System Under Test (SUT). In this topology the three routers are considered a System Under
NOTE: All routers in the SUT must be the same model and identically Test (SUT). NOTE: All routers in the SUT must be the same model and
configured. identically configured.
----- --------- ----- ---------
| | Preferred | | | | Preferred | |
----- |R2 |---------------------->| | ----- |R2 |---------------------->| |
| |-->| | Egress Interface | | | |-->| | Egress Interface | |
| | ----- | | | | ----- | |
|R1 | |Tester | |R1 | |Tester |
| | ----- | | | | ----- | |
| |-->| | Next-Best | | | |-->| | Next-Best | |
----- |R3 |~~~~~~~~~~~~~~~~~~~~~~>| | ----- |R3 |~~~~~~~~~~~~~~~~~~~~~~>| |
skipping to change at page 3, line 56 skipping to change at page 3, line 56
| ----- --------- | ----- ---------
| | | |
|-------------------------------------- |--------------------------------------
Ingress Interface Ingress Interface
Figure 2. IGP Route Convergence Test Topology Figure 2. IGP Route Convergence Test Topology
for Remote Changes for Remote Changes
Figure 3 shows the test topology to measure IGP Route Convergence Figure 3 shows the test topology to measure IGP Route Convergence
time with members of an Equal Cost Multipath (ECMP) Set. These time with members of an Equal Cost Multipath (ECMP) Set. These
times are measured by observing packet loss in the data plane. times are measured by observing packet loss in the data plane at
In this topology, the DUT is configured with each Egress interface the Tester. In this topology, the DUT is configured with each
Egress interface
IGP Data Plane Route Convergence IGP Data Plane Route Convergence
as a member of an ECMP set and the Tester emulates multiple as a member of an ECMP set and the Tester emulates multiple
next-hop routers (emulates one router for each member). next-hop routers (emulates one router for each member).
--------- Ingress Interface --------- --------- Ingress Interface ---------
| |<--------------------------------| | | |<--------------------------------| |
| | | | | | | |
| | ECMP Set Interface 1 | | | | ECMP Set Interface 1 | |
| DUT |-------------------------------->| Tester| | DUT |-------------------------------->| Tester|
skipping to change at page 4, line 26 skipping to change at page 4, line 26
| | . | | | | . | |
| |~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~>| | | |~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~>| |
| | ECMP Set Interface N | | | | ECMP Set Interface N | |
--------- --------- --------- ---------
Figure 3. IGP Route Convergence Test Topology Figure 3. IGP Route Convergence Test Topology
for ECMP Convergence for ECMP Convergence
Figure 4 shows the test topology to measure IGP Route Convergence Figure 4 shows the test topology to measure IGP Route Convergence
time with members of a Parallel Link. These times are measured by time with members of a Parallel Link. These times are measured by
observing packet loss in the data plane. In this topology, the DUT observing packet loss in the data plane at the Tester. In this
is configured with each Egress interface as a member of a Parallel topology, the DUT is configured with each Egress interface as a
Link and the Tester emulates the single next-hop router. member of a Parallel Link and the Tester emulates the single
next-hop router.
--------- Ingress Interface --------- --------- Ingress Interface ---------
| |<--------------------------------| | | |<--------------------------------| |
| | | | | | | |
| | Parallel Link Interface 1 | | | | Parallel Link Interface 1 | |
| DUT |-------------------------------->| Tester| | DUT |-------------------------------->| Tester|
| | . | | | | . | |
| | . | | | | . | |
| | . | | | | . | |
| |~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~>| | | |~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~>| |
skipping to change at page 4, line 52 skipping to change at page 4, line 53
Figure 4. IGP Route Convergence Test Topology Figure 4. IGP Route Convergence Test Topology
for Parallel Link Convergence for Parallel Link Convergence
3.2 Test Considerations 3.2 Test Considerations
3.2.1 IGP Selection 3.2.1 IGP Selection
The test cases described in section 4 can be used for ISIS or The test cases described in section 4 can be used for ISIS or
OSPF. The Route Convergence test methodology for both is OSPF. The Route Convergence test methodology for both is
identical. The IGP adjacencies are established on the Preferred identical. The IGP adjacencies are established on the Preferred
Egress Interface and Next-Best Egress Interface. Egress Interface and Next-Best Egress Interface.
3.2.2 BGP Configuration 3.2.2 Routing Protocol Configuration
The obtained results for IGP Route Convergence may vary if The obtained results for IGP Route Convergence may vary if
BGP routes are installed. It is recommended that the IGP other routing protocols are enabled and routes learned via those
Convergence times are benchmarked without BGP routes installed. protocols are installed. IGP convergence times MUST be benchmarked
without routes installed from other protocols.
IGP Data Plane Route Convergence IGP Data Plane Route Convergence
3.2.3 IGP Route Scaling 3.2.3 IGP Route Scaling
The number of IGP routes will impact the measured IGP Route The number of IGP routes will impact the measured IGP Route
Convergence because convergence for the entire IGP route table Convergence. To obtain results similar to those that would be
is measured. To obtain results similar to those that would be
observed in an operational network, it is recommended that the observed in an operational network, it is recommended that the
number of installed routes closely approximates that for routers number of installed routes closely approximates that the network.
in the network. The number of areas (for OSPF) and levels (for The number of areas (for OSPF) and levels (for ISIS) can impact
ISIS) can impact the benchmark results. the benchmark results.
3.2.4 Timers 3.2.4 Timers
There are some timers that will impact the measured IGP Convergence There are some timers that will impact the measured IGP Convergence
time. The following timers should be configured to the minimum value time. Benchmarking metrics may be measured at any fixed values for
prior to beginning execution of the test cases: these timers. It is RECOMMENDED that the following timers be
configured to the minimum values listed:
Timer Recommended Value Timer Recommended Value
----- ----------------- ----- -----------------
Link Failure Indication Delay <10milliseconds Link Failure Indication Delay <10milliseconds
IGP Hello Timer 1 second IGP Hello Timer 1 second
IGP Dead-Interval 3 seconds IGP Dead-Interval 3 seconds
LSA Generation Delay 0 LSA Generation Delay 0
LSA Flood Packet Pacing 0 LSA Flood Packet Pacing 0
LSA Retransmission Packet Pacing 0 LSA Retransmission Packet Pacing 0
SPF Delay 0 SPF Delay 0
skipping to change at page 5, line 46 skipping to change at page 5, line 46
<= 100 milliseconds. If the test equipment does not permit <= 100 milliseconds. If the test equipment does not permit
the Packet Sampling Interval to be set as low as 100 msec, the Packet Sampling Interval to be set as low as 100 msec,
then both the Rate-Derived Convergence Time and Loss-Derived then both the Rate-Derived Convergence Time and Loss-Derived
Convergence Time [Po07t] must be reported. The Packet Sampling Convergence Time [Po07t] must be reported. The Packet Sampling
Interval value MUST be reported as the smallest measurable Interval value MUST be reported as the smallest measurable
convergence time. convergence time.
3.2.6 Interface Types 3.2.6 Interface Types
All test cases in this methodology document may be executed with All test cases in this methodology document may be executed with
any interface type. All interfaces MUST be the same media and any interface type. All interfaces MUST be the same media and
Throughput [Br91][Br99] for each test case. Media and protocols MUST Throughput [Br91][Br99] for each test case. This is because each
be configured for minimum failure detection delay to minimize interface type has a unique mechanism for detecting link failures
the contribution to the measured Convergence time. For example, and the speed at which that mechanism operates will influence
configure SONET with minimum carrier-loss-delay or Bi-directional the measure results. Media and protocols MUST be configured for
Forwarding Detection (BFD). minimum failure detection delay to minimize the contribution to
the measured Convergence time. For example, configure SONET with
the minimum carrier-loss-delay.
IGP Data Plane Route Convergence IGP Data Plane Route Convergence
3.2.7 Offered Load 3.2.7 Offered Load
The offered Load MUST be the Throughput of the device as defined The offered Load MUST be the Throughput of the device as defined
in [Br91] and benchmarked in [Br99] at a fixed packet size. in [Br91] and benchmarked in [Br99] at a fixed packet size.
Packet size is measured in bytes and includes the IP header and Packet size is measured in bytes and includes the IP header and
payload. The packet size is selectable and MUST be recorded. payload. The packet size is selectable and MUST be recorded.
The Forwarding Rate [Ma98] MUST be measured at the Preferred Egress The Forwarding Rate [Ma98] MUST be measured at the Preferred Egress
Interface and the Next-Best Egress Interface. The duration of Interface and the Next-Best Egress Interface. The duration of
skipping to change at page 9, line 39 skipping to change at page 9, line 39
Interface is the preferred next-hop. Interface is the preferred next-hop.
2. Send offered load at measured Throughput with fixed packet 2. Send offered load at measured Throughput with fixed packet
size to destinations matching all IGP routes from Tester to size to destinations matching all IGP routes from Tester to
DUT on Ingress Interface [Po07t]. DUT on Ingress Interface [Po07t].
3. Verify traffic routed over Preferred Egress Interface. 3. Verify traffic routed over Preferred Egress Interface.
4. Remove Layer 2 session from Tester's Neighbor Interface [Po07t] 4. Remove Layer 2 session from Tester's Neighbor Interface [Po07t]
connected to Preferred Egress Interface. connected to Preferred Egress Interface.
5. Measure Rate-Derived Convergence Time [Po07t] as DUT detects the 5. Measure Rate-Derived Convergence Time [Po07t] as DUT detects the
Layer 2 session down event and converges all IGP routes and Layer 2 session down event and converges all IGP routes and
traffic over the Next-Best Egress Interface. traffic over the Next-Best Egress Interface.
6. Restore Layer 2 session on DUT's Preferred Egress Interface. 6. Stop offered load. Wait 30 seconds for queues to drain.
7. Measure Restoration Convergence Time [Po07t] as DUT detects the Restart Offered Load.
7. Restore Layer 2 session on DUT's Preferred Egress Interface.
8. Measure Restoration Convergence Time [Po07t] as DUT detects the
session up event and converges all IGP routes and traffic session up event and converges all IGP routes and traffic
over the Preferred Egress Interface. over the Preferred Egress Interface.
Results Results
The measured IGP Convergence time is influenced by the Layer 2 The measured IGP Convergence time is influenced by the Layer 2
failure indication, SPF delay, SPF Hold time, SPF Execution failure indication, SPF delay, SPF Hold time, SPF Execution
Time, Tree Build Time, and Hardware Update Time [Po07a]. Time, Tree Build Time, and Hardware Update Time [Po07a].
IGP Data Plane Route Convergence IGP Data Plane Route Convergence
skipping to change at page 13, line 49 skipping to change at page 13, line 49
[Ca90] Callon, R., "Use of OSI IS-IS for Routing in TCP/IP and Dual [Ca90] Callon, R., "Use of OSI IS-IS for Routing in TCP/IP and Dual
Environments", RFC 1195, IETF, December 1990. Environments", RFC 1195, IETF, December 1990.
[Ma98] Mandeville, R., "Benchmarking Terminology for LAN [Ma98] Mandeville, R., "Benchmarking Terminology for LAN
Switching Devices", RFC 2285, February 1998. Switching Devices", RFC 2285, February 1998.
[Mo98] Moy, J., "OSPF Version 2", RFC 2328, IETF, April 1998. [Mo98] Moy, J., "OSPF Version 2", RFC 2328, IETF, April 1998.
[Po07a] Poretsky, S., "Considerations for Benchmarking IGP [Po07a] Poretsky, S., "Considerations for Benchmarking IGP
Convergence", draft-ietf-bmwg-igp-dataplane-conv-app-12, Convergence", draft-ietf-bmwg-igp-dataplane-conv-app-13,
work in progress, February 2007. work in progress, July 2007.
[Po07t] Poretsky, S., Imhoff, B., "Benchmarking Terminology for IGP [Po07t] Poretsky, S., Imhoff, B., "Benchmarking Terminology for IGP
Convergence", draft-ietf-bmwg-igp-dataplane-conv-term-12, Convergence", draft-ietf-bmwg-igp-dataplane-conv-term-13,
work in progress, February 2007. work in progress, July 2007.
8.2 Informative References 8.2 Informative References
None None
IGP Data Plane Route Convergence IGP Data Plane Route Convergence
9. Author's Address 9. Author's Address
Scott Poretsky Scott Poretsky
Reef Point Systems Reef Point Systems
8 New England Executive Park 8 New England Executive Park
 End of changes. 19 change blocks. 
38 lines changed or deleted 46 lines changed or added

This html diff was produced by rfcdiff 1.33. The latest version is available from http://tools.ietf.org/tools/rfcdiff/