draft-ietf-bmwg-igp-dataplane-conv-meth-17.txt   draft-ietf-bmwg-igp-dataplane-conv-meth-18.txt 
Network Working Group S. Poretsky
Internet Draft Allot Communications
Expires: September 08, 2009
Intended Status: Informational Brent Imhoff
Juniper Networks
March 08, 2009
Benchmarking Methodology for Network Working Group S. Poretsky
Link-State IGP Data Plane Route Convergence Internet-Draft Allot Communications
Intended status: Informational B. Imhoff
Expires: January 14, 2010 Juniper Networks
K. Michielsen
Cisco Systems
July 13, 2009
<draft-ietf-bmwg-igp-dataplane-conv-meth-17.txt> Benchmarking Methodology for Link-State IGP Data Plane Route Convergence
draft-ietf-bmwg-igp-dataplane-conv-meth-18
Status of this Memo Status of this Memo
This Internet-Draft is submitted to IETF in full conformance with the This Internet-Draft is submitted to IETF in full conformance with the
provisions of BCP 78 and BCP 79. provisions of BCP 78 and BCP 79. This document may contain material
from IETF Documents or IETF Contributions published or made publicly
available before November 10, 2008. The person(s) controlling the
copyright in some of this material may not have granted the IETF
Trust the right to allow modifications of such material outside the
IETF Standards Process. Without obtaining an adequate license from
the person(s) controlling the copyright in such materials, this
document may not be modified outside the IETF Standards Process, and
derivative works of it may not be created outside the IETF Standards
Process, except to format it for publication as an RFC or to
translate it into languages other than English.
Internet-Drafts are working documents of the Internet Engineering Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF), its areas, and its working groups. Note that Task Force (IETF), its areas, and its working groups. Note that
other groups may also distribute working documents as Internet- other groups may also distribute working documents as Internet-
Drafts. Drafts.
Internet-Drafts are draft documents valid for a maximum of six months Internet-Drafts are draft documents valid for a maximum of six months
and may be updated, replaced, or obsoleted by other documents at any and may be updated, replaced, or obsoleted by other documents at any
time. It is inappropriate to use Internet-Drafts as reference time. It is inappropriate to use Internet-Drafts as reference
material or to cite them other than as "work in progress." material or to cite them other than as "work in progress."
skipping to change at page 1, line 31 skipping to change at page 1, line 41
other groups may also distribute working documents as Internet- other groups may also distribute working documents as Internet-
Drafts. Drafts.
Internet-Drafts are draft documents valid for a maximum of six months Internet-Drafts are draft documents valid for a maximum of six months
and may be updated, replaced, or obsoleted by other documents at any and may be updated, replaced, or obsoleted by other documents at any
time. It is inappropriate to use Internet-Drafts as reference time. It is inappropriate to use Internet-Drafts as reference
material or to cite them other than as "work in progress." material or to cite them other than as "work in progress."
The list of current Internet-Drafts can be accessed at The list of current Internet-Drafts can be accessed at
http://www.ietf.org/ietf/1id-abstracts.txt. http://www.ietf.org/ietf/1id-abstracts.txt.
The list of Internet-Draft Shadow Directories can be accessed at The list of Internet-Draft Shadow Directories can be accessed at
http://www.ietf.org/shadow.html. http://www.ietf.org/shadow.html.
This Internet-Draft will expire on September 8, 2009. This Internet-Draft will expire on January 14, 2010.
Copyright Notice Copyright Notice
Copyright (c) 2009 IETF Trust and the persons identified as the Copyright (c) 2009 IETF Trust and the persons identified as the
document authors. All rights reserved. document authors. All rights reserved.
This document is subject to BCP 78 and the IETF Trust's Legal This document is subject to BCP 78 and the IETF Trust's Legal
Provisions Relating to IETF Documents in effect on the date of Provisions Relating to IETF Documents in effect on the date of
publication of this document (http://trustee.ietf.org/license-info). publication of this document (http://trustee.ietf.org/license-info).
Please review these documents carefully, as they describe your rights Please review these documents carefully, as they describe your rights
and restrictions with respect to this document. and restrictions with respect to this document.
ABSTRACT Abstract
This document describes the methodology for benchmarking Interior
Gateway Protocol (IGP) Route Convergence. The methodology is to
be used for benchmarking IGP convergence time through externally
observable (black box) data plane measurements. The methodology
can be applied to any link-state IGP, such as ISIS and OSPF.
Link-State IGP Data Plane Route Convergence This document describes the methodology for benchmarking Link-State
Interior Gateway Protocol (IGP) Route Convergence. The methodology
is to be used for benchmarking IGP convergence time through
externally observable (black box) data plane measurements. The
methodology can be applied to any link-state IGP, such as ISIS and
OSPF.
Table of Contents Table of Contents
1. Introduction and Scope......................................2
2. Existing Definitions .......................................2 1. Introduction and Scope . . . . . . . . . . . . . . . . . . . . 5
3. Test Setup..................................................3 2. Existing Definitions . . . . . . . . . . . . . . . . . . . . . 5
3.1 Test Topologies............................................3 3. Test Topologies . . . . . . . . . . . . . . . . . . . . . . . 5
3.2 Test Considerations........................................5 3.1. Test topology for local changes . . . . . . . . . . . . . 5
3.3 Reporting Format...........................................8 3.2. Test topology for remote changes . . . . . . . . . . . . . 6
4. Test Cases..................................................9 3.3. Test topology for local ECMP changes . . . . . . . . . . . 7
4.1 Convergence Due to Local Interface Failure.................9 3.4. Test topology for remote ECMP changes . . . . . . . . . . 7
4.2 Convergence Due to Remote Interface Failure................10 3.5. Test topology for Parallel Link changes . . . . . . . . . 8
4.3 Convergence Due to Local Administrative Shutdown...........11 4. Convergence Time and Loss of Connectivity Period . . . . . . . 9
4.4 Convergence Due to Layer 2 Session Loss....................11 5. Test Considerations . . . . . . . . . . . . . . . . . . . . . 13
4.5 Convergence Due to Loss of IGP Adjacency...................12 5.1. IGP Selection . . . . . . . . . . . . . . . . . . . . . . 13
4.6 Convergence Due to Route Withdrawal........................13 5.2. Routing Protocol Configuration . . . . . . . . . . . . . . 13
4.7 Convergence Due to Cost Change.............................14 5.3. IGP Topology . . . . . . . . . . . . . . . . . . . . . . . 13
4.8 Convergence Due to ECMP Member Interface Failure...........15 5.4. Timers . . . . . . . . . . . . . . . . . . . . . . . . . . 14
4.9 Convergence Due to ECMP Member Remote Interface Failure....16 5.5. Interface Types . . . . . . . . . . . . . . . . . . . . . 14
4.10 Convergence Due to Parallel Link Interface Failure........16 5.6. Offered Load . . . . . . . . . . . . . . . . . . . . . . . 14
5. IANA Considerations.........................................17 5.7. Measurement Accuracy . . . . . . . . . . . . . . . . . . . 15
6. Security Considerations.....................................17 5.8. Measurement Statistics . . . . . . . . . . . . . . . . . . 15
7. Acknowledgements............................................17 5.9. Tester Capabilities . . . . . . . . . . . . . . . . . . . 15
8. References..................................................18 6. Selection of Convergence Time Benchmark Metrics and Methods . 16
9. Author's Address............................................18 6.1. Loss-Derived Method . . . . . . . . . . . . . . . . . . . 16
6.1.1. Tester capabilities . . . . . . . . . . . . . . . . . 16
6.1.2. Benchmark Metrics . . . . . . . . . . . . . . . . . . 16
6.1.3. Measurement Accuracy . . . . . . . . . . . . . . . . . 16
6.2. Rate-Derived Method . . . . . . . . . . . . . . . . . . . 17
6.2.1. Tester Capabilities . . . . . . . . . . . . . . . . . 17
6.2.2. Benchmark Metrics . . . . . . . . . . . . . . . . . . 17
6.2.3. Measurement Accuracy . . . . . . . . . . . . . . . . . 17
6.3. Route-Specific Loss-Derived Method . . . . . . . . . . . . 17
6.3.1. Tester Capabilities . . . . . . . . . . . . . . . . . 17
6.3.2. Benchmark Metrics . . . . . . . . . . . . . . . . . . 18
6.3.3. Measurement Accuracy . . . . . . . . . . . . . . . . . 18
7. Reporting Format . . . . . . . . . . . . . . . . . . . . . . . 18
8. Test Cases . . . . . . . . . . . . . . . . . . . . . . . . . . 20
8.1. Interface failures . . . . . . . . . . . . . . . . . . . . 21
8.1.1. Convergence Due to Local Interface Failure . . . . . . 21
8.1.2. Convergence Due to Remote Interface Failure . . . . . 22
8.1.3. Convergence Due to ECMP Member Local Interface
Failure . . . . . . . . . . . . . . . . . . . . . . . 24
8.1.4. Convergence Due to ECMP Member Remote Interface
Failure . . . . . . . . . . . . . . . . . . . . . . . 25
8.1.5. Convergence Due to Parallel Link Interface Failure . . 26
8.2. Other failures . . . . . . . . . . . . . . . . . . . . . . 27
8.2.1. Convergence Due to Layer 2 Session Loss . . . . . . . 27
8.2.2. Convergence Due to Loss of IGP Adjacency . . . . . . . 28
8.2.3. Convergence Due to Route Withdrawal . . . . . . . . . 30
8.3. Administrative changes . . . . . . . . . . . . . . . . . . 31
8.3.1. Convergence Due to Local Adminstrative Shutdown . . . 31
8.3.2. Convergence Due to Cost Change . . . . . . . . . . . . 32
9. Security Considerations . . . . . . . . . . . . . . . . . . . 34
10. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 34
11. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . 34
12. Normative References . . . . . . . . . . . . . . . . . . . . . 34
Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 35
1. Introduction and Scope 1. Introduction and Scope
This document describes the methodology for benchmarking Interior
Gateway Protocol (IGP) Route Convergence. The motivation and This document describes the methodology for benchmarking Link-State
applicability for this benchmarking is described in [Po09a]. Interior Gateway Protocol (IGP) convergence. The motivation and
The terminology to be used for this benchmarking is described applicability for this benchmarking is described in [Po09a]. The
in [Po09t]. Service Providers use IGP Convergence time as a key terminology to be used for this benchmarking is described in [Po09t].
metric of router design and architecture. Customers of Service
Providers observe convergence time by packet loss, so IGP Route IGP convergence time is measured on the data plane at the Tester by
Convergence is considered a Direct Measure of Quality (DMOQ). The observing packet loss through the DUT. All factors contributing to
test cases in this document are black-box tests that emulate the convergence time are accounted for by measuring on the data plane, as
network events that cause route convergence, as described in discussed in [Po09a]. The test cases in this document are black-box
[Po09a]. The black-box test designs benchmark the data plane and tests that emulate the network events that cause convergence, as
account for all of the factors contributing to convergence time, described in [Po09a].
as discussed in [Po09a]. Convergence times are measured at the
Tester on the data plane by observing packet loss through the DUT. The methodology described in this document can be applied to IPv4 and
The methodology (and terminology) for benchmarking route IPv6 traffic and link-state IGPs such as ISIS [Ca90][Ho08], OSPF
convergence can be applied to any link-state IGP such as ISIS [Mo98][Co08], and others.
[Ca90] and OSPF [Mo98] and others. These methodologies apply to
IPv4 and IPv6 traffic and IGPs.
2. Existing Definitions 2. Existing Definitions
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
"SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
document are to be interpreted as described in BCP 14, RFC 2119 document are to be interpreted as described in BCP 14, RFC 2119
[Br97]. RFC 2119 defines the use of these key words to help make the [Br97]. RFC 2119 defines the use of these key words to help make the
Link-State IGP Data Plane Route Convergence
intent of standards track documents as clear as possible. While this intent of standards track documents as clear as possible. While this
document uses these keywords, this document is not a standards track document uses these keywords, this document is not a standards track
document. document.
This document adopts the definition format in Section 2 of RFC 1242 This document uses much of the terminology defined in [Po09t] and
[Br91]. This document uses much of the terminology defined in uses existing terminology defined in other BMWG work. Examples
[Po09t]. This document uses existing terminology defined in other include, but are not limited to:
BMWG work. Examples include, but are not limited to:
Throughput [Ref.[Br91], section 3.17] Throughput [Ref.[Br91], section 3.17]
Device Under Test (DUT) [Ref.[Ma98], section 3.1.1] Device Under Test (DUT) [Ref.[Ma98], section 3.1.1]
System Under Test (SUT) [Ref.[Ma98], section 3.1.2] System Under Test (SUT) [Ref.[Ma98], section 3.1.2]
Out-of-order Packet [Ref.[Po06], section 3.3.2] Out-of-order Packet [Ref.[Po06], section 3.3.2]
Duplicate Packet [Ref.[Po06], section 3.3.3] Duplicate Packet [Ref.[Po06], section 3.3.3]
Packet Loss [Ref.[Po09t], Section 3.5] Stream [Ref.[Po06], section 3.3.2]
Loss Period [Ref.[Ko02], section 4]
3. Test Setup
3.1 Test Topologies 3. Test Topologies
Convergence times are measured at the Tester on the data plane 3.1. Test topology for local changes
by observing packet loss through the DUT. Figure 1 shows the test
topology to measure IGP Route Convergence due to local Convergence
Events such as Link Failure, Layer 2 Session Failure, IGP
Adjacency Failure, Route Withdrawal, and route cost change. These
test cases discussed in section 4 provide route convergence times
that include the Event Detection time, SPF Processing time, and
FIB Update time.
Figure 2 shows the test topology to measure IGP Route Convergence Figure 1 shows the test topology to measure IGP convergence time due
time due to remote changes in the network topology. These times to local Convergence Events such as Local Interface failure
are measured by observing packet loss in the data plane at the (Section 8.1.1), layer 2 session failure (Section 8.2.1), and IGP
Tester. In this topology the three routers are considered a System adjacency failure (Section 8.2.2). This topology is also used to
Under Test (SUT). A Remote Interface [Po09t] failure on router R2 measure IGP convergence time due to the route withdrawal
MUST result in convergence of traffic to router R3. NOTE: All (Section 8.2.3), and route cost change (Section 8.3.2) Convergence
routers in the SUT must be the same model and identically Events. IGP adjancencies MUST be established between Tester and DUT,
configured. one on the Preferred Egress Interface and one on the Next-Best Egress
Interface. For this purpose the Tester emulates two routers, each
establishing one adjacency with the DUT. An IGP adjacency MAY be
established on the Ingress Interface between Tester and DUT.
--------- Ingress Interface --------- --------- Ingress Interface ----------
| |<--------------------------------| | | |<--------------------------------| |
| | | | | | | |
| | Preferred Egress Interface | | | | Preferred Egress Interface | |
| DUT |-------------------------------->| Tester| | DUT |-------------------------------->| Tester|
| | | | | | | |
| |~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~>| | | |-------------------------------->| |
| | Next-Best Egress Interface | | | | Next-Best Egress Interface | |
--------- --------- --------- ----------
Figure 1. Test Topology 1: IGP Convergence Test Topology Figure 1: IGP convergence test topology for local changes
for Local Changes
Link-State IGP Data Plane Route Convergence
----- --------- 3.2. Test topology for remote changes
Figure 2 shows the test topology to measure IGP convergence time due
to Remote Interface failure (Section 8.1.2). In this topology the
two routers R1 and R2 are considered System Under Test (SUT) and
SHOULD be identically configured devices of the same model. IGP
adjancencies MUST be established between Tester and SUT, one on the
Preferred Egress Interface and one on the Next-Best Egress Interface.
For this purpose the Tester emulates one or two routers. An IGP
adjacency MAY be established on the Ingress Interface between Tester
and SUT. In this topology there is a possibility of a transient
microloop between R1 and R2 during convergence.
------ ----------
| | Preferred | | | | Preferred | |
----- |R2 |---------------------->| | ------ | R2 |--------------------->| |
| |-->| | Egress Interface | | | |-->| | Egress Interface | |
| | ----- | | | | ------ | |
|R1 | |Tester | |R1 | |Tester |
| | ----- | | | | Next-Best | |
| |-->| | Next-Best | | | |------------------------------>| |
----- |R3 |~~~~~~~~~~~~~~~~~~~~~~>| | ------ Egress Interface | |
^ | | Egress Interface | | ^ ----------
| ----- ---------
| | | |
|-------------------------------------- ---------------------------------------
Ingress Interface Ingress Interface
Figure 2. Test Topology 2: IGP Convergence Test Topology Figure 2: IGP convergence test topology for remote changes
for Convergence Due to Remote Changes
--------- Ingress Interface --------- 3.3. Test topology for local ECMP changes
Figure 3 shows the test topology to measure IGP convergence time due
to local Convergence Events with members of an Equal Cost Multipath
(ECMP) set (Section 8.1.3). In this topology, the DUT is configured
with each egress interface as a member of a single ECMP set and the
Tester emulates N next-hop routers, one router for each member. IGP
adjancencies MUST be established between Tester and DUT, one on each
member of the ECMP set. For this purpose each of the N routers
emulated by the Tester establishes one adjacency with the DUT. An
IGP adjacency MAY be established on the Ingress Interface between
Tester and DUT.
--------- Ingress Interface ----------
| |<--------------------------------| | | |<--------------------------------| |
| | | | | | | |
| | ECMP Set Interface 1 | | | | ECMP set interface 1 | |
| DUT |-------------------------------->| Tester| | |-------------------------------->| |
| | . | | | DUT | . | Tester |
| | . | | | | . | |
| | . | | | | . | |
| |~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~>| | | |-------------------------------->| |
| | ECMP Set Interface N | | | | ECMP set interface N | |
--------- --------- --------- ----------
Figure 3. Test Topology 3: IGP Convergence Test Topology Figure 3: IGP convergence test topology for local ECMP change
for ECMP Convergence
Figure 3 shows the test topology to measure IGP Route Convergence 3.4. Test topology for remote ECMP changes
time with members of an Equal Cost Multipath (ECMP) Set. These
times are measured by observing packet loss in the data plane at
the Tester. In this topology, the DUT is configured with each
Egress interface as a member of an ECMP set and the Tester emulates
multiple next-hop routers (emulates one router for each member).
Figure 4 shows the test topology to measure IGP Route Convergence Figure 4 shows the test topology to measure IGP convergence time due
time with members of a Parallel Link. These times are measured by to remote Convergence Events with members of an Equal Cost Multipath
observing packet loss in the data plane at the Tester. In this (ECMP) set (Section 8.1.4). In this topology the two routers R1 and
topology, the DUT is configured with each Egress interface as a R2 are considered System Under Test (SUT) and MUST be identically
member of a Parallel Link and the Tester emulates the single configured devices of the same model. Route R1 is configured with
next-hop router. each egress interface as a member of a single ECMP set and the Tester
emulates N next-hop routers, one router for each member. IGP
adjancencies MUST be established between Tester and SUT, one on each
egress interface of SUT. For this purpose each of the N routers
emulated by the Tester establishes one adjacency with the SUT. An
IGP adjacency MAY be established on the Ingress Interface between
Tester and SUT. In this topology there is a possibility of a
transient microloop between R1 and R2 during convergence.
Link-State IGP Data Plane Route Convergence ------ ----------
| | | |
------ ECMP set | R2 |---->| |
| |------------------->| | | |
| | Interface 1 ------ | |
| | | |
| | ECMP set interface 2 | |
| R1 |------------------------------>| Tester |
| | . | |
| | . | |
| | . | |
| |------------------------------>| |
------ ECMP set interface N | |
^ ----------
| |
---------------------------------------
Ingress Interface
--------- Ingress Interface --------- Figure 4: IGP convergence test topology for remote ECMP convergence
3.5. Test topology for Parallel Link changes
Figure 5 shows the test topology to measure IGP convergence time due
to local Convergence Events with members of a Parallel Link
(Section 8.1.5). In this topology, the DUT is configured with each
egress interface as a member of a Parallel Link and the Tester
emulates the single next-hop router. IGP adjancencies MUST be
established on all N members of the Parallel Link between Tester and
DUT. For this purpose the router emulated by the Tester establishes
N adjacencies with the DUT. An IGP adjacency MAY be established on
the Ingress Interface between Tester and DUT.
--------- Ingress Interface ----------
| |<--------------------------------| | | |<--------------------------------| |
| | | | | | | |
| | Parallel Link Interface 1 | | | | Parallel Link Interface 1 | |
| DUT |-------------------------------->| Tester| | |-------------------------------->| |
| | . | | | DUT | . | Tester |
| | . | | | | . | |
| | . | | | | . | |
| |~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~>| | | |-------------------------------->| |
| | Parallel Link Interface N | | | | Parallel Link Interface N | |
--------- --------- --------- ----------
Figure 4. Test Topology 4: IGP Convergence Test Topology Figure 5: IGP convergence test topology for Parallel Link changes
for Parallel Link Convergence
4. Convergence Time and Loss of Connectivity Period
Two concepts will be highlighted in this section: convergence time
and loss of connectivity period.
The Route Convergence [Po09t] time indicates the period in time
between the Convergence Event Instant [Po09t] and the instant in time
the DUT is ready to forward traffic for a specific route on its Next-
Best Egress Interface and maintains this state for the duration of
the Sustained Convergence Validation Time [Po09t]. To measure Route
Convergence time, the Convergence Event Instant and the traffic
received from the Next-Best Egress Interface need to be observed.
The Route Loss of Connectivity Period [Po09t] indicates the time
during which traffic to a specific route is lost following a
Convergence Event until Full Convergence [Po09t] completes. This
Route Loss of Connectivity Period can consist of one or more Loss
Periods [Ko02]. For the testcases described in this document it is
expected to have a single Loss Period. To measure Route Loss of
Connectivity Period, the traffic received from the Preferred Egress
Interface and the traffic received from the Next-Best Egress
Interface need to be observed.
The Route Loss of Connectivity Period is most important since that
has a direct impact on the network user's application performance.
In general the Route Convergence time is larger than or equal to the
Route Loss of Connectivity Period. Depending on which Convergence
Event occurs and how this Convergence Event is applied, traffic for a
route may still be forwarded over the Preferred Egress Interface
after the Convergence Event Instant, before converging to the Next-
Best Egress Interface. In that case the Route Loss of Connectivity
Period is shorter than the Route Convergence time.
At least one condition need to be fulfilled for Route Convergence
time to be equal to Route Loss of Connectivity Period. The condition
is that the Convergence Event causes an instantaneous traffic loss
for the measured route. A fiber cut on the Preferred Egress
Interface is an example of such a Convergence Event. For Convergence
Events caused by the Tester, such as an IGP cost change, the Tester
may start to drop all traffic received from the Preferred Egress
Interface at the Convergence Event Instant to achieve the same
result.
A second condition applies to Route Convergence time measurements
based on Connectivity Packet Loss [Po09t].This second condition is
that there is only a single Loss Period during Route Convergence.
For the testcases described in this document this is expected to be
the case.
To measure convergence time without real instantaneous traffic loss
at the Convergence Event Instant, such as a reversion of a link
failure Convergence Event, the Tester SHOULD collect a timestamp at
the time instant traffic starts and a timestamp at the Convergence
Event Instant, and only observe packet statistics on the Next-Best
Egress Interface.
The Convergence Event Instant together with the receive rate
observations on the Next-Best Egress Interface allow to derive the
convergence benchmarks using the Rate-Derived Method [Po09t].
By observing lost packets on the Next-Best Egress Interface only, the
measured packet loss is the number of lost packets between traffic
start and Convergence Recovery Instant. To measure convergence times
using a loss-derived method, packet loss between the Convergence
Event Instant and the Convergence Recovery Instant is needed. The
time between traffic start and Convergence Event Instant must be
accounted for
Figure 6 illustrates a Convergence Event without instantaneous
traffic loss for all routes. The top graph shows the Forwarding Rate
over all routes, the bottem graph shows the Forwarding Rate for a
single route Rta. Some time after the Convergence Event Instant,
Forwarding Rate observed on the Preferred Egress Interface starts to
decrease. In the example route Rta is the first route to experience
packet loss at time Ta. Some time later, the Forwarding Rate
observed on the Next-Best Egress Interface starts to increase. In
the example route Rta is the first route to complete convergence at
time Ta'.
^
Fwd |
Rate |------------- ............
| \ .
| \ .
| \ .
| \ .
|.................-.-.-.-.-.-.----------------
+----+-------+---------------+----------------->
^ ^ ^ ^ time
T0 CEI Ta Ta'
^
Fwd |
Rate |------------- .................
Rta | | .
| | .
|.............-.-.-.-.-.-.-.-.----------------
+----+-------+---------------+----------------->
^ ^ ^ ^ time
T0 CEI Ta Ta'
Preferred Egress Interface: ---
Next-Best Egress Interface: ...
With CEI the Convergence Event Instant; T0 the time instant traffic
starts; Ta the time instant traffic loss for route Rta starts; Ta'
the time instant traffic loss for route Rta ends.
Figure 6
If only packets received on the Next-Best Egress Interface are
observed, the duration of the packet loss period for route Rta
observed on the Next-Best Egress Interface can be calculated from the
received packets as in Equation 1. Since the Convergence Event
Instant is the start time for convergence time measurement, the
period in time between T0 and CEI needs to be substracted from the
calculated result to become the convergence time, as in Equation 2.
Next-Best Egress Interface packet loss period
= (packets transmitted
- packets received from Next-Best Egress Interface) / tx rate
= Ta' - T0
Equation 1
convergence time
= Next-Best Egress Interface packet loss period - (CEI - T0)
= Ta' - CEI
Equation 2
Route Loss of Connectivity Period SHOULD be measured using the Route-
Specific Loss-Derived Method. Since the start instant and end
instant of the Route Loss of Connectivity Period can be different for
each route, these can not be accurately derived by only observing
global statistics over all routes. An example may clarify this.
Following a Convergence Event, route Rta is the first route for which
packet loss starts, the Route Loss of Connectivity Period for route
Rta starts at time Ta. Route Rtb is the last route for which packet
loss starts, the Route Loss of Connectivity Period for route Rtb
starts at time Tb with Tb>Ta.
^
Fwd |
Rate |-------- -----------
| \ /
| \ /
| \ /
| \ /
| ---------------
+------------------------------------------>
^ ^ ^ ^ time
Ta Tb Ta' Tb'
Tb'' Ta''
Figure 7: Example Route Loss Of Connectivity Period
If the DUT implementation would be such that Route Rta would be the
first route for which traffic loss ends at time Ta' with Ta'>Tb.
Route Rtb would be the last route for which traffic loss ends at time
Tb' with Tb'>Ta'. By using only observing global traffic statistics
over all routes, the minimum Route Loss of Connectivity Period would
be measured as Ta'-Ta. The maximum calculated Route Loss of
Connectivity Period would be Tb'-Ta. The real minimum and maximum
Route Loss of Connectivity Periods are Ta'-Ta and Tb'-Tb.
Illustrating this with the numbers Ta=0, Tb=1, Ta'=3, and Tb'=5,
would give a LoC Period between 3 and 5 derived from the global
traffic statistics, versus the real LoC Period between 3 and 4.
If the DUT implementation would be such that route Rtb would be the
first for which packet loss ends at time Tb'' and route Rta would be
the last for which packet loss ends at time Ta'', then the minimum
and maximum Route Loss of Connectivity Periods derived by observing
only global traffic statistics would be Tb''-Ta, and Ta''-Ta. The
real minimum and maximum Route Loss of Connectivity Periods are
Tb''-Tb and Ta''-Ta. Illustrating this with the numbers Ta=0, Tb=1,
Ta''=5, Tb''=3, would give a LoC Period between 3 and 5 derived from
the global traffic statistics, versus the real LoC Period between 2
and 5.
The two implementation variations in the above example would result
in the same derived minimum and maximum Route Loss of Connectivity
Periods when only observing the global packet statistics, while the
real Route Loss of Connectivity Periods are different.
5. Test Considerations
5.1. IGP Selection
3.2 Test Considerations
3.2.1 IGP Selection
The test cases described in section 4 MAY be used for link-state The test cases described in section 4 MAY be used for link-state
IGPs, such as ISIS or OSPF. The Route Convergence test methodology IGPs, such as ISIS or OSPF. The IGP convergence time test
is identical. The IGP adjacencies are established on the Preferred methodology is identical.
Egress Interface and Next-Best Egress Interface.
3.2.2 Routing Protocol Configuration 5.2. Routing Protocol Configuration
The obtained results for IGP Route Convergence may vary if
other routing protocols are enabled and routes learned via those
protocols are installed. IGP convergence times MUST be benchmarked
without routes installed from other protocols.
When performing test cases, advertise a single IGP topology from The obtained results for IGP convergence time may vary if other
Tester to DUT on the Preferred Egress Interface [Po09t] and routing protocols are enabled and routes learned via those protocols
Next-Best Egress Interface [Po09t] using the test setup shown in are installed. IGP convergence times MUST be benchmarked without
Figure 1. These two interfaces on the DUT must peer with routes installed from other protocols.
different emulated neighbor routers for their IGP adjacencies.
The IGP topology learned on both interfaces MUST be the same
topology with the same nodes and routes.
3.2.3 IGP Route Scaling 5.3. IGP Topology
The number of IGP routes will impact the measured IGP Route
Convergence. To obtain results similar to those that would be
observed in an operational network, it is RECOMMENDED that the
number of installed routes and nodes closely approximates that
of the network (e.g. thousands of routes with tens of nodes).
The number of areas (for OSPF) and levels (for ISIS) can impact
the benchmark results.
Link-State IGP Data Plane Route Convergence The Tester emulates a single IGP topology. The DUT establishes IGP
adjacencies with one or more of the emulated routers in this single
IGP topology emulated by the Tester. See topology details in
Section 3. The emulated topology SHOULD only be advertised on the
DUT egress interfaces.
3.2.4 Timers The number of IGP routes will impact the measured IGP route
There are some timers that will impact the measured IGP Convergence convergence time. To obtain results similar to those that would be
time. Benchmarking metrics may be measured at any fixed values for observed in an operational network, it is RECOMMENDED that the number
these timers. It is RECOMMENDED that the following timers be of installed routes and nodes closely approximates that of the
configured to the minimum values listed: network (e.g. thousands of routes with tens or hundreds of nodes).
Timer Recommended Value The number of areas (for OSPF) and levels (for ISIS) can impact the
----- ----------------- benchmark results.
Link Failure Indication Delay <10milliseconds
IGP Hello Timer 1 second
IGP Dead-Interval 3 seconds
LSA Generation Delay 0
LSA Flood Packet Pacing 0
LSA Retransmission Packet Pacing 0
SPF Delay 0
3.2.5 Interface Types 5.4. Timers
All test cases in this methodology document may be executed with any
interface type. All interfaces MUST be the same media and Throughput
[Br91][Br99] for each test case. The type of media may dictate which
test cases may be executed. This is because each interface type has
a unique mechanism for detecting link failures and the speed at which
that mechanism operates will influence the measure results. Media
and protocols MUST be configured for minimum failure detection delay
to minimize the contribution to the measured Convergence time. For
example, configure SONET with the minimum carrier-loss-delay. All
interfaces SHOULD be configured as point-to-point.
3.2.6 Packet Sampling Interval There are timers that may impact the measured IGP convergence times.
The Packet Sampling Interval [Po09t] value is the fastest measurable The benchmark metrics MAY be measured at any fixed values for these
convergence time. The RECOMMENDED value for the Packet Sampling timers. To obtain results similar to those that would be observed in
Interval to be set on the Tester is 10 milliseconds. The Packet an operational network, it is RECOMMENDED to configure the timers
Sampling Interval MUST be reported. with the values as configured in the operational network.
3.2.7 Offered Load Examples of timers that may impact measured IGP convergence time
The offered load MUST be the Throughput of the device as defined in include, but are not limited to:
[Br91] and benchmarked in [Br99] at a fixed packet size. At least
one packet per route in the FIB for all routes in the FIB MUST be
offered to the DUT within the Packet Sampling interval. Packet
size is measured in bytes and includes the IP header and payload.
The packet size is selectable and MUST be recorded. The Throughput
MUST be measured at the Preferred Egress Interface and the
Next-Best Egress Interface. The duration of offered load MUST be
greater than the convergence time.
The destination addresses for the offered load MUST be distributed Interface failure indication
such that all routes are matched and each route is offered an equal
share of the total Offered Load. This requirement for the Offered
Load to be distributed to match all destinations in the route table
creates separate flows that are offered to the DUT. The capability
of the Tester to measure packet loss for each individual flow
Link-State IGP Data Plane Route Convergence
(identified by the destination address matching a route entry) and IGP hello timer
the scale for the number of individual flows for which it can
measure packet loss should be considered when benchmarking
Route-Specific Convergence [Po09t].
3.2.8 Selection of Convergence Time Benchmark Metrics and Methods IGP dead-interval or hold-timer
The methodologies in the section 4 test cases MAY be applied to LSA or LSP generation delay
benchmark Full Convergence Time, First Route Convergence Time,
Reversion Convergence Time, and Route-Specific Convergence Time
[Po09t]. The First Route Convergence Time benchmark metric MAY
be measured while measuring any of these convergence benchmarks.
The benchmarking metrics may be obtained using either the
Loss-Derived Convergence Method or Rate-Derived Convergence
Method. It is RECOMMENDED that the Rate-Derived Convergence
Method be measured when benchmarking convergence times. The
Loss-Derived Convergence Method is not the preferred method to
measure convergence benchmarks because it can produce a result
that is faster than the actual convergence time. When the
Packet Sampling Interval is too large, the Rate-Derived
Convergence Method may produce a larger than actual convergence
time. In such cases the Loss-Derived Convergence Method may
produce a more accurate result.
3.2.9 Tester Capabilities LSA or LSP flood packet pacing
It is RECOMMENDED that the Tester used to execute each test case
have the following capabilities: LSA or LSP retransmission packet pacing
1. Ability to establish IGP adjacencies and advertise a single
IGP topology to one or more peers. SPF delay
2. Ability to produce convergence Event Triggers [Po09t].
3. Ability to insert a timestamp in each data packet's IP 5.5. Interface Types
payload.
2. An internal time clock to control timestamping, time All test cases in this methodology document MAY be executed with any
interface type. The type of media may dictate which test cases may
be executed. This is because each interface type has a unique
mechanism for detecting link failures and the speed at which that
mechanism operates will influence the measurement results. All
interfaces MUST be the same media and Throughput [Br91][Br99] for
each test case. All interfaces SHOULD be configured as point-to-
point.
5.6. Offered Load
The Throughput of the device, as defined in [Br91] and benchmarked in
[Br99] at a fixed packet size, needs to be determined over the
preferred path and over the next-best path. The Offered Load SHOULD
be the minumum of the measured Throughput of the device over the
primary path and over the backup path. The packet size is selectable
and MUST be recorded. Packet size is measured in bytes and includes
the IP header and payload.
In the Remote Interface failure testcases using topologies 2 and 4
there is a possibility of a transient microloop between R1 and R2
during convergence. The TTL value of the packets send by the Tester
may influence the benchmark measurements since it determines which
device in the topology may send an ICMP Time Exceeded Message for
looped packets.
The duration of the Offered Load MUST be greater than the convergence
time.
5.7. Measurement Accuracy
Since packet loss is observed to measure the Route Convergence Time,
the time between two successive packets offered to each individual
route is the highest possible accuracy of any packet loss based
measurement. When packet jitter is much less than the convergence
time, it is a negligible source of error and therefor it will be
ignored here.
5.8. Measurement Statistics
The benchmark measurements may vary for each trial, due to the
statistical nature of timer expirations, cpu scheduling, etc.
Evaluation of the test data must be done with an understanding of
generally accepted testing practices regarding repeatability,
variance and statistical significance of a small number of trials.
5.9. Tester Capabilities
It is RECOMMENDED that the Tester used to execute each test case has
the following capabilities:
1. Ability to establish IGP adjacencies and advertise a single IGP
topology to one or more peers.
2. Ability to insert a timestamp in each data packet's IP payload.
3. An internal time clock to control timestamping, time
measurements, and time calculations. measurements, and time calculations.
3. Ability to distinguish traffic load received on the
Preferred and Next-Best Interfaces [Po09t].
4. Ability to disable or tune specific Layer-2 and Layer-3
protocol functions on any interface(s).
It is not required that the Tester be capable of making non-data 4. Ability to distinguish traffic load received on the Preferred and
plane convergence observations nor to use those observations for Next-Best Interfaces [Po09t].
measurements.
Link-State IGP Data Plane Route Convergence 5. Ability to disable or tune specific Layer-2 and Layer-3 protocol
functions on any interface(s).
3.3 Reporting Format The Tester MAY be capable to make non-data plane convergence
For each test case, it is recommended that the reporting table below observations and use those observations for measurements. The Tester
is completed and all time values SHOULD be reported with resolution MAY be capable to send and receive multiple traffic Streams [Po06].
6. Selection of Convergence Time Benchmark Metrics and Methods
Different convergence time benchmark methods MAY be used to measure
convergence time benchmark metrics. The Tester capabilities are
important criteria to select a specific convergence time benchmark
method. The criteria to select a specific benchmark method include,
but are not limited to:
Tester capabilities: Sampling Interval, number of
Stream statistics to collect
Measurement accuracy: Sampling Interval, Offered Load
Test specification: number of routes
DUT capabilities: Throughput
6.1. Loss-Derived Method
6.1.1. Tester capabilities
The Offered Load SHOULD consist of a single Stream [Po06]. If
sending multiple Streams, the measured packet loss statistics for all
Streams MUST be added together.
The destination addresses for the Offered Load MUST be distributed
such that all routes are matched and each route is offered an equal
share of the total Offered Load.
In order to verify Full Convergence completion and the Sustained
Convergence Validation Time, the Tester MUST measure Forwarding Rate
each Packet Sampling Interval.
The total number of packets lost between the start of the traffic and
the end of the Sustained Convergence Validation Time is used to
calculate the Loss-Derived Convergence Time.
6.1.2. Benchmark Metrics
The Loss-Derived Method can be used to measure the Loss-Derived
Convergence Time, which is the average convergence time over all
routes, and to measure the Loss-Derived Loss of Connectivity Period,
which is the average Route Loss of Connectivity Period over all
routes.
6.1.3. Measurement Accuracy
TBD
6.2. Rate-Derived Method
6.2.1. Tester Capabilities
The Offered Load SHOULD consist of a single Stream. If sending
multiple Streams, the measured traffic rate statistics for all
Streams MUST be added together.
The destination addresses for the Offered Load MUST be distributed
such that all routes are matched and each route is offered an equal
share of the total Offered Load.
The Tester measures Forwarding Rate each Sampling Interval. The
Packet Sampling Interval influences the observation of the different
convergence time instants. If the Packet Sampling Interval is large
in comparison to the time between the convergence time instants, then
the different time instants may not be easily identifiable from the
Forwarding Rate observation. The requirements for the Packet
Sampling Interval are specified in [Po09t]. The RECOMMENDED value
for the Packet Sampling Interval is 10 milliseconds. The Packet
Sampling Interval MUST be reported.
6.2.2. Benchmark Metrics
The Rate-Derived Method SHOULD be used to measure First Route
Convergence Time and Full Convergence Time. It SHOULD NOT be used to
measure Loss of Connectivity Period (see Section Section 4).
6.2.3. Measurement Accuracy
The measurement accuracy of the Rate-Derived Method for transitions
that occur for all routes at the same instant is equal to the Packet
Sampling Interval and for other transitions the measurement accuracy
is equal to the Packet Sampling Interval plus the time between two
consecutive packets to the same destination. The latter is the case
since packets are sent in a particular order to all destinations in a
stream and when part of the routes experience packet loss, it is
unknown where in the transmit cycle packets to these routes are sent.
This uncertainty adds to the error.
6.3. Route-Specific Loss-Derived Method
6.3.1. Tester Capabilities
The Offered Load consists of multiple Streams. To measure Route-
Specific Convergence Times, the Tester sends one Stream to each route
in the FIB. The Tester MUST measure packet loss for each Stream
seperately.
In order to verify Full Convergence completion and the Sustained
Convergence Validation Time, the Tester MUST measure packet loss each
Packet Sampling Interval. This measurement at each Packet Sampling
Interval MAY be per Stream.
Only the total packet loss measured per Stream at the end of the
Sustained Convergence Validation Time is used to calculate the
benchmark metrics with this method.
6.3.2. Benchmark Metrics
The Route-Specific Loss-Derived Method SHOULD be used to measure
Route-Specific Convergence Times. It is the RECOMMENDED method to
measure Route Loss of Connectivity Period.
Under the conditions explained in Section 4, First Route Convergence
Time and Full Convergence Time as benchmarked using Rate-Derived
Method, may be equal to the minimum resp. maximum of the Route-
Specific Convergence Times.
6.3.3. Measurement Accuracy
The measurement accuracy of the Route-Specific Loss-Derived Method is
equal to the time between two consecutive packets to the same route.
7. Reporting Format
For each test case, it is recommended that the reporting tables below
are completed and all time values SHOULD be reported with resolution
as specified in [Po09t]. as specified in [Po09t].
Parameter Units Parameter Units
--------- ----- ----------------------------------- -----------------------
Test Case test case number Test Case test case number
Test Topology (1, 2, 3, or 4) Test Topology (1, 2, 3, 4, or 5)
IGP (ISIS, OSPF, other) IGP (ISIS, OSPF, other)
Interface Type (GigE, POS, ATM, other) Interface Type (GigE, POS, ATM, other)
Packet Size offered to DUT bytes Packet Size offered to DUT bytes
Offered Load packets per second
IGP Routes advertised to DUT number of IGP routes IGP Routes advertised to DUT number of IGP routes
Nodes in emulated network number of nodes Nodes in emulated network number of nodes
Packet Sampling Interval on Tester milliseconds Packet Sampling Interval on Tester seconds
IGP Timer Values configured on DUT: Maximum Packet Delay Threshold seconds
Interface Failure Indication Delay seconds
Timer Values configured on DUT:
Interface failure indication delay seconds
IGP Hello Timer seconds IGP Hello Timer seconds
IGP Dead-Interval seconds IGP Dead-Interval or hold-time seconds
LSA Generation Delay seconds LSA Generation Delay seconds
LSA Flood Packet Pacing seconds LSA Flood Packet Pacing seconds
LSA Retransmission Packet Pacing seconds LSA Retransmission Packet Pacing seconds
SPF Delay seconds SPF Delay seconds
Forwarding Metrics
Total Packets Offered to DUT number of Packets Complete the table below for the initial Convergence Event and the
Total Packets Routed by DUT number of Packets reversion Convergence Event.
Parameter Units
------------------------------------------ ----------------------
Conversion Event (initial or reversion)
Traffic Forwarding Metrics:
Total number of packets offered to DUT number of Packets
Total number of packets forwarded by DUT number of Packets
Connectivity Packet Loss number of Packets
Convergence Packet Loss number of Packets Convergence Packet Loss number of Packets
Out-of-Order Packets number of Packets Out-of-Order Packets number of Packets
Duplicate Packets number of Packets Duplicate Packets number of Packets
Convergence Benchmarks
Full Convergence Convergence Benchmarks:
First Route Convergence Time seconds Rate-Derived Method:
Full Convergence Time (Rate-Derived) seconds
Full Convergence Time (Loss-Derived) seconds
Route-Specific Convergence
Number of Routes Measured number of flows
Route-Specific Convergence Time[n] array of seconds
Minimum R-S Convergence Time seconds
Maximum R-S Convergence Time seconds
Median R-S Convergence Time seconds
Average R-S Convergence Time seconds
Reversion
Reversion Convergence Time seconds
First Route Convergence Time seconds First Route Convergence Time seconds
Route-Specific Convergence Full Convergence Time seconds
Number of Routes Measured number of flows Loss-Derived Method:
Loss-Derived Convergence Time seconds
Route-Specific Loss-Derived Method:
Number of Routes Measured number of routes
Route-Specific Convergence Time[n] array of seconds Route-Specific Convergence Time[n] array of seconds
Minimum R-S Convergence Time seconds Minimum R-S Convergence Time seconds
Maximum R-S Convergence Time seconds Maximum R-S Convergence Time seconds
Median R-S Convergence Time seconds Median R-S Convergence Time seconds
Average R-S Convergence Time seconds Average R-S Convergence Time seconds
Link-State IGP Data Plane Route Convergence
4. Test Cases Loss of Connectivity Benchmarks:
Loss-Derived Method:
Loss-Derived Loss of Connectivity Period seconds
Route-Specific Loss-Derived Method:
Number of Routes Measured number of routes
Route LoC Period[n] array of seconds
Minimum Route LoC Period seconds
Maximum Route LoC Period seconds
Median Route LoC Period seconds
Average Route LoC Period seconds
8. Test Cases
It is RECOMMENDED that all applicable test cases be performed for It is RECOMMENDED that all applicable test cases be performed for
best characterization of the DUT. The test cases follow a generic best characterization of the DUT. The test cases follow a generic
procedure tailored to the specific DUT configuration and Convergence procedure tailored to the specific DUT configuration and Convergence
Event[Po09t]. This generic procedure is as follows: Event[Po09t]. This generic procedure is as follows:
1. Establish DUT configuration and install routes. 1. Establish DUT and Tester configurations and advertise an IGP
2. Send offered load with traffic traversing Preferred Egress topology from Tester to DUT.
Interface [Po09t].
3. Introduce Convergence Event to force traffic to Next-Best 2. Send Offered Load from Tester to DUT on ingress interface.
Egress Interface [Po09t].
4. Measure First Route Convergence Time. 3. Verify traffic is routed correctly.
5. Measure Full Convergence Time and, optionally, the
Route-Specific Convergence Times. 4. Introduce Convergence Event [Po09t].
6. Wait the Sustained Convergence Validation Time to ensure there
is no residual packet loss. 5. Measure First Route Convergence Time [Po09t].
7. Recover from Convergence Event.
8. Measure Reversion Convergence Time, and optionally the First 6. Measure Full Convergence Time [Po09t].
Route Convergence Time and Route-Specific Convergence Times.
7. Stop Offered Load.
8. Measure Route-Specific Convergence Times, Loss-Derived
Convergence Time, Route LoC Periods, and Loss-Derived LoC Period
[Po09t].
9. Wait sufficient time for queues to drain.
10. Restart Offered Load.
11. Reverse Convergence Event.
12. Measure First Route Convergence Time.
13. Measure Full Convergence Time.
14. Stop Offered Load.
15. Measure Route-Specific Convergence Times, Loss-Derived
Convergence Time, Route LoC Periods, and Loss-Derived LoC
Period.
8.1. Interface failures
8.1.1. Convergence Due to Local Interface Failure
4.1 Convergence Due to Local Interface Failure
Objective Objective
To obtain the IGP Route Convergence due to a local link failure event
at the DUT's Local Interface. To obtain the IGP convergence times due to a Local Interface failure
event.
Procedure Procedure
1. Advertise matching IGP routes and topology from Tester to DUT on
the Preferred Egress Interface [Po09t] and Next-Best Egress 1. Advertise an IGP topology from Tester to DUT using the topology
Interface [Po09t] using the topology shown in Figure 1. Set the shown in Figure 1.
cost of the routes so that the Preferred Egress Interface is the
preferred next-hop. 2. Send Offered Load from Tester to DUT on ingress interface.
2. Send offered load at measured Throughput with fixed packet
size to destinations matching all IGP routes from Tester to 3. Verify traffic is forwarded over Preferred Egress Interface.
DUT on Ingress Interface [Po09t].
3. Verify traffic is routed over Preferred Egress Interface.
4. Remove link on DUT's Preferred Egress Interface. This is the 4. Remove link on DUT's Preferred Egress Interface. This is the
Convergence Event Trigger[Po09t] that produces the Convergence Convergence Event.
Event Instant [Po09t].
5. Measure First Route Convergence Time [Po09t] as DUT detects the
link down event and begins to converge IGP routes and traffic
over the Next-Best Egress Interface.
6. Measure Full Convergence Time [Po09t] as DUT detects the
link down event and converges all IGP routes and traffic over
the Next-Best Egress Interface. Optionally, Route-Specific
Convergence Times [Po09t] MAY be measured.
7. Stop offered load. Wait 30 seconds for queues to drain.
Restart offered load.
8. Restore link on DUT's Preferred Egress Interface.
9. Measure Reversion Convergence Time [Po09t], and optionally
measure First Route Convergence Time and Route-Specific
Convergence Times, as DUT detects the link up event and
converges all IGP routes and traffic back to the Preferred
Egress Interface.
Link-State IGP Data Plane Route Convergence 5. Measure First Route Convergence Time.
6. Measure Full Convergence Time.
7. Stop Offered Load.
8. Measure Route-Specific Convergence Times and Loss-Derived
Convergence Time.
9. Wait sufficient time for queues to drain.
10. Restart Offered Load.
11. Restore link on DUT's Preferred Egress Interface.
12. Measure First Route Convergence Time.
13. Measure Full Convergence Time.
14. Stop Offered Load.
15. Measure Route-Specific Convergence Times, Loss-Derived
Convergence Time, Route LoC Periods, and Loss-Derived LoC
Period.
Results Results
The measured IGP Convergence time is influenced by the Local
link failure indication, SPF delay, SPF Hold time, SPF Execution
Time, Tree Build Time, and Hardware Update Time [Po09a].
4.2 Convergence Due to Remote Interface Failure The measured IGP convergence time may be influenced by the link
failure indication time, LSA/LSP delay, LSA/LSP generation time, LSA/
LSP flood packet pacing, SPF delay, SPF execution time, and routing
and forwarding tables update time [Po09a].
8.1.2. Convergence Due to Remote Interface Failure
Objective Objective
To obtain the IGP Route Convergence due to a Remote Interface
Failure event. To obtain the IGP convergence time due to a Remote Interface failure
event.
Procedure Procedure
1. Advertise matching IGP routes and topology from Tester to 1. Advertise an IGP topology from Tester to SUT using the topology
SUT on Preferred Egress Interface [Po09t] and Next-Best Egress shown in Figure 2.
Interface [Po09t] using the topology shown in Figure 2.
Set the cost of the routes so that the Preferred Egress 2. Send Offered Load from Tester to SUT on ingress interface.
Interface is the preferred next-hop.
2. Send offered load at measured Throughput with fixed packet 3. Verify traffic is forwarded over Preferred Egress Interface.
size to destinations matching all IGP routes from Tester to
SUT on Ingress Interface [Po09t]. 4. Remove link on Tester's interface [Po09t] connected to SUT's
3. Verify traffic is routed over Preferred Egress Interface. Preferred Egress Interface. This is the Convergence Event.
4. Remove link on Tester's Neighbor Interface [Po09t] connected to
SUT's Preferred Egress Interface. This is the Convergence Event 5. Measure First Route Convergence Time.
Trigger [Po09t] that produces the Convergence Event Instant
[Po09t]. 6. Measure Full Convergence Time.
5. Measure First Route Convergence Time [Po09t] as SUT detects the
link down event and begins to converge IGP routes and traffic 7. Stop Offered Load.
over the Next-Best Egress Interface.
6. Measure Full Convergence Time [Po09t] as SUT detects 8. Measure Route-Specific Convergence Times and Loss-Derived
the link down event and converges all IGP routes and traffic Convergence Time.
over the Next-Best Egress Interface. Optionally, Route-Specific
Convergence Times [Po09t] MAY be measured. 9. Wait sufficient time for queues to drain.
7. Stop offered load. Wait 30 seconds for queues to drain.
Restart offered load. 10. Restart Offered Load.
8. Restore link on Tester's Neighbor Interface connected to
DUT's Preferred Egress Interface. 11. Restore link on Tester's interface connected to DUT's Preferred
9. Measure Reversion Convergence Time [Po09t], and optionally Egress Interface.
measure First Route Convergence Time [Po09t] and Route-Specific
Convergence Times [Po09t], as DUT detects the link up event and 12. Measure First Route Convergence Time.
converges all IGP routes and traffic back to the Preferred Egress
Interface. 13. Measure Full Convergence Time.
14. Stop Offered Load.
15. Measure Route-Specific Convergence Times, Loss-Derived
Convergence Time, Route LoC Periods, and Loss-Derived LoC
Period.
Results Results
The measured IGP Convergence time is influenced by the link failure
indication, LSA/LSP Flood Packet Pacing, LSA/LSP Retransmission
Packet Pacing, LSA/LSP Generation time, SPF delay, SPF Hold time,
SPF Execution Time, Tree Build Time, and Hardware Update Time
[Po09a]. This test case may produce Stale Forwarding [Po09t] due to
microloops which may increase the measured convergence times.
Link-State IGP Data Plane Route Convergence The measured IGP convergence time may be influenced by the link
failure indication time, LSA/LSP delay, LSA/LSP generation time, LSA/
LSP flood packet pacing, SPF delay, SPF execution time, and routing
and forwarding tables update time. This test case may produce Stale
Forwarding [Po09t] due to a transient microloop between R1 and R2
during convergence, which may increase the measured convergence times
and loss of connectivity periods.
8.1.3. Convergence Due to ECMP Member Local Interface Failure
4.3 Convergence Due to Local Adminstrative Shutdown
Objective Objective
To obtain the IGP Route Convergence due to a administrative shutdown
at the DUT's Local Interface. To obtain the IGP convergence time due to a Local Interface link
failure event of an ECMP Member.
Procedure Procedure
1. Advertise matching IGP routes and topology from Tester to DUT on
Preferred Egress Interface [Po09t] and Next-Best Egress Interface 1. Advertise an IGP topology from Tester to DUT using the test
[Po09t] using the topology shown in Figure 1. Set the cost of setup shown in Figure 3.
the routes so that the Preferred Egress Interface is the
preferred next-hop. 2. Send Offered Load from Tester to DUT on ingress interface.
2. Send offered load at measured Throughput with fixed packet
size to destinations matching all IGP routes from Tester to 3. Verify traffic is forwarded over the DUT's ECMP member interface
DUT on Ingress Interface [Po09t]. that will be failed in the next step.
3. Verify traffic is routed over Preferred Egress Interface.
4. Perform adminstrative shutdown on the DUT's Preferred Egress 4. Remove link on one of the DUT's ECMP member interfaces. This is
Interface. This is the Convergence Event Trigger [Po09t] that the Convergence Event.
produces the Convergence Event Instant [Po09t].
5. Measure First Route Convergence Time [Po09t] as DUT detects the 5. Measure First Route Convergence Time.
link down event and begins to converge IGP routes and traffic
over the Next-Best Egress Interface. 6. Measure Full Convergence Time.
6. Measure Full Convergence Time [Po09t] as DUT converges
all IGP routes and traffic over the Next-Best Egress Interface. 7. Stop Offered Load.
Optionally, Route-Specific Convergence Times [Po09t] MAY be
measured. 8. Measure Route-Specific Convergence Times and Loss-Derived
7. Stop offered load. Wait 30 seconds for queues to drain. Convergence Time. At the same time measure Out-of-Order Packets
Restart offered load. [Po06] and Duplicate Packets [Po06].
8. Restore Preferred Egress Interface by administratively enabling
the interface. 9. Wait sufficient time for queues to drain.
9. Measure Reversion Convergence Time [Po09t], and optionally
measure First Route Convergence Time [Po09t] and Route-Specific 10. Restart Offered Load.
Convergence Times [Po09t], as DUT detects the link up event and
converges all IGP routes and traffic back to the Preferred 11. Restore link on DUT's ECMP member interface.
Egress Interface.
12. Measure First Route Convergence Time.
13. Measure Full Convergence Time.
14. Stop Offered Load.
15. Measure Route-Specific Convergence Times, Loss-Derived
Convergence Time, Route LoC Periods, and Loss-Derived LoC
Period. At the same time measure Out-of-Order Packets [Po06]
and Duplicate Packets [Po06].
Results Results
The measured IGP Convergence time is influenced by SPF delay, The measured IGP Convergence time may be influenced by link failure
SPF Hold time, SPF Execution Time, Tree Build Time, and Hardware indication time, LSA/LSP delay, LSA/LSP generation time, LSA/LSP
Update Time [Po09a]. flood packet pacing, SPF delay, SPF execution time, and routing and
forwarding tables update time [Po09a].
8.1.4. Convergence Due to ECMP Member Remote Interface Failure
4.4 Convergence Due to Layer 2 Session Loss
Objective Objective
To obtain the IGP Route Convergence due to a local Layer 2 loss.
To obtain the IGP convergence time due to a Remote Interface link
failure event for an ECMP Member.
Procedure Procedure
1. Advertise matching IGP routes and topology from Tester to DUT on
Preferred Egress Interface [Po09t] and Next-Best Egress Interface
[Po09t] using the topology shown in Figure 1. Set the cost of
the routes so that the IGP routes along the Preferred Egress
Interface is the preferred next-hop.
2. Send offered load at measured Throughput with fixed packet
size to destinations matching all IGP routes from Tester to
DUT on Ingress Interface [Po09t].
Link-State IGP Data Plane Route Convergence 1. Advertise an IGP topology from Tester to DUT using the test
setup shown in Figure 4.
3. Verify traffic is routed over Preferred Egress Interface. 2. Send Offered Load from Tester to DUT on ingress interface.
4. Tester removes Layer 2 session from DUT's Preferred Egress
Interface [Po09t]. It is RECOMMENDED that this be achieved with 3. Verify traffic is forwarded over the DUT's ECMP member interface
messaging, but the method MAY vary with the Layer 2 protocol. that will be failed in the next step.
This is the Convergence Event Trigger [Po09t] that produces the
Convergence Event Instant [Po09t]. 4. Remove link on Tester's interface to R2. This is the
5. Measure First Route Convergence Time [Po09t] as DUT detects the Convergence Event Trigger.
Layer 2 session down event and begins to converge IGP routes and
traffic over the Next-Best Egress Interface. 5. Measure First Route Convergence Time.
6. Measure Full Convergence Time [Po09t] as DUT detects the
Layer 2 session down event and converges all IGP routes and 6. Measure Full Convergence Time.
traffic over the Next-Best Egress Interface. Optionally,
Route-Specific Convergence Times [Po09t] MAY be measured. 7. Stop Offered Load.
7. Stop offered load. Wait 30 seconds for queues to drain.
Restart offered load. 8. Measure Route-Specific Convergence Times and Loss-Derived
8. Restore Layer 2 session on DUT's Preferred Egress Interface. Convergence Time. At the same time measure Out-of-Order Packets
9. Measure Reversion Convergence Time [Po09t], and optionally [Po06] and Duplicate Packets [Po06].
measure First Route Convergence Time [Po09t] and Route-Specific
Convergence Times [Po09t], as DUT detects the session up event 9. Wait sufficient time for queues to drain.
and converges all IGP routes and traffic over the Preferred Egress
Interface. 10. Restart Offered Load.
11. Restore link on Tester's interface to R2.
12. Measure First Route Convergence Time.
13. Measure Full Convergence Time.
14. Stop Offered Load.
15. Measure Route-Specific Convergence Times, Loss-Derived
Convergence Time, Route LoC Periods, and Loss-Derived LoC
Period. At the same time measure Out-of-Order Packets [Po06]
and Duplicate Packets [Po06].
Results Results
The measured IGP Convergence time is influenced by the Layer 2
failure indication, SPF delay, SPF Hold time, SPF Execution
Time, Tree Build Time, and Hardware Update Time [Po09a].
4.5 Convergence Due to Loss of IGP Adjacency The measured IGP convergence time may influenced by the link failure
indication time, LSA/LSP delay, LSA/LSP generation time, LSA/LSP
flood packet pacing, SPF delay, SPF execution time, and routing and
forwarding tables update time. This test case may produce Stale
Forwarding [Po09t] due to a transient microloop between R1 and R2
during convergence, which may increase the measured convergence times
and loss of connectivity periods.
8.1.5. Convergence Due to Parallel Link Interface Failure
Objective Objective
To obtain the IGP Route Convergence due to loss of the IGP
Adjacency. To obtain the IGP convergence due to a local link failure event for a
member of a parallel link. The links can be used for data load
balancing
Procedure Procedure
1. Advertise matching IGP routes and topology from Tester to DUT on
Preferred Egress Interface [Po09t] and Next-Best Egress Interface
[Po09t] using the topology shown in Figure 1. Set the cost of
the routes so that the Preferred Egress Interface is the
preferred next-hop.
2. Send offered load at measured Throughput with fixed packet
size to destinations matching all IGP routes from Tester to
DUT on Ingress Interface [Po09t].
3. Verify traffic is routed over Preferred Egress Interface.
4. Remove IGP adjacency from Tester's Neighbor Interface [Po09t]
connected to Preferred Egress Interface. The Layer 2 session
MUST be maintained. This is the Convergence Event Trigger
[Po09t] that produces the Convergence Event Instant [Po09t].
5. Measure First Route Convergence Time [Po09t] as DUT detects the
loss of IGP adjacency and begins to converge IGP routes and
traffic over the Next-Best Egress Interface.
6. Measure Full Convergence Time [Po09t] as DUT detects the
IGP session failure event and converges all IGP routes and
traffic over the Next-Best Egress Interface. Optionally,
Route-Specific Convergence Times [Po09t] MAY be measured.
Link-State IGP Data Plane Route Convergence 1. Advertise an IGP topology from Tester to DUT using the test
setup shown in Figure 5.
7. Stop offered load. Wait 30 seconds for queues to drain. 2. Send Offered Load from Tester to DUT on ingress interface.
Restart offered load.
8. Restore IGP session on DUT's Preferred Egress Interface. 3. Verify traffic is forwarded over the parallel link member that
9. Measure Reversion Convergence Time [Po09t], and optionally will be failed in the next step.
measure First Route Convergence Time [Po09t] and Route-Specific
Convergence Times [Po09t], as DUT detects the session recovery 4. Remove link on one of the DUT's parallel link member interfaces.
event and converges all IGP routes and traffic over the This is the Convergence Event.
Preferred Egress Interface.
5. Measure First Route Convergence Time.
6. Measure Full Convergence Time.
7. Stop Offered Load.
8. Measure Route-Specific Convergence Times and Loss-Derived
Convergence Time. At the same time measure Out-of-Order Packets
[Po06] and Duplicate Packets [Po06].
9. Wait sufficient time for queues to drain.
10. Restart Offered Load.
11. Restore link on DUT's Parallel Link member interface.
12. Measure First Route Convergence Time.
13. Measure Full Convergence Time.
14. Stop Offered Load.
15. Measure Route-Specific Convergence Times, Loss-Derived
Convergence Time, Route LoC Periods, and Loss-Derived LoC
Period. At the same time measure Out-of-Order Packets [Po06]
and Duplicate Packets [Po06].
Results Results
The measured IGP Convergence time is influenced by the IGP Hello
Interval, IGP Dead Interval, SPF delay, SPF Hold time, SPF
Execution Time, Tree Build Time, and Hardware Update Time [Po09a].
4.6 Convergence Due to Route Withdrawal The measured IGP convergence time may be influenced by the link
failure indication time, LSA/LSP delay, LSA/LSP generation time, LSA/
LSP flood packet pacing, SPF delay, SPF execution time, and routing
and forwarding tables update time [Po09a].
8.2. Other failures
8.2.1. Convergence Due to Layer 2 Session Loss
Objective Objective
To obtain the IGP Route Convergence due to Route Withdrawal.
To obtain the IGP convergence time due to a local layer 2 loss.
Procedure Procedure
1. Advertise a single IGP topology from Tester to DUT on Preferred
Egress Interface [Po09t] and Next-Best Egress Interface [Po09t] 1. Advertise an IGP topology from Tester to DUT using the topology
using the test setup shown in Figure 1. These two interfaces shown in Figure 1.
on the DUT must peer with different emulated neighbor routers
for their IGP adjacency. The IGP topology learned on both 2. Send Offered Load from Tester to DUT on ingress interface.
interfaces MUST be the same topology with the same nodes and
routes. It is RECOMMENDED that the IGP routes be IGP external
routes for which the Tester would be emulating a preferred and
a next-best Autonomous System Border Router (ASBR). Set the
cost of the routes so that the Preferred Egress Interface is
the preferred next-hop.
2. Send offered load at measured Throughput with fixed packet
size to destinations matching all IGP routes from Tester to
DUT on Ingress Interface [Po09t].
3. Verify traffic is routed over Preferred Egress Interface. 3. Verify traffic is routed over Preferred Egress Interface.
4. The Tester, emulating the neighbor node, withdraws one or
more IGP leaf routes from the DUT's Preferred Egress Interface.
The withdrawal update message MUST be a single unfragmented
packet. This is the Convergence Event Trigger [Po09t] that
produces the Convergence Event Instant [Po09t]. The Tester
MAY record the time it sends the withdrawal message(s).
5. Measure First Route Convergence Time [Po09t] as DUT detects the
route withdrawal event and begins to converge IGP routes and
traffic over the Next-Best Egress Interface.
6. Measure Full Convergence Time [Po09t] as DUT withdraws
routes and converges all IGP routes and traffic over the
Next-Best Egress Interface. Optionally, Route-Specific
Convergence Times [Po09t] MAY be measured.
7. Stop offered load. Wait 30 seconds for queues to drain.
Restart offered load.
8. Re-advertise the withdrawn IGP leaf routes to DUT's Preferred
Egress Interface.
Link-State IGP Data Plane Route Convergence 4. Remove Layer 2 session from DUT's Preferred Egress Interface.
This is the Convergence Event.
9. Measure Reversion Convergence Time [Po09t], and optionally 5. Measure First Route Convergence Time.
measure First Route Convergence Time [Po09t] and Route-Specific
Convergence Times [Po09t], as DUT converges all IGP routes and 6. Measure Full Convergence Time.
traffic over the Preferred Egress Interface.
7. Stop Offered Load.
8. Measure Route-Specific Convergence Times, Loss-Derived
Convergence Time, Route LoC Periods, and Loss-Derived LoC
Period.
9. Wait sufficient time for queues to drain.
10. Restart Offered Load.
11. Restore Layer 2 session on DUT's Preferred Egress Interface.
12. Measure First Route Convergence Time.
13. Measure Full Convergence Time.
14. Stop Offered Load.
15. Measure Route-Specific Convergence Times, Loss-Derived
Convergence Time, Route LoC Periods, and Loss-Derived LoC
Period.
Results Results
The measured IGP Convergence time is the SPF Processing and FIB
Update time as influenced by the SPF or route calculation delay,
Hold time, Execution Time, and Hardware Update Time [Po09a].
4.7 Convergence Due to Cost Change The measured IGP Convergence time may be influenced by the Layer 2
failure indication time, LSA/LSP delay, LSA/LSP generation time, LSA/
LSP flood packet pacing, SPF delay, SPF execution time, and routing
and forwarding tables update time [Po09a].
Discussion
Configure IGP timers such that the IGP adjacency does not time out
before layer 2 failure is detected.
To measure convergence time, traffic SHOULD start dropping on the
Preferred Egress Interface on the instant the layer 2 session is
removed. Alternatively the Tester SHOULD record the time the instant
layer 2 session is removed and traffic loss SHOULD only be measured
on the Next-Best Egress Interface.
8.2.2. Convergence Due to Loss of IGP Adjacency
Objective Objective
To obtain the IGP Route Convergence due to route cost change.
To obtain the IGP convergence time due to loss of an IGP Adjacency.
Procedure Procedure
1. Advertise a single IGP topology from Tester to DUT on 1. Advertise an IGP topology from Tester to DUT using the topology
Preferred Egress Interface [Po09t] and Next-Best Egress shown in Figure 1.
Interface [Po09t] using the test setup shown in Figure 1.
These two interfaces on the DUT must peer with different 2. Send Offered Load from Tester to DUT on ingress interface.
emulated neighbor routers for their IGP adjacency. The
IGP topology learned on both interfaces MUST be the same
topology with the same nodes and routes. It is RECOMMENDED
that the IGP routes be IGP external routes for which the
Tester would be emulating a preferred and a next-best
Autonomous System Border Router (ASBR). Set the cost of
the routes so that the Preferred Egress Interface is the
preferred next-hop.
2. Send offered load at measured Throughput with fixed packet
size to destinations matching all IGP routes from Tester to
DUT on Ingress Interface [Po09t].
3. Verify traffic is routed over Preferred Egress Interface. 3. Verify traffic is routed over Preferred Egress Interface.
4. The Tester, emulating the neighbor node, increases the cost for
all IGP routes at DUT's Preferred Egress Interface so that the
Next-Best Egress Interface has lower cost and becomes preferred
path. The update message advertising the higher cost MUST be a
single unfragmented packet. This is the Convergence Event
Trigger [Po09t] that produces the Convergence Event Instant
[Po09t]. The Tester MAY record the time it sends the message
advertising the higher cost on the Preferred Egress Interface.
5. Measure First Route Convergence Time [Po09t] as DUT detects the
cost change event and begins to converge IGP routes and traffic
over the Next-Best Egress Interface.
6. Measure Full Convergence Time [Po09t] as DUT detects the
cost change event and converges all IGP routes and traffic
over the Next-Best Egress Interface. Optionally, Route-Specific
Convergence Times [Po09t] MAY be measured.
7. Stop offered load. Wait 30 seconds for queues to drain.
Restart offered load.
8. Re-advertise IGP routes to DUT's Preferred Egress Interface
with original lower cost metric.
Link-State IGP Data Plane Route Convergence 4. Remove IGP adjacency from the Preferred Egress Interface while
the layer 2 session MUST be maintained. This is the Convergence
Event.
9. Measure Reversion Convergence Time [Po09t], and optionally 5. Measure First Route Convergence Time.
measure First Route Convergence Time [Po09t] and Route-Specific
Convergence Times [Po09t], as DUT converges all IGP routes and 6. Measure Full Convergence Time.
traffic over the Preferred Egress Interface.
7. Stop Offered Load.
8. Measure Route-Specific Convergence Times, Loss-Derived
Convergence Time, Route LoC Periods, and Loss-Derived LoC
Period.
9. Wait sufficient time for queues to drain.
10. Restart Offered Load.
11. Restore IGP session on DUT's Preferred Egress Interface.
12. Measure First Route Convergence Time.
13. Measure Full Convergence Time.
14. Stop Offered Load.
15. Measure Route-Specific Convergence Times, Loss-Derived
Convergence Time, Route LoC Periods, and Loss-Derived LoC
Period.
Results Results
It is possible that no measured packet loss will be observed for
this test case.
4.8 Convergence Due to ECMP Member Interface Failure The measured IGP Convergence time may be influenced by the IGP Hello
Interval, IGP Dead Interval, LSA/LSP delay, LSA/LSP generation time,
LSA/LSP flood packet pacing, SPF delay, SPF execution time, and
routing and forwarding tables update time [Po09a].
Discussion
Configure layer 2 such that layer 2 does not time out before IGP
adjacency failure is detected.
To measure convergence time, traffic SHOULD start dropping on the
Preferred Egress Interface on the instant the IGP adjacency is
removed. Alternatively the Tester SHOULD record the time the instant
the IGP adjacency is removed and traffic loss SHOULD only be measured
on the Next-Best Egress Interface.
8.2.3. Convergence Due to Route Withdrawal
Objective Objective
To obtain the IGP Route Convergence due to a local link failure event
of an ECMP Member. To obtain the IGP convergence time due to route withdrawal.
Procedure Procedure
1. Configure ECMP Set as shown in Figure 3.
2. Advertise matching IGP routes and topology from Tester to DUT on 1. Advertise an IGP topology from Tester to DUT using the topology
each ECMP member. shown in Figure 1. The routes that will be withdrawn MUST be a
3. Send offered load at measured Throughput with fixed packet size to set of leaf routes advertised by at least two nodes in the
destinations matching all IGP routes from Tester to DUT on Ingress emulated topology. The topology SHOULD be such that before the
Interface [Po09t]. withdrawal the DUT prefers the leaf routes advertised by a node
4. Verify traffic is routed over all members of ECMP Set. "nodeA" via the Preferred Egress Interface, and after the
5. Remove link on Tester's Neighbor Interface [Po09t] connected to withdrawal the DUT prefers the leaf routes advertised by a node
one of the DUT's ECMP member interfaces. This is the Convergence "nodeB" via the Next-Best Egress Interface.
Event Trigger [Po09t] that produces the Convergence Event Instant
[Po09t]. 2. Send Offered Load from Tester to DUT on Ingress Interface.
6. Measure First Route Convergence Time [Po09t] as DUT detects the
link down event and begins to converge IGP routes and traffic 3. Verify traffic is routed over Preferred Egress Interface.
over the other ECMP members.
7. Measure Full Convergence Time [Po09t] as DUT detects 4. The Tester withdraws the set of IGP leaf routes from nodeA. The
the link down event and converges all IGP routes and traffic withdrawal update message MUST be a single unfragmented packet.
over the other ECMP members. At the same time measure This is the Convergence Event. The Tester MAY record the time
Out-of-Order Packets [Po06] and Duplicate Packets [Po06]. it sends the withdrawal message(s).
Optionally, Route-Specific Convergence Times [Po09t] MAY be
measured. 5. Measure First Route Convergence Time.
8. Stop offered load. Wait 30 seconds for queues to drain.
Restart offered load. 6. Measure Full Convergence Time.
9. Restore link on Tester's Neighbor Interface connected to
DUT's ECMP member interface. 7. Stop Offered Load.
10. Measure Reversion Convergence Time [Po09t], and optionally
measure First Route Convergence Time [Po09t] and Route-Specific 8. Measure Route-Specific Convergence Times, Loss-Derived
Convergence Times [Po09t], as DUT detects the link up event and Convergence Time, Route LoC Periods, and Loss-Derived LoC
converges IGP routes and some distribution of traffic over the Period.
restored ECMP member.
9. Wait sufficient time for queues to drain.
10. Restart Offered Load.
11. Re-advertise the set of withdrawn IGP leaf routes from nodeA
emulated by the Tester. The update message MUST be a single
unfragmented packet.
12. Measure First Route Convergence Time.
13. Measure Full Convergence Time.
14. Stop Offered Load.
15. Measure Route-Specific Convergence Times, Loss-Derived
Convergence Time, Route LoC Periods, and Loss-Derived LoC
Period.
Results Results
The measured IGP Convergence time is influenced by Local link
failure indication, Tree Build Time, and Hardware Update Time
[Po09a].
Link-State IGP Data Plane Route Convergence The measured IGP convergence time is influenced by SPF or route
calculation delay, SPF or route calculation execution time, and
routing and forwarding tables update time [Po09a].
4.9 Convergence Due to ECMP Member Remote Interface Failure Discussion
To measure convergence time, traffic SHOULD start dropping on the
Preferred Egress Interface on the instant the routes are withdrawn by
the Tester. Alternatively the Tester SHOULD record the time the
instant the routes are withdrawn and traffic loss SHOULD only be
measured on the Next-Best Egress Interface.
8.3. Administrative changes
8.3.1. Convergence Due to Local Adminstrative Shutdown
Objective Objective
To obtain the IGP Route Convergence due to a remote interface
failure event for an ECMP Member. To obtain the IGP convergence time due to taking the DUT's Local
Interface administratively out of service.
Procedure Procedure
1. Configure ECMP Set as shown in Figure 2 in which the links
from R1 to R2 and R1 to R3 are members of an ECMP Set. 1. Advertise an IGP topology from Tester to DUT using the topology
2. Advertise matching IGP routes and topology from Tester to shown in Figure 1.
SUT to balance traffic to each ECMP member.
3. Send offered load at measured Throughput with fixed packet 2. Send Offered Load from Tester to DUT on ingress interface.
size to destinations matching all IGP routes from Tester to
SUT on Ingress Interface [Po09t]. 3. Verify traffic is routed over Preferred Egress Interface.
4. Verify traffic is routed over all members of ECMP Set.
5. Remove link on Tester's Neighbor Interface to R2 or R3. 4. Take the DUT's Preferred Egress Interface administratively out
This is the Convergence Event Trigger [Po09t] that produces of service. This is the Convergence Event.
the Convergence Event Instant [Po09t].
6. Measure First Route Convergence Time [Po09t] as SUT detects 5. Measure First Route Convergence Time.
the link down event and begins to converge IGP routes and
traffic over the other ECMP members. 6. Measure Full Convergence Time.
7. Measure Full Convergence Time [Po09t] as SUT detects
the link down event and converges all IGP routes and traffic 7. Stop Offered Load.
over the other ECMP members. At the same time measure
Out-of-Order Packets [Po06] and Duplicate Packets [Po06]. 8. Measure Route-Specific Convergence Times, Loss-Derived
Optionally, Route-Specific Convergence Times [Po09t] MAY be Convergence Time, Route LoC Periods, and Loss-Derived LoC
measured. Period.
8. Stop offered load. Wait 30 seconds for queues to drain.
Restart offered load. 9. Wait sufficient time for queues to drain.
9. Restore link on Tester's Neighbor Interface to R2 or R3.
10. Measure Reversion Convergence Time [Po09t], and optionally 10. Restart Offered Load.
measure First Route Convergence Time [Po09t] and
Route-Specific Convergence Times [Po09t], as SUT detects 11. Restore Preferred Egress Interface by administratively enabling
the link up event and converges IGP routes and some the interface.
distribution of traffic over the restored ECMP member.
12. Measure First Route Convergence Time.
13. Measure Full Convergence Time.
14. Stop Offered Load.
15. Measure Route-Specific Convergence Times, Loss-Derived
Convergence Time, Route LoC Periods, and Loss-Derived LoC
Period.
16. It is possible that no measured packet loss will be observed for
this test case.
Results Results
The measured IGP Convergence time is influenced by Local link
failure indication, Tree Build Time, and Hardware Update Time The measured IGP Convergence time may be influenced by LSA/LSP delay,
LSA/LSP generation time, LSA/LSP flood packet pacing, SPF delay, SPF
execution time, and routing and forwarding tables update time
[Po09a]. [Po09a].
4.10 Convergence Due to Parallel Link Interface Failure 8.3.2. Convergence Due to Cost Change
Objective Objective
To obtain the IGP Route Convergence due to a local link failure
event for a Member of a Parallel Link. The links can be used To obtain the IGP convergence time due to route cost change.
for data Load Balancing
Procedure Procedure
1. Configure Parallel Link as shown in Figure 4.
2. Advertise matching IGP routes and topology from Tester to DUT
on each Parallel Link member.
Link-State IGP Data Plane Route Convergence 1. Advertise an IGP topology from Tester to DUT using the topology
shown in Figure 1.
3. Send offered load at measured Throughput with fixed packet 2. Send Offered Load from Tester to DUT on ingress interface.
size to destinations matching all IGP routes from Tester to
DUT on Ingress Interface [Po09t]. 3. Verify traffic is routed over Preferred Egress Interface.
4. Verify traffic is routed over all members of Parallel Link.
5. Remove link on Tester's Neighbor Interface [Po09t] connected to 4. The Tester, emulating the neighbor node, increases the cost for
one of the DUT's Parallel Link member interfaces. This is the all IGP routes at DUT's Preferred Egress Interface so that the
Convergence Event Trigger [Po09t] that produces the Convergence Next-Best Egress Interface becomes preferred path. The update
Event Instant [Po09t]. message advertising the higher cost MUST be a single
6. Measure First Route Convergence Time [Po09t] as DUT detects the unfragmented packet. This is the Convergence Event. The Tester
link down event and begins to converge IGP routes and traffic MAY record the time it sends the update message advertising the
over the other Parallel Link members. higher cost on the Preferred Egress Interface.
7. Measure Full Convergence Time [Po09t] as DUT detects the
link down event and converges all IGP routes and traffic over 5. Measure First Route Convergence Time.
the other Parallel Link members. At the same time measure
Out-of-Order Packets [Po06] and Duplicate Packets [Po06]. 6. Measure Full Convergence Time.
Optionally, Route-Specific Convergence Times [Po09t] MAY be
measured. 7. Stop Offered Load.
8. Stop offered load. Wait 30 seconds for queues to drain.
Restart offered load. 8. Measure Route-Specific Convergence Times, Loss-Derived
9. Restore link on Tester's Neighbor Interface connected to Convergence Time, Route LoC Periods, and Loss-Derived LoC
DUT's Parallel Link member interface. Period.
10. Measure Reversion Convergence Time [Po09t], and optionally
measure First Route Convergence Time [Po09t] and 9. Wait sufficient time for queues to drain.
Route-Specific Convergence Times [Po09t], as DUT
detects the link up event and converges IGP routes and some 10. Restart Offered Load.
distribution of traffic over the restored Parallel Link member.
11. The Tester, emulating the neighbor node, decreases the cost for
all IGP routes at DUT's Preferred Egress Interface so that the
Preferred Egress Interface becomes preferred path. The update
message advertising the lower cost MUST be a single unfragmented
packet.
12. Measure First Route Convergence Time.
13. Measure Full Convergence Time.
14. Stop Offered Load.
15. Measure Route-Specific Convergence Times, Loss-Derived
Convergence Time, Route LoC Periods, and Loss-Derived LoC
Period.
Results Results
The measured IGP Convergence time is influenced by the Local
link failure indication, Tree Build Time, and Hardware Update
Time [Po09a].
5. IANA Considerations The measured IGP Convergence time may be influenced by SPF delay, SPF
execution time, and routing and forwarding tables update time
[Po09a].
This document requires no IANA considerations. Discussion
To measure convergence time, traffic SHOULD start dropping on the
Preferred Egress Interface on the instant the cost is changed by the
Tester. Alternatively the Tester SHOULD record the time the instant
the cost is changed and traffic loss SHOULD only be measured on the
Next-Best Egress Interface.
9. Security Considerations
6. Security Considerations
Documents of this type do not directly affect the security of Documents of this type do not directly affect the security of
Internet or corporate networks as long as benchmarking is not Internet or corporate networks as long as benchmarking is not
performed on devices or systems connected to production networks. performed on devices or systems connected to production networks.
Security threats and how to counter these in SIP and the media Security threats and how to counter these in SIP and the media layer
layer is discussed in RFC3261, RFC3550, and RFC3711 and various is discussed in RFC3261, RFC3550, and RFC3711 and various other
other drafts. This document attempts to formalize a set of drafts. This document attempts to formalize a set of common
common methodology for benchmarking IGP convergence performance methodology for benchmarking IGP convergence performance in a lab
in a lab environment. environment.
7. Acknowledgements 10. IANA Considerations
Thanks to Sue Hares, Al Morton, Kevin Dubray, Ron Bonica, David Ward,
Kris Michielsen, Peter De Vriendt and the BMWG for their
contributions to this work.
Link-State IGP Data Plane Route Convergence This document requires no IANA considerations.
8. References 11. Acknowledgements
8.1 Normative References
[Br91] Bradner, S., "Benchmarking Terminology for Network Thanks to Sue Hares, Al Morton, Kevin Dubray, Ron Bonica, David Ward,
Interconnection Devices", RFC 1242, IETF, March 1991. Peter De Vriendt and the BMWG for their contributions to this work.
12. Normative References
[Br91] Bradner, S., "Benchmarking terminology for network
interconnection devices", RFC 1242, July 1991.
[Br97] Bradner, S., "Key words for use in RFCs to Indicate [Br97] Bradner, S., "Key words for use in RFCs to Indicate
[Br99] Bradner, S. and McQuaid, J., "Benchmarking Methodology for Requirement Levels", BCP 14, RFC 2119, March 1997.
Network Interconnect Devices", RFC 2544, IETF, March 1999.
[Ca90] Callon, R., "Use of OSI IS-IS for Routing in TCP/IP and Dual [Br99] Bradner, S. and J. McQuaid, "Benchmarking Methodology for
Environments", RFC 1195, IETF, December 1990. Network Interconnect Devices", RFC 2544, March 1999.
[Ma98] Mandeville, R., "Benchmarking Terminology for LAN [Ca90] Callon, R., "Use of OSI IS-IS for routing in TCP/IP and dual
Switching Devices", RFC 2285, February 1998. environments", RFC 1195, December 1990.
[Mo98] Moy, J., "OSPF Version 2", RFC 2328, IETF, April 1998. [Co08] Coltun, R., Ferguson, D., Moy, J., and A. Lindem, "OSPF for
IPv6", RFC 5340, July 2008.
[Po06] Poretsky, S., et al., "Terminology for Benchmarking [Ho08] Hopps, C., "Routing IPv6 with IS-IS", RFC 5308,
Network-layer Traffic Control Mechanisms", RFC 4689, October 2008.
November 2006.
[Po09a] Poretsky, S., "Considerations for Benchmarking Link-State [Ko02] Koodli, R. and R. Ravikanth, "One-way Loss Pattern Sample
IGP Convergence", draft-ietf-bmwg-igp-dataplane-conv-app-17, Metrics", RFC 3357, August 2002.
work in progress, March 2009.
[Po09t] Poretsky, S., Imhoff, B., "Benchmarking Terminology for [Ma98] Mandeville, R., "Benchmarking Terminology for LAN Switching
Link-State IGP Convergence", Devices", RFC 2285, February 1998.
draft-ietf-bmwg-igp-dataplane-conv-term-17, work in
progress, March 2009.
8.2 Informative References [Mo98] Moy, J., "OSPF Version 2", STD 54, RFC 2328, April 1998.
None
9. Author's Address [Po06] Poretsky, S., Perser, J., Erramilli, S., and S. Khurana,
"Terminology for Benchmarking Network-layer Traffic Control
Mechanisms", RFC 4689, October 2006.
[Po09a] Poretsky, S., "Considerations for Benchmarking Link-State
IGP Data Plane Route Convergence",
draft-ietf-bmwg-igp-dataplane-conv-app-17 (work in
progress), March 2009.
[Po09t] Poretsky, S. and B. Imhoff, "Terminology for Benchmarking
Link-State IGP Data Plane Route Convergence",
draft-ietf-bmwg-igp-dataplane-conv-term-18 (work in
progress), July 2009.
Authors' Addresses
Scott Poretsky Scott Poretsky
Allot Communications Allot Communications
67 South Bedford Street, Suite 400 67 South Bedford Street, Suite 400
Burlington, MA 01803 Burlington, MA 01803
USA USA
Phone: + 1 508 309 2179 Phone: + 1 508 309 2179
Email: sporetsky@allot.com Email: sporetsky@allot.com
Brent Imhoff Brent Imhoff
Juniper Networks Juniper Networks
1194 North Mathilda Ave 1194 North Mathilda Ave
Sunnyvale, CA 94089 Sunnyvale, CA 94089
USA USA
Phone: + 1 314 378 2571 Phone: + 1 314 378 2571
EMail: bimhoff@planetspork.com Email: bimhoff@planetspork.com
Kris Michielsen
Cisco Systems
6A De Kleetlaan
Diegem, BRABANT 1831
Belgium
Email: kmichiel@cisco.com
 End of changes. 148 change blocks. 
688 lines changed or deleted 1268 lines changed or added

This html diff was produced by rfcdiff 1.35. The latest version is available from http://tools.ietf.org/tools/rfcdiff/