draft-ietf-bmwg-igp-dataplane-conv-meth-23.txt   rfc6413.txt 
Network Working Group S. Poretsky Internet Engineering Task Force (IETF) S. Poretsky
Internet-Draft Allot Communications Request for Comments: 6413 Allot Communications
Intended status: Informational B. Imhoff Category: Informational B. Imhoff
Expires: August 13, 2011 Juniper Networks ISSN: 2070-1721 Juniper Networks
K. Michielsen K. Michielsen
Cisco Systems Cisco Systems
February 16, 2011 November 2011
Benchmarking Methodology for Link-State IGP Data Plane Route Convergence Benchmarking Methodology for Link-State IGP Data-Plane Route Convergence
draft-ietf-bmwg-igp-dataplane-conv-meth-23
Abstract Abstract
This document describes the methodology for benchmarking Link-State This document describes the methodology for benchmarking Link-State
Interior Gateway Protocol (IGP) Route Convergence. The methodology Interior Gateway Protocol (IGP) Route Convergence. The methodology
is to be used for benchmarking IGP convergence time through is to be used for benchmarking IGP convergence time through
externally observable (black box) data plane measurements. The externally observable (black-box) data-plane measurements. The
methodology can be applied to any link-state IGP, such as IS-IS and methodology can be applied to any link-state IGP, such as IS-IS and
OSPF. OSPF.
Status of this Memo Status of This Memo
This Internet-Draft is submitted in full conformance with the
provisions of BCP 78 and BCP 79.
Internet-Drafts are working documents of the Internet Engineering This document is not an Internet Standards Track specification; it is
Task Force (IETF). Note that other groups may also distribute published for informational purposes.
working documents as Internet-Drafts. The list of current Internet-
Drafts is at http://datatracker.ietf.org/drafts/current/.
Internet-Drafts are draft documents valid for a maximum of six months This document is a product of the Internet Engineering Task Force
and may be updated, replaced, or obsoleted by other documents at any (IETF). It represents the consensus of the IETF community. It has
time. It is inappropriate to use Internet-Drafts as reference received public review and has been approved for publication by the
material or to cite them other than as "work in progress." Internet Engineering Steering Group (IESG). Not all documents
approved by the IESG are a candidate for any level of Internet
Standard; see Section 2 of RFC 5741.
This Internet-Draft will expire on August 13, 2011. Information about the current status of this document, any errata,
and how to provide feedback on it may be obtained at
http://www.rfc-editor.org/info/rfc6413.
Copyright Notice Copyright Notice
Copyright (c) 2011 IETF Trust and the persons identified as the Copyright (c) 2011 IETF Trust and the persons identified as the
document authors. All rights reserved. document authors. All rights reserved.
This document is subject to BCP 78 and the IETF Trust's Legal This document is subject to BCP 78 and the IETF Trust's Legal
Provisions Relating to IETF Documents Provisions Relating to IETF Documents
(http://trustee.ietf.org/license-info) in effect on the date of (http://trustee.ietf.org/license-info) in effect on the date of
publication of this document. Please review these documents publication of this document. Please review these documents
skipping to change at page 3, line 7 skipping to change at page 2, line 21
modifications of such material outside the IETF Standards Process. modifications of such material outside the IETF Standards Process.
Without obtaining an adequate license from the person(s) controlling Without obtaining an adequate license from the person(s) controlling
the copyright in such materials, this document may not be modified the copyright in such materials, this document may not be modified
outside the IETF Standards Process, and derivative works of it may outside the IETF Standards Process, and derivative works of it may
not be created outside the IETF Standards Process, except to format not be created outside the IETF Standards Process, except to format
it for publication as an RFC or to translate it into languages other it for publication as an RFC or to translate it into languages other
than English. than English.
Table of Contents Table of Contents
1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 5 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 4
1.1. Motivation . . . . . . . . . . . . . . . . . . . . . . . . 5 1.1. Motivation . . . . . . . . . . . . . . . . . . . . . . . . 4
1.2. Factors for IGP Route Convergence Time . . . . . . . . . . 5 1.2. Factors for IGP Route Convergence Time . . . . . . . . . . 4
1.3. Use of Data Plane for IGP Route Convergence 1.3. Use of Data Plane for IGP Route Convergence
Benchmarking . . . . . . . . . . . . . . . . . . . . . . . 6 Benchmarking . . . . . . . . . . . . . . . . . . . . . . . 5
1.4. Applicability and Scope . . . . . . . . . . . . . . . . . 7 1.4. Applicability and Scope . . . . . . . . . . . . . . . . . 6
2. Existing Definitions . . . . . . . . . . . . . . . . . . . . . 7 2. Existing Definitions . . . . . . . . . . . . . . . . . . . . . 6
3. Test Topologies . . . . . . . . . . . . . . . . . . . . . . . 8 3. Test Topologies . . . . . . . . . . . . . . . . . . . . . . . 7
3.1. Test topology for local changes . . . . . . . . . . . . . 8 3.1. Test Topology for Local Changes . . . . . . . . . . . . . 7
3.2. Test topology for remote changes . . . . . . . . . . . . . 9 3.2. Test Topology for Remote Changes . . . . . . . . . . . . . 8
3.3. Test topology for local ECMP changes . . . . . . . . . . . 11 3.3. Test Topology for Local ECMP Changes . . . . . . . . . . . 10
3.4. Test topology for remote ECMP changes . . . . . . . . . . 11 3.4. Test Topology for Remote ECMP Changes . . . . . . . . . . 11
3.5. Test topology for Parallel Link changes . . . . . . . . . 12 3.5. Test topology for Parallel Link Changes . . . . . . . . . 11
4. Convergence Time and Loss of Connectivity Period . . . . . . . 13 4. Convergence Time and Loss of Connectivity Period . . . . . . . 12
4.1. Convergence Events without instant traffic loss . . . . . 14 4.1. Convergence Events without Instant Traffic Loss . . . . . 13
4.2. Loss of Connectivity (LoC) . . . . . . . . . . . . . . . . 16 4.2. Loss of Connectivity (LoC) . . . . . . . . . . . . . . . . 16
5. Test Considerations . . . . . . . . . . . . . . . . . . . . . 17 5. Test Considerations . . . . . . . . . . . . . . . . . . . . . 17
5.1. IGP Selection . . . . . . . . . . . . . . . . . . . . . . 17 5.1. IGP Selection . . . . . . . . . . . . . . . . . . . . . . 17
5.2. Routing Protocol Configuration . . . . . . . . . . . . . . 17 5.2. Routing Protocol Configuration . . . . . . . . . . . . . . 17
5.3. IGP Topology . . . . . . . . . . . . . . . . . . . . . . . 17 5.3. IGP Topology . . . . . . . . . . . . . . . . . . . . . . . 17
5.4. Timers . . . . . . . . . . . . . . . . . . . . . . . . . . 18 5.4. Timers . . . . . . . . . . . . . . . . . . . . . . . . . . 18
5.5. Interface Types . . . . . . . . . . . . . . . . . . . . . 18 5.5. Interface Types . . . . . . . . . . . . . . . . . . . . . 18
5.6. Offered Load . . . . . . . . . . . . . . . . . . . . . . . 19 5.6. Offered Load . . . . . . . . . . . . . . . . . . . . . . . 18
5.7. Measurement Accuracy . . . . . . . . . . . . . . . . . . . 20 5.7. Measurement Accuracy . . . . . . . . . . . . . . . . . . . 19
5.8. Measurement Statistics . . . . . . . . . . . . . . . . . . 20 5.8. Measurement Statistics . . . . . . . . . . . . . . . . . . 20
5.9. Tester Capabilities . . . . . . . . . . . . . . . . . . . 20 5.9. Tester Capabilities . . . . . . . . . . . . . . . . . . . 20
6. Selection of Convergence Time Benchmark Metrics and Methods . 21 6. Selection of Convergence Time Benchmark Metrics and Methods . 20
6.1. Loss-Derived Method . . . . . . . . . . . . . . . . . . . 21 6.1. Loss-Derived Method . . . . . . . . . . . . . . . . . . . 21
6.1.1. Tester capabilities . . . . . . . . . . . . . . . . . 21 6.1.1. Tester Capabilities . . . . . . . . . . . . . . . . . 21
6.1.2. Benchmark Metrics . . . . . . . . . . . . . . . . . . 21 6.1.2. Benchmark Metrics . . . . . . . . . . . . . . . . . . 21
6.1.3. Measurement Accuracy . . . . . . . . . . . . . . . . . 21 6.1.3. Measurement Accuracy . . . . . . . . . . . . . . . . . 21
6.2. Rate-Derived Method . . . . . . . . . . . . . . . . . . . 22 6.2. Rate-Derived Method . . . . . . . . . . . . . . . . . . . 22
6.2.1. Tester Capabilities . . . . . . . . . . . . . . . . . 22 6.2.1. Tester Capabilities . . . . . . . . . . . . . . . . . 22
6.2.2. Benchmark Metrics . . . . . . . . . . . . . . . . . . 23 6.2.2. Benchmark Metrics . . . . . . . . . . . . . . . . . . 23
6.2.3. Measurement Accuracy . . . . . . . . . . . . . . . . . 23 6.2.3. Measurement Accuracy . . . . . . . . . . . . . . . . . 23
6.3. Route-Specific Loss-Derived Method . . . . . . . . . . . . 24 6.3. Route-Specific Loss-Derived Method . . . . . . . . . . . . 24
6.3.1. Tester Capabilities . . . . . . . . . . . . . . . . . 24 6.3.1. Tester Capabilities . . . . . . . . . . . . . . . . . 24
6.3.2. Benchmark Metrics . . . . . . . . . . . . . . . . . . 24 6.3.2. Benchmark Metrics . . . . . . . . . . . . . . . . . . 24
6.3.3. Measurement Accuracy . . . . . . . . . . . . . . . . . 24 6.3.3. Measurement Accuracy . . . . . . . . . . . . . . . . . 24
7. Reporting Format . . . . . . . . . . . . . . . . . . . . . . . 24 7. Reporting Format . . . . . . . . . . . . . . . . . . . . . . . 25
8. Test Cases . . . . . . . . . . . . . . . . . . . . . . . . . . 26 8. Test Cases . . . . . . . . . . . . . . . . . . . . . . . . . . 26
8.1. Interface Failure and Recovery . . . . . . . . . . . . . . 27 8.1. Interface Failure and Recovery . . . . . . . . . . . . . . 27
8.1.1. Convergence Due to Local Interface Failure and 8.1.1. Convergence Due to Local Interface Failure and
Recovery . . . . . . . . . . . . . . . . . . . . . . . 27 Recovery . . . . . . . . . . . . . . . . . . . . . . . 27
8.1.2. Convergence Due to Remote Interface Failure and 8.1.2. Convergence Due to Remote Interface Failure and
Recovery . . . . . . . . . . . . . . . . . . . . . . . 28 Recovery . . . . . . . . . . . . . . . . . . . . . . . 28
8.1.3. Convergence Due to ECMP Member Local Interface 8.1.3. Convergence Due to ECMP Member Local Interface
Failure and Recovery . . . . . . . . . . . . . . . . . 30 Failure and Recovery . . . . . . . . . . . . . . . . . 30
8.1.4. Convergence Due to ECMP Member Remote Interface 8.1.4. Convergence Due to ECMP Member Remote Interface
Failure and Recovery . . . . . . . . . . . . . . . . . 31 Failure and Recovery . . . . . . . . . . . . . . . . . 31
8.1.5. Convergence Due to Parallel Link Interface Failure 8.1.5. Convergence Due to Parallel Link Interface Failure
and Recovery . . . . . . . . . . . . . . . . . . . . . 32 and Recovery . . . . . . . . . . . . . . . . . . . . . 32
8.2. Other Failures and Recoveries . . . . . . . . . . . . . . 33 8.2. Other Failures and Recoveries . . . . . . . . . . . . . . 33
8.2.1. Convergence Due to Layer 2 Session Loss and 8.2.1. Convergence Due to Layer 2 Session Loss and
Recovery . . . . . . . . . . . . . . . . . . . . . . . 33 Recovery . . . . . . . . . . . . . . . . . . . . . . . 33
8.2.2. Convergence Due to Loss and Recovery of IGP 8.2.2. Convergence Due to Loss and Recovery of IGP
skipping to change at page 4, line 17 skipping to change at page 3, line 32
8.1.4. Convergence Due to ECMP Member Remote Interface 8.1.4. Convergence Due to ECMP Member Remote Interface
Failure and Recovery . . . . . . . . . . . . . . . . . 31 Failure and Recovery . . . . . . . . . . . . . . . . . 31
8.1.5. Convergence Due to Parallel Link Interface Failure 8.1.5. Convergence Due to Parallel Link Interface Failure
and Recovery . . . . . . . . . . . . . . . . . . . . . 32 and Recovery . . . . . . . . . . . . . . . . . . . . . 32
8.2. Other Failures and Recoveries . . . . . . . . . . . . . . 33 8.2. Other Failures and Recoveries . . . . . . . . . . . . . . 33
8.2.1. Convergence Due to Layer 2 Session Loss and 8.2.1. Convergence Due to Layer 2 Session Loss and
Recovery . . . . . . . . . . . . . . . . . . . . . . . 33 Recovery . . . . . . . . . . . . . . . . . . . . . . . 33
8.2.2. Convergence Due to Loss and Recovery of IGP 8.2.2. Convergence Due to Loss and Recovery of IGP
Adjacency . . . . . . . . . . . . . . . . . . . . . . 34 Adjacency . . . . . . . . . . . . . . . . . . . . . . 34
8.2.3. Convergence Due to Route Withdrawal and 8.2.3. Convergence Due to Route Withdrawal and
Re-advertisement . . . . . . . . . . . . . . . . . . . 35 Re-Advertisement . . . . . . . . . . . . . . . . . . . 35
8.3. Administrative changes . . . . . . . . . . . . . . . . . . 37 8.3. Administrative Changes . . . . . . . . . . . . . . . . . . 37
8.3.1. Convergence Due to Local Interface Adminstrative 8.3.1. Convergence Due to Local Interface Administrative
Changes . . . . . . . . . . . . . . . . . . . . . . . 37 Changes . . . . . . . . . . . . . . . . . . . . . . . 37
8.3.2. Convergence Due to Cost Change . . . . . . . . . . . . 38 8.3.2. Convergence Due to Cost Change . . . . . . . . . . . . 38
9. Security Considerations . . . . . . . . . . . . . . . . . . . 39 9. Security Considerations . . . . . . . . . . . . . . . . . . . 39
10. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 40 10. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . 40
11. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . 40 11. References . . . . . . . . . . . . . . . . . . . . . . . . . . 40
12. References . . . . . . . . . . . . . . . . . . . . . . . . . . 40 11.1. Normative References . . . . . . . . . . . . . . . . . . . 40
12.1. Normative References . . . . . . . . . . . . . . . . . . . 40 11.2. Informative References . . . . . . . . . . . . . . . . . . 41
12.2. Informative References . . . . . . . . . . . . . . . . . . 41
Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 42
1. Introduction 1. Introduction
1.1. Motivation 1.1. Motivation
Convergence time is a critical performance parameter. Service Convergence time is a critical performance parameter. Service
Providers use IGP convergence time as a key metric of router design Providers use IGP convergence time as a key metric of router design
and architecture. Fast network convergence can be optimally achieved and architecture. Fast network convergence can be optimally achieved
through deployment of fast converging routers. Customers of Service through deployment of fast converging routers. Customers of Service
Providers use packet loss due to Interior Gateway Protocol (IGP) Providers use packet loss due to Interior Gateway Protocol (IGP)
convergence as a key metric of their network service quality. IGP convergence as a key metric of their network service quality. IGP
route convergence is a Direct Measure of Quality (DMOQ) when route convergence is a Direct Measure of Quality (DMOQ) when
benchmarking the data plane. The fundamental basis by which network benchmarking the data plane. The fundamental basis by which network
users and operators benchmark convergence is packet loss and other users and operators benchmark convergence is packet loss and other
packet impairments, which are externally observable events having packet impairments, which are externally observable events having
direct impact on their application performance. For this reason it direct impact on their application performance. For this reason, it
is important to develop a standard methodology for benchmarking link- is important to develop a standard methodology for benchmarking link-
state IGP convergence time through externally observable (black-box) state IGP convergence time through externally observable (black-box)
data plane measurements. All factors contributing to convergence data-plane measurements. All factors contributing to convergence
time are accounted for by measuring on the data plane. time are accounted for by measuring on the data plane.
1.2. Factors for IGP Route Convergence Time 1.2. Factors for IGP Route Convergence Time
There are four major categories of factors contributing to the There are four major categories of factors contributing to the
measured IGP convergence time. As discussed in [Vi02], [Ka02], measured IGP convergence time. As discussed in [Vi02], [Ka02],
[Fi02], [Al00], [Al02], and [Fr05], these categories are Event [Fi02], [Al00], [Al02], and [Fr05], these categories are Event
Detection, Shortest Path First (SPF) Processing, Link State Detection, Shortest Path First (SPF) Processing, Link State
Advertisement (LSA) / Link State Packet (LSP) Advertisement, and Advertisement (LSA) / Link State Packet (LSP) Advertisement, and
Forwarding Information Base (FIB) Update. These have numerous Forwarding Information Base (FIB) Update. These have numerous
components that influence the convergence time, including but not components that influence the convergence time, including but not
limited to the list below: limited to the list below:
o Event Detection o Event Detection
* Physical Layer failure/recovery indication time * Physical-Layer Failure/Recovery Indication Time
* Layer 2 failure/recovery indication time * Layer 2 Failure/Recovery Indication Time
* IGP Hello Dead Interval * IGP Hello Dead Interval
o SPF Processing o SPF Processing
* SPF Delay Time * SPF Delay Time
* SPF Hold time * SPF Hold Time
* SPF Execution time * SPF Execution Time
o LSA/LSP Advertisement o LSA/LSP Advertisement
* LSA/LSP Generation time * LSA/LSP Generation Time
* LSA/LSP Flood Packet Pacing * LSA/LSP Flood Packet Pacing
* LSA/LSP Retransmission Packet Pacing * LSA/LSP Retransmission Packet Pacing
o FIB Update o FIB Update
* Tree Build time * Tree Build Time
* Hardware Update time * Hardware Update Time
o Increased Forwarding Delay due to Queueing o Increased Forwarding Delay due to Queueing
The contribution of each of these factors listed above will vary with The contribution of each of the factors listed above will vary with
each router vendors' architecture and IGP implementation. Routers each router vendor's architecture and IGP implementation. Routers
may have a centralized forwarding architecture, in which one may have a centralized forwarding architecture, in which one
forwarding table is calculated and referenced for all arriving forwarding table is calculated and referenced for all arriving
packets, or a distributed forwarding architecture, in which the packets, or a distributed forwarding architecture, in which the
central forwarding table is calculated and distributed to the central forwarding table is calculated and distributed to the
interfaces for local look-up as packets arrive. The distributed interfaces for local look-up as packets arrive. The distributed
forwarding tables are typically maintained in hardware. forwarding tables are typically maintained (loaded and changed) in
software.
The variation in router architecture and implementation necessitates The variation in router architecture and implementation necessitates
the design of a convergence test that considers all of these the design of a convergence test that considers all of these
components contributing to convergence time and is independent of the components contributing to convergence time and is independent of the
Device Under Test (DUT) architecture and implementation. The benefit Device Under Test (DUT) architecture and implementation. The benefit
of designing a test for these considerations is that it enables of designing a test for these considerations is that it enables
black-box testing in which knowledge of the routers' internal black-box testing in which knowledge of the routers' internal
implementation is not required. It is then possible to make valid implementation is not required. It is then possible to make valid
use of the convergence benchmarking metrics when comparing routers use of the convergence benchmarking metrics when comparing routers
from different vendors. from different vendors.
Convergence performance is tightly linked to the number of tasks a Convergence performance is tightly linked to the number of tasks a
router has to deal with. As the most impacting tasks are mainly router has to deal with. As the most important tasks are mainly
related to the control plane and the data plane, the more the DUT is related to the control plane and the data plane, the more the DUT is
stressed as in a live production environment, the closer performance stressed as in a live production environment, the closer performance
measurement results match the ones that would be observed in a live measurement results match the ones that would be observed in a live
production environment. production environment.
1.3. Use of Data Plane for IGP Route Convergence Benchmarking 1.3. Use of Data Plane for IGP Route Convergence Benchmarking
Customers of Service Providers use packet loss and other packet Customers of Service Providers use packet loss and other packet
impairments as metrics to calculate convergence time. Packet loss impairments as metrics to calculate convergence time. Packet loss
and other packet impairments are externally observable events having and other packet impairments are externally observable events having
direct impact on customers' application performance. For this reason direct impact on customers' application performance. For this
it is important to develop a standard router benchmarking methodology reason, it is important to develop a standard router benchmarking
that is a Direct Measure of Quality (DMOQ) for measuring IGP methodology that is a Direct Measure of Quality (DMOQ) for measuring
convergence. An additional benefit of using packet loss for IGP convergence. An additional benefit of using packet loss for
calculation of IGP Route Convergence time is that it enables black- calculation of IGP Route Convergence time is that it enables black-
box tests to be designed. Data traffic can be offered to the Device box tests to be designed. Data traffic can be offered to the Device
Under Test (DUT), an emulated network event can be forced to occur, Under Test (DUT), an emulated network event can be forced to occur,
and packet loss and other impaired packets can be externally measured and packet loss and other impaired packets can be externally measured
to calculate the convergence time. Knowledge of the DUT architecture to calculate the convergence time. Knowledge of the DUT architecture
and IGP implementation is not required. There is no need to rely on and IGP implementation is not required. There is no need to rely on
the DUT to produce the test results. There is no need to build the DUT to produce the test results. There is no need to build
intrusive test harnesses for the DUT. All factors contributing to intrusive test harnesses for the DUT. All factors contributing to
convergence time are accounted for by measuring on the dataplane. convergence time are accounted for by measuring on the data plane.
Other work of the Benchmarking Methodology Working Group (BMWG) Other work of the Benchmarking Methodology Working Group (BMWG)
focuses on characterizing single router control plane convergence. focuses on characterizing single router control-plane convergence.
See [Ma05], [Ma05t], and [Ma05c]. See [Ma05], [Ma05t], and [Ma05c].
1.4. Applicability and Scope 1.4. Applicability and Scope
The methodology described in this document can be applied to IPv4 and The methodology described in this document can be applied to IPv4 and
IPv6 traffic and link-state IGPs such as IS-IS [Ca90][Ho08], OSPF IPv6 traffic and link-state IGPs such as IS-IS [Ca90][Ho08], OSPF
[Mo98][Co08], and others. IGP adjacencies established over any kind [Mo98][Co08], and others. IGP adjacencies established over any kind
of tunnel (such as Traffic Engineering tunnels) are outside the scope of tunnel (such as Traffic Engineering tunnels) are outside the scope
of this document. Convergence time benchmarking in topologies with of this document. Convergence time benchmarking in topologies with
non point-to-point IGP adjacencies will be covered in a later IGP adjacencies that are not point-to-point will be covered in a
document. Convergence from Bidirectional Forwarding Detection (BFD) later document. Convergence from Bidirectional Forwarding Detection
is outside the scope of this document. Non-Stop Forwarding (NSF), (BFD) is outside the scope of this document. Non-Stop Forwarding
Non-Stop Routing (NSR), Graceful Restart (GR), or any other High (NSF), Non-Stop Routing (NSR), Graceful Restart (GR), and any other
Availability mechanism are outside the scope of this document. Fast High Availability mechanism are outside the scope of this document.
reroute mechanisms such as IP Fast-Reroute [Sh10i] or MPLS Fast- Fast reroute mechanisms such as IP Fast-Reroute [Sh10i] or MPLS Fast-
Reroute [Pa05] are outside the scope of this document. Reroute [Pa05] are outside the scope of this document.
2. Existing Definitions 2. Existing Definitions
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", The keywords "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
"SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
document are to be interpreted as described in BCP 14, RFC 2119 document are to be interpreted as described in BCP 14, RFC 2119
[Br97]. RFC 2119 defines the use of these key words to help make the [Br97]. RFC 2119 defines the use of these keywords to help make the
intent of standards track documents as clear as possible. While this intent of Standards Track documents as clear as possible. While this
document uses these keywords, this document is not a standards track document uses these keywords, this document is not a Standards Track
document. document.
This document uses much of the terminology defined in [Po11t]. For This document uses much of the terminology defined in [Po11t]. For
any conflicting content, this document supersedes [Po11t]. This any conflicting content, this document supersedes [Po11t]. This
document uses existing terminology defined in other documents issued document uses existing terminology defined in other documents issued
by the Benchmarking Methodology Working Group (BMWG). Examples by the Benchmarking Methodology Working Group (BMWG). Examples
include, but are not limited to: include, but are not limited to:
Throughput [Ref.[Br91], section 3.17] Throughput [Br91], Section 3.17
Device Under Test (DUT) [Ref.[Ma98], section 3.1.1] Offered Load [Ma98], Section 3.5.2
System Under Test (SUT) [Ref.[Ma98], section 3.1.2] Forwarding Rate [Ma98], Section 3.6.1
Out-of-Order Packet [Ref.[Po06], section 3.3.4] Device Under Test (DUT) [Ma98], Section 3.1.1
Duplicate Packet [Ref.[Po06], section 3.3.5] System Under Test (SUT) [Ma98], Section 3.1.2
Stream [Ref.[Po06], section 3.3.2] Out-of-Order Packet [Po06], Section 3.3.4
Loss Period [Ref.[Ko02], section 4] Duplicate Packet [Po06], Section 3.3.5
Forwarding Delay [Ref.[Po06], section 3.2.4] Stream [Po06], Section 3.3.2
IP Packet Delay Variation (IPDV) [Ref.[De02], section 1.2] Forwarding Delay [Po06], Section 3.2.4
IP Packet Delay Variation (IPDV) [De02], Section 1.2
Loss Period [Ko02], Section 4
3. Test Topologies 3. Test Topologies
3.1. Test topology for local changes 3.1. Test Topology for Local Changes
Figure 1 shows the test topology to measure IGP convergence time due Figure 1 shows the test topology to measure IGP convergence time due
to local Convergence Events such as Local Interface failure and to local Convergence Events such as Local Interface failure and
recovery (Section 8.1.1), layer 2 session failure and recovery recovery (Section 8.1.1), Layer 2 session failure and recovery
(Section 8.2.1), and IGP adjacency failure and recovery (Section 8.2.1), and IGP adjacency failure and recovery
(Section 8.2.2). This topology is also used to measure IGP (Section 8.2.2). This topology is also used to measure IGP
convergence time due to route withdrawal and readvertisement convergence time due to route withdrawal and re-advertisement
(Section 8.2.3), and route cost change (Section 8.3.2) Convergence (Section 8.2.3) and to measure IGP convergence time due to route cost
Events. IGP adjacencies MUST be established between Tester and DUT: change (Section 8.3.2) Convergence Events. IGP adjacencies MUST be
one on the Ingress Interface, one on the Preferred Egress Interface, established between Tester and DUT: one on the Ingress Interface, one
and one on the Next-Best Egress Interface. For this purpose the on the Preferred Egress Interface, and one on the Next-Best Egress
Tester emulates three routers (RTa, RTb, and RTc), each establishing Interface. For this purpose, the Tester emulates three routers (RTa,
one adjacency with the DUT. RTb, and RTc), each establishing one adjacency with the DUT.
------- -------
| | Preferred ....... | | Preferred .......
| |------------------. RTb . | |------------------. RTb .
....... Ingress | | Egress Interface ....... ....... Ingress | | Egress Interface .......
. RTa .------------| DUT | . RTa .------------| DUT |
....... Interface | | Next-Best ....... ....... Interface | | Next-Best .......
| |------------------. RTc . | |------------------. RTc .
| | Egress Interface ....... | | Egress Interface .......
------- -------
Figure 1: IGP convergence test topology for local changes Figure 1: IGP convergence test topology for local changes
Figure 2 shows the test topology to measure IGP convergence time due Figure 2 shows the test topology to measure IGP convergence time due
to local Convergence Events with a non-Equal Cost Multipath (ECMP) to local Convergence Events with a non-Equal Cost Multipath (ECMP)
Preferred Egress Interface and Equal Cost Multipath (ECMP) Next-Best Preferred Egress Interface and ECMP Next-Best Egress Interfaces
Egress Interfaces (Section 8.1.1). In this topology, the DUT is (Section 8.1.1). In this topology, the DUT is configured with each
configured with each Next-Best Egress interface as a member of a Next-Best Egress Interface as a member of a single ECMP set. The
single ECMP set. The Preferred Egress Interface is not a member of Preferred Egress Interface is not a member of an ECMP set. The
an ECMP set. The Tester emulates N+2 neighbor routers (N>0): one Tester emulates N+2 neighbor routers (N>0): one router for the
router for the Ingress Interface (RTa), one router for the Preferred Ingress Interface (RTa), one router for the Preferred Egress
Egress Interface (RTb), and N routers for the members of the ECMP set Interface (RTb), and N routers for the members of the ECMP set
(RTc1...RTcN). IGP adjacencies MUST be established between Tester (RTc1...RTcN). IGP adjacencies MUST be established between Tester
and DUT: one on the Ingress Interface, one on the Preferred Egress and DUT: one on the Ingress Interface, one on the Preferred Egress
Interface, and one on each member of the ECMP set. When the test Interface, and one on each member of the ECMP set. When the test
specifies to observe the Next-Best Egress Interface statistics, the specifies to observe the Next-Best Egress Interface statistics, the
combined statistics for all ECMP members should be observed. combined statistics for all ECMP members should be observed.
------- -------
| | Preferred ....... | | Preferred .......
| |------------------. RTb . | |------------------. RTb .
| | Egress Interface ....... | | Egress Interface .......
skipping to change at page 9, line 30 skipping to change at page 8, line 31
| | . | | .
| | . | | .
| | ECMP Set ........ | | ECMP Set ........
| |------------------. RTcN . | |------------------. RTcN .
| | Interface N ........ | | Interface N ........
------- -------
Figure 2: IGP convergence test topology for local changes with non- Figure 2: IGP convergence test topology for local changes with non-
ECMP to ECMP convergence ECMP to ECMP convergence
3.2. Test topology for remote changes 3.2. Test Topology for Remote Changes
Figure 3 shows the test topology to measure IGP convergence time due Figure 3 shows the test topology to measure IGP convergence time due
to Remote Interface failure and recovery (Section 8.1.2). In this to Remote Interface failure and recovery (Section 8.1.2). In this
topology the two routers DUT1 and DUT2 are considered System Under topology, the two routers DUT1 and DUT2 are considered the System
Test (SUT) and SHOULD be identically configured devices of the same Under Test (SUT) and SHOULD be identically configured devices of the
model. IGP adjacencies MUST be established between Tester and SUT, same model. IGP adjacencies MUST be established between Tester and
one on the Ingress Interface, one on the Preferred Egress Interface, SUT, one on the Ingress Interface, one on the Preferred Egress
and one on the Next-Best Egress Interface. For this purpose the Interface, and one on the Next-Best Egress Interface. For this
Tester emulates three routers (RTa, RTb, and RTc). In this topology purpose, the Tester emulates three routers (RTa, RTb, and RTc). In
there is a possibility of a packet forwarding loop that may occur this topology, a packet forwarding loop, also known as micro-loop
transiently between DUT1 and DUT2 during convergence (micro-loop, see (see [Sh10]), may occur transiently between DUT1 and DUT2 during
[Sh10]). convergence.
-------- --------
| | -------- Preferred ....... | | -------- Preferred .......
| |--| DUT2 |------------------. RTb . | |--| DUT2 |------------------. RTb .
....... Ingress | | -------- Egress Interface ....... ....... Ingress | | -------- Egress Interface .......
. RTa .------------| DUT1 | . RTa .------------| DUT1 |
....... Interface | | Next-Best ....... ....... Interface | | Next-Best .......
| |----------------------------. RTc . | |----------------------------. RTc .
| | Egress Interface ....... | | Egress Interface .......
-------- --------
skipping to change at page 10, line 28 skipping to change at page 9, line 28
to remote Convergence Events with a non-ECMP Preferred Egress to remote Convergence Events with a non-ECMP Preferred Egress
Interface and ECMP Next-Best Egress Interfaces (Section 8.1.2). In Interface and ECMP Next-Best Egress Interfaces (Section 8.1.2). In
this topology the two routers DUT1 and DUT2 are considered System this topology the two routers DUT1 and DUT2 are considered System
Under Test (SUT) and MUST be identically configured devices of the Under Test (SUT) and MUST be identically configured devices of the
same model. Router DUT1 is configured with the Next-Best Egress same model. Router DUT1 is configured with the Next-Best Egress
Interface an ECMP set of interfaces. The Preferred Egress Interface Interface an ECMP set of interfaces. The Preferred Egress Interface
of DUT1 is not a member of an ECMP set. The Tester emulates N+2 of DUT1 is not a member of an ECMP set. The Tester emulates N+2
neighbor routers (N>0), one for the Ingress Interface (RTa), one for neighbor routers (N>0), one for the Ingress Interface (RTa), one for
DUT2 (RTb) and one for each member of the ECMP set (RTc1...RTcN). DUT2 (RTb) and one for each member of the ECMP set (RTc1...RTcN).
IGP adjacencies MUST be established between Tester and SUT, one on IGP adjacencies MUST be established between Tester and SUT, one on
each interface of SUT. For this purpose each of the N+2 routers each interface of the SUT. For this purpose each of the N+2 routers
emulated by the Tester establishes one adjacency with the SUT. In emulated by the Tester establishes one adjacency with the SUT. In
this topology there is a possibility of a packet forwarding loop that this topology, there is a possibility of a packet-forwarding loop
may occur transiently between DUT1 and DUT2 during convergence that may occur transiently between DUT1 and DUT2 during convergence
(micro-loop, see [Sh10]). When the test specifies to observe the (micro-loop, see [Sh10]). When the test specifies to observe the
Next-Best Egress Interface statistics, the combined statistics for Next-Best Egress Interface statistics, the combined statistics for
all members of the ECMP set should be observed. all members of the ECMP set should be observed.
-------- --------
| | -------- Preferred ....... | | -------- Preferred .......
| |--| DUT2 |------------------. RTb . | |--| DUT2 |------------------. RTb .
| | -------- Egress Interface ....... | | -------- Egress Interface .......
| | | |
| | ECMP Set ........ | | ECMP Set ........
....... Ingress | |----------------------------. RTc1 . ....... Ingress | |----------------------------. RTc1 .
. RTa .------------| DUT1 | Interface 1 ........ . RTa .------------| DUT1 | Interface 1 ........
....... Interface | | . ....... Interface | | .
| | . | | .
| | . | | .
| | ECMP Set ........ | | ECMP Set ........
| |----------------------------. RTcN . | |----------------------------. RTcN .
| | Interface N ........ | | Interface N ........
-------- --------
Figure 4: IGP convergence test topology for remote changes with non- Figure 4: IGP convergence test topology for remote changes with
ECMP to ECMP convergence non-ECMP to ECMP convergence
3.3. Test topology for local ECMP changes 3.3. Test Topology for Local ECMP Changes
Figure 5 shows the test topology to measure IGP convergence time due Figure 5 shows the test topology to measure IGP convergence time due
to local Convergence Events of a member of an Equal Cost Multipath to local Convergence Events of a member of an Equal Cost Multipath
(ECMP) set (Section 8.1.3). In this topology, the DUT is configured (ECMP) set (Section 8.1.3). In this topology, the DUT is configured
with each egress interface as a member of a single ECMP set and the with each egress interface as a member of a single ECMP set and the
Tester emulates N+1 next-hop routers, one for the Ingress Interface Tester emulates N+1 next-hop routers, one for the Ingress Interface
(RTa) and one for each member of the ECMP set (RTb1...RTbN). IGP (RTa) and one for each member of the ECMP set (RTb1...RTbN). IGP
adjacencies MUST be established between Tester and DUT, one on the adjacencies MUST be established between Tester and DUT, one on the
Ingress Interface and one on each member of the ECMP set. For this Ingress Interface and one on each member of the ECMP set. For this
purpose each of the N+1 routers emulated by the Tester establishes purpose, each of the N+1 routers emulated by the Tester establishes
one adjacency with the DUT. When the test specifies to observe the one adjacency with the DUT. When the test specifies to observe the
Next-Best Egress Interface statistics, the combined statistics for Next-Best Egress Interface statistics, the combined statistics for
all ECMP members except the one affected by the Convergence Event, all ECMP members except the one affected by the Convergence Event
should be observed. should be observed.
------- -------
| | ECMP Set ........ | | ECMP Set ........
| |-------------. RTb1 . | |-------------. RTb1 .
| | Interface 1 ........ | | Interface 1 ........
....... Ingress | | . ....... Ingress | | .
. RTa .------------| DUT | . . RTa .------------| DUT | .
....... Interface | | . ....... Interface | | .
| | ECMP Set ........ | | ECMP Set ........
| |-------------. RTbN . | |-------------. RTbN .
| | Interface N ........ | | Interface N ........
------- -------
Figure 5: IGP convergence test topology for local ECMP changes Figure 5: IGP convergence test topology for local ECMP changes
3.4. Test topology for remote ECMP changes 3.4. Test Topology for Remote ECMP Changes
Figure 6 shows the test topology to measure IGP convergence time due Figure 6 shows the test topology to measure IGP convergence time due
to remote Convergence Events of a member of an Equal Cost Multipath to remote Convergence Events of a member of an Equal Cost Multipath
(ECMP) set (Section 8.1.4). In this topology the two routers DUT1 (ECMP) set (Section 8.1.4). In this topology, the two routers DUT1
and DUT2 are considered System Under Test (SUT) and MUST be and DUT2 are considered the System Under Test (SUT) and MUST be
identically configured devices of the same model. Router DUT1 is identically configured devices of the same model. Router DUT1 is
configured with each egress interface as a member of a single ECMP configured with each egress interface as a member of a single ECMP
set and the Tester emulates N+1 neighbor routers (N>0), one for the set, and the Tester emulates N+1 neighbor routers (N>0), one for the
Ingress Interface (RTa) and one for each member of the ECMP set Ingress Interface (RTa) and one for each member of the ECMP set
(RTb1...RTbN). IGP adjacencies MUST be established between Tester (RTb1...RTbN). IGP adjacencies MUST be established between Tester
and SUT, one on each interface of SUT. For this purpose each of the and SUT, one on each interface of the SUT. For this purpose, each of
N+1 routers emulated by the Tester establishes one adjacency with the the N+1 routers emulated by the Tester establishes one adjacency with
SUT (N-1 emulated routers are adjacent to DUT1 egress interfaces, one the SUT (N-1 emulated routers are adjacent to DUT1 egress interfaces,
emulated router is adjacent to DUT1 Ingress Interface, and one one emulated router is adjacent to DUT1 Ingress Interface, and one
emulated router is adjacent to DUT2). In this topology there is a emulated router is adjacent to DUT2). In this topology, there is a
possibility of a packet forwarding loop that may occur transiently possibility of a packet-forwarding loop that may occur transiently
between DUT1 and DUT2 during convergence (micro-loop, see [Sh10]). between DUT1 and DUT2 during convergence (micro-loop, see [Sh10]).
When the test specifies to observe the Next-Best Egress Interface When the test specifies to observe the Next-Best Egress Interface
statistics, the combined statistics for all ECMP members except the statistics, the combined statistics for all ECMP members except the
one affected by the Convergence Event, should be observed. one affected by the Convergence Event should be observed.
-------- --------
| | ECMP Set -------- ........ | | ECMP Set -------- ........
| |-------------| DUT2 |---. RTb1 . | |-------------| DUT2 |---. RTb1 .
| | Interface 1 -------- ........ | | Interface 1 -------- ........
| | | |
| | ECMP Set ........ | | ECMP Set ........
....... Ingress | |------------------------. RTb2 . ....... Ingress | |------------------------. RTb2 .
. RTa .------------| DUT1 | Interface 2 ........ . RTa .------------| DUT1 | Interface 2 ........
....... Interface | | . ....... Interface | | .
| | . | | .
| | . | | .
| | ECMP Set ........ | | ECMP Set ........
| |------------------------. RTbN . | |------------------------. RTbN .
| | Interface N ........ | | Interface N ........
-------- --------
Figure 6: IGP convergence test topology for remote ECMP changes Figure 6: IGP convergence test topology for remote ECMP changes
3.5. Test topology for Parallel Link changes 3.5. Test topology for Parallel Link Changes
Figure 7 shows the test topology to measure IGP convergence time due Figure 7 shows the test topology to measure IGP convergence time due
to local Convergence Events with members of a Parallel Link to local Convergence Events with members of a Parallel Link
(Section 8.1.5). In this topology, the DUT is configured with each (Section 8.1.5). In this topology, the DUT is configured with each
egress interface as a member of a Parallel Link and the Tester egress interface as a member of a Parallel Link and the Tester
emulates two neighbor routers, one for the Ingress Interface (RTa) emulates two neighbor routers, one for the Ingress Interface (RTa)
and one for the Parallel Link members (RTb). IGP adjacencies MUST be and one for the Parallel Link members (RTb). IGP adjacencies MUST be
established on the Ingress Interface and on all N members of the established on the Ingress Interface and on all N members of the
Parallel Link between Tester and DUT (N>0). For this purpose the Parallel Link between Tester and DUT (N>0). For this purpose, the
routers emulated by the Tester establishes N+1 adjacencies with the routers emulated by the Tester establishes N+1 adjacencies with the
DUT. When the test specifies to observe the Next-Best Egress DUT. When the test specifies to observe the Next-Best Egress
Interface statistics, the combined statistics for all Parallel Link Interface statistics, the combined statistics for all Parallel Link
members except the one affected by the Convergence Event, should be members except the one affected by the Convergence Event should be
observed. observed.
------- ....... ------- .......
| | Parallel Link . . | | Parallel Link . .
| |----------------. . | |----------------. .
| | Interface 1 . . | | Interface 1 . .
....... Ingress | | . . . ....... Ingress | | . . .
. RTa .------------| DUT | . . RTb . . RTa .------------| DUT | . . RTb .
....... Interface | | . . . ....... Interface | | . . .
| | Parallel Link . . | | Parallel Link . .
skipping to change at page 13, line 36 skipping to change at page 12, line 43
the DUT is ready to forward traffic for a specific route on its Next- the DUT is ready to forward traffic for a specific route on its Next-
Best Egress Interface and maintains this state for the duration of Best Egress Interface and maintains this state for the duration of
the Sustained Convergence Validation Time [Po11t]. To measure Route the Sustained Convergence Validation Time [Po11t]. To measure Route
Convergence time, the Convergence Event Instant and the traffic Convergence time, the Convergence Event Instant and the traffic
received from the Next-Best Egress Interface need to be observed. received from the Next-Best Egress Interface need to be observed.
The Route Loss of Connectivity Period [Po11t] indicates the time The Route Loss of Connectivity Period [Po11t] indicates the time
during which traffic to a specific route is lost following a during which traffic to a specific route is lost following a
Convergence Event until Full Convergence [Po11t] completes. This Convergence Event until Full Convergence [Po11t] completes. This
Route Loss of Connectivity Period can consist of one or more Loss Route Loss of Connectivity Period can consist of one or more Loss
Periods [Ko02]. For the testcases described in this document it is Periods [Ko02]. For the test cases described in this document, it is
expected to have a single Loss Period. To measure Route Loss of expected to have a single Loss Period. To measure the Route Loss of
Connectivity Period, the traffic received from the Preferred Egress Connectivity Period, the traffic received from the Preferred Egress
Interface and the traffic received from the Next-Best Egress Interface and the traffic received from the Next-Best Egress
Interface need to be observed. Interface need to be observed.
The Route Loss of Connectivity Period is most important since that The Route Loss of Connectivity Period is most important since that
has a direct impact on the network user's application performance. has a direct impact on the network user's application performance.
In general the Route Convergence time is larger than or equal to the In general, the Route Convergence time is larger than or equal to the
Route Loss of Connectivity Period. Depending on which Convergence Route Loss of Connectivity Period. Depending on which Convergence
Event occurs and how this Convergence Event is applied, traffic for a Event occurs and how this Convergence Event is applied, traffic for a
route may still be forwarded over the Preferred Egress Interface route may still be forwarded over the Preferred Egress Interface
after the Convergence Event Instant, before converging to the Next- after the Convergence Event Instant, before converging to the Next-
Best Egress Interface. In that case the Route Loss of Connectivity Best Egress Interface. In that case, the Route Loss of Connectivity
Period is shorter than the Route Convergence time. Period is shorter than the Route Convergence time.
At least one condition needs to be fulfilled for Route Convergence At least one condition needs to be fulfilled for Route Convergence
time to be equal to Route Loss of Connectivity Period. The condition time to be equal to Route Loss of Connectivity Period. The condition
is that the Convergence Event causes an instantaneous traffic loss is that the Convergence Event causes an instantaneous traffic loss
for the measured route. A fiber cut on the Preferred Egress for the measured route. A fiber cut on the Preferred Egress
Interface is an example of such a Convergence Event. Interface is an example of such a Convergence Event.
A second condition applies to Route Convergence time measurements A second condition applies to Route Convergence time measurements
based on Connectivity Packet Loss [Po11t]. This second condition is based on Connectivity Packet Loss [Po11t]. This second condition is
that there is only a single Loss Period during Route Convergence. that there is only a single Loss Period during Route Convergence.
For the testcases described in this document this is expected to be For the test cases described in this document, the second condition
the case. is expected to apply.
4.1. Convergence Events without instant traffic loss 4.1. Convergence Events without Instant Traffic Loss
To measure convergence time benchmarks for Convergence Events caused To measure convergence time benchmarks for Convergence Events caused
by a Tester, such as an IGP cost change, the Tester MAY start to by a Tester, such as an IGP cost change, the Tester MAY start to
discard all traffic received from the Preferred Egress Interface at discard all traffic received from the Preferred Egress Interface at
the Convergence Event Instant, or MAY separately observe packets the Convergence Event Instant, or MAY separately observe packets
received from the Preferred Egress Interface prior to the Convergence received from the Preferred Egress Interface prior to the Convergence
Event Instant. This way these Convergence Events can be treated the Event Instant. This way, these Convergence Events can be treated the
same as Convergence Events that cause instantaneous traffic loss. same as Convergence Events that cause instantaneous traffic loss.
To measure convergence time benchmarks without instantaneous traffic To measure convergence time benchmarks without instantaneous traffic
loss (either real or induced by the Tester) at the Convergence Event loss (either real or induced by the Tester) at the Convergence Event
Instant, such as a reversion of a link failure Convergence Event, the Instant, such as a reversion of a link failure Convergence Event, the
Tester SHALL only observe packet statistics on the Next-Best Egress Tester SHALL only observe packet statistics on the Next-Best Egress
Interface. If using the Rate-Derived method to benchmark convergence Interface. If using the Rate-Derived method to benchmark convergence
times for such Convergence Events, the Tester MUST collect a times for such Convergence Events, the Tester MUST collect a
timestamp at the Convergence Event Instant. If using a loss-derived timestamp at the Convergence Event Instant. If using a loss-derived
method to benchmark convergence times for such Convergence Events, method to benchmark convergence times for such Convergence Events,
the Tester MUST measure the period in time between the Start Traffic the Tester MUST measure the period in time between the Start Traffic
Instant and the Convergence Event Instant. To measure this period in Instant and the Convergence Event Instant. To measure this period in
time the Tester can collect timestamps at the Start Traffic Instant time, the Tester can collect timestamps at the Start Traffic Instant
and the Convergence Event Instant. and the Convergence Event Instant.
The Convergence Event Instant together with the receive rate The Convergence Event Instant together with the receive rate
observations on the Next-Best Egress Interface allow to derive the observations on the Next-Best Egress Interface allow the derivation
convergence time benchmarks using the Rate-Derived Method [Po11t]. of the convergence time benchmarks using the Rate-Derived Method
[Po11t].
By observing packets on the Next-Best Egress Interface only, the By observing packets on the Next-Best Egress Interface only, the
observed Impaired Packet count is the number of Impaired Packets observed Impaired Packet count is the number of Impaired Packets
between Traffic Start Instant and Convergence Recovery Instant. To between Traffic Start Instant and Convergence Recovery Instant. To
measure convergence times using a loss-derived method, the Impaired measure convergence times using a loss-derived method, the Impaired
Packet count between the Convergence Event Instant and the Packet count between the Convergence Event Instant and the
Convergence Recovery Instant is needed. The time between Traffic Convergence Recovery Instant is needed. The time between Traffic
Start Instant and Convergence Event Instant must be accounted for. Start Instant and Convergence Event Instant must be accounted for.
An example may clarify this. An example may clarify this.
Figure 8 illustrates a Convergence Event without instantaneous Figure 8 illustrates a Convergence Event without instantaneous
traffic loss for all routes. The top graph shows the Forwarding Rate traffic loss for all routes. The top graph shows the Forwarding Rate
over all routes, the bottom graph shows the Forwarding Rate for a over all routes, the bottom graph shows the Forwarding Rate for a
single route Rta. Some time after the Convergence Event Instant, single route Rta. Some time after the Convergence Event Instant, the
Forwarding Rate observed on the Preferred Egress Interface starts to Forwarding Rate observed on the Preferred Egress Interface starts to
decrease. In the example, route Rta is the first route to experience decrease. In the example, route Rta is the first route to experience
packet loss at time Ta. Some time later, the Forwarding Rate packet loss at time Ta. Some time later, the Forwarding Rate
observed on the Next-Best Egress Interface starts to increase. In observed on the Next-Best Egress Interface starts to increase. In
the example, route Rta is the first route to complete convergence at the example, route Rta is the first route to complete convergence at
time Ta'. time Ta'.
^ ^
Fwd | Fwd |
Rate |------------- ............ Rate |------------- ............
| \ . | \ .
| \ . | \ .
| \ . | \ .
| \ . | \ .
|.................-.-.-.-.-.-.---------------- |.................-.-.-.-.-.-.----------------
+----+-------+---------------+-----------------> +----+-------+---------------+----------------->
^ ^ ^ ^ time ^ ^ ^ ^ time
T0 CEI Ta Ta' T0 CEI Ta Ta'
^ ^
Fwd | Fwd |
Rate |------------- ................. Rate |------------- .................
Rta | | . Rta | | .
| | . | | .
|.............-.-.-.-.-.-.-.-.---------------- |.............-.-.-.-.-.-.-.-.----------------
+----+-------+---------------+-----------------> +----+-------+---------------+----------------->
^ ^ ^ ^ time ^ ^ ^ ^ time
T0 CEI Ta Ta' T0 CEI Ta Ta'
Preferred Egress Interface: --- Preferred Egress Interface: ---
Next-Best Egress Interface: ... Next-Best Egress Interface: ...
With T0 the Start Traffic Instant; CEI the Convergence Event Instant; T0 : Start Traffic Instant
Ta the time instant packet loss for route Rta starts; Ta' the time CEI : Convergence Event Instant
instant packet impairment for route Rta ends. Ta : the time instant packet loss for route Rta starts
Ta' : the time instant packet impairment for route Rta ends
Figure 8 Figure 8
If only packets received on the Next-Best Egress Interface are If only packets received on the Next-Best Egress Interface are
observed, the duration of the loss period for route Rta can be observed, the duration of the loss period for route Rta can be
calculated from the received packets as in Equation 1. Since the calculated from the received packets as in Equation 1. Since the
Convergence Event Instant is the start time for convergence time Convergence Event Instant is the start time for convergence time
measurement, the period in time between T0 and CEI needs to be measurement, the period in time between T0 and CEI needs to be
subtracted from the calculated result to become the convergence time, subtracted from the calculated result to become the convergence time,
as in Equation 2. as in Equation 2.
skipping to change at page 16, line 24 skipping to change at page 16, line 16
= Next-Best Egress Interface loss period - (CEI - T0) = Next-Best Egress Interface loss period - (CEI - T0)
= Ta' - CEI = Ta' - CEI
Equation 2 Equation 2
4.2. Loss of Connectivity (LoC) 4.2. Loss of Connectivity (LoC)
Route Loss of Connectivity Period SHOULD be measured using the Route- Route Loss of Connectivity Period SHOULD be measured using the Route-
Specific Loss-Derived Method. Since the start instant and end Specific Loss-Derived Method. Since the start instant and end
instant of the Route Loss of Connectivity Period can be different for instant of the Route Loss of Connectivity Period can be different for
each route, these can not be accurately derived by only observing each route, these cannot be accurately derived by only observing
global statistics over all routes. An example may clarify this. global statistics over all routes. An example may clarify this.
Following a Convergence Event, route Rta is the first route for which Following a Convergence Event, route Rta is the first route for which
packet impairment starts, the Route Loss of Connectivity Period for packet impairment starts; the Route Loss of Connectivity Period for
route Rta starts at time Ta. Route Rtb is the last route for which route Rta starts at time Ta. Route Rtb is the last route for which
packet impairment starts, the Route Loss of Connectivity Period for packet impairment starts; the Route Loss of Connectivity Period for
route Rtb starts at time Tb with Tb>Ta. route Rtb starts at time Tb with Tb>Ta.
^ ^
Fwd | Fwd |
Rate |-------- ----------- Rate |-------- -----------
| \ / | \ /
| \ / | \ /
| \ / | \ /
| \ / | \ /
| --------------- | ---------------
+------------------------------------------> +------------------------------------------>
^ ^ ^ ^ time ^ ^ ^ ^ time
Ta Tb Ta' Tb' Ta Tb Ta' Tb'
Tb'' Ta'' Tb'' Ta''
Figure 9: Example Route Loss Of Connectivity Period Figure 9: Example Route Loss Of Connectivity Period
If the DUT implementation were such that route Rta would be the first If the DUT implementation were such that route Rta would be the first
route for which traffic loss ends at time Ta' (with Ta'>Tb) and route route for which traffic loss ends at time Ta' (with Ta'>Tb), and
Rtb would be the last route for which traffic loss ends at time Tb' route Rtb would be the last route for which traffic loss ends at time
(with Tb'>Ta'). By only observing global traffic statistics over all Tb' (with Tb'>Ta'). By only observing global traffic statistics over
routes, the minimum Route Loss of Connectivity Period would be all routes, the minimum Route Loss of Connectivity Period would be
measured as Ta'-Ta. The maximum calculated Route Loss of measured as Ta'-Ta. The maximum calculated Route Loss of
Connectivity Period would be Tb'-Ta. The real minimum and maximum Connectivity Period would be Tb'-Ta. The real minimum and maximum
Route Loss of Connectivity Periods are Ta'-Ta and Tb'-Tb. Route Loss of Connectivity Periods are Ta'-Ta and Tb'-Tb.
Illustrating this with the numbers Ta=0, Tb=1, Ta'=3, and Tb'=5, Illustrating this with the numbers Ta=0, Tb=1, Ta'=3, and Tb'=5 would
would give a Loss of Connectivity Period between 3 and 5 derived from give a Loss of Connectivity Period between 3 and 5 derived from the
the global traffic statistics, versus the real Loss of Connectivity global traffic statistics, versus the real Loss of Connectivity
Period between 3 and 4. Period between 3 and 4.
If the DUT implementation were such that route Rtb would be the first If the DUT implementation were such that route Rtb would be the first
for which packet loss ends at time Tb'' and route Rta would be the for which packet loss ends at time Tb'' and route Rta would be the
last for which packet impairment ends at time Ta'', then the minimum last for which packet impairment ends at time Ta'', then the minimum
and maximum Route Loss of Connectivity Periods derived by observing and maximum Route Loss of Connectivity Periods derived by observing
only global traffic statistics would be Tb''-Ta, and Ta''-Ta. The only global traffic statistics would be Tb''-Ta and Ta''-Ta. The
real minimum and maximum Route Loss of Connectivity Periods are real minimum and maximum Route Loss of Connectivity Periods are
Tb''-Tb and Ta''-Ta. Illustrating this with the numbers Ta=0, Tb=1, Tb''-Tb and Ta''-Ta. Illustrating this with the numbers Ta=0, Tb=1,
Ta''=5, Tb''=3, would give a Loss of Connectivity Period between 3 Ta''=5, Tb''=3 would give a Loss of Connectivity Period between 3 and
and 5 derived from the global traffic statistics, versus the real 5 derived from the global traffic statistics, versus the real Loss of
Loss of Connectivity Period between 2 and 5. Connectivity Period between 2 and 5.
The two implementation variations in the above example would result The two implementation variations in the above example would result
in the same derived minimum and maximum Route Loss of Connectivity in the same derived minimum and maximum Route Loss of Connectivity
Periods when only observing the global packet statistics, while the Periods when only observing the global packet statistics, while the
real Route Loss of Connectivity Periods are different. real Route Loss of Connectivity Periods are different.
5. Test Considerations 5. Test Considerations
5.1. IGP Selection 5.1. IGP Selection
skipping to change at page 18, line 12 skipping to change at page 17, line 51
The Tester emulates a single IGP topology. The DUT establishes IGP The Tester emulates a single IGP topology. The DUT establishes IGP
adjacencies with one or more of the emulated routers in this single adjacencies with one or more of the emulated routers in this single
IGP topology emulated by the Tester. See test topology details in IGP topology emulated by the Tester. See test topology details in
Section 3. The emulated topology SHOULD only be advertised on the Section 3. The emulated topology SHOULD only be advertised on the
DUT egress interfaces. DUT egress interfaces.
The number of IGP routes and number of nodes in the topology, and the The number of IGP routes and number of nodes in the topology, and the
type of topology will impact the measured IGP convergence time. To type of topology will impact the measured IGP convergence time. To
obtain results similar to those that would be observed in an obtain results similar to those that would be observed in an
operational network, it is RECOMMENDED that the number of installed operational network, it is RECOMMENDED that the number of installed
routes and nodes closely approximate that of the network (e.g. routes and nodes closely approximate that of the network (e.g.,
thousands of routes with tens or hundreds of nodes). thousands of routes with tens or hundreds of nodes).
The number of areas (for OSPF) and levels (for IS-IS) can impact the The number of areas (for OSPF) and levels (for IS-IS) can impact the
benchmark results. benchmark results.
5.4. Timers 5.4. Timers
There are timers that may impact the measured IGP convergence times. There are timers that may impact the measured IGP convergence times.
The benchmark metrics MAY be measured at any fixed values for these The benchmark metrics MAY be measured at any fixed values for these
timers. To obtain results similar to those that would be observed in timers. To obtain results similar to those that would be observed in
skipping to change at page 18, line 40 skipping to change at page 18, line 30
IGP hello timer IGP hello timer
IGP dead-interval or hold-timer IGP dead-interval or hold-timer
Link State Advertisement (LSA) or Link State Packet (LSP) Link State Advertisement (LSA) or Link State Packet (LSP)
generation delay generation delay
LSA or LSP flood packet pacing LSA or LSP flood packet pacing
route calculation delay Route calculation delay
5.5. Interface Types 5.5. Interface Types
All test cases in this methodology document can be executed with any All test cases in this methodology document can be executed with any
interface type. The type of media may dictate which test cases may interface type. The type of media may dictate which test cases may
be executed. Each interface type has a unique mechanism for be executed. Each interface type has a unique mechanism for
detecting link failures and the speed at which that mechanism detecting link failures, and the speed at which that mechanism
operates will influence the measurement results. All interfaces MUST operates will influence the measurement results. All interfaces MUST
be the same media and Throughput [Br91][Br99] for each test case. be the same media and Throughput [Br91][Br99] for each test case.
All interfaces SHOULD be configured as point-to-point. All interfaces SHOULD be configured as point-to-point.
5.6. Offered Load 5.6. Offered Load
The Throughput of the device, as defined in [Br91] and benchmarked in The Throughput of the device, as defined in [Br91] and benchmarked in
[Br99] at a fixed packet size, needs to be determined over the [Br99] at a fixed packet size, needs to be determined over the
preferred path and over the next-best path. The Offered Load SHOULD preferred path and over the next-best path. The Offered Load SHOULD
be the minimum of the measured Throughput of the device over the be the minimum of the measured Throughput of the device over the
skipping to change at page 19, line 29 skipping to change at page 19, line 19
routes, but a statistically representative subset of all routes can routes, but a statistically representative subset of all routes can
be used if required. be used if required.
Splitting traffic flows across multiple paths (as with ECMP or Splitting traffic flows across multiple paths (as with ECMP or
Parallel Link sets) is in general done by hashing on various fields Parallel Link sets) is in general done by hashing on various fields
on the IP or contained headers. The hashing is typically based on on the IP or contained headers. The hashing is typically based on
the IP source and destination addresses, the protocol ID, and higher- the IP source and destination addresses, the protocol ID, and higher-
layer flow-dependent fields such as TCP/UDP ports. In practice, layer flow-dependent fields such as TCP/UDP ports. In practice,
within a network core, the hashing is based mainly or exclusively on within a network core, the hashing is based mainly or exclusively on
the IP source and destination addresses. Knowledge of the hashing the IP source and destination addresses. Knowledge of the hashing
algorithm used by the DUT is not always possible beforehand, and algorithm used by the DUT is not always possible beforehand and would
would violate the black-box spirit of this document. Therefor it is violate the black-box spirit of this document. Therefore, it is
RECOMMENDED to use a randomly distributed range of source and RECOMMENDED to use a randomly distributed range of source and
destination IP addresses, protocol IDs, and higher-layer flow- destination IP addresses, protocol IDs, and higher-layer flow-
dependent fields for the packets of the Offered Load (see also dependent fields for the packets of the Offered Load (see also
[Ne07]). The content of the Offered Load MUST remain the same during [Ne07]). The content of the Offered Load MUST remain the same during
the test. It is RECOMMENDED to repeat a test multiple times with the test. It is RECOMMENDED to repeat a test multiple times with
different random ranges of the header fields such that convergence different random ranges of the header fields such that convergence
time benchmarks are measured for different distributions of traffic time benchmarks are measured for different distributions of traffic
over the available paths. over the available paths.
In the Remote Interface failure testcases using topologies 3, 4, and In the Remote Interface failure test cases using topologies 3, 4, and
6 there is a possibility of a packet forwarding loop that may occur 6, there is a possibility of a packet-forwarding loop that may occur
transiently between DUT1 and DUT2 during convergence (micro-loop, see transiently between DUT1 and DUT2 during convergence (micro-loop, see
[Sh10]). The Time To Live (TTL) or Hop Limit value of the packets [Sh10]). The Time To Live (TTL) or Hop Limit value of the packets
sent by the Tester may influence the benchmark measurements since it sent by the Tester may influence the benchmark measurements since it
determines which device in the topology may send an ICMP Time determines which device in the topology may send an ICMP Time
Exceeded Message for looped packets. Exceeded Message for looped packets.
The duration of the Offered Load MUST be greater than the convergence The duration of the Offered Load MUST be greater than the convergence
time plus the Sustained Convergence Validation Time. time plus the Sustained Convergence Validation Time.
Offered load should send a packet to each destination before sending Offered load should send a packet to each destination before sending
another packet to the same destination. It is RECOMMENDED that the another packet to the same destination. It is RECOMMENDED that the
packets be transmitted in a round-robin fashion with a uniform packets be transmitted in a round-robin fashion with a uniform
interpacket delay. interpacket delay.
5.7. Measurement Accuracy 5.7. Measurement Accuracy
Since Impaired Packet count is observed to measure the Route Since Impaired Packet count is observed to measure the Route
Convergence Time, the time between two successive packets offered to Convergence Time, the time between two successive packets offered to
each individual route is the highest possible accuracy of any each individual route is the highest possible accuracy of any
Impaired Packet based measurement. The higher the traffic rate Impaired-Packet-based measurement. The higher the traffic rate
offered to each route the higher the possible measurement accuracy. offered to each route, the higher the possible measurement accuracy.
Also see Section 6 for method-specific measurement accuracy. Also see Section 6 for method-specific measurement accuracy.
5.8. Measurement Statistics 5.8. Measurement Statistics
The benchmark measurements may vary for each trial, due to the The benchmark measurements may vary for each trial, due to the
statistical nature of timer expirations, cpu scheduling, etc. statistical nature of timer expirations, CPU scheduling, etc.
Evaluation of the test data must be done with an understanding of Evaluation of the test data must be done with an understanding of
generally accepted testing practices regarding repeatability, generally accepted testing practices regarding repeatability,
variance and statistical significance of a small number of trials. variance, and statistical significance of a small number of trials.
5.9. Tester Capabilities 5.9. Tester Capabilities
It is RECOMMENDED that the Tester used to execute each test case have It is RECOMMENDED that the Tester used to execute each test case have
the following capabilities: the following capabilities:
1. Ability to establish IGP adjacencies and advertise a single IGP 1. Ability to establish IGP adjacencies and advertise a single IGP
topology to one or more peers. topology to one or more peers.
2. Ability to measure Forwarding Delay, Duplicate Packets and Out- 2. Ability to measure Forwarding Delay, Duplicate Packets, and Out-
of-Order Packets. of-Order Packets.
3. An internal time clock to control timestamping, time 3. An internal time clock to control timestamping, time
measurements, and time calculations. measurements, and time calculations.
4. Ability to distinguish traffic load received on the Preferred and 4. Ability to distinguish traffic load received on the Preferred and
Next-Best Interfaces [Po11t]. Next-Best Interfaces [Po11t].
5. Ability to disable or tune specific Layer-2 and Layer-3 protocol 5. Ability to disable or tune specific Layer 2 and Layer 3 protocol
functions on any interface(s). functions on any interface(s).
The Tester MAY be capable to make non-data plane convergence The Tester MAY be capable of making non-data-plane convergence
observations and use those observations for measurements. The Tester observations and using those observations for measurements. The
MAY be capable to send and receive multiple traffic Streams [Po06]. Tester MAY be capable of sending and receiving multiple traffic
Streams [Po06].
Also see Section 6 for method-specific capabilities. Also see Section 6 for method-specific capabilities.
6. Selection of Convergence Time Benchmark Metrics and Methods 6. Selection of Convergence Time Benchmark Metrics and Methods
Different convergence time benchmark methods MAY be used to measure Different convergence time benchmark methods MAY be used to measure
convergence time benchmark metrics. The Tester capabilities are convergence time benchmark metrics. The Tester capabilities are
important criteria to select a specific convergence time benchmark important criteria to select a specific convergence time benchmark
method. The criteria to select a specific benchmark method include, method. The criteria to select a specific benchmark method include,
but are not limited to: but are not limited to:
skipping to change at page 21, line 23 skipping to change at page 21, line 15
Tester capabilities: Sampling Interval, number of Tester capabilities: Sampling Interval, number of
Stream statistics to collect Stream statistics to collect
Measurement accuracy: Sampling Interval, Offered Load, Measurement accuracy: Sampling Interval, Offered Load,
number of routes number of routes
Test specification: number of routes Test specification: number of routes
DUT capabilities: Throughput, IP Packet Delay DUT capabilities: Throughput, IP Packet Delay
Variation Variation
6.1. Loss-Derived Method 6.1. Loss-Derived Method
6.1.1. Tester capabilities 6.1.1. Tester Capabilities
To enable collecting statistics of Out-of-Order Packets per flow (See To enable collecting statistics of Out-of-Order Packets per flow (see
[Th00], Section 3) the Offered Load SHOULD consist of multiple [Th00], Section 3), the Offered Load SHOULD consist of multiple
Streams [Po06] and each Stream SHOULD consist of a single flow . If Streams [Po06], and each Stream SHOULD consist of a single flow. If
sending multiple Streams, the measured traffic statistics for all sending multiple Streams, the measured traffic statistics for all
Streams MUST be added together. Streams MUST be added together.
In order to verify Full Convergence completion and the Sustained In order to verify Full Convergence completion and the Sustained
Convergence Validation Time, the Tester MUST measure Forwarding Rate Convergence Validation Time, the Tester MUST measure Forwarding Rate
each Packet Sampling Interval. each Packet Sampling Interval.
The total number of Impaired Packets between the start of the traffic The total number of Impaired Packets between the start of the traffic
and the end of the Sustained Convergence Validation Time is used to and the end of the Sustained Convergence Validation Time is used to
calculate the Loss-Derived Convergence Time. calculate the Loss-Derived Convergence Time.
skipping to change at page 22, line 9 skipping to change at page 22, line 9
6.1.3. Measurement Accuracy 6.1.3. Measurement Accuracy
The actual value falls within the accuracy interval [-(number of The actual value falls within the accuracy interval [-(number of
destinations/Offered Load), +(number of destinations/Offered Load)] destinations/Offered Load), +(number of destinations/Offered Load)]
around the value as measured using the Loss-Derived Method. around the value as measured using the Loss-Derived Method.
6.2. Rate-Derived Method 6.2. Rate-Derived Method
6.2.1. Tester Capabilities 6.2.1. Tester Capabilities
To enable collecting statistics of Out-of-Order Packets per flow (See To enable collecting statistics of Out-of-Order Packets per flow (see
[Th00], Section 3) the Offered Load SHOULD consist of multiple [Th00], Section 3), the Offered Load SHOULD consist of multiple
Streams [Po06] and each Stream SHOULD consist of a single flow . If Streams [Po06], and each Stream SHOULD consist of a single flow. If
sending multiple Streams, the measured traffic statistics for all sending multiple Streams, the measured traffic statistics for all
Streams MUST be added together. Streams MUST be added together.
The Tester measures Forwarding Rate each Sampling Interval. The The Tester measures Forwarding Rate each Sampling Interval. The
Packet Sampling Interval influences the observation of the different Packet Sampling Interval influences the observation of the different
convergence time instants. If the Packet Sampling Interval is large convergence time instants. If the Packet Sampling Interval is large
compared to the time between the convergence time instants, then the compared to the time between the convergence time instants, then the
different time instants may not be easily identifiable from the different time instants may not be easily identifiable from the
Forwarding Rate observation. The presence of IP Packet Delay Forwarding Rate observation. The presence of IP Packet Delay
Variation (IPDV) [De02] may cause fluctuations of the Forwarding Rate Variation (IPDV) [De02] may cause fluctuations of the Forwarding Rate
observation and can prevent correct observation of the different observation and can prevent correct observation of the different
convergence time instants. convergence time instants.
The Packet Sampling Interval MUST be larger than or equal to the time The Packet Sampling Interval MUST be larger than or equal to the time
between two consecutive packets to the same destination. For maximum between two consecutive packets to the same destination. For maximum
accuracy the value for the Packet Sampling Interval SHOULD be as accuracy, the value for the Packet Sampling Interval SHOULD be as
small as possible, but the presence of IPDV may enforce using a small as possible, but the presence of IPDV may require the use of a
larger Packet Sampling Interval. The Packet Sampling Interval MUST larger Packet Sampling Interval. The Packet Sampling Interval MUST
be reported. be reported.
IPDV causes fluctuations in the number of received packets during IPDV causes fluctuations in the number of received packets during
each Packet Sampling Interval. To account for the presence of IPDV each Packet Sampling Interval. To account for the presence of IPDV
in determining if a convergence instant has been reached, Forwarding in determining if a convergence instant has been reached, Forwarding
Delay SHOULD be observed during each Packet Sampling Interval. The Delay SHOULD be observed during each Packet Sampling Interval. The
minimum and maximum number of packets expected in a Packet Sampling minimum and maximum number of packets expected in a Packet Sampling
Interval in presence of IPDV can be calculated with Equation 3. Interval in presence of IPDV can be calculated with Equation 3.
number of packets expected in a Packet Sampling Interval number of packets expected in a Packet Sampling Interval
in presence of IP Packet Delay Variation in presence of IP Packet Delay Variation
= expected number of packets without IP Packet Delay Variation = expected number of packets without IP Packet Delay Variation
+/-( (maxDelay - minDelay) * Offered Load) +/-( (maxDelay - minDelay) * Offered Load)
with minDelay and maxDelay the minimum resp. maximum Forwarding Delay where minDelay and maxDelay indicate (respectively) the minimum and
of packets received during the Packet Sampling Interval maximum Forwarding Delay of packets received during the Packet
Sampling Interval
Equation 3 Equation 3
To determine if a convergence instant has been reached the number of To determine if a convergence instant has been reached, the number of
packets received in a Packet Sampling Interval is compared with the packets received in a Packet Sampling Interval is compared with the
range of expected number of packets calculated in Equation 3. range of expected number of packets calculated in Equation 3.
6.2.2. Benchmark Metrics 6.2.2. Benchmark Metrics
The Rate-Derived Method SHOULD be used to measure First Route The Rate-Derived Method SHOULD be used to measure First Route
Convergence Time and Full Convergence Time. It SHOULD NOT be used to Convergence Time and Full Convergence Time. It SHOULD NOT be used to
measure Loss of Connectivity Period (see Section 4). measure Loss of Connectivity Period (see Section 4).
6.2.3. Measurement Accuracy 6.2.3. Measurement Accuracy
The measurement accuracy interval of the Rate-Derived Method depends The measurement accuracy interval of the Rate-Derived Method depends
on the metric being measured or calculated and the characteristics of on the metric being measured or calculated and the characteristics of
the related transition. IP Packet Delay Variation (IPDV) [De02] adds the related transition. IP Packet Delay Variation (IPDV) [De02] adds
uncertainty to the amount of packets received in a Packet Sampling uncertainty to the amount of packets received in a Packet Sampling
Interval and this uncertainty adds to the measurement error. The Interval, and this uncertainty adds to the measurement error. The
effect of IPDV is not accounted for in the calculation of the effect of IPDV is not accounted for in the calculation of the
accuracy intervals below. IPDV is of importance for the convergence accuracy intervals below. IPDV is of importance for the convergence
instants were a variation in Forwarding Rate needs to be observed instants where a variation in Forwarding Rate needs to be observed.
(Convergence Recovery Instant and for topologies with ECMP also This is applicable to the Convergence Recovery Instant for all
Convergence Event Instant and First Route Convergence Instant). topologies, and for topologies with ECMP it also applies to the
Convergence Event Instant and the First Route Convergence Instant.
and for topologies with ECMP also Convergence Event Instant and First
Route Convergence Instant).
If the Convergence Event Instant is observed on the dataplane using If the Convergence Event Instant is observed on the data plane using
the Rate Derived Method, it needs to be instantaneous for all routes the Rate Derived Method, it needs to be instantaneous for all routes
(see Section 4.1). The actual value of the Convergence Event Instant (see Section 4.1). The actual value of the Convergence Event Instant
falls within the accuracy interval [-(Packet Sampling Interval + falls within the accuracy interval [-(Packet Sampling Interval +
1/Offered Load), +0] around the value as measured using the Rate- 1/Offered Load), +0] around the value as measured using the Rate-
Derived Method. Derived Method.
If the Convergence Recovery Transition is non-instantaneous for all If the Convergence Recovery Transition is non-instantaneous for all
routes then the actual value of the First Route Convergence Instant routes, then the actual value of the First Route Convergence Instant
falls within the accuracy interval [-(Packet Sampling Interval + time falls within the accuracy interval [-(Packet Sampling Interval + time
between two consecutive packets to the same destination), +0] around between two consecutive packets to the same destination), +0] around
the value as measured using the Rate-Derived Method, and the actual the value as measured using the Rate-Derived Method, and the actual
value of the Convergence Recovery Instant falls within the accuracy value of the Convergence Recovery Instant falls within the accuracy
interval [-(2 * Packet Sampling Interval), -(Packet Sampling Interval interval [-(2 * Packet Sampling Interval), -(Packet Sampling Interval
- time between two consecutive packets to the same destination)] - time between two consecutive packets to the same destination)]
around the value as measured using the Rate-Derived Method. around the value as measured using the Rate-Derived Method.
The term "time between two consecutive packets to the same The term "time between two consecutive packets to the same
destination" is added in the above accuracy intervals since packets destination" is added in the above accuracy intervals since packets
are sent in a particular order to all destinations in a stream and are sent in a particular order to all destinations in a stream, and
when part of the routes experience packet loss, it is unknown where when part of the routes experience packet loss, it is unknown where
in the transmit cycle packets to these routes are sent. This in the transmit cycle packets to these routes are sent. This
uncertainty adds to the error. uncertainty adds to the error.
The accuracy intervals of the derived metrics First Route Convergence The accuracy intervals of the derived metrics First Route Convergence
Time and Rate-Derived Convergence Time are calculated from the above Time and Rate-Derived Convergence Time are calculated from the above
convergence instants accuracy intervals. The actual value of First convergence instants accuracy intervals. The actual value of First
Route Convergence Time falls within the accuracy interval [-(Packet Route Convergence Time falls within the accuracy interval [-(Packet
Sampling Interval + time between two consecutive packets to the same Sampling Interval + time between two consecutive packets to the same
destination), +(Packet Sampling Interval + 1/Offered Load)] around destination), +(Packet Sampling Interval + 1/Offered Load)] around
skipping to change at page 24, line 33 skipping to change at page 24, line 39
end of the Sustained Convergence Validation Time is used to calculate end of the Sustained Convergence Validation Time is used to calculate
the benchmark metrics with this method. the benchmark metrics with this method.
6.3.2. Benchmark Metrics 6.3.2. Benchmark Metrics
The Route-Specific Loss-Derived Method SHOULD be used to measure The Route-Specific Loss-Derived Method SHOULD be used to measure
Route-Specific Convergence Times. It is the RECOMMENDED method to Route-Specific Convergence Times. It is the RECOMMENDED method to
measure Route Loss of Connectivity Period. measure Route Loss of Connectivity Period.
Under the conditions explained in Section 4, First Route Convergence Under the conditions explained in Section 4, First Route Convergence
Time and Full Convergence Time as benchmarked using Rate-Derived Time and Full Convergence Time, as benchmarked using Rate-Derived
Method, may be equal to the minimum resp. maximum of the Route- Method, may be equal to the minimum and maximum (respectively) of the
Specific Convergence Times. Route-Specific Convergence Times.
6.3.3. Measurement Accuracy 6.3.3. Measurement Accuracy
The actual value falls within the accuracy interval [-(number of The actual value falls within the accuracy interval [-(number of
destinations/Offered Load), +(number of destinations/Offered Load)] destinations/Offered Load), +(number of destinations/Offered Load)]
around the value as measured using the Route-Specific Loss-Derived around the value as measured using the Route-Specific Loss-Derived
Method. Method.
7. Reporting Format 7. Reporting Format
For each test case, it is RECOMMENDED that the reporting tables below For each test case, it is RECOMMENDED that the reporting tables below
be completed and all time values SHOULD be reported with a be completed. All time values SHOULD be reported with a sufficiently
sufficiently high resolution. high resolution (fractions of a second sufficient to distinguish
significant differences between measured values).
Parameter Units Parameter Units
------------------------------------- --------------------------- ------------------------------------- ---------------------------
Test Case test case number Test Case test case number
Test Topology Test Topology Figure number Test Topology Test Topology Figure number
IGP (IS-IS, OSPF, other) IGP (IS-IS, OSPF, other)
Interface Type (GigE, POS, ATM, other) Interface Type (GigE, POS, ATM, other)
Packet Size offered to DUT bytes Packet Size offered to DUT bytes
Offered Load packets per second Offered Load packets per second
IGP Routes advertised to DUT number of IGP routes IGP Routes Advertised to DUT number of IGP routes
Nodes in emulated network number of nodes Nodes in Emulated Network number of nodes
Number of Parallel or ECMP links number of links Number of Parallel or ECMP links number of links
Number of Routes measured number of routes Number of Routes Measured number of routes
Packet Sampling Interval on Tester seconds Packet Sampling Interval on Tester seconds
Forwarding Delay Threshold seconds Forwarding Delay Threshold seconds
Timer Values configured on DUT: Timer Values configured on DUT:
Interface failure indication delay seconds Interface Failure Indication Delay seconds
IGP Hello Timer seconds IGP Hello Timer seconds
IGP Dead-Interval or hold-time seconds IGP Dead-Interval or Hold-Time seconds
LSA/LSP Generation Delay seconds LSA/LSP Generation Delay seconds
LSA/LSP Flood Packet Pacing seconds LSA/LSP Flood Packet Pacing seconds
LSA/LSP Retransmission Packet Pacing seconds LSA/LSP Retransmission Packet Pacing seconds
route calculation Delay seconds Route Calculation Delay seconds
Test Details: Test Details:
Describe the IGP extensions and IGP security mechanisms that are Describe the IGP extensions and IGP security mechanisms that are
configured on the DUT. configured on the DUT.
Describe how the various fields on the IP and contained headers Describe how the various fields on the IP and contained headers
for the packets for the Offered Load are generated (Section 5.6). for the packets for the Offered Load are generated (Section 5.6).
If the Offered Load matches a subset of routes, describe how this If the Offered Load matches a subset of routes, describe how this
skipping to change at page 26, line 10 skipping to change at page 26, line 10
instantaneous traffic loss or not? instantaneous traffic loss or not?
The table below should be completed for the initial Convergence Event The table below should be completed for the initial Convergence Event
and the reversion Convergence Event. and the reversion Convergence Event.
Parameter Units Parameter Units
------------------------------------------- ---------------------- ------------------------------------------- ----------------------
Convergence Event (initial or reversion) Convergence Event (initial or reversion)
Traffic Forwarding Metrics: Traffic Forwarding Metrics:
Total number of packets offered to DUT number of Packets Total number of packets offered to DUT number of packets
Total number of packets forwarded by DUT number of Packets Total number of packets forwarded by DUT number of packets
Connectivity Packet Loss number of Packets Connectivity Packet Loss number of packets
Convergence Packet Loss number of Packets Convergence Packet Loss number of packets
Out-of-Order Packets number of Packets Out-of-Order Packets number of packets
Duplicate Packets number of Packets Duplicate Packets number of packets
excessive Forwarding Delay Packets number of Packets Excessive Forwarding Delay Packets number of packets
Convergence Benchmarks: Convergence Benchmarks:
Rate-Derived Method: Rate-Derived Method:
First Route Convergence Time seconds First Route Convergence Time seconds
Full Convergence Time seconds Full Convergence Time seconds
Loss-Derived Method: Loss-Derived Method:
Loss-Derived Convergence Time seconds Loss-Derived Convergence Time seconds
Route-Specific Loss-Derived Method: Route-Specific Loss-Derived Method:
Route-Specific Convergence Time[n] array of seconds Route-Specific Convergence Time[n] array of seconds
Minimum Route-Specific Convergence Time seconds Minimum Route-Specific Convergence Time seconds
skipping to change at page 27, line 5 skipping to change at page 26, line 51
8. Test Cases 8. Test Cases
It is RECOMMENDED that all applicable test cases be performed for It is RECOMMENDED that all applicable test cases be performed for
best characterization of the DUT. The test cases follow a generic best characterization of the DUT. The test cases follow a generic
procedure tailored to the specific DUT configuration and Convergence procedure tailored to the specific DUT configuration and Convergence
Event [Po11t]. This generic procedure is as follows: Event [Po11t]. This generic procedure is as follows:
1. Establish DUT and Tester configurations and advertise an IGP 1. Establish DUT and Tester configurations and advertise an IGP
topology from Tester to DUT. topology from Tester to DUT.
2. Send Offered Load from Tester to DUT on ingress interface. 2. Send Offered Load from Tester to DUT on Ingress Interface.
3. Verify traffic is routed correctly. Verify if traffic is 3. Verify traffic is routed correctly. Verify if traffic is
forwarded without Impaired Packets [Po06]. forwarded without Impaired Packets [Po06].
4. Introduce Convergence Event [Po11t]. 4. Introduce Convergence Event [Po11t].
5. Measure First Route Convergence Time [Po11t]. 5. Measure First Route Convergence Time [Po11t].
6. Measure Full Convergence Time [Po11t]. 6. Measure Full Convergence Time [Po11t].
7. Stop Offered Load. 7. Stop Offered Load.
8. Measure Route-Specific Convergence Times, Loss-Derived 8. Measure Route-Specific Convergence Times, Loss-Derived
Convergence Time, Route Loss of Connectivity Periods, and Loss- Convergence Time, Route Loss of Connectivity Periods, and Loss-
Derived Loss of Connectivity Period [Po11t]. At the same time Derived Loss of Connectivity Period [Po11t]. At the same time,
measure number of Impaired Packets [Po11t]. measure number of Impaired Packets [Po11t].
9. Wait sufficient time for queues to drain. The duration of this 9. Wait sufficient time for queues to drain. The duration of this
time period MUST be larger than or equal to the Forwarding Delay time period MUST be larger than or equal to the Forwarding Delay
Threshold. Threshold.
10. Restart Offered Load. 10. Restart Offered Load.
11. Reverse Convergence Event. 11. Reverse Convergence Event.
12. Measure First Route Convergence Time. 12. Measure First Route Convergence Time.
13. Measure Full Convergence Time. 13. Measure Full Convergence Time.
14. Stop Offered Load. 14. Stop Offered Load.
15. Measure Route-Specific Convergence Times, Loss-Derived 15. Measure Route-Specific Convergence Times, Loss-Derived
Convergence Time, Route Loss of Connectivity Periods, and Loss- Convergence Time, Route Loss of Connectivity Periods, and Loss-
Derived Loss of Connectivity Period. At the same time measure Derived Loss of Connectivity Period. At the same time, measure
number of Impaired Packets [Po11t]. number of Impaired Packets [Po11t].
8.1. Interface Failure and Recovery 8.1. Interface Failure and Recovery
8.1.1. Convergence Due to Local Interface Failure and Recovery 8.1.1. Convergence Due to Local Interface Failure and Recovery
Objective Objective:
To obtain the IGP convergence measurements for Local Interface To obtain the IGP convergence measurements for Local Interface
failure and recovery events. The Next-Best Egress Interface can be a failure and recovery events. The Next-Best Egress Interface can
single interface (Figure 1) or an ECMP set (Figure 2). The test with be a single interface (Figure 1) or an ECMP set (Figure 2). The
ECMP topology (Figure 2) is OPTIONAL. test with ECMP topology (Figure 2) is OPTIONAL.
Procedure Procedure:
1. Advertise an IGP topology from Tester to DUT using the topology 1. Advertise an IGP topology from Tester to DUT using the topology
shown in Figure 1 or 2. shown in Figures 1 or 2.
2. Send Offered Load from Tester to DUT on ingress interface. 2. Send Offered Load from Tester to DUT on Ingress Interface.
3. Verify traffic is forwarded over Preferred Egress Interface. 3. Verify traffic is forwarded over Preferred Egress Interface.
4. Remove link on the Preferred Egress Interface of the DUT. This 4. Remove link on the Preferred Egress Interface of the DUT. This
is the Convergence Event. is the Convergence Event.
5. Measure First Route Convergence Time. 5. Measure First Route Convergence Time.
6. Measure Full Convergence Time. 6. Measure Full Convergence Time.
7. Stop Offered Load. 7. Stop Offered Load.
8. Measure Route-Specific Convergence Times and Loss-Derived 8. Measure Route-Specific Convergence Times and Loss-Derived
Convergence Time. At the same time measure number of Impaired Convergence Time. At the same time, measure number of Impaired
Packets. Packets.
9. Wait sufficient time for queues to drain. 9. Wait sufficient time for queues to drain.
10. Restart Offered Load. 10. Restart Offered Load.
11. Restore link on the Preferred Egress Interface of the DUT. 11. Restore link on the Preferred Egress Interface of the DUT.
12. Measure First Route Convergence Time. 12. Measure First Route Convergence Time.
13. Measure Full Convergence Time. 13. Measure Full Convergence Time.
14. Stop Offered Load. 14. Stop Offered Load.
15. Measure Route-Specific Convergence Times, Loss-Derived 15. Measure Route-Specific Convergence Times, Loss-Derived
Convergence Time, Route Loss of Connectivity Periods, and Loss- Convergence Time, Route Loss of Connectivity Periods, and Loss-
Derived Loss of Connectivity Period.At the same time measure Derived Loss of Connectivity Period. At the same time, measure
number of Impaired Packets. number of Impaired Packets.
8.1.2. Convergence Due to Remote Interface Failure and Recovery 8.1.2. Convergence Due to Remote Interface Failure and Recovery
Objective Objective:
To obtain the IGP convergence measurements for Remote Interface To obtain the IGP convergence measurements for Remote Interface
failure and recovery events. The Next-Best Egress Interface can be a failure and recovery events. The Next-Best Egress Interface can
single interface (Figure 3) or an ECMP set (Figure 4). The test with be a single interface (Figure 3) or an ECMP set (Figure 4). The
ECMP topology (Figure 4) is OPTIONAL. test with ECMP topology (Figure 4) is OPTIONAL.
Procedure Procedure:
1. Advertise an IGP topology from Tester to SUT using the topology 1. Advertise an IGP topology from Tester to SUT using the topology
shown in Figure 3 or 4. shown in Figures 3 or 4.
2. Send Offered Load from Tester to SUT on ingress interface. 2. Send Offered Load from Tester to SUT on Ingress Interface.
3. Verify traffic is forwarded over Preferred Egress Interface. 3. Verify traffic is forwarded over Preferred Egress Interface.
4. Remove link on the interface of the Tester connected to the 4. Remove link on the interface of the Tester connected to the
Preferred Egress Interface of the SUT. This is the Convergence Preferred Egress Interface of the SUT. This is the Convergence
Event. Event.
5. Measure First Route Convergence Time. 5. Measure First Route Convergence Time.
6. Measure Full Convergence Time. 6. Measure Full Convergence Time.
7. Stop Offered Load. 7. Stop Offered Load.
8. Measure Route-Specific Convergence Times and Loss-Derived 8. Measure Route-Specific Convergence Times and Loss-Derived
Convergence Time. At the same time measure number of Impaired Convergence Time. At the same time, measure number of Impaired
Packets. Packets.
9. Wait sufficient time for queues to drain. 9. Wait sufficient time for queues to drain.
10. Restart Offered Load. 10. Restart Offered Load.
11. Restore link on the interface of the Tester connected to the 11. Restore link on the interface of the Tester connected to the
Preferred Egress Interface of the SUT. Preferred Egress Interface of the SUT.
12. Measure First Route Convergence Time. 12. Measure First Route Convergence Time.
13. Measure Full Convergence Time. 13. Measure Full Convergence Time.
14. Stop Offered Load. 14. Stop Offered Load.
15. Measure Route-Specific Convergence Times, Loss-Derived 15. Measure Route-Specific Convergence Times, Loss-Derived
Convergence Time, Route Loss of Connectivity Periods, and Loss- Convergence Time, Route Loss of Connectivity Periods, and Loss-
Derived Loss of Connectivity Period. At the same time measure Derived Loss of Connectivity Period. At the same time, measure
number of Impaired Packets. number of Impaired Packets.
Discussion Discussion:
In this test case there is a possibility of a packet forwarding loop In this test case, there is a possibility of a packet-forwarding
that may occur transiently between DUT1 and DUT2 during convergence loop that may occur transiently between DUT1 and DUT2 during
(micro-loop, see [Sh10]), which may increase the measured convergence convergence (micro-loop, see [Sh10]), which may increase the
times and loss of connectivity periods. measured convergence times and loss of connectivity periods.
8.1.3. Convergence Due to ECMP Member Local Interface Failure and 8.1.3. Convergence Due to ECMP Member Local Interface Failure and
Recovery Recovery
Objective Objective:
To obtain the IGP convergence measurements for Local Interface link To obtain the IGP convergence measurements for Local Interface
failure and recovery events of an ECMP Member. link failure and recovery events of an ECMP Member.
Procedure Procedure:
1. Advertise an IGP topology from Tester to DUT using the test 1. Advertise an IGP topology from Tester to DUT using the test
setup shown in Figure 5. setup shown in Figure 5.
2. Send Offered Load from Tester to DUT on ingress interface. 2. Send Offered Load from Tester to DUT on Ingress Interface.
3. Verify traffic is forwarded over the ECMP member interface of 3. Verify traffic is forwarded over the ECMP member interface of
the DUT that will be failed in the next step. the DUT that will be failed in the next step.
4. Remove link on one of the ECMP member interfaces of the DUT. 4. Remove link on one of the ECMP member interfaces of the DUT.
This is the Convergence Event. This is the Convergence Event.
5. Measure First Route Convergence Time. 5. Measure First Route Convergence Time.
6. Measure Full Convergence Time. 6. Measure Full Convergence Time.
7. Stop Offered Load. 7. Stop Offered Load.
8. Measure Route-Specific Convergence Times and Loss-Derived 8. Measure Route-Specific Convergence Times and Loss-Derived
Convergence Time. At the same time measure number of Impaired Convergence Time. At the same time, measure number of Impaired
Packets. Packets.
9. Wait sufficient time for queues to drain. 9. Wait sufficient time for queues to drain.
10. Restart Offered Load. 10. Restart Offered Load.
11. Restore link on the ECMP member interface of the DUT. 11. Restore link on the ECMP member interface of the DUT.
12. Measure First Route Convergence Time. 12. Measure First Route Convergence Time.
13. Measure Full Convergence Time. 13. Measure Full Convergence Time.
14. Stop Offered Load. 14. Stop Offered Load.
15. Measure Route-Specific Convergence Times, Loss-Derived 15. Measure Route-Specific Convergence Times, Loss-Derived
Convergence Time, Route Loss of Connectivity Periods, and Loss- Convergence Time, Route Loss of Connectivity Periods, and Loss-
Derived Loss of Connectivity Period. At the same time measure Derived Loss of Connectivity Period. At the same time, measure
number of Impaired Packets. number of Impaired Packets.
8.1.4. Convergence Due to ECMP Member Remote Interface Failure and 8.1.4. Convergence Due to ECMP Member Remote Interface Failure and
Recovery Recovery
Objective Objective:
To obtain the IGP convergence measurements for Remote Interface link To obtain the IGP convergence measurements for Remote Interface
failure and recovery events for an ECMP Member. link failure and recovery events for an ECMP Member.
Procedure Procedure:
1. Advertise an IGP topology from Tester to DUT using the test 1. Advertise an IGP topology from Tester to DUT using the test
setup shown in Figure 6. setup shown in Figure 6.
2. Send Offered Load from Tester to DUT on ingress interface. 2. Send Offered Load from Tester to DUT on Ingress Interface.
3. Verify traffic is forwarded over the ECMP member interface of 3. Verify traffic is forwarded over the ECMP member interface of
the DUT that will be failed in the next step. the DUT that will be failed in the next step.
4. Remove link on the interface of the Tester to R2. This is the 4. Remove link on the interface of the Tester to R2. This is the
Convergence Event Trigger. Convergence Event Trigger.
5. Measure First Route Convergence Time. 5. Measure First Route Convergence Time.
6. Measure Full Convergence Time. 6. Measure Full Convergence Time.
7. Stop Offered Load. 7. Stop Offered Load.
8. Measure Route-Specific Convergence Times and Loss-Derived 8. Measure Route-Specific Convergence Times and Loss-Derived
Convergence Time. At the same time measure number of Impaired Convergence Time. At the same time, measure number of Impaired
Packets. Packets.
9. Wait sufficient time for queues to drain. 9. Wait sufficient time for queues to drain.
10. Restart Offered Load. 10. Restart Offered Load.
11. Restore link on the interface of the Tester to R2. 11. Restore link on the interface of the Tester to R2.
12. Measure First Route Convergence Time. 12. Measure First Route Convergence Time.
13. Measure Full Convergence Time. 13. Measure Full Convergence Time.
14. Stop Offered Load. 14. Stop Offered Load.
15. Measure Route-Specific Convergence Times, Loss-Derived 15. Measure Route-Specific Convergence Times, Loss-Derived
Convergence Time, Route Loss of Connectivity Periods, and Loss- Convergence Time, Route Loss of Connectivity Periods, and Loss-
Derived Loss of Connectivity Period. At the same time measure Derived Loss of Connectivity Period. At the same time, measure
number of Impaired Packets. number of Impaired Packets.
Discussion Discussion:
In this test case there is a possibility of a packet forwarding loop In this test case, there is a possibility of a packet-forwarding
that may occur temporarily between DUT1 and DUT2 during convergence loop that may occur temporarily between DUT1 and DUT2 during
(micro-loop, see [Sh10]), which may increase the measured convergence convergence (micro-loop, see [Sh10]), which may increase the
times and loss of connectivity periods. measured convergence times and loss of connectivity periods.
8.1.5. Convergence Due to Parallel Link Interface Failure and Recovery 8.1.5. Convergence Due to Parallel Link Interface Failure and Recovery
Objective Objective:
To obtain the IGP convergence measurements for local link failure and To obtain the IGP convergence measurements for local link failure
recovery events for a member of a parallel link. The links can be and recovery events for a member of a parallel link. The links
used for data load balancing can be used for data load-balancing
Procedure Procedure:
1. Advertise an IGP topology from Tester to DUT using the test 1. Advertise an IGP topology from Tester to DUT using the test
setup shown in Figure 7. setup shown in Figure 7.
2. Send Offered Load from Tester to DUT on ingress interface. 2. Send Offered Load from Tester to DUT on Ingress Interface.
3. Verify traffic is forwarded over the parallel link member that 3. Verify traffic is forwarded over the parallel link member that
will be failed in the next step. will be failed in the next step.
4. Remove link on one of the parallel link member interfaces of the 4. Remove link on one of the parallel link member interfaces of the
DUT. This is the Convergence Event. DUT. This is the Convergence Event.
5. Measure First Route Convergence Time. 5. Measure First Route Convergence Time.
6. Measure Full Convergence Time. 6. Measure Full Convergence Time.
7. Stop Offered Load. 7. Stop Offered Load.
8. Measure Route-Specific Convergence Times and Loss-Derived 8. Measure Route-Specific Convergence Times and Loss-Derived
Convergence Time. At the same time measure number of Impaired Convergence Time. At the same time, measure number of Impaired
Packets. Packets.
9. Wait sufficient time for queues to drain. 9. Wait sufficient time for queues to drain.
10. Restart Offered Load. 10. Restart Offered Load.
11. Restore link on the Parallel Link member interface of the DUT. 11. Restore link on the Parallel Link member interface of the DUT.
12. Measure First Route Convergence Time. 12. Measure First Route Convergence Time.
13. Measure Full Convergence Time. 13. Measure Full Convergence Time.
14. Stop Offered Load. 14. Stop Offered Load.
15. Measure Route-Specific Convergence Times, Loss-Derived 15. Measure Route-Specific Convergence Times, Loss-Derived
Convergence Time, Route Loss of Connectivity Periods, and Loss- Convergence Time, Route Loss of Connectivity Periods, and Loss-
Derived Loss of Connectivity Period. At the same time measure Derived Loss of Connectivity Period. At the same time, measure
number of Impaired Packets. number of Impaired Packets.
8.2. Other Failures and Recoveries 8.2. Other Failures and Recoveries
8.2.1. Convergence Due to Layer 2 Session Loss and Recovery 8.2.1. Convergence Due to Layer 2 Session Loss and Recovery
Objective Objective:
To obtain the IGP convergence measurements for a local layer 2 loss To obtain the IGP convergence measurements for a local Layer 2
and recovery. loss and recovery.
Procedure Procedure:
1. Advertise an IGP topology from Tester to DUT using the topology 1. Advertise an IGP topology from Tester to DUT using the topology
shown in Figure 1. shown in Figure 1.
2. Send Offered Load from Tester to DUT on ingress interface. 2. Send Offered Load from Tester to DUT on Ingress Interface.
3. Verify traffic is routed over Preferred Egress Interface. 3. Verify traffic is routed over Preferred Egress Interface.
4. Remove Layer 2 session from Preferred Egress Interface of the 4. Remove Layer 2 session from Preferred Egress Interface of the
DUT. This is the Convergence Event. DUT. This is the Convergence Event.
5. Measure First Route Convergence Time. 5. Measure First Route Convergence Time.
6. Measure Full Convergence Time. 6. Measure Full Convergence Time.
7. Stop Offered Load. 7. Stop Offered Load.
8. Measure Route-Specific Convergence Times, Loss-Derived 8. Measure Route-Specific Convergence Times, Loss-Derived
Convergence Time, Route Loss of Connectivity Periods, and Loss- Convergence Time, Route Loss of Connectivity Periods, and Loss-
Derived Loss of Connectivity Period. At the same time measure Derived Loss of Connectivity Period. At the same time, measure
number of Impaired Packets. number of Impaired Packets.
9. Wait sufficient time for queues to drain. 9. Wait sufficient time for queues to drain.
10. Restart Offered Load. 10. Restart Offered Load.
11. Restore Layer 2 session on Preferred Egress Interface of the 11. Restore Layer 2 session on Preferred Egress Interface of the
DUT. DUT.
12. Measure First Route Convergence Time. 12. Measure First Route Convergence Time.
13. Measure Full Convergence Time. 13. Measure Full Convergence Time.
14. Stop Offered Load. 14. Stop Offered Load.
15. Measure Route-Specific Convergence Times, Loss-Derived 15. Measure Route-Specific Convergence Times, Loss-Derived
Convergence Time, Route Loss of Connectivity Periods, and Loss- Convergence Time, Route Loss of Connectivity Periods, and Loss-
Derived Loss of Connectivity Period. At the same time measure Derived Loss of Connectivity Period. At the same time, measure
number of Impaired Packets. number of Impaired Packets.
Discussion Discussion:
When removing the layer 2 session, the physical layer must stay up. When removing the Layer 2 session, the physical layer must stay
Configure IGP timers such that the IGP adjacency does not time out up. Configure IGP timers such that the IGP adjacency does not
before layer 2 failure is detected. time out before Layer 2 failure is detected.
To measure convergence time, traffic SHOULD start dropping on the To measure convergence time, traffic SHOULD start dropping on the
Preferred Egress Interface on the instant the layer 2 session is Preferred Egress Interface on the instant the Layer 2 session is
removed. Alternatively the Tester SHOULD record the time the instant removed. Alternatively, the Tester SHOULD record the time the
layer 2 session is removed and traffic loss SHOULD only be measured instant Layer 2 session is removed, and traffic loss SHOULD only
on the Next-Best Egress Interface. For loss-derived benchmarks the be measured on the Next-Best Egress Interface. For loss-derived
time of the Start Traffic Instant SHOULD be recorded as well. See benchmarks, the time of the Start Traffic Instant SHOULD be
Section 4.1. recorded as well. See Section 4.1.
8.2.2. Convergence Due to Loss and Recovery of IGP Adjacency 8.2.2. Convergence Due to Loss and Recovery of IGP Adjacency
Objective Objective:
To obtain the IGP convergence measurements for loss and recovery of To obtain the IGP convergence measurements for loss and recovery
an IGP Adjacency. The IGP adjacency is removed on the Tester by of an IGP Adjacency. The IGP adjacency is removed on the Tester
disabling processing of IGP routing protocol packets on the Tester. by disabling processing of IGP routing protocol packets on the
Tester.
Procedure Procedure:
1. Advertise an IGP topology from Tester to DUT using the topology 1. Advertise an IGP topology from Tester to DUT using the topology
shown in Figure 1. shown in Figure 1.
2. Send Offered Load from Tester to DUT on ingress interface. 2. Send Offered Load from Tester to DUT on Ingress Interface.
3. Verify traffic is routed over Preferred Egress Interface. 3. Verify traffic is routed over Preferred Egress Interface.
4. Remove IGP adjacency from the Preferred Egress Interface while 4. Remove IGP adjacency from the Preferred Egress Interface while
the layer 2 session MUST be maintained. This is the Convergence the Layer 2 session MUST be maintained. This is the Convergence
Event. Event.
5. Measure First Route Convergence Time. 5. Measure First Route Convergence Time.
6. Measure Full Convergence Time. 6. Measure Full Convergence Time.
7. Stop Offered Load. 7. Stop Offered Load.
8. Measure Route-Specific Convergence Times, Loss-Derived 8. Measure Route-Specific Convergence Times, Loss-Derived
Convergence Time, Route Loss of Connectivity Periods, and Loss- Convergence Time, Route Loss of Connectivity Periods, and Loss-
Derived Loss of Connectivity Period. At the same time measure Derived Loss of Connectivity Period. At the same time, measure
number of Impaired Packets. number of Impaired Packets.
9. Wait sufficient time for queues to drain. 9. Wait sufficient time for queues to drain.
10. Restart Offered Load. 10. Restart Offered Load.
11. Restore IGP session on Preferred Egress Interface of the DUT. 11. Restore IGP session on Preferred Egress Interface of the DUT.
12. Measure First Route Convergence Time. 12. Measure First Route Convergence Time.
13. Measure Full Convergence Time. 13. Measure Full Convergence Time.
14. Stop Offered Load. 14. Stop Offered Load.
15. Measure Route-Specific Convergence Times, Loss-Derived 15. Measure Route-Specific Convergence Times, Loss-Derived
Convergence Time, Route Loss of Connectivity Periods, and Loss- Convergence Time, Route Loss of Connectivity Periods, and Loss-
Derived Loss of Connectivity Period. At the same time measure Derived Loss of Connectivity Period. At the same time, measure
number of Impaired Packets. number of Impaired Packets.
Discussion Discussion:
Configure layer 2 such that layer 2 does not time out before IGP Configure Layer 2 such that Layer 2 does not time out before IGP
adjacency failure is detected. adjacency failure is detected.
To measure convergence time, traffic SHOULD start dropping on the To measure convergence time, traffic SHOULD start dropping on the
Preferred Egress Interface on the instant the IGP adjacency is Preferred Egress Interface on the instant the IGP adjacency is
removed. Alternatively the Tester SHOULD record the time the instant removed. Alternatively, the Tester SHOULD record the time the
the IGP adjacency is removed and traffic loss SHOULD only be measured instant the IGP adjacency is removed and traffic loss SHOULD only
on the Next-Best Egress Interface. For loss-derived benchmarks the be measured on the Next-Best Egress Interface. For loss-derived
time of the Start Traffic Instant SHOULD be recorded as well. See benchmarks, the time of the Start Traffic Instant SHOULD be
Section 4.1. recorded as well. See Section 4.1.
8.2.3. Convergence Due to Route Withdrawal and Re-advertisement 8.2.3. Convergence Due to Route Withdrawal and Re-Advertisement
Objective Objective:
To obtain the IGP convergence measurements for route withdrawal and To obtain the IGP convergence measurements for route withdrawal
re-advertisement. and re-advertisement.
Procedure:
Procedure
1. Advertise an IGP topology from Tester to DUT using the topology 1. Advertise an IGP topology from Tester to DUT using the topology
shown in Figure 1. The routes that will be withdrawn MUST be a shown in Figure 1. The routes that will be withdrawn MUST be a
set of leaf routes advertised by at least two nodes in the set of leaf routes advertised by at least two nodes in the
emulated topology. The topology SHOULD be such that before the emulated topology. The topology SHOULD be such that before the
withdrawal the DUT prefers the leaf routes advertised by a node withdrawal the DUT prefers the leaf routes advertised by a node
"nodeA" via the Preferred Egress Interface, and after the "nodeA" via the Preferred Egress Interface, and after the
withdrawal the DUT prefers the leaf routes advertised by a node withdrawal the DUT prefers the leaf routes advertised by a node
"nodeB" via the Next-Best Egress Interface. "nodeB" via the Next-Best Egress Interface.
2. Send Offered Load from Tester to DUT on Ingress Interface. 2. Send Offered Load from Tester to DUT on Ingress Interface.
skipping to change at page 36, line 32 skipping to change at page 36, line 35
record the time it sends the withdrawal message(s). record the time it sends the withdrawal message(s).
5. Measure First Route Convergence Time. 5. Measure First Route Convergence Time.
6. Measure Full Convergence Time. 6. Measure Full Convergence Time.
7. Stop Offered Load. 7. Stop Offered Load.
8. Measure Route-Specific Convergence Times, Loss-Derived 8. Measure Route-Specific Convergence Times, Loss-Derived
Convergence Time, Route Loss of Connectivity Periods, and Loss- Convergence Time, Route Loss of Connectivity Periods, and Loss-
Derived Loss of Connectivity Period. At the same time measure Derived Loss of Connectivity Period. At the same time, measure
number of Impaired Packets. number of Impaired Packets.
9. Wait sufficient time for queues to drain. 9. Wait sufficient time for queues to drain.
10. Restart Offered Load. 10. Restart Offered Load.
11. Re-advertise the set of withdrawn IGP leaf routes from nodeA 11. Re-advertise the set of withdrawn IGP leaf routes from nodeA
emulated by the Tester. The update message SHOULD be a single emulated by the Tester. The update message SHOULD be a single
unfragmented packet. If the routes cannot be advertised by a unfragmented packet. If the routes cannot be advertised by a
single packet, the messages SHOULD be sent using the same pacing single packet, the messages SHOULD be sent using the same pacing
skipping to change at page 37, line 7 skipping to change at page 37, line 9
sends the update message(s). sends the update message(s).
12. Measure First Route Convergence Time. 12. Measure First Route Convergence Time.
13. Measure Full Convergence Time. 13. Measure Full Convergence Time.
14. Stop Offered Load. 14. Stop Offered Load.
15. Measure Route-Specific Convergence Times, Loss-Derived 15. Measure Route-Specific Convergence Times, Loss-Derived
Convergence Time, Route Loss of Connectivity Periods, and Loss- Convergence Time, Route Loss of Connectivity Periods, and Loss-
Derived Loss of Connectivity Period. At the same time measure Derived Loss of Connectivity Period. At the same time, measure
number of Impaired Packets. number of Impaired Packets.
Discussion Discussion:
To measure convergence time, traffic SHOULD start dropping on the To measure convergence time, traffic SHOULD start dropping on the
Preferred Egress Interface on the instant the routes are withdrawn by Preferred Egress Interface on the instant the routes are withdrawn
the Tester. Alternatively the Tester SHOULD record the time the by the Tester. Alternatively, the Tester SHOULD record the time
instant the routes are withdrawn and traffic loss SHOULD only be the instant the routes are withdrawn, and traffic loss SHOULD only
measured on the Next-Best Egress Interface. For loss-derived be measured on the Next-Best Egress Interface. For loss-derived
benchmarks the time of the Start Traffic Instant SHOULD be recorded benchmarks, the time of the Start Traffic Instant SHOULD be
as well. See Section 4.1. recorded as well. See Section 4.1.
8.3. Administrative changes 8.3. Administrative Changes
8.3.1. Convergence Due to Local Interface Adminstrative Changes 8.3.1. Convergence Due to Local Interface Administrative Changes
Objective Objective:
To obtain the IGP convergence measurements for administratively To obtain the IGP convergence measurements for administratively
disabling and enabling a Local Interface. disabling and enabling a Local Interface.
Procedure Procedure:
1. Advertise an IGP topology from Tester to DUT using the topology 1. Advertise an IGP topology from Tester to DUT using the topology
shown in Figure 1. shown in Figure 1.
2. Send Offered Load from Tester to DUT on ingress interface. 2. Send Offered Load from Tester to DUT on Ingress Interface.
3. Verify traffic is routed over Preferred Egress Interface. 3. Verify traffic is routed over Preferred Egress Interface.
4. Administratively disable the Preferred Egress Interface of the 4. Administratively disable the Preferred Egress Interface of the
DUT. This is the Convergence Event. DUT. This is the Convergence Event.
5. Measure First Route Convergence Time. 5. Measure First Route Convergence Time.
6. Measure Full Convergence Time. 6. Measure Full Convergence Time.
7. Stop Offered Load. 7. Stop Offered Load.
8. Measure Route-Specific Convergence Times, Loss-Derived 8. Measure Route-Specific Convergence Times, Loss-Derived
Convergence Time, Route Loss of Connectivity Periods, and Loss- Convergence Time, Route Loss of Connectivity Periods, and Loss-
Derived Loss of Connectivity Period. At the same time measure Derived Loss of Connectivity Period. At the same time, measure
number of Impaired Packets. number of Impaired Packets.
9. Wait sufficient time for queues to drain. 9. Wait sufficient time for queues to drain.
10. Restart Offered Load. 10. Restart Offered Load.
11. Administratively enable the Preferred Egress Interface of the 11. Administratively enable the Preferred Egress Interface of the
DUT.. DUT.
12. Measure First Route Convergence Time. 12. Measure First Route Convergence Time.
13. Measure Full Convergence Time. 13. Measure Full Convergence Time.
14. Stop Offered Load. 14. Stop Offered Load.
15. Measure Route-Specific Convergence Times, Loss-Derived 15. Measure Route-Specific Convergence Times, Loss-Derived
Convergence Time, Route Loss of Connectivity Periods, and Loss- Convergence Time, Route Loss of Connectivity Periods, and Loss-
Derived Loss of Connectivity Period. At the same time measure Derived Loss of Connectivity Period. At the same time, measure
number of Impaired Packets. number of Impaired Packets.
8.3.2. Convergence Due to Cost Change 8.3.2. Convergence Due to Cost Change
Objective Objective:
To obtain the IGP convergence measurements for route cost change. To obtain the IGP convergence measurements for route cost change.
Procedure Procedure:
1. Advertise an IGP topology from Tester to DUT using the topology 1. Advertise an IGP topology from Tester to DUT using the topology
shown in Figure 1. shown in Figure 1.
2. Send Offered Load from Tester to DUT on ingress interface. 2. Send Offered Load from Tester to DUT on Ingress Interface.
3. Verify traffic is routed over Preferred Egress Interface. 3. Verify traffic is routed over Preferred Egress Interface.
4. The Tester, emulating the neighbor node, increases the cost for 4. The Tester, emulating the neighbor node, increases the cost for
all IGP routes at Preferred Egress Interface of the DUT so that all IGP routes at the Preferred Egress Interface of the DUT so
the Next-Best Egress Interface becomes preferred path. The that the Next-Best Egress Interface becomes the preferred path.
update message advertising the higher cost MUST be a single The update message advertising the higher cost MUST be a single
unfragmented packet. This is the Convergence Event. The Tester unfragmented packet. This is the Convergence Event. The Tester
MAY record the time it sends the update message advertising the MAY record the time it sends the update message advertising the
higher cost on the Preferred Egress Interface. higher cost on the Preferred Egress Interface.
5. Measure First Route Convergence Time. 5. Measure First Route Convergence Time.
6. Measure Full Convergence Time. 6. Measure Full Convergence Time.
7. Stop Offered Load. 7. Stop Offered Load.
8. Measure Route-Specific Convergence Times, Loss-Derived 8. Measure Route-Specific Convergence Times, Loss-Derived
Convergence Time, Route Loss of Connectivity Periods, and Loss- Convergence Time, Route Loss of Connectivity Periods, and Loss-
Derived Loss of Connectivity Period. At the same time measure Derived Loss of Connectivity Period. At the same time, measure
number of Impaired Packets. number of Impaired Packets.
9. Wait sufficient time for queues to drain. 9. Wait sufficient time for queues to drain.
10. Restart Offered Load. 10. Restart Offered Load.
11. The Tester, emulating the neighbor node, decreases the cost for 11. The Tester, emulating the neighbor node, decreases the cost for
all IGP routes at Preferred Egress Interface of the DUT so that all IGP routes at the Preferred Egress Interface of the DUT so
the Preferred Egress Interface becomes preferred path. The that the Preferred Egress Interface becomes the preferred path.
update message advertising the lower cost MUST be a single The update message advertising the lower cost MUST be a single
unfragmented packet. unfragmented packet.
12. Measure First Route Convergence Time. 12. Measure First Route Convergence Time.
13. Measure Full Convergence Time. 13. Measure Full Convergence Time.
14. Stop Offered Load. 14. Stop Offered Load.
15. Measure Route-Specific Convergence Times, Loss-Derived 15. Measure Route-Specific Convergence Times, Loss-Derived
Convergence Time, Route Loss of Connectivity Periods, and Loss- Convergence Time, Route Loss of Connectivity Periods, and Loss-
Derived Loss of Connectivity Period. At the same time measure Derived Loss of Connectivity Period. At the same time, measure
number of Impaired Packets. number of Impaired Packets.
Discussion Discussion:
To measure convergence time, traffic SHOULD start dropping on the To measure convergence time, traffic SHOULD start dropping on the
Preferred Egress Interface on the instant the cost is changed by the Preferred Egress Interface on the instant the cost is changed by
Tester. Alternatively the Tester SHOULD record the time the instant the Tester. Alternatively, the Tester SHOULD record the time the
the cost is changed and traffic loss SHOULD only be measured on the instant the cost is changed, and traffic loss SHOULD only be
Next-Best Egress Interface. For loss-derived benchmarks the time of measured on the Next-Best Egress Interface. For loss-derived
the Start Traffic Instant SHOULD be recorded as well. See Section benchmarks, the time of the Start Traffic Instant SHOULD be
4.1. recorded as well. See Section 4.1.
9. Security Considerations 9. Security Considerations
Benchmarking activities as described in this memo are limited to Benchmarking activities as described in this memo are limited to
technology characterization using controlled stimuli in a laboratory technology characterization using controlled stimuli in a laboratory
environment, with dedicated address space and the constraints environment, with dedicated address space and the constraints
specified in the sections above. specified in the sections above.
The benchmarking network topology will be an independent test setup The benchmarking network topology will be an independent test setup
and MUST NOT be connected to devices that may forward the test and MUST NOT be connected to devices that may forward the test
traffic into a production network, or misroute traffic to the test traffic into a production network or misroute traffic to the test
management network. management network.
Further, benchmarking is performed on a "black-box" basis, relying Further, benchmarking is performed on a "black-box" basis, relying
solely on measurements observable external to the DUT/SUT. solely on measurements observable external to the DUT/SUT.
Special capabilities SHOULD NOT exist in the DUT/SUT specifically for Special capabilities SHOULD NOT exist in the DUT/SUT specifically for
benchmarking purposes. Any implications for network security arising benchmarking purposes. Any implications for network security arising
from the DUT/SUT SHOULD be identical in the lab and in production from the DUT/SUT SHOULD be identical in the lab and in production
networks. networks.
10. IANA Considerations 10. Acknowledgements
This document requires no IANA considerations.
11. Acknowledgements
Thanks to Sue Hares, Al Morton, Kevin Dubray, Ron Bonica, David Ward, Thanks to Sue Hares, Al Morton, Kevin Dubray, Ron Bonica, David Ward,
Peter De Vriendt, Anuj Dewagan, Julien Meuric, Adrian Farrel, Stewart Peter De Vriendt, Anuj Dewagan, Julien Meuric, Adrian Farrel, Stewart
Bryant, and the Benchmarking Methodology Working Group for their Bryant, and the Benchmarking Methodology Working Group for their
contributions to this work. contributions to this work.
12. References 11. References
12.1. Normative References 11.1. Normative References
[Br91] Bradner, S., "Benchmarking terminology for network [Br91] Bradner, S., "Benchmarking terminology for network
interconnection devices", RFC 1242, July 1991. interconnection devices", RFC 1242, July 1991.
[Br97] Bradner, S., "Key words for use in RFCs to Indicate [Br97] Bradner, S., "Key words for use in RFCs to Indicate
Requirement Levels", BCP 14, RFC 2119, March 1997. Requirement Levels", BCP 14, RFC 2119, March 1997.
[Br99] Bradner, S. and J. McQuaid, "Benchmarking Methodology for [Br99] Bradner, S. and J. McQuaid, "Benchmarking Methodology for
Network Interconnect Devices", RFC 2544, March 1999. Network Interconnect Devices", RFC 2544, March 1999.
skipping to change at page 41, line 36 skipping to change at page 41, line 29
March 2007. March 2007.
[Pa05] Pan, P., Swallow, G., and A. Atlas, "Fast Reroute Extensions [Pa05] Pan, P., Swallow, G., and A. Atlas, "Fast Reroute Extensions
to RSVP-TE for LSP Tunnels", RFC 4090, May 2005. to RSVP-TE for LSP Tunnels", RFC 4090, May 2005.
[Po06] Poretsky, S., Perser, J., Erramilli, S., and S. Khurana, [Po06] Poretsky, S., Perser, J., Erramilli, S., and S. Khurana,
"Terminology for Benchmarking Network-layer Traffic Control "Terminology for Benchmarking Network-layer Traffic Control
Mechanisms", RFC 4689, October 2006. Mechanisms", RFC 4689, October 2006.
[Po11t] Poretsky, S., Imhoff, B., and K. Michielsen, "Terminology [Po11t] Poretsky, S., Imhoff, B., and K. Michielsen, "Terminology
for Benchmarking Link-State IGP Data Plane Route for Benchmarking Link-State IGP Data-Plane Route
Convergence", draft-ietf-bmwg-igp-dataplane-conv-term-23 Convergence", RFC 6412, November 2011.
(work in progress), January 2011.
[Sh10] Shand, M. and S. Bryant, "A Framework for Loop-Free [Sh10] Shand, M. and S. Bryant, "A Framework for Loop-Free
Convergence", RFC 5715, January 2010. Convergence", RFC 5715, January 2010.
[Sh10i] Shand, M. and S. Bryant, "IP Fast Reroute Framework", [Sh10i] Shand, M. and S. Bryant, "IP Fast Reroute Framework",
RFC 5714, January 2010. RFC 5714, January 2010.
[Th00] Thaler, D. and C. Hopps, "Multipath Issues in Unicast and [Th00] Thaler, D. and C. Hopps, "Multipath Issues in Unicast and
Multicast Next-Hop Selection", RFC 2991, November 2000. Multicast Next-Hop Selection", RFC 2991, November 2000.
12.2. Informative References 11.2. Informative References
[Al00] Alaettinoglu, C., Jacobson, V., and H. Yu, "Towards [Al00] Alaettinoglu, C., Jacobson, V., and H. Yu, "Towards
Millisecond IGP Convergence", NANOG 20, October 2000. Millisecond IGP Convergence", NANOG 20, October 2000.
[Al02] Alaettinoglu, C. and S. Casner, "ISIS Routing on the Qwest [Al02] Alaettinoglu, C. and S. Casner, "ISIS Routing on the Qwest
Backbone: a Recipe for Subsecond ISIS Convergence", Backbone: a Recipe for Subsecond ISIS Convergence",
NANOG 24, February 2002. NANOG 24, February 2002.
[Fi02] Filsfils, C., "Tutorial: Deploying Tight-SLA Services on an [Fi02] Filsfils, C., "Tutorial: Deploying Tight-SLA Services on an
Internet Backbone: ISIS Fast Convergence and Differentiated Internet Backbone: ISIS Fast Convergence and Differentiated
skipping to change at page 42, line 28 skipping to change at page 42, line 20
[Ka02] Katz, D., "Why are we scared of SPF? IGP Scaling and [Ka02] Katz, D., "Why are we scared of SPF? IGP Scaling and
Stability", NANOG 25, June 2002. Stability", NANOG 25, June 2002.
[Vi02] Villamizar, C., "Convergence and Restoration Techniques for [Vi02] Villamizar, C., "Convergence and Restoration Techniques for
ISP Interior Routing", NANOG 25, June 2002. ISP Interior Routing", NANOG 25, June 2002.
Authors' Addresses Authors' Addresses
Scott Poretsky Scott Poretsky
Allot Communications Allot Communications
67 South Bedford Street, Suite 400 300 TradeCenter
Burlington, MA 01803 Woburn, MA 01801
USA USA
Phone: + 1 508 309 2179 Phone: + 1 508 309 2179
Email: sporetsky@allot.com EMail: sporetsky@allot.com
Brent Imhoff Brent Imhoff
Juniper Networks Juniper Networks
1194 North Mathilda Ave 1194 North Mathilda Ave
Sunnyvale, CA 94089 Sunnyvale, CA 94089
USA USA
Phone: + 1 314 378 2571 Phone: + 1 314 378 2571
Email: bimhoff@planetspork.com EMail: bimhoff@planetspork.com
Kris Michielsen Kris Michielsen
Cisco Systems Cisco Systems
6A De Kleetlaan 6A De Kleetlaan
Diegem, BRABANT 1831 Diegem, BRABANT 1831
Belgium Belgium
Email: kmichiel@cisco.com EMail: kmichiel@cisco.com
 End of changes. 202 change blocks. 
397 lines changed or deleted 402 lines changed or added

This html diff was produced by rfcdiff 1.41. The latest version is available from http://tools.ietf.org/tools/rfcdiff/