draft-ietf-bmwg-igp-dataplane-conv-meth-20.txt   draft-ietf-bmwg-igp-dataplane-conv-meth-21.txt 
Network Working Group S. Poretsky Network Working Group S. Poretsky
Internet-Draft Allot Communications Internet-Draft Allot Communications
Intended status: Informational B. Imhoff Intended status: Informational B. Imhoff
Expires: September 9, 2010 Juniper Networks Expires: November 11, 2010 Juniper Networks
K. Michielsen K. Michielsen
Cisco Systems Cisco Systems
March 8, 2010 May 10, 2010
Benchmarking Methodology for Link-State IGP Data Plane Route Convergence Benchmarking Methodology for Link-State IGP Data Plane Route Convergence
draft-ietf-bmwg-igp-dataplane-conv-meth-20 draft-ietf-bmwg-igp-dataplane-conv-meth-21
Abstract Abstract
This document describes the methodology for benchmarking Link-State This document describes the methodology for benchmarking Link-State
Interior Gateway Protocol (IGP) Route Convergence. The methodology Interior Gateway Protocol (IGP) Route Convergence. The methodology
is to be used for benchmarking IGP convergence time through is to be used for benchmarking IGP convergence time through
externally observable (black box) data plane measurements. The externally observable (black box) data plane measurements. The
methodology can be applied to any link-state IGP, such as ISIS and methodology can be applied to any link-state IGP, such as ISIS and
OSPF. OSPF.
Status of this Memo Status of this Memo
This Internet-Draft is submitted to IETF in full conformance with the This Internet-Draft is submitted in full conformance with the
provisions of BCP 78 and BCP 79. provisions of BCP 78 and BCP 79.
Internet-Drafts are working documents of the Internet Engineering Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF), its areas, and its working groups. Note that Task Force (IETF). Note that other groups may also distribute
other groups may also distribute working documents as Internet- working documents as Internet-Drafts. The list of current Internet-
Drafts. Drafts is at http://datatracker.ietf.org/drafts/current/.
Internet-Drafts are draft documents valid for a maximum of six months Internet-Drafts are draft documents valid for a maximum of six months
and may be updated, replaced, or obsoleted by other documents at any and may be updated, replaced, or obsoleted by other documents at any
time. It is inappropriate to use Internet-Drafts as reference time. It is inappropriate to use Internet-Drafts as reference
material or to cite them other than as "work in progress." material or to cite them other than as "work in progress."
The list of current Internet-Drafts can be accessed at This Internet-Draft will expire on November 11, 2010.
http://www.ietf.org/ietf/1id-abstracts.txt.
The list of Internet-Draft Shadow Directories can be accessed at
http://www.ietf.org/shadow.html.
This Internet-Draft will expire on September 9, 2010.
Copyright Notice Copyright Notice
Copyright (c) 2010 IETF Trust and the persons identified as the Copyright (c) 2010 IETF Trust and the persons identified as the
document authors. All rights reserved. document authors. All rights reserved.
This document is subject to BCP 78 and the IETF Trust's Legal This document is subject to BCP 78 and the IETF Trust's Legal
Provisions Relating to IETF Documents Provisions Relating to IETF Documents
(http://trustee.ietf.org/license-info) in effect on the date of (http://trustee.ietf.org/license-info) in effect on the date of
publication of this document. Please review these documents publication of this document. Please review these documents
carefully, as they describe your rights and restrictions with respect carefully, as they describe your rights and restrictions with respect
to this document. Code Components extracted from this document must to this document. Code Components extracted from this document must
include Simplified BSD License text as described in Section 4.e of include Simplified BSD License text as described in Section 4.e of
the Trust Legal Provisions and are provided without warranty as the Trust Legal Provisions and are provided without warranty as
described in the BSD License. described in the Simplified BSD License.
This document may contain material from IETF Documents or IETF This document may contain material from IETF Documents or IETF
Contributions published or made publicly available before November Contributions published or made publicly available before November
10, 2008. The person(s) controlling the copyright in some of this 10, 2008. The person(s) controlling the copyright in some of this
material may not have granted the IETF Trust the right to allow material may not have granted the IETF Trust the right to allow
modifications of such material outside the IETF Standards Process. modifications of such material outside the IETF Standards Process.
Without obtaining an adequate license from the person(s) controlling Without obtaining an adequate license from the person(s) controlling
the copyright in such materials, this document may not be modified the copyright in such materials, this document may not be modified
outside the IETF Standards Process, and derivative works of it may outside the IETF Standards Process, and derivative works of it may
not be created outside the IETF Standards Process, except to format not be created outside the IETF Standards Process, except to format
skipping to change at page 3, line 40 skipping to change at page 3, line 40
6.1.1. Tester capabilities . . . . . . . . . . . . . . . . . 18 6.1.1. Tester capabilities . . . . . . . . . . . . . . . . . 18
6.1.2. Benchmark Metrics . . . . . . . . . . . . . . . . . . 19 6.1.2. Benchmark Metrics . . . . . . . . . . . . . . . . . . 19
6.1.3. Measurement Accuracy . . . . . . . . . . . . . . . . . 19 6.1.3. Measurement Accuracy . . . . . . . . . . . . . . . . . 19
6.2. Rate-Derived Method . . . . . . . . . . . . . . . . . . . 19 6.2. Rate-Derived Method . . . . . . . . . . . . . . . . . . . 19
6.2.1. Tester Capabilities . . . . . . . . . . . . . . . . . 19 6.2.1. Tester Capabilities . . . . . . . . . . . . . . . . . 19
6.2.2. Benchmark Metrics . . . . . . . . . . . . . . . . . . 20 6.2.2. Benchmark Metrics . . . . . . . . . . . . . . . . . . 20
6.2.3. Measurement Accuracy . . . . . . . . . . . . . . . . . 20 6.2.3. Measurement Accuracy . . . . . . . . . . . . . . . . . 20
6.3. Route-Specific Loss-Derived Method . . . . . . . . . . . . 21 6.3. Route-Specific Loss-Derived Method . . . . . . . . . . . . 21
6.3.1. Tester Capabilities . . . . . . . . . . . . . . . . . 21 6.3.1. Tester Capabilities . . . . . . . . . . . . . . . . . 21
6.3.2. Benchmark Metrics . . . . . . . . . . . . . . . . . . 21 6.3.2. Benchmark Metrics . . . . . . . . . . . . . . . . . . 21
6.3.3. Measurement Accuracy . . . . . . . . . . . . . . . . . 21 6.3.3. Measurement Accuracy . . . . . . . . . . . . . . . . . 22
7. Reporting Format . . . . . . . . . . . . . . . . . . . . . . . 22 7. Reporting Format . . . . . . . . . . . . . . . . . . . . . . . 22
8. Test Cases . . . . . . . . . . . . . . . . . . . . . . . . . . 23 8. Test Cases . . . . . . . . . . . . . . . . . . . . . . . . . . 23
8.1. Interface failures . . . . . . . . . . . . . . . . . . . . 24 8.1. Interface failures . . . . . . . . . . . . . . . . . . . . 24
8.1.1. Convergence Due to Local Interface Failure . . . . . . 24 8.1.1. Convergence Due to Local Interface Failure . . . . . . 24
8.1.2. Convergence Due to Remote Interface Failure . . . . . 25 8.1.2. Convergence Due to Remote Interface Failure . . . . . 25
8.1.3. Convergence Due to ECMP Member Local Interface 8.1.3. Convergence Due to ECMP Member Local Interface
Failure . . . . . . . . . . . . . . . . . . . . . . . 27 Failure . . . . . . . . . . . . . . . . . . . . . . . 27
8.1.4. Convergence Due to ECMP Member Remote Interface 8.1.4. Convergence Due to ECMP Member Remote Interface
Failure . . . . . . . . . . . . . . . . . . . . . . . 28 Failure . . . . . . . . . . . . . . . . . . . . . . . 28
8.1.5. Convergence Due to Parallel Link Interface Failure . . 29 8.1.5. Convergence Due to Parallel Link Interface Failure . . 29
skipping to change at page 5, line 10 skipping to change at page 5, line 10
10. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 38 10. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 38
11. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . 38 11. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . 38
12. Normative References . . . . . . . . . . . . . . . . . . . . . 38 12. Normative References . . . . . . . . . . . . . . . . . . . . . 38
Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 39 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 39
1. Introduction and Scope 1. Introduction and Scope
This document describes the methodology for benchmarking Link-State This document describes the methodology for benchmarking Link-State
Interior Gateway Protocol (IGP) convergence. The motivation and Interior Gateway Protocol (IGP) convergence. The motivation and
applicability for this benchmarking is described in [Po09a]. The applicability for this benchmarking is described in [Po09a]. The
terminology to be used for this benchmarking is described in [Po09t]. terminology to be used for this benchmarking is described in [Po10t].
IGP convergence time is measured on the data plane at the Tester by IGP convergence time is measured on the data plane at the Tester by
observing packet loss through the DUT. All factors contributing to observing packet loss through the DUT. All factors contributing to
convergence time are accounted for by measuring on the data plane, as convergence time are accounted for by measuring on the data plane, as
discussed in [Po09a]. The test cases in this document are black-box discussed in [Po09a]. The test cases in this document are black-box
tests that emulate the network events that cause convergence, as tests that emulate the network events that cause convergence, as
described in [Po09a]. described in [Po09a].
The methodology described in this document can be applied to IPv4 and The methodology described in this document can be applied to IPv4 and
IPv6 traffic and link-state IGPs such as ISIS [Ca90][Ho08], OSPF IPv6 traffic and link-state IGPs such as ISIS [Ca90][Ho08], OSPF
skipping to change at page 5, line 33 skipping to change at page 5, line 33
2. Existing Definitions 2. Existing Definitions
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
"SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
document are to be interpreted as described in BCP 14, RFC 2119 document are to be interpreted as described in BCP 14, RFC 2119
[Br97]. RFC 2119 defines the use of these key words to help make the [Br97]. RFC 2119 defines the use of these key words to help make the
intent of standards track documents as clear as possible. While this intent of standards track documents as clear as possible. While this
document uses these keywords, this document is not a standards track document uses these keywords, this document is not a standards track
document. document.
This document uses much of the terminology defined in [Po09t] and This document uses much of the terminology defined in [Po10t] and
uses existing terminology defined in other BMWG work. Examples uses existing terminology defined in other BMWG work. Examples
include, but are not limited to: include, but are not limited to:
Throughput [Ref.[Br91], section 3.17] Throughput [Ref.[Br91], section 3.17]
Device Under Test (DUT) [Ref.[Ma98], section 3.1.1] Device Under Test (DUT) [Ref.[Ma98], section 3.1.1]
System Under Test (SUT) [Ref.[Ma98], section 3.1.2] System Under Test (SUT) [Ref.[Ma98], section 3.1.2]
Out-of-Order Packet [Ref.[Po06], section 3.3.4] Out-of-Order Packet [Ref.[Po06], section 3.3.4]
Duplicate Packet [Ref.[Po06], section 3.3.5] Duplicate Packet [Ref.[Po06], section 3.3.5]
Stream [Ref.[Po06], section 3.3.2] Stream [Ref.[Po06], section 3.3.2]
Loss Period [Ref.[Ko02], section 4] Loss Period [Ref.[Ko02], section 4]
Forwarding Delay [Ref.[Po06], section 3.2.4] Forwarding Delay [Ref.[Po06], section 3.2.4]
Jitter [Ref.[Po06], section 3.2.5] IP Packet Delay Variation (IPDV) [Ref.[De02], section 1.2]
3. Test Topologies 3. Test Topologies
3.1. Test topology for local changes 3.1. Test topology for local changes
Figure 1 shows the test topology to measure IGP convergence time due Figure 1 shows the test topology to measure IGP convergence time due
to local Convergence Events such as Local Interface failure to local Convergence Events such as Local Interface failure
(Section 8.1.1), layer 2 session failure (Section 8.2.1), and IGP (Section 8.1.1), layer 2 session failure (Section 8.2.1), and IGP
adjacency failure (Section 8.2.2). This topology is also used to adjacency failure (Section 8.2.2). This topology is also used to
measure IGP convergence time due to the route withdrawal measure IGP convergence time due to the route withdrawal
(Section 8.2.3), and route cost change (Section 8.3.2) Convergence (Section 8.2.3), and route cost change (Section 8.3.2) Convergence
skipping to change at page 11, line 10 skipping to change at page 11, line 10
| | Parallel Link Interface N | | | | Parallel Link Interface N | |
--------- ---------- --------- ----------
Figure 7: IGP convergence test topology for Parallel Link changes Figure 7: IGP convergence test topology for Parallel Link changes
4. Convergence Time and Loss of Connectivity Period 4. Convergence Time and Loss of Connectivity Period
Two concepts will be highlighted in this section: convergence time Two concepts will be highlighted in this section: convergence time
and loss of connectivity period. and loss of connectivity period.
The Route Convergence [Po09t] time indicates the period in time The Route Convergence [Po10t] time indicates the period in time
between the Convergence Event Instant [Po09t] and the instant in time between the Convergence Event Instant [Po10t] and the instant in time
the DUT is ready to forward traffic for a specific route on its Next- the DUT is ready to forward traffic for a specific route on its Next-
Best Egress Interface and maintains this state for the duration of Best Egress Interface and maintains this state for the duration of
the Sustained Convergence Validation Time [Po09t]. To measure Route the Sustained Convergence Validation Time [Po10t]. To measure Route
Convergence time, the Convergence Event Instant and the traffic Convergence time, the Convergence Event Instant and the traffic
received from the Next-Best Egress Interface need to be observed. received from the Next-Best Egress Interface need to be observed.
The Route Loss of Connectivity Period [Po09t] indicates the time The Route Loss of Connectivity Period [Po10t] indicates the time
during which traffic to a specific route is lost following a during which traffic to a specific route is lost following a
Convergence Event until Full Convergence [Po09t] completes. This Convergence Event until Full Convergence [Po10t] completes. This
Route Loss of Connectivity Period can consist of one or more Loss Route Loss of Connectivity Period can consist of one or more Loss
Periods [Ko02]. For the testcases described in this document it is Periods [Ko02]. For the testcases described in this document it is
expected to have a single Loss Period. To measure Route Loss of expected to have a single Loss Period. To measure Route Loss of
Connectivity Period, the traffic received from the Preferred Egress Connectivity Period, the traffic received from the Preferred Egress
Interface and the traffic received from the Next-Best Egress Interface and the traffic received from the Next-Best Egress
Interface need to be observed. Interface need to be observed.
The Route Loss of Connectivity Period is most important since that The Route Loss of Connectivity Period is most important since that
has a direct impact on the network user's application performance. has a direct impact on the network user's application performance.
skipping to change at page 11, line 46 skipping to change at page 11, line 46
Best Egress Interface. In that case the Route Loss of Connectivity Best Egress Interface. In that case the Route Loss of Connectivity
Period is shorter than the Route Convergence time. Period is shorter than the Route Convergence time.
At least one condition needs to be fulfilled for Route Convergence At least one condition needs to be fulfilled for Route Convergence
time to be equal to Route Loss of Connectivity Period. The condition time to be equal to Route Loss of Connectivity Period. The condition
is that the Convergence Event causes an instantaneous traffic loss is that the Convergence Event causes an instantaneous traffic loss
for the measured route. A fiber cut on the Preferred Egress for the measured route. A fiber cut on the Preferred Egress
Interface is an example of such a Convergence Event. Interface is an example of such a Convergence Event.
A second condition applies to Route Convergence time measurements A second condition applies to Route Convergence time measurements
based on Connectivity Packet Loss [Po09t]. This second condition is based on Connectivity Packet Loss [Po10t]. This second condition is
that there is only a single Loss Period during Route Convergence. that there is only a single Loss Period during Route Convergence.
For the testcases described in this document this is expected to be For the testcases described in this document this is expected to be
the case. the case.
4.1. Convergence Events without instant traffic loss 4.1. Convergence Events without instant traffic loss
To measure convergence time benchmarks for Convergence Events caused To measure convergence time benchmarks for Convergence Events caused
by a Tester, such as an IGP cost change, the Tester MAY start to by a Tester, such as an IGP cost change, the Tester MAY start to
discard all traffic received from the Preferred Egress Interface at discard all traffic received from the Preferred Egress Interface at
the Convergence Event Instant, or MAY separately observe packets the Convergence Event Instant, or MAY separately observe packets
skipping to change at page 12, line 30 skipping to change at page 12, line 30
times for such Convergence Events, the Tester MUST collect a times for such Convergence Events, the Tester MUST collect a
timestamp at the Convergence Event Instant. If using a loss-derived timestamp at the Convergence Event Instant. If using a loss-derived
method to benchmark convergence times for such Convergence Events, method to benchmark convergence times for such Convergence Events,
the Tester MUST measure the period in time between the Start Traffic the Tester MUST measure the period in time between the Start Traffic
Instant and the Convergence Event Instant. To measure this period in Instant and the Convergence Event Instant. To measure this period in
time the Tester can collect timestamps at the Start Traffic Instant time the Tester can collect timestamps at the Start Traffic Instant
and the Convergence Event Instant. and the Convergence Event Instant.
The Convergence Event Instant together with the receive rate The Convergence Event Instant together with the receive rate
observations on the Next-Best Egress Interface allow to derive the observations on the Next-Best Egress Interface allow to derive the
convergence time benchmarks using the Rate-Derived Method [Po09t]. convergence time benchmarks using the Rate-Derived Method [Po10t].
By observing lost packets on the Next-Best Egress Interface only, the By observing lost packets on the Next-Best Egress Interface only, the
observed packet loss is the number of lost packets between Traffic observed packet loss is the number of lost packets between Traffic
Start Instant and Convergence Recovery Instant. To measure Start Instant and Convergence Recovery Instant. To measure
convergence times using a loss-derived method, packet loss between convergence times using a loss-derived method, packet loss between
the Convergence Event Instant and the Convergence Recovery Instant is the Convergence Event Instant and the Convergence Recovery Instant is
needed. The time between Traffic Start Instant and Convergence Event needed. The time between Traffic Start Instant and Convergence Event
Instant must be accounted for. An example may clarify this. Instant must be accounted for. An example may clarify this.
Figure 8 illustrates a Convergence Event without instantaneous Figure 8 illustrates a Convergence Event without instantaneous
skipping to change at page 18, line 9 skipping to change at page 18, line 9
1. Ability to establish IGP adjacencies and advertise a single IGP 1. Ability to establish IGP adjacencies and advertise a single IGP
topology to one or more peers. topology to one or more peers.
2. Ability to measure Forwarding Delay, Duplicate Packets and Out- 2. Ability to measure Forwarding Delay, Duplicate Packets and Out-
of-Order Packets. of-Order Packets.
3. An internal time clock to control timestamping, time 3. An internal time clock to control timestamping, time
measurements, and time calculations. measurements, and time calculations.
4. Ability to distinguish traffic load received on the Preferred and 4. Ability to distinguish traffic load received on the Preferred and
Next-Best Interfaces [Po09t]. Next-Best Interfaces [Po10t].
5. Ability to disable or tune specific Layer-2 and Layer-3 protocol 5. Ability to disable or tune specific Layer-2 and Layer-3 protocol
functions on any interface(s). functions on any interface(s).
The Tester MAY be capable to make non-data plane convergence The Tester MAY be capable to make non-data plane convergence
observations and use those observations for measurements. The Tester observations and use those observations for measurements. The Tester
MAY be capable to send and receive multiple traffic Streams [Po06]. MAY be capable to send and receive multiple traffic Streams [Po06].
Also see Section 6 for method-specific capabilities. Also see Section 6 for method-specific capabilities.
skipping to change at page 18, line 33 skipping to change at page 18, line 33
convergence time benchmark metrics. The Tester capabilities are convergence time benchmark metrics. The Tester capabilities are
important criteria to select a specific convergence time benchmark important criteria to select a specific convergence time benchmark
method. The criteria to select a specific benchmark method include, method. The criteria to select a specific benchmark method include,
but are not limited to: but are not limited to:
Tester capabilities: Sampling Interval, number of Tester capabilities: Sampling Interval, number of
Stream statistics to collect Stream statistics to collect
Measurement accuracy: Sampling Interval, Offered Load, Measurement accuracy: Sampling Interval, Offered Load,
number of routes number of routes
Test specification: number of routes Test specification: number of routes
DUT capabilities: Throughput, Jitter DUT capabilities: Throughput, IP Packet Delay
Variation
6.1. Loss-Derived Method 6.1. Loss-Derived Method
6.1.1. Tester capabilities 6.1.1. Tester capabilities
The Offered Load SHOULD consist of a single Stream [Po06]. If The Offered Load SHOULD consist of a single Stream [Po06]. If
sending multiple Streams, the measured packet loss statistics for all sending multiple Streams, the measured packet loss statistics for all
Streams MUST be added together. Streams MUST be added together.
In order to verify Full Convergence completion and the Sustained In order to verify Full Convergence completion and the Sustained
skipping to change at page 19, line 32 skipping to change at page 19, line 32
The Offered Load SHOULD consist of a single Stream. If sending The Offered Load SHOULD consist of a single Stream. If sending
multiple Streams, the measured traffic rate statistics for all multiple Streams, the measured traffic rate statistics for all
Streams MUST be added together. Streams MUST be added together.
The Tester measures Forwarding Rate each Sampling Interval. The The Tester measures Forwarding Rate each Sampling Interval. The
Packet Sampling Interval influences the observation of the different Packet Sampling Interval influences the observation of the different
convergence time instants. If the Packet Sampling Interval is large convergence time instants. If the Packet Sampling Interval is large
compared to the time between the convergence time instants, then the compared to the time between the convergence time instants, then the
different time instants may not be easily identifiable from the different time instants may not be easily identifiable from the
Forwarding Rate observation. The presence of Jitter [Po06] may cause Forwarding Rate observation. The presence of IPDV [De02] may cause
fluctuations of the Forwarding Rate observation and can prevent fluctuations of the Forwarding Rate observation and can prevent
correct observation of the different convergence time instants. correct observation of the different convergence time instants.
The Packet Sampling Interval MUST be larger than or equal to the time The Packet Sampling Interval MUST be larger than or equal to the time
between two consecutive packets to the same destination. For maximum between two consecutive packets to the same destination. For maximum
accuracy the value for the Packet Sampling Interval SHOULD be as accuracy the value for the Packet Sampling Interval SHOULD be as
small as possible, but the presence of Jitter may enforce using a small as possible, but the presence of IPDV may enforce using a
larger Packet Sampling Interval. The Packet Sampling Interval MUST larger Packet Sampling Interval. The Packet Sampling Interval MUST
be reported. be reported.
Jitter causes fluctuations in the number of received packets during IPDV causes fluctuations in the number of received packets during
each Packet Sampling Interval. To account for the presence of Jitter each Packet Sampling Interval. To account for the presence of IPDV
in determining if a convergence instant has been reached, Jitter in determining if a convergence instant has been reached, Forwarding
SHOULD be observed during each Packet Sampling Interval. The minimum Delay SHOULD be observed during each Packet Sampling Interval. The
and maximum number of packets expected in a Packet Sampling Interval minimum and maximum number of packets expected in a Packet Sampling
in presence of Jitter can be calculated with Equation 3. Interval in presence of IPDV can be calculated with Equation 3.
number of packets expected in a Packet Sampling Interval number of packets expected in a Packet Sampling Interval
in presence of Jitter in presence of IP Packet Delay Variation
= expected number of packets without Jitter = expected number of packets without IP Packet Delay Variation
+/-(max Jitter during Packet Sampling Interval * Offered Load) +/-( (maxDelay - minDelay) * Offered Load)
with minDelay and maxDelay the minimum resp. maximum Forwarding Delay
of packets received during the Packet Sampling Interval
Equation 3 Equation 3
To determine if a convergence instant has been reached the number of To determine if a convergence instant has been reached the number of
packets received in a Packet Sampling Interval is compared with the packets received in a Packet Sampling Interval is compared with the
range of expected number of packets calculated in Equation 3. range of expected number of packets calculated in Equation 3.
6.2.2. Benchmark Metrics 6.2.2. Benchmark Metrics
The Rate-Derived Method SHOULD be used to measure First Route The Rate-Derived Method SHOULD be used to measure First Route
Convergence Time and Full Convergence Time. It SHOULD NOT be used to Convergence Time and Full Convergence Time. It SHOULD NOT be used to
measure Loss of Connectivity Period (see Section 4). measure Loss of Connectivity Period (see Section 4).
6.2.3. Measurement Accuracy 6.2.3. Measurement Accuracy
The measurement accuracy interval of the Rate-Derived Method depends The measurement accuracy interval of the Rate-Derived Method depends
on the metric being measured or calculated and the characteristics of on the metric being measured or calculated and the characteristics of
the related transition. Jitter [Po06] adds uncertainty to the amount the related transition. IPDV [De02] adds uncertainty to the amount
of packets received in a Packet Sampling Interval and this of packets received in a Packet Sampling Interval and this
uncertainty adds to the measurement error. The effect of Jitter is uncertainty adds to the measurement error. The effect of IPDV is not
not accounted for in the calculation of the accuracy intervals below. accounted for in the calculation of the accuracy intervals below.
Jitter is of importance for the convergence instants were a variation IPDV is of importance for the convergence instants were a variation
in Forwarding Rate needs to be observed (Convergence Recovery Instant in Forwarding Rate needs to be observed (Convergence Recovery Instant
and for topologies with ECMP also Convergence Event Instant and First and for topologies with ECMP also Convergence Event Instant and First
Route Convergence Instant). Route Convergence Instant).
If the Convergence Event Instant is observed on the dataplane using If the Convergence Event Instant is observed on the dataplane using
the Rate Derived Method, it needs to be instantaneous for all routes the Rate Derived Method, it needs to be instantaneous for all routes
(see Section 4.1). The actual value of the Convergence Event Instant (see Section 4.1). The actual value of the Convergence Event Instant
falls within the accuracy interval [-(Packet Sampling Interval + falls within the accuracy interval [-(Packet Sampling Interval +
1/Offered Load), +0] around the value as measured using the Rate- 1/Offered Load), +0] around the value as measured using the Rate-
Derived Method. Derived Method.
skipping to change at page 22, line 11 skipping to change at page 22, line 16
The actual value falls within the accuracy interval [-(number of The actual value falls within the accuracy interval [-(number of
destinations/Offered Load), +(number of destinations/Offered Load)] destinations/Offered Load), +(number of destinations/Offered Load)]
around the value as measured using the Route-Specific Loss-Derived around the value as measured using the Route-Specific Loss-Derived
Method. Method.
7. Reporting Format 7. Reporting Format
For each test case, it is recommended that the reporting tables below For each test case, it is recommended that the reporting tables below
are completed and all time values SHOULD be reported with resolution are completed and all time values SHOULD be reported with resolution
as specified in [Po09t]. as specified in [Po10t].
Parameter Units Parameter Units
----------------------------------- --------------------------- ----------------------------------- ---------------------------
Test Case test case number Test Case test case number
Test Topology Test Topology Figure number Test Topology Test Topology Figure number
IGP (ISIS, OSPF, other) IGP (ISIS, OSPF, other)
Interface Type (GigE, POS, ATM, other) Interface Type (GigE, POS, ATM, other)
Packet Size offered to DUT bytes Packet Size offered to DUT bytes
Offered Load packets per second Offered Load packets per second
IGP Routes advertised to DUT number of IGP routes IGP Routes advertised to DUT number of IGP routes
skipping to change at page 23, line 45 skipping to change at page 23, line 45
Minimum Route LoC Period seconds Minimum Route LoC Period seconds
Maximum Route LoC Period seconds Maximum Route LoC Period seconds
Median Route LoC Period seconds Median Route LoC Period seconds
Average Route LoC Period seconds Average Route LoC Period seconds
8. Test Cases 8. Test Cases
It is RECOMMENDED that all applicable test cases be performed for It is RECOMMENDED that all applicable test cases be performed for
best characterization of the DUT. The test cases follow a generic best characterization of the DUT. The test cases follow a generic
procedure tailored to the specific DUT configuration and Convergence procedure tailored to the specific DUT configuration and Convergence
Event [Po09t]. This generic procedure is as follows: Event [Po10t]. This generic procedure is as follows:
1. Establish DUT and Tester configurations and advertise an IGP 1. Establish DUT and Tester configurations and advertise an IGP
topology from Tester to DUT. topology from Tester to DUT.
2. Send Offered Load from Tester to DUT on ingress interface. 2. Send Offered Load from Tester to DUT on ingress interface.
3. Verify traffic is routed correctly. Verify if traffic is 3. Verify traffic is routed correctly. Verify if traffic is
forwarded without drops, without Out-of-Order Packets, and forwarded without drops, without Out-of-Order Packets, and
without exceeding the Forwarding Delay Threshold [Po06]. without exceeding the Forwarding Delay Threshold [Po06].
4. Introduce Convergence Event [Po09t]. 4. Introduce Convergence Event [Po10t].
5. Measure First Route Convergence Time [Po09t]. 5. Measure First Route Convergence Time [Po10t].
6. Measure Full Convergence Time [Po09t]. 6. Measure Full Convergence Time [Po10t].
7. Stop Offered Load. 7. Stop Offered Load.
8. Measure Route-Specific Convergence Times, Loss-Derived 8. Measure Route-Specific Convergence Times, Loss-Derived
Convergence Time, Route LoC Periods, and Loss-Derived LoC Period Convergence Time, Route LoC Periods, and Loss-Derived LoC Period
[Po09t]. [Po10t].
9. Wait sufficient time for queues to drain. The duration of this 9. Wait sufficient time for queues to drain. The duration of this
time period is equal to the Forwarding Delay Threshold. In time period is equal to the Forwarding Delay Threshold. In
absence of a Forwarding Delay Threshold specification the absence of a Forwarding Delay Threshold specification the
duration of this time period is 2 seconds [Br99]. duration of this time period is 2 seconds [Br99].
10. Restart Offered Load. 10. Restart Offered Load.
11. Reverse Convergence Event. 11. Reverse Convergence Event.
skipping to change at page 26, line 16 skipping to change at page 26, line 16
Procedure Procedure
1. Advertise an IGP topology from Tester to SUT using the topology 1. Advertise an IGP topology from Tester to SUT using the topology
shown in Figure 3 or 4. shown in Figure 3 or 4.
2. Send Offered Load from Tester to SUT on ingress interface. 2. Send Offered Load from Tester to SUT on ingress interface.
3. Verify traffic is forwarded over Preferred Egress Interface. 3. Verify traffic is forwarded over Preferred Egress Interface.
4. Remove link on Tester's interface [Po09t] connected to SUT's 4. Remove link on Tester's interface [Po10t] connected to SUT's
Preferred Egress Interface. This is the Convergence Event. Preferred Egress Interface. This is the Convergence Event.
5. Measure First Route Convergence Time. 5. Measure First Route Convergence Time.
6. Measure Full Convergence Time. 6. Measure Full Convergence Time.
7. Stop Offered Load. 7. Stop Offered Load.
8. Measure Route-Specific Convergence Times and Loss-Derived 8. Measure Route-Specific Convergence Times and Loss-Derived
Convergence Time. Convergence Time.
skipping to change at page 26, line 51 skipping to change at page 26, line 51
15. Measure Route-Specific Convergence Times, Loss-Derived 15. Measure Route-Specific Convergence Times, Loss-Derived
Convergence Time, Route LoC Periods, and Loss-Derived LoC Convergence Time, Route LoC Periods, and Loss-Derived LoC
Period. Period.
Results Results
The measured IGP convergence time may be influenced by the link The measured IGP convergence time may be influenced by the link
failure indication time, LSA/LSP delay, LSA/LSP generation time, LSA/ failure indication time, LSA/LSP delay, LSA/LSP generation time, LSA/
LSP flood packet pacing, SPF delay, SPF execution time, and routing LSP flood packet pacing, SPF delay, SPF execution time, and routing
and forwarding tables update time. This test case may produce Stale and forwarding tables update time. This test case may produce Stale
Forwarding [Po09t] due to a transient microloop between R1 and R2 Forwarding [Po10t] due to a transient microloop between R1 and R2
during convergence, which may increase the measured convergence times during convergence, which may increase the measured convergence times
and loss of connectivity periods. and loss of connectivity periods.
8.1.3. Convergence Due to ECMP Member Local Interface Failure 8.1.3. Convergence Due to ECMP Member Local Interface Failure
Objective Objective
To obtain the IGP convergence time due to a Local Interface link To obtain the IGP convergence time due to a Local Interface link
failure event of an ECMP Member. failure event of an ECMP Member.
skipping to change at page 29, line 20 skipping to change at page 29, line 20
Convergence Time, Route LoC Periods, and Loss-Derived LoC Convergence Time, Route LoC Periods, and Loss-Derived LoC
Period. At the same time measure Out-of-Order Packets [Po06] Period. At the same time measure Out-of-Order Packets [Po06]
and Duplicate Packets [Po06]. and Duplicate Packets [Po06].
Results Results
The measured IGP convergence time may influenced by the link failure The measured IGP convergence time may influenced by the link failure
indication time, LSA/LSP delay, LSA/LSP generation time, LSA/LSP indication time, LSA/LSP delay, LSA/LSP generation time, LSA/LSP
flood packet pacing, SPF delay, SPF execution time, and routing and flood packet pacing, SPF delay, SPF execution time, and routing and
forwarding tables update time. This test case may produce Stale forwarding tables update time. This test case may produce Stale
Forwarding [Po09t] due to a transient microloop between R1 and R2 Forwarding [Po10t] due to a transient microloop between R1 and R2
during convergence, which may increase the measured convergence times during convergence, which may increase the measured convergence times
and loss of connectivity periods. and loss of connectivity periods.
8.1.5. Convergence Due to Parallel Link Interface Failure 8.1.5. Convergence Due to Parallel Link Interface Failure
Objective Objective
To obtain the IGP convergence due to a local link failure event for a To obtain the IGP convergence due to a local link failure event for a
member of a parallel link. The links can be used for data load member of a parallel link. The links can be used for data load
balancing balancing
skipping to change at page 38, line 32 skipping to change at page 38, line 32
[Br99] Bradner, S. and J. McQuaid, "Benchmarking Methodology for [Br99] Bradner, S. and J. McQuaid, "Benchmarking Methodology for
Network Interconnect Devices", RFC 2544, March 1999. Network Interconnect Devices", RFC 2544, March 1999.
[Ca90] Callon, R., "Use of OSI IS-IS for routing in TCP/IP and dual [Ca90] Callon, R., "Use of OSI IS-IS for routing in TCP/IP and dual
environments", RFC 1195, December 1990. environments", RFC 1195, December 1990.
[Co08] Coltun, R., Ferguson, D., Moy, J., and A. Lindem, "OSPF for [Co08] Coltun, R., Ferguson, D., Moy, J., and A. Lindem, "OSPF for
IPv6", RFC 5340, July 2008. IPv6", RFC 5340, July 2008.
[De02] Demichelis, C. and P. Chimento, "IP Packet Delay Variation
Metric for IP Performance Metrics (IPPM)", RFC 3393,
November 2002.
[Ho08] Hopps, C., "Routing IPv6 with IS-IS", RFC 5308, [Ho08] Hopps, C., "Routing IPv6 with IS-IS", RFC 5308,
October 2008. October 2008.
[Ko02] Koodli, R. and R. Ravikanth, "One-way Loss Pattern Sample [Ko02] Koodli, R. and R. Ravikanth, "One-way Loss Pattern Sample
Metrics", RFC 3357, August 2002. Metrics", RFC 3357, August 2002.
[Ma98] Mandeville, R., "Benchmarking Terminology for LAN Switching [Ma98] Mandeville, R., "Benchmarking Terminology for LAN Switching
Devices", RFC 2285, February 1998. Devices", RFC 2285, February 1998.
[Mo98] Moy, J., "OSPF Version 2", STD 54, RFC 2328, April 1998. [Mo98] Moy, J., "OSPF Version 2", STD 54, RFC 2328, April 1998.
[Po06] Poretsky, S., Perser, J., Erramilli, S., and S. Khurana, [Po06] Poretsky, S., Perser, J., Erramilli, S., and S. Khurana,
"Terminology for Benchmarking Network-layer Traffic Control "Terminology for Benchmarking Network-layer Traffic Control
Mechanisms", RFC 4689, October 2006. Mechanisms", RFC 4689, October 2006.
[Po09a] Poretsky, S., "Considerations for Benchmarking Link-State [Po09a] Poretsky, S., "Considerations for Benchmarking Link-State
IGP Data Plane Route Convergence", IGP Data Plane Route Convergence",
draft-ietf-bmwg-igp-dataplane-conv-app-17 (work in draft-ietf-bmwg-igp-dataplane-conv-app-17 (work in
progress), March 2009. progress), March 2009.
[Po09t] Poretsky, S. and B. Imhoff, "Terminology for Benchmarking [Po10t] Poretsky, S., Imhoff, B., and K. Michielsen, "Terminology
Link-State IGP Data Plane Route Convergence", for Benchmarking Link-State IGP Data Plane Route
draft-ietf-bmwg-igp-dataplane-conv-term-18 (work in Convergence", draft-ietf-bmwg-igp-dataplane-conv-term-20
progress), July 2009. (work in progress), March 2010.
Authors' Addresses Authors' Addresses
Scott Poretsky Scott Poretsky
Allot Communications Allot Communications
67 South Bedford Street, Suite 400 67 South Bedford Street, Suite 400
Burlington, MA 01803 Burlington, MA 01803
USA USA
Phone: + 1 508 309 2179 Phone: + 1 508 309 2179
 End of changes. 37 change blocks. 
58 lines changed or deleted 59 lines changed or added

This html diff was produced by rfcdiff 1.38. The latest version is available from http://tools.ietf.org/tools/rfcdiff/