draft-ietf-bmwg-bgp-basic-convergence-05.txt   rfc7747.txt 
Benchmarking Working Group R. Papneja Internet Engineering Task Force (IETF) R. Papneja
Internet-Draft Huawei Technologies Request for Comments: 7747 Huawei Technologies
Intended status: Informational B. Parise Category: Informational B. Parise
Expires: July 20, 2015 Cisco Systems ISSN: 2070-1721 Skyport Systems
S. Hares S. Hares
Huawei Technologies Huawei Technologies
D. Lee D. Lee
IXIA IXIA
I. Varlashkin I. Varlashkin
Google Google
January 16, 2015 April 2016
Basic BGP Convergence Benchmarking Methodology for Data Plane Basic BGP Convergence Benchmarking Methodology
Convergence for Data-Plane Convergence
draft-ietf-bmwg-bgp-basic-convergence-05.txt
Abstract Abstract
BGP is widely deployed and used by several service providers as the BGP is widely deployed and used by several service providers as the
default Inter AS routing protocol. It is of utmost importance to default inter-AS (Autonomous System) routing protocol. It is of
ensure that when a BGP peer or a downstream link of a BGP peer fails, utmost importance to ensure that when a BGP peer or a downstream link
the alternate paths are rapidly used and routes via these alternate of a BGP peer fails, the alternate paths are rapidly used and routes
paths are installed. This document provides the basic BGP via these alternate paths are installed. This document provides the
Benchmarking Methodology using existing BGP Convergence Terminology, basic BGP benchmarking methodology using existing BGP convergence
RFC 4098. terminology as defined in RFC 4098.
Status of this Memo
This Internet-Draft is submitted in full conformance with the Status of This Memo
provisions of BCP 78 and BCP 79.
Internet-Drafts are working documents of the Internet Engineering This document is not an Internet Standards Track specification; it is
Task Force (IETF). Note that other groups may also distribute published for informational purposes.
working documents as Internet-Drafts. The list of current Internet-
Drafts is at http://datatracker.ietf.org/drafts/current/.
Internet-Drafts are draft documents valid for a maximum of six months This document is a product of the Internet Engineering Task Force
and may be updated, replaced, or obsoleted by other documents at any (IETF). It represents the consensus of the IETF community. It has
time. It is inappropriate to use Internet-Drafts as reference received public review and has been approved for publication by the
material or to cite them other than as "work in progress." Internet Engineering Steering Group (IESG). Not all documents
approved by the IESG are a candidate for any level of Internet
Standard; see Section 2 of RFC 5741.
This Internet-Draft will expire on July 20, 2015. Information about the current status of this document, any errata,
and how to provide feedback on it may be obtained at
http://www.rfc-editor.org/info/rfc7747.
Copyright Notice Copyright Notice
Copyright (c) 2015 IETF Trust and the persons identified as the Copyright (c) 2016 IETF Trust and the persons identified as the
document authors. All rights reserved. document authors. All rights reserved.
This document is subject to BCP 78 and the IETF Trust's Legal This document is subject to BCP 78 and the IETF Trust's Legal
Provisions Relating to IETF Documents Provisions Relating to IETF Documents
(http://trustee.ietf.org/license-info) in effect on the date of (http://trustee.ietf.org/license-info) in effect on the date of
publication of this document. Please review these documents publication of this document. Please review these documents
carefully, as they describe your rights and restrictions with respect carefully, as they describe your rights and restrictions with respect
to this document. Code Components extracted from this document must to this document. Code Components extracted from this document must
include Simplified BSD License text as described in Section 4.e of include Simplified BSD License text as described in Section 4.e of
the Trust Legal Provisions and are provided without warranty as the Trust Legal Provisions and are provided without warranty as
skipping to change at page 3, line 7 skipping to change at page 3, line 7
modifications of such material outside the IETF Standards Process. modifications of such material outside the IETF Standards Process.
Without obtaining an adequate license from the person(s) controlling Without obtaining an adequate license from the person(s) controlling
the copyright in such materials, this document may not be modified the copyright in such materials, this document may not be modified
outside the IETF Standards Process, and derivative works of it may outside the IETF Standards Process, and derivative works of it may
not be created outside the IETF Standards Process, except to format not be created outside the IETF Standards Process, except to format
it for publication as an RFC or to translate it into languages other it for publication as an RFC or to translate it into languages other
than English. than English.
Table of Contents Table of Contents
1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 4 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 4
1.1. Benchmarking Definitions . . . . . . . . . . . . . . . . . 4 1.1. Benchmarking Definitions . . . . . . . . . . . . . . . . 4
1.2. Purpose of BGP FIB (Data Plane) Convergence . . . . . . . 4 1.2. Purpose of BGP FIB (Data-Plane) Convergence . . . . . . . 4
1.3. Control Plane Convergence . . . . . . . . . . . . . . . . 5 1.3. Control-Plane Convergence . . . . . . . . . . . . . . . . 5
1.4. Benchmarking Testing . . . . . . . . . . . . . . . . . . . 5 1.4. Benchmarking Testing . . . . . . . . . . . . . . . . . . 5
2. Existing Definitions and Requirements . . . . . . . . . . . . 5 2. Existing Definitions and Requirements . . . . . . . . . . . . 5
3. Test Topologies . . . . . . . . . . . . . . . . . . . . . . . 6 3. Test Topologies . . . . . . . . . . . . . . . . . . . . . . . 6
3.1. General Reference Topologies . . . . . . . . . . . . . . . 6 3.1. General Reference Topologies . . . . . . . . . . . . . . 7
4. Test Considerations . . . . . . . . . . . . . . . . . . . . . 8 4. Test Considerations . . . . . . . . . . . . . . . . . . . . . 8
4.1. Number of Peers . . . . . . . . . . . . . . . . . . . . . 9 4.1. Number of Peers . . . . . . . . . . . . . . . . . . . . . 9
4.2. Number of Routes per Peer . . . . . . . . . . . . . . . . 9 4.2. Number of Routes per Peer . . . . . . . . . . . . . . . . 9
4.3. Policy Processing/Reconfiguration . . . . . . . . . . . . 9 4.3. Policy Processing/Reconfiguration . . . . . . . . . . . . 9
4.4. Configured Parameters (Timers, etc..) . . . . . . . . . . 9 4.4. Configured Parameters (Timers, etc.) . . . . . . . . . . 9
4.5. Interface Types . . . . . . . . . . . . . . . . . . . . . 11 4.5. Interface Types . . . . . . . . . . . . . . . . . . . . . 11
4.6. Measurement Accuracy . . . . . . . . . . . . . . . . . . . 11 4.6. Measurement Accuracy . . . . . . . . . . . . . . . . . . 11
4.7. Measurement Statistics . . . . . . . . . . . . . . . . . . 11 4.7. Measurement Statistics . . . . . . . . . . . . . . . . . 11
4.8. Authentication . . . . . . . . . . . . . . . . . . . . . . 12 4.8. Authentication . . . . . . . . . . . . . . . . . . . . . 11
4.9. Convergence Events . . . . . . . . . . . . . . . . . . . . 12 4.9. Convergence Events . . . . . . . . . . . . . . . . . . . 12
4.10. High Availability . . . . . . . . . . . . . . . . . . . . 12 4.10. High Availability . . . . . . . . . . . . . . . . . . . . 12
5. Test Cases . . . . . . . . . . . . . . . . . . . . . . . . . . 12 5. Test Cases . . . . . . . . . . . . . . . . . . . . . . . . . 12
5.1. Basic Convergence Tests . . . . . . . . . . . . . . . . . 13 5.1. Basic Convergence Tests . . . . . . . . . . . . . . . . . 13
5.1.1. RIB-IN Convergence . . . . . . . . . . . . . . . . . . 13 5.1.1. RIB-IN Convergence . . . . . . . . . . . . . . . . . 13
5.1.2. RIB-OUT Convergence . . . . . . . . . . . . . . . . . 15 5.1.2. RIB-OUT Convergence . . . . . . . . . . . . . . . . . 15
5.1.3. eBGP Convergence . . . . . . . . . . . . . . . . . . . 16 5.1.3. eBGP Convergence . . . . . . . . . . . . . . . . . . 16
5.1.4. iBGP Convergence . . . . . . . . . . . . . . . . . . . 17 5.1.4. iBGP Convergence . . . . . . . . . . . . . . . . . . 16
5.1.5. eBGP Multihop Convergence . . . . . . . . . . . . . . 17 5.1.5. eBGP Multihop Convergence . . . . . . . . . . . . . . 17
5.2. BGP Failure/Convergence Events . . . . . . . . . . . . . . 18 5.2. BGP Failure/Convergence Events . . . . . . . . . . . . . 18
5.2.1. Physical Link Failure on DUT End . . . . . . . . . . . 18 5.2.1. Physical Link Failure on DUT End . . . . . . . . . . 18
5.2.2. Physical Link Failure on Remote/Emulator End . . . . . 20 5.2.2. Physical Link Failure on Remote/Emulator End . . . . 19
5.2.3. ECMP Link Failure on DUT End . . . . . . . . . . . . . 20 5.2.3. ECMP Link Failure on DUT End . . . . . . . . . . . . 20
5.3. BGP Adjacency Failure (Non-Physical Link Failure) on 5.3. BGP Adjacency Failure (Non-Physical Link Failure) on
Emulator . . . . . . . . . . . . . . . . . . . . . . . . . 20 Emulator . . . . . . . . . . . . . . . . . . . . . . . . 20
5.4. BGP Hard Reset Test Cases . . . . . . . . . . . . . . . . 21 5.4. BGP Hard Reset Test Cases . . . . . . . . . . . . . . . . 21
5.4.1. BGP Non-Recovering Hard Reset Event on DUT . . . . . . 21 5.4.1. BGP Non-Recovering Hard Reset Event on DUT . . . . . 21
5.5. BGP Soft Reset . . . . . . . . . . . . . . . . . . . . . . 23 5.5. BGP Soft Reset . . . . . . . . . . . . . . . . . . . . . 22
5.6. BGP Route Withdrawal Convergence Time . . . . . . . . . . 24 5.6. BGP Route Withdrawal Convergence Time . . . . . . . . . . 24
5.7. BGP Path Attribute Change Convergence Time . . . . . . . . 26 5.7. BGP Path Attribute Change Convergence Time . . . . . . . 26
5.8. BGP Graceful Restart Convergence Time . . . . . . . . . . 27 5.8. BGP Graceful Restart Convergence Time . . . . . . . . . . 27
6. Reporting Format . . . . . . . . . . . . . . . . . . . . . . . 29 6. Reporting Format . . . . . . . . . . . . . . . . . . . . . . 29
7. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 32 7. Security Considerations . . . . . . . . . . . . . . . . . . . 32
8. Security Considerations . . . . . . . . . . . . . . . . . . . 32 8. References . . . . . . . . . . . . . . . . . . . . . . . . . 32
9. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . 32 8.1. Normative References . . . . . . . . . . . . . . . . . . 32
10. References . . . . . . . . . . . . . . . . . . . . . . . . . . 33 8.2. Informative References . . . . . . . . . . . . . . . . . 33
10.1. Normative References . . . . . . . . . . . . . . . . . . . 33 Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . . 34
10.2. Informative References . . . . . . . . . . . . . . . . . . 33 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 35
Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 34
1. Introduction 1. Introduction
This document defines the methodology for benchmarking data plane FIB This document defines the methodology for benchmarking data-plane
convergence performance of BGP in routers and switches using Forwarding Information Base (FIB) convergence performance of BGP in
topologies of 3 or 4 nodes. The methodology proposed in this routers and switches using topologies of three or four nodes. The
document applies to both IPv4 and IPv6 and if a particular test is methodology proposed in this document applies to both IPv4 and IPv6,
unique to one version, it is marked accordingly. For IPv6 and if a particular test is unique to one version, it is marked
benchmarking the device under test will require the support of Multi- accordingly. For IPv6 benchmarking, the Device Under Test (DUT) will
Protocol BGP (MP-BGP) [RFC4760, RFC2545]. Similarly both iBGP & eBGP require the support of Multiprotocol BGP (MP-BGP) [RFC4760]
are covered in the tests as applicable. [RFC2545]. Similarly, both Internal BGP (iBGP) and External BGP
(eBGP) are covered in the tests as applicable.
The scope of this document is to provide methodology for BGP protocol The scope of this document is to provide methodology for BGP FIB
FIB convergence measurements with BGP functionality limited to IPv4 & convergence measurements with BGP functionality limited to IPv4 and
IPv6 as defined in RFC 4271 and Multi-Protocol BGP (MP-BGP) [RFC4760, IPv6 as defined in [RFC4271] and MP-BGP [RFC4760] [RFC2545]. Other
RFC2545]. Other BGP extensions to support layer-2, layer-3 virtual BGP extensions to support Layer 2 and Layer 3 Virtual Private
private networks (VPN) are outside the scope of this document. Networks (VPNs) are outside the scope of this document. Interaction
Interaction with IGPs (IGP interworking) is outside the scope of this with IGPs (IGP interworking) is outside the scope of this document.
document.
1.1. Benchmarking Definitions 1.1. Benchmarking Definitions
The terminology used in this document is defined in [RFC4098]. One The terminology used in this document is defined in [RFC4098]. One
additional term is defined in this draft: FIB (Data plane) BGP additional term is defined in this document as follows.
Convergence.
FIB (Data plane) convergence is defined as the completion of all FIB FIB (data-plane) convergence is defined as the completion of all FIB
changes so that all forwarded traffic now takes the new proposed changes so that all forwarded traffic then takes the newly proposed
route. RFC 4098 defines the terms BGP device, FIB and the forwarded route. RFC 4098 defines the terms 'BGP device', 'FIB', and
traffic. Data plane convergence is different than control plane 'forwarded traffic'. Data-plane convergence is different than
convergence within a node. control-plane convergence within a node.
This document defines methodology to test This document defines methodology to test
- Data plane convergence on a single BGP device that supports the BGP o data-plane convergence on a single BGP device that supports the
functionality with scope as outlined above BGP functionality with a scope as outlined above; and
- using test topology of 3 or 4 nodes which are sufficient to o using test topology of three or four nodes that are sufficient to
recreate the Convergence events used in the various tests of this recreate the convergence events used in the various tests of this
draft document.
1.2. Purpose of BGP FIB (Data Plane) Convergence 1.2. Purpose of BGP FIB (Data-Plane) Convergence
In the current Internet architecture the Inter-Autonomous System In the current Internet architecture, the inter-AS transit is
(inter-AS) transit is primarily available through BGP. To maintain primarily available through BGP. To maintain reliable connectivity
reliable connectivity within intra-domains or across inter-domains, within intra-domains or across inter-domains, fast recovery from
fast recovery from failures remains most critical. To ensure minimal failures remains most critical. To ensure minimal traffic losses,
traffic losses, many service providers are requiring BGP many service providers are requiring BGP implementations to converge
implementations to converge the entire Internet routing table within the entire Internet routing table within sub-seconds at FIB level.
sub-seconds at FIB level.
Furthermore, to compare these numbers amongst various devices, Furthermore, to compare these numbers amongst various devices,
service providers are also looking at ways to standardize the service providers are also looking at ways to standardize the
convergence measurement methods. This document offers test methods convergence measurement methods. This document offers test methods
for simple topologies. These simple tests will provide a quick high- for simple topologies. These simple tests will provide a quick high-
level check of the BGP data plane convergence across multiple level check of BGP data-plane convergence across multiple
implementations from different vendors. implementations from different vendors.
1.3. Control Plane Convergence 1.3. Control-Plane Convergence
The convergence of BGP occurs at two levels: RIB and FIB convergence. The convergence of BGP occurs at two levels: Routing Information Base
RFC 4098 defines terms for BGP control plane convergence. (RIB) and FIB convergence. RFC 4098 defines terms for BGP control-
Methodologies which test control plane convergence are out of scope plane convergence. Methodologies that test control-plane convergence
for this draft. are out of scope for this document.
1.4. Benchmarking Testing 1.4. Benchmarking Testing
In order to ensure that the results obtained in tests are repeatable, In order to ensure that the results obtained in tests are repeatable,
careful setup of initial conditions and exact steps are required. careful setup of initial conditions and exact steps are required.
This document proposes these initial conditions, test steps, and This document proposes these initial conditions, test steps, and
result checking. To ensure uniformity of the results all optional result checking. To ensure uniformity of the results, all optional
parameters SHOULD be disabled and all settings SHOULD be changed to parameters SHOULD be disabled and all settings SHOULD be changed to
default, these may include BGP timers as well. default; these may include BGP timers as well.
2. Existing Definitions and Requirements 2. Existing Definitions and Requirements
RFC 1242, "Benchmarking Terminology for Network Interconnect Devices" "Benchmarking Terminology for Network Interconnect Devices" [RFC1242]
[RFC1242] and RFC 2285, "Benchmarking Terminology for LAN Switching and "Benchmarking Terminology for LAN Switching Devices" [RFC2285]
Devices" [RFC2285] SHOULD be reviewed in conjunction with this SHOULD be reviewed in conjunction with this document. WLAN-specific
document. WLAN-specific terms and definitions are also provided in terms and definitions are also provided in Clauses 3 and 4 of the
Clauses 3 and 4 of the IEEE 802.11 standard [802.11]. Commonly used IEEE 802.11 standard [IEEE.802.11]. Commonly used terms may also be
terms may also be found in RFC 1983 [RFC1983]. found in RFC 1983 [RFC1983].
For the sake of clarity and continuity, this document adopts the For the sake of clarity and continuity, this document adopts the
general template for benchmarking terminology set out in Section 2 of general template for benchmarking terminology set out in Section 2 of
RFC 1242. Definitions are organized in alphabetical order, and [RFC1242]. Definitions are organized in alphabetical order and
grouped into sections for ease of reference. The following terms are grouped into sections for ease of reference. The following terms are
assumed to be taken as defined in RFC 1242 [RFC1242]: Throughput, assumed to be taken as defined in RFC 1242 [RFC1242]: Throughput,
Latency, Constant Load, Frame Loss Rate, and Overhead Behavior. In Latency, Constant Load, Frame Loss Rate, and Overhead Behavior. In
addition, the following terms are taken as defined in [RFC2285]: addition, the following terms are taken as defined in [RFC2285]:
Forwarding Rates, Maximum Forwarding Rate, Loads, Device Under Test Forwarding Rates, Maximum Forwarding Rate, Loads, Device Under Test
(DUT), and System Under Test (SUT). (DUT), and System Under Test (SUT).
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
"SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
document are to be interpreted as described in RFC 2119 [RFC2119]. document are to be interpreted as described in RFC 2119 [RFC2119].
3. Test Topologies 3. Test Topologies
This section describes the test setups for use in BGP benchmarking This section describes the test setups for use in BGP benchmarking
tests measuring convergence of the FIB (data plane) after the BGP tests measuring convergence of the FIB (data-plane) after BGP updates
updates has been received. have been received.
These test setups have 3 or 4 nodes with the following configuration: These test setups have three or four nodes with the following
configuration:
1. Basic Test Setup 1. Basic test setup
2. Three node setup for iBGP or eBGP convergence 2. Three-node setup for iBGP or eBGP convergence
3. Setup for eBGP multihop test scenario 3. Setup for eBGP multihop test Scenario
4. Four node setup for iBGP or eBGP convergence 4. Four-node setup for iBGP or eBGP convergence
Individual tests refer to these topologies. Individual tests refer to these topologies.
Figures 1-4 use the following conventions Figures 1 through 4 use the following conventions:
o AS-X: Autonomous System X o AS-X: Autonomous System X
o Loopback Int: Loopback interface on the BGP enabled device o Loopback Int: Loopback interface on a BGP-enabled device
o HLP,HLP1,HLP2: Helper routers running the same version of BGP as o HLP, HLP1, HLP2: Helper routers running the same version of BGP as
DUT the DUT
o Enable NTP or use any external clock source to synchronize to the o All devices MUST be synchronized using NTP or some other clock
nodes synchronization mechanism
3.1. General Reference Topologies 3.1. General Reference Topologies
Emulator acts as 1 or more BGP peers for different testcases. Emulator acts as one or more BGP peers for different test cases.
+----------+ +------------+ +----------+ +------------+
| | traffic interfaces | | | | Traffic Interfaces | |
| |-----------------------1---- | tx | | |-----------------------1---- | tx |
| |-----------------------2---- | tr1 | | |-----------------------2---- | tr1 |
| |-----------------------3-----| tr2 | | |-----------------------3-----| tr2 |
| DUT | | Emulator | | DUT | | Emulator |
| | routing interfaces | | | | Routing Interfaces | |
| Dp1 |--------------------------- |Emp1 | | Dp1 |--------------------------- |Emp1 |
| | BGP Peering | | | | BGP Peering | |
| Dp2 |---------------------------- |Emp2 | | Dp2 |---------------------------- |Emp2 |
| | BGP Peering | | | | BGP Peering | |
+----------+ +------------+ +----------+ +------------+
Figure 1 Basic Test Setup Figure 1: Basic Test Setup
+------------+ +-----------+ +-----------+ +------------+ +-----------+ +-----------+
| | | | | | | | | | | |
| | | | | | | | | | | |
| HLP | | DUT | | Emulator | | HLP | | DUT | | Emulator |
| (AS-X) |--------| (AS-Y) |-----------| (AS-Z) | | (AS-X) |--------| (AS-Y) |-----------| (AS-Z) |
| | | | | | | | | | | |
| | | | | | | | | | | |
| | | | | | | | | | | |
+------------+ +-----------+ +-----------+ +------------+ +-----------+ +-----------+
| | | |
| | | |
+--------------------------------------------+ +--------------------------------------------+
Figure 2 Three Node Setup for eBGP and iBGP Convergence Figure 2: Three-Node Setup for eBGP and iBGP Convergence
+----------------------------------------------+ +----------------------------------------------+
| | | |
| | | |
+------------+ +-----------+ +-----------+ +------------+ +-----------+ +-----------+
| | | | | | | | | | | |
| | | | | | | | | | | |
| HLP | | DUT | | Emulator | | HLP | | DUT | | Emulator |
| (AS-X) |--------| (AS-Y) |-----------| (AS-Z) | | (AS-X) |--------| (AS-Y) |-----------| (AS-Z) |
| | | | | | | | | | | |
| | | | | | | | | | | |
| | | | | | | | | | | |
+------------+ +-----------+ +-----------+ +------------+ +-----------+ +-----------+
|Loopback-Int |Loopback-Int |Loopback-Int |Loopback-Int
| | | |
+ + + +
Figure 3 BGP Convergence for eBGP Multihop Scenario Figure 3: BGP Convergence for eBGP Multihop Scenario
+---------+ +--------+ +--------+ +---------+ +---------+ +--------+ +--------+ +---------+
| | | | | | | | | | | | | | | |
| | | | | | | | | | | | | | | |
| HLP1 | | DUT | | HLP2 | |Emulator | | HLP1 | | DUT | | HLP2 | |Emulator |
| (AS-X) |-----| (AS-X) |-----| (AS-Y) |-----| (AS-Z) | | (AS-X) |-----| (AS-X) |-----| (AS-Y) |-----| (AS-Z) |
| | | | | | | | | | | | | | | |
| | | | | | | | | | | | | | | |
| | | | | | | | | | | | | | | |
+---------+ +--------+ +--------+ +---------+ +---------+ +--------+ +--------+ +---------+
| | | |
| | | |
+---------------------------------------------+ +---------------------------------------------+
Figure 4 Four Node Setup for EBGP and IBGP Convergence Figure 4: Four-Node Setup for eBGP and iBGP Convergence
4. Test Considerations 4. Test Considerations
The test cases for measuring convergence for iBGP and eBGP are The test cases for measuring convergence for iBGP and eBGP are
different. Both iBGP and eBGP use different mechanisms to advertise, different. Both iBGP and eBGP use different mechanisms to advertise,
install and learn the routes. Typically, an iBGP route on the DUT is install, and learn the routes. Typically, an iBGP route on the DUT
installed and exported when the next-hop is valid. For eBGP the is installed and exported when the next hop is valid. For eBGP, the
route is installed on the DUT with the remote interface address as route is installed on the DUT with the remote interface address as
the next-hop, with the exception of the multihop test case (as the next hop, with the exception of the multihop test case (as
specified in the test). specified in the test).
4.1. Number of Peers 4.1. Number of Peers
Number of Peers is defined as the number of BGP neighbors or sessions "Number of Peers" is defined as the number of BGP neighbors or
the DUT has at the beginning of the test. The peers are established sessions the DUT has at the beginning of the test. The peers are
before the tests begin. The relationship could be either, iBGP or established before the tests begin. The relationship could be either
eBGP peering depending upon the test case requirement. iBGP or eBGP peering depending upon the test case requirement.
The DUT establishes one or more BGP sessions with one more emulated The DUT establishes one or more BGP peer sessions with one or more
routers or helper nodes. Additional peers can be added based on the emulated routers or Helper Nodes. Additional peers can be added
testing requirements. The number of peers enabled during the testing based on the testing requirements. The number of peers enabled
should be well documented in the report matrix. during the testing should be well documented in the report matrix.
4.2. Number of Routes per Peer 4.2. Number of Routes per Peer
Number of Routes per Peer is defined as the number of routes "Number of Routes per Peer" is defined as the number of routes
advertised or learnt by the DUT per session or through a neighbor advertised or learned by the DUT per session or through a neighbor
relationship with an emulator or helper node. The tester, emulating relationship with an emulator or Helper Node. The Tester, emulating
as neighbor MUST advertise at least one route per peer. as a BGP neighbor, MUST advertise at least one route per BGP peer.
Each test run must identify the route stream in terms of route Each test run must identify the route stream in terms of route
packing, route mixture, and number of routes. This route stream must packing, route mixture, and number of routes. This route stream must
be well documented in the reporting stream. RFC 4098 defines these be well documented in the reporting stream. RFC 4098 defines these
terms. terms.
It is RECOMMENDED that the user consider advertising the entire It is RECOMMENDED that the user consider advertising the entire
current Internet routing table per peering session using an Internet current Internet routing table per peering session using an Internet
route mixture with unique or non-unique routes. If multiple peers route mixture with unique or non-unique routes. If multiple peers
are used, it is important to precisely document the timing sequence are used, it is important to precisely document the timing sequence
between the peer sending routes (as defined in RFC 4098). between the peer sending routes (as defined in RFC 4098).
4.3. Policy Processing/Reconfiguration 4.3. Policy Processing/Reconfiguration
The DUT MUST run one baseline test where policy is Minimal policy as The DUT MUST run one baseline test where policy is the Minimal policy
defined in RFC 4098. Additional runs may be done with policy set-up as defined in RFC 4098. Additional runs may be done with the policy
before the tests begin. Exact policy settings MUST be documented as that was set up before the tests began. Exact policy settings MUST
part of the test. be documented as part of the test.
4.4. Configured Parameters (Timers, etc..) 4.4. Configured Parameters (Timers, etc.)
There are configured parameters and timers that may impact the There are configured parameters and timers that may impact the
measured BGP convergence times. measured BGP convergence times.
The benchmark metrics MAY be measured at any fixed values for these The benchmark metrics MAY be measured at any fixed values for these
configured parameters. configured parameters.
It is RECOMMENDED these configure parameters have the following It is RECOMMENDED these configure parameters have the following
settings: a) default values specified by the respective RFC b) settings: a) default values specified by the respective RFC, b)
platform-specific default parameters and c) values as expected in the platform-specific default parameters, and c) values as expected in
operational network. All optional BGP settings MUST be kept the operational network. All optional BGP settings MUST be kept
consistent across iterations of any specific tests consistent across iterations of any specific tests
Examples of the configured parameters that may impact measured BGP Examples of the configured parameters that may impact measured BGP
convergence time include, but are not limited to: convergence time include, but are not limited to:
1. Interface failure detection timer 1. Interface failure detection timer
2. BGP Keepalive timer 2. BGP keepalive timer
3. BGP Holdtime 3. BGP holdtime
4. BGP update delay timer 4. BGP update delay timer
5. ConnectRetry timer 5. ConnectRetry timer
6. TCP Segment Size 6. TCP segment size
7. Minimum Route Advertisement Interval (MRAI) 7. Minimum Route Advertisement Interval (MRAI)
8. MinASOriginationInterval (MAOI) 8. MinASOriginationInterval (MAOI)
9. Route Flap Dampening parameters 9. Route flap damping parameters
10. TCP MD5 10. TCP Authentication Option (TCP AO or TCP MD5)
11. Maximum TCP Window Size 11. Maximum TCP window size
12. MTU 12. MTU
The basic-test settings for the parameters should be: The basic-test settings for the parameters should be:
1. Interface failure detection timer (0 ms) 1. Interface failure detection timer (0 ms)
2. BGP Keepalive timer (1 min) 2. BGP keepalive timer (1 min)
3. BGP Holdtime (3 min) 3. BGP holdtime (3 min)
4. BGP update delay timer (0 s) 4. BGP update delay timer (0 s)
5. ConnectRetry timer (1 s)
6. TCP Segment Size (4096) 5. ConnectRetry timer (1 s)
7. Minimum Route Advertisement Interval (MRAI) (0 s) 6. TCP segment size (4096 bytes)
8. MinASOriginationInterval (MAOI)(0 s) 7. Minimum Route Advertisement Interval (MRAI) (0 s)
8. MinASOriginationInterval (MAOI) (0 s)
9. Route Flap Dampening parameters (off) 9. Route flap damping parameters (off)
10. TCP MD5 (off) 10. TCP Authentication Option (off)
4.5. Interface Types 4.5. Interface Types
The type of media dictate which test cases may be executed, each The type of media dictates which test cases may be executed; each
interface type has unique mechanism for detecting link failures and interface type has a unique mechanism for detecting link failures,
the speed at which that mechanism operates will influence the and the speed at which that mechanism operates will influence the
measurement results. All interfaces MUST be of the same media and measurement results. All interfaces MUST be of the same media and
throughput for all iterations of each test case. throughput for all iterations of each test case.
4.6. Measurement Accuracy 4.6. Measurement Accuracy
Since observed packet loss is used to measure the route convergence Since observed packet loss is used to measure the route convergence
time, the time between two successive packets offered to each time, the time between two successive packets offered to each
individual route is the highest possible accuracy of any packet-loss individual route is the highest possible accuracy of any packet-loss-
based measurement. When packet jitter is much less than the based measurement. When packet jitter is much less than the
convergence time, it is a negligible source of error and hence it convergence time, it is a negligible source of error, and hence, it
will be treated as within tolerance. will be treated as within tolerance.
Other options to measure convergence are the Time-Based Loss Method Other options to measure convergence are the Time-Based Loss Method
(TBLM) and Timestamp Based Method(TBM)[MPLSProt]. (TBLM) and Timestamp-Based Method (TBM) [RFC6414].
An exterior measurement on the input media (such as Ethernet) is An exterior measurement on the input media (such as Ethernet) is
defined by this specification. defined by this specification.
4.7. Measurement Statistics 4.7. Measurement Statistics
The benchmark measurements may vary for each trial, due to the The benchmark measurements may vary for each trial due to the
statistical nature of timer expirations, CPU scheduling, etc. It is statistical nature of timer expirations, CPU scheduling, etc. It is
recommended to repeat the test multiple times. Evaluation of the recommended to repeat the test multiple times. Evaluation of the
test data must be done with an understanding of generally accepted test data must be done with an understanding of generally accepted
testing practices regarding repeatability, variance and statistical testing practices regarding repeatability, variance, and statistical
significance of a small number of trials. significance of a small number of trials.
For any repeated tests that are averaged to remove variance, all For any repeated tests that are averaged to remove variance, all
parameters MUST remain the same. parameters MUST remain the same.
4.8. Authentication 4.8. Authentication
Authentication in BGP is done using the TCP MD5 Signature Option Authentication in BGP is done using the TCP Authentication Option
[RFC5925]. The processing of the MD5 hash, particularly in devices [RFC5925]. (In some legacy situations, the authentication may still
with a large number of BGP peers and a large amount of update be with TCP MD5). The processing of the authentication hash,
traffic, can have an impact on the control plane of the device. If particularly in devices with a large number of BGP peers and a large
authentication is enabled, it MUST be documented correctly in the amount of update traffic, can have an impact on the control plane of
reporting format. the device. If authentication is enabled, it MUST be documented
correctly in the reporting format.
Also it is recommended that trials MUST be with the same SIDR Also, it is recommended that trials MUST be with the same Secure
features (RFC7115 & BGPSec). The best convergence tests would be Inter-Domain Routing (SIDR) features [RFC7115] [BGPsec]. The best
with No SIDR features, and then with the same SIDR features. convergence tests would be with no SIDR features and then to repeat
the convergence tests with the same SIDR features.
4.9. Convergence Events 4.9. Convergence Events
Convergence events or triggers are defined as abnormal occurrences in Convergence events or triggers are defined as abnormal occurrences in
the network, which initiate route flapping in the network, and hence the network, which initiate route flapping in the network and hence
forces the re-convergence of a steady state network. In a real forces the reconvergence of a steady state network. In a real
network, a series of convergence events may cause convergence latency network, a series of convergence events may cause convergence latency
operators desire to test. operators desire to test.
These convergence events must be defined in terms of the sequences These convergence events must be defined in terms of the sequences
defined in RFC 4098. This basic document begins all tests with a defined in RFC 4098. This basic document begins all tests with a
router initial set-up. Additional documents will define BGP data router initial setup. Additional documents will define BGP data-
plane convergence based on peer initialization. plane convergence based on peer initialization.
The convergence events may or may not be tied to the actual failure A The convergence events may or may not be tied to the actual failure.
Soft Reset (RFC 4098) does not clear the RIB or FIB tables. A Hard A soft reset [RFC4098] does not clear the RIB or FIB tables. A hard
reset clears the BGP peer sessions, the RIB tables, and FIB tables. reset clears BGP peer sessions, RIB tables, and FIB tables.
4.10. High Availability 4.10. High Availability
Due to the different Non-Stop-Routing (sometimes referred to High- Due to the different Non-Stop-Routing (sometimes referred to High-
Availability) solutions available from different vendors, it is Availability) solutions available from different vendors, it is
RECOMMENDED that any redundancy available in the routing processors RECOMMENDED that any redundancy available in the routing processors
should be disabled during the convergence measurements. For cases should be disabled during the convergence measurements. For cases
where the redundancy cannot be disabled, the results are no longer where the redundancy cannot be disabled, the results are no longer
comparable and the level of impacts on the measurements is out of comparable and the level of impact on the measurements is out of
scope of this document. scope of this document.
5. Test Cases 5. Test Cases
All tests defined under this section assume the following: All tests defined under this section assume the following:
a. BGP peers are in established state a. BGP peers are in Established state.
b. BGP state should be cleared from established state to idle prior
b. BGP state should be cleared from Established state to Idle prior
to each test. This is recommended to ensure that all tests start to each test. This is recommended to ensure that all tests start
with the BGP peers being forced back to idle state and databases with BGP peers being forced back to Idle state and databases
flushed. flushed.
c. Furthermore the traffic generation and routing should be verified c. Furthermore, the traffic generation and routing should be
in the topology to ensure there is no packet loss observed on any verified in the topology to ensure there is no packet loss
advertised routes observed on any advertised routes.
d. The arrival timestamp of advertised routes can be measured by d. The arrival timestamp of advertised routes can be measured by
installing an inline monitoring device between the emulator and installing an inline monitoring device between the emulator and
DUT, or by the span port of DUT connected with an external the DUT or by using the span port of the DUT connected with an
analyzer. The time base of such inline monitor or external external analyzer. The time base of such an inline monitor or
analyzer needs to be synchronized with the protocol and traffic external analyzer needs to be synchronized with the protocol and
emulator. Some modern emulator may have the capability to traffic emulator. Some modern emulators may have the capability
capture and timestamp every NLRI packets leaving and arriving at to capture and timestamp every NLRI packet leaving and arriving
the emulator ports. The timestamps of these NLRI packets will be at the emulator ports. The timestamps of these NLRI packets will
almost identical to the arrival time at DUT if the cable distance be almost identical to the arrival time at the DUT if the cable
between the emulator and DUT is relatively short. distance between the emulator and DUT is relatively short.
5.1. Basic Convergence Tests 5.1. Basic Convergence Tests
These test cases measure characteristics of a BGP implementation in These test cases measure characteristics of a BGP implementation in
non-failure scenarios like: non-failure scenarios like:
1. RIB-IN Convergence 1. RIB-IN Convergence
2. RIB-OUT Convergence 2. RIB-OUT Convergence
3. eBGP Convergence 3. eBGP Convergence
4. iBGP Convergence 4. iBGP Convergence
5.1.1. RIB-IN Convergence 5.1.1. RIB-IN Convergence
Objective: Objective:
This test measures the convergence time taken to receive and This test measures the convergence time taken to receive and
install a route in RIB using BGP. install a route in RIB using BGP.
Reference Test Setup: Reference Test Setup:
This test uses the setup as shown in figure 1 This test uses the setup as shown in Figure 1
Procedure: Procedure:
A. All variables affecting Convergence should be set to a basic A. All variables affecting convergence should be set to a basic test
test state (as defined in section 4-4). state (as defined in Section 4.4).
B. Establish BGP adjacency between DUT and one peer of Emulator, B. Establish BGP adjacency between the DUT and one peer of the
Emp1. emulator, Emp1.
C. To ensure adjacency establishment, wait for 3 KeepAlives from C. To ensure adjacency establishment, wait for three keepalives to
the DUT or a configurable delay before proceeding with the be received from the DUT or a configurable delay before
rest of the test. proceeding with the rest of the test.
D. Start the traffic from the Emulator tx towards the DUT D. Start the traffic from the emulator tx towards the DUT targeted
targeted at a routes specified in route mixture (ex. routeA) at a route specified in the route mixture (e.g., routeA).
Initially no traffic SHOULD be observed on the egress Initially, no traffic SHOULD be observed on the egress interface
interface as the routeA is not installed in the forwarding as routeA is not installed in the forwarding database of the DUT.
database of the DUT.
E. Advertise routeA from the peer(Emp1) to the DUT and record the E. Advertise routeA from the peer (Emp1) to the DUT and record the
time. time.
This is Tup(EMp1,Rt-A) also named 'XMT-Rt-time(Rt-A)'. This is Tup(Emp1,Rt-A), also named XMT-Rt-time(Rt-A).
F. Record the time when the routeA from Emp1 is received at the F. Record the time when routeA from Emp1 is received at the DUT.
DUT.
This Tup(DUT,Rt-A) also named 'RCV-Rt-time(Rt-A)'. This is Tup(DUT,Rt-A), also named RCV-Rt-time(Rt-A).
G. Record the time when the traffic targeted towards routeA is G. Record the time when the traffic targeted towards routeA is
received by Emulator on appropriate traffic egress interface. received by the emulator on the appropriate traffic egress
interface.
This is TR(TDr,Rt-A). This is also named DUT-XMT-Data- This is TR(TDr,Rt-A), also named DUT-XMT-Data-Time(Rt-A).
Time(Rt-A).
H. The difference between the Tup(DUT,RT-A) and traffic received H. The difference between the Tup(DUT,RT-A) and traffic received
time (TR (TDr, Rt-A) is the FIB Convergence Time for routeA in time (TR (TDr, Rt-A) is the FIB convergence time for routeA in
the route mixture. A full convergence for the route update is the route mixture. A full convergence for the route update is
the measurement between the 1st route (Rt-A) and the last the measurement between the first route (Rt-A) and the last route
route (Rt-last) (Rt-last).
Route update convergence is Route update convergence is
TR(TDr, Rt-last)- Tup(DUT, Rt-A) or TR(TDr, Rt-last)- Tup(DUT, Rt-A), or
(DUT-XMT-Data-Time - RCV-Rt-Time)(Rt-A)
(DUT-XMT-Data-Time - RCV-Rt-Time)(Rt-A).
Note: It is recommended that a single test with the same route Note: It is recommended that a single test with the same route
mixture be repeated several times. A report should provide the mixture be repeated several times. A report should provide the
Standard Deviation of all tests and the Average. standard deviation and the average of all tests.
Running tests with a varying number of routes and route mixtures is Running tests with a varying number of routes and route mixtures is
important to get a full characterization of a single peer. important to get a full characterization of a single peer.
5.1.2. RIB-OUT Convergence 5.1.2. RIB-OUT Convergence
Objective: Objective:
This test measures the convergence time taken by an implementation This test measures the convergence time taken by an implementation
to receive, install and advertise a route using BGP. to receive, install, and advertise a route using BGP.
Reference Test Setup: Reference Test Setup:
This test uses the setup as shown in figure 2. This test uses the setup as shown in Figure 2.
Procedure: Procedure:
A. The Helper node (HLP) MUST run same version of BGP as DUT. A. The Helper Node (HLP) MUST run same version of BGP as the DUT.
B. All devices MUST be synchronized using NTP or some local B. All devices MUST be synchronized using NTP or some local
reference clock. reference clock.
C. All configuration variables for HLP, DUT and Emulator SHOULD C. All configuration variables for the Helper Node, DUT, and
be set to the same values. These values MAY be basic-test or emulator SHOULD be set to the same values. These values MAY be
a unique set completely described in the test set-up. basic test or a unique set completely described in the test
setup.
D. Establish BGP adjacency between DUT and Emulator. D. Establish BGP adjacency between the DUT and the emulator.
E. Establish BGP adjacency between DUT and Helper Node. E. Establish BGP adjacency between the DUT and the Helper Node.
F. To ensure adjacency establishment, wait for 3 KeepAlives from F. To ensure adjacency establishment, wait for three keepalives to
the DUT or a configurable delay before proceeding with the be received from the DUT or a configurable delay before
rest of the test. proceeding with the rest of the test.
G. Start the traffic from the Emulator towards the Helper Node G. Start the traffic from the emulator towards the Helper Node
targeted at a specific route (e.g. routeA). Initially no targeted at a specific route (e.g., routeA). Initially, no
traffic SHOULD be observed on the egress interface as the traffic SHOULD be observed on the egress interface as routeA is
routeA is not installed in the forwarding database of the DUT. not installed in the forwarding database of the DUT.
H. Advertise routeA from the Emulator to the DUT and note the H. Advertise routeA from the emulator to the DUT and note the time.
time.
This is Tup(EMx, Rt-A), also named EM-XMT-Data-Time(Rt-A) This is Tup(EMx, Rt-A), also named EM-XMT-Data-Time(Rt-A).
I. Record when routeA is received by DUT. I. Record when routeA is received by the DUT.
This is Tup(DUTr, Rt-A), also named DUT-RCV-Rt-Time(Rt-A) This is Tup(DUTr, Rt-A), also named DUT-RCV-Rt-Time(Rt-A).
J. Record the time when the routeA is forwarded by DUT towards J. Record the time when routeA is forwarded by the DUT towards the
the Helper node. Helper Node.
This is Tup(DUTx, Rt-A), also named DUT-XMT-Rt-Time(Rt-A) This is Tup(DUTx, Rt-A), also named DUT-XMT-Rt-Time(Rt-A).
K. Record the time when the traffic targeted towards routeA is K. Record the time when the traffic targeted towards routeA is
received on the Route Egress Interface. This is TR(EMr, received on the Route Egress Interface. This is TR(EMr, Rt-A),
Rt-A), also named DUT-XMT-Data Time(Rt-A). also named DUT-XMT-Data Time(Rt-A).
FIB convergence = (DUT-XMT-Data-Time FIB convergence = (DUT-XMT-Data-Time -DUT-RCV-Rt-Time)(Rt-A)
-DUT-RCV-Rt-Time)(Rt-A)
RIB convergence = (DUT-XMT-Rt-Time - DUT-RCV-Rt-Time)(Rt-A) RIB convergence = (DUT-XMT-Rt-Time - DUT-RCV-Rt-Time)(Rt-A)
Convergence for a route stream is characterized by Convergence for a route stream is characterized by
a) Individual route convergence for FIB, RIB a) individual route convergence for FIB and RIB, and
b) All route convergence of b) all route convergence of
FIB-convergence = DUT-XMT-Data-Time(last) - DUT-RCV-Rt- FIB-convergence = DUT-XMT-Data-Time(last) - DUT-RCV-Rt-
Time(first) Time(first), and
RIB-convergence = DUT-XMT-Rt-Time(last) - DUT-RCV-Rt- RIB-convergence = DUT-XMT-Rt-Time(last) - DUT-RCV-Rt-
Time(first) Time(first).
5.1.3. eBGP Convergence 5.1.3. eBGP Convergence
Objective: Objective:
This test measures the convergence time taken by an implementation This test measures the convergence time taken by an implementation
to receive, install and advertise a route in an eBGP Scenario. to receive, install, and advertise a route in an eBGP Scenario.
Reference Test Setup: Reference Test Setup:
This test uses the setup as shown in figure 2 and the scenarios This test uses the setup as shown in Figure 2, and the scenarios
described in RIB-IN and RIB-OUT are applicable to this test case. described in RIB-IN and RIB-OUT are applicable to this test case.
5.1.4. iBGP Convergence 5.1.4. iBGP Convergence
Objective: Objective:
This test measures the convergence time taken by an implementation This test measures the convergence time taken by an implementation
to receive, install and advertise a route in an iBGP Scenario. to receive, install, and advertise a route in an iBGP Scenario.
Reference Test Setup: Reference Test Setup:
This test uses the setup as shown in figure 2 and the scenarios This test uses the setup as shown in Figure 2, and the scenarios
described in RIB-IN and RIB-OUT are applicable to this test case. described in RIB-IN and RIB-OUT are applicable to this test case.
5.1.5. eBGP Multihop Convergence 5.1.5. eBGP Multihop Convergence
Objective: Objective:
This test measures the convergence time taken by an implementation This test measures the convergence time taken by an implementation
to receive, install and advertise a route in an eBGP Multihop to receive, install, and advertise a route in an eBGP Multihop
Scenario. Scenario.
Reference Test Setup: Reference Test Setup:
This test uses the setup as shown in figure 3. DUT is used along This test uses the setup as shown in Figure 3. The DUT is used
with a helper node. along with a Helper Node.
Procedure: Procedure:
A. The Helper Node (HLP) MUST run the same version of BGP as DUT. A. The Helper Node MUST run the same version of BGP as the DUT.
B. All devices MUST be synchronized using NTP or some local B. All devices MUST be synchronized using NTP or some local
reference clock. reference clock.
C. All variables affecting Convergence like authentication, C. All variables affecting convergence, like authentication,
policies, timers SHOULD be set to basic-settings policies, and timers, SHOULD be set to basic settings.
D. All 3 devices, DUT, Emulator and Helper Node are configured D. All three devices, the DUT, emulator, and Helper Node, are
with different Autonomous Systems. configured with different ASs.
E. Loopback Interfaces are configured on DUT and Helper Node and E. Loopback interfaces are configured on the DUT and Helper Node,
connectivity is established between them using any config and connectivity is established between them using any config
options available on the DUT. options available on the DUT.
F. Establish BGP adjacency between DUT and Emulator. F. Establish BGP adjacency between the DUT and the emulator.
G. Establish BGP adjacency between DUT and Helper Node. G. Establish BGP adjacency between the DUT and the Helper Node.
H. To ensure adjacency establishment, wait for 3 KeepAlives from H. To ensure adjacency establishment, wait for three keepalives to
the DUT or a configurable delay before proceeding with the be received from the DUT or a configurable delay before
rest of the test proceeding with the rest of the test
I. Start the traffic from the Emulator towards the DUT targeted I. Start the traffic from the emulator towards the DUT targeted at a
at a specific route (e.g. routeA). specific route (e.g., routeA).
J. Initially no traffic SHOULD be observed on the egress J. Initially, no traffic SHOULD be observed on the egress interface
interface as the routeA is not installed in the forwarding as routeA is not installed in the forwarding database of the DUT.
database of the DUT.
K. Advertise routeA from the Emulator to the DUT and note the K. Advertise routeA from the emulator to the DUT and note the time
time (Tup(EMx,RouteA) also named Route-Tx-time(Rt-A). (Tup(EMx,RouteA), also named Route-Tx-time(Rt-A).
L. Record the time when the route is received by the DUT. This L. Record the time when the route is received by the DUT. This is
is Tup(EMr,DUT) named Route-Rcv-time(Rt-A). Tup(EMr,DUT), also named Route-Rcv-time(Rt-A).
M. Record the time when the traffic targeted towards routeA is M. Record the time when the traffic targeted towards routeA is
received from Egress Interface of DUT on emulator. This is received from the egress interface of the DUT on the emulator.
Tup(EMd,DUT) named Data-Rcv-time(Rt-A) This is Tup(EMd,DUT) named Data-Rcv-time(Rt-A)
N. Record the time when the routeA is forwarded by DUT towards N. Record the time when routeA is forwarded by the DUT towards the
the Helper node. This is Tup(EMf,DUT) also named Route-Fwd- Helper Node. This is Tup(EMf,DUT), also named Route-Fwd-time(Rt-
time(Rt-A) A).
FIB Convergence = (Data-Rcv-time - Route-Rcv-time)(Rt-A) FIB Convergence = (Data-Rcv-time - Route-Rcv-time)(Rt-A)
RIB Convergence = (Route-Fwd-time - Route-Rcv-time)(Rt-A) RIB Convergence = (Route-Fwd-time - Route-Rcv-time)(Rt-A)
Note: It is recommended that the test be repeated with varying number Note: It is recommended that the test be repeated with a varying
of routes and route mixtures. With each set route mixture, the test number of routes and route mixtures. With each set route mixture,
should be repeated multiple times. The results should record the test should be repeated multiple times. The results should
average, mean, Standard Deviation record the average, mean, standard deviation.
5.2. BGP Failure/Convergence Events 5.2. BGP Failure/Convergence Events
5.2.1. Physical Link Failure on DUT End 5.2.1. Physical Link Failure on DUT End
Objective: Objective:
This test measures the route convergence time due to local link This test measures the route convergence time due to a local link
failure event at DUT's Local Interface. failure event at the DUT's Local Interface.
Reference Test Setup: Reference Test Setup:
This test uses the setup as shown in figure 1. Shutdown event is This test uses the setup as shown in Figure 1. The shutdown event
defined as an administrative shutdown event on the DUT. is defined as an administrative shutdown event on the DUT.
Procedure: Procedure:
A. All variables affecting Convergence like authentication, A. All variables affecting convergence, like authentication,
policies, timers should be set to basic-test policy. policies, and timers, should be set to basic-test policy.
B. Establish 2 BGP adjacencies from DUT to Emulator, one over the B. Establish two BGP adjacencies from the DUT to the emulator, one
peer interface and the other using a second peer interface. over the peer interface and the other using a second peer
interface.
C. Advertise the same route, routeA over both the adjacencies and C. Advertise the same route, routeA, over both adjacencies with
(Emp1) Interface to be the preferred next hop. preferences so that the Best Egress Interface for the preferred
next hop is (Emp1) interface.
D. To ensure adjacency establishment, wait for 3 KeepAlives from D. To ensure adjacency establishment, wait for three keepalives to
the DUT or a configurable delay before proceeding with the be received from the DUT or a configurable delay before
rest of the test. proceeding with the rest of the test.
E. Start the traffic from the Emulator towards the DUT targeted E. Start the traffic from the emulator towards the DUT targeted at a
at a specific route (e.g. routeA). Initially traffic would be specific route (e.g., routeA). Initially, traffic would be
observed on the best egress route (Emp1) instead of Emp2. observed on the best egress route, Emp1, instead of Emp2.
F. Trigger the shutdown event of Best Egress Interface on DUT F. Trigger the shutdown event of Best Egress Interface on the DUT
(Dp1). This time is called Shutdown time (Dp1). This time is called Shutdown time.
G. Measure the Convergence Time for the event to be detected and G. Measure the convergence time for the event to be detected and
traffic to be forwarded to Next-Best Egress Interface (Dp2) traffic to be forwarded to Next-Best Egress Interface (Dp2).
Time = Data-detect(Emp2) - Shutdown time Time = Data-detect(Emp2) - Shutdown time
H. Stop the offered load and wait for the queues to drain. H. Stop the offered load and wait for the queues to drain. Restart
Restart the data flow. the data flow.
I. Bring up the link on DUT Best Egress Interface. I. Bring up the link on the DUT's Best Egress Interface.
J. Measure the convergence time taken for the traffic to be J. Measure the convergence time taken for the traffic to be rerouted
rerouted from (Dp2) to Best Interface (Dp1) from Dp2 to Best Egress Interface, Dp1.
Time = Data-detect(Emp1) - Bring Up time Time = Data-detect(Emp1) - Bring Up time
K. It is recommended that the test be repeated with varying K. It is recommended that the test be repeated with a varying number
number of routes and route mixtures or with number of routes & of routes and route mixtures or with a number of routes and route
route mixtures closer to what is deployed in operational mixtures closer to what is deployed in operational networks.
networks.
5.2.2. Physical Link Failure on Remote/Emulator End 5.2.2. Physical Link Failure on Remote/Emulator End
Objective: Objective:
This test measures the route convergence time due to local link This test measures the route convergence time due to a local link
failure event at Tester's Local Interface. failure event at the Tester's Local Interface.
Reference Test Setup: Reference Test Setup:
This test uses the setup as shown in figure 1. Shutdown event is This test uses the setup as shown in Figure 1. The shutdown event
defined as shutdown of the local interface of Tester via logical is defined as a shutdown of the local interface of the Tester via
shutdown event. The procedure used in 5.2.1 is used for the a logical shutdown event. The procedure used in Section 5.2.1 is
termination. used for the termination.
5.2.3. ECMP Link Failure on DUT End 5.2.3. ECMP Link Failure on DUT End
Objective: Objective:
This test measures the route convergence time due to local link This test measures the route convergence time due to a local link
failure event at ECMP Member. The FIB configuration and BGP is failure event at the ECMP member. The FIB configuration and BGP
set to allow two ECMP routes to be installed. However, policy are set to allow two ECMP routes to be installed. However, policy
directs the routes to be sent only over one of the paths directs the routes to be sent only over one of the paths.
Reference Test Setup: Reference Test Setup:
This test uses the setup as shown in figure 1 and the procedure This test uses the setup as shown in Figure 1, and the procedure
uses 5.2.1. used in Section 5.2.1.
5.3. BGP Adjacency Failure (Non-Physical Link Failure) on Emulator 5.3. BGP Adjacency Failure (Non-Physical Link Failure) on Emulator
Objective: Objective:
This test measures the route convergence time due to BGP Adjacency This test measures the route convergence time due to BGP Adjacency
Failure on Emulator. Failure on the emulator.
Reference Test Setup: Reference Test Setup:
This test uses the setup as shown in figure 1. This test uses the setup as shown in Figure 1.
Procedure: Procedure:
A. All variables affecting Convergence like authentication, A. All variables affecting convergence, like authentication,
policies, timers should be basic-policy set. policies, and timers, should be set to basic-policy.
B. Establish 2 BGP adjacencies from DUT to Emulator, one over the B. Establish two BGP adjacencies from the DUT to the emulator: one
Best Egress Interface and the other using the Next-Best Egress over the Best Egress Interface and the other using the Next-Best
Interface. Egress Interface.
C. Advertise the same route, routeA over both the adjacencies and C. Advertise the same route, routeA, over both adjacencies with
make Best Egress Interface to be the preferred next hop preferences so that the Best Egress Interface for the preferred
next hop is (Emp1) interface.
D. To ensure adjacency establishment, wait for 3 KeepAlives from D. To ensure adjacency establishment, wait for three keepalives to
the DUT or a configurable delay before proceeding with the be received from the DUT or a configurable delay before
rest of the test. proceeding with the rest of the test.
E. Start the traffic from the Emulator towards the DUT targeted E. Start the traffic from the emulator towards the DUT targeted at a
at a specific route (e.g. routeA). Initially traffic would be specific route (e.g., routeA). Initially, traffic would be
observed on the Best Egress interface. observed on the Best Egress Interface.
F. Remove BGP adjacency via a software adjacency down on the F. Remove BGP adjacency via a software adjacency down on the
Emulator on the Best Egress Interface. This time is called emulator on the Best Egress Interface. This time is called
BGPadj-down-time also termed BGPpeer-down BGPadj-down-time, also termed BGPpeer-down.
G. Measure the Convergence Time for the event to be detected and G. Measure the convergence time for the event to be detected and
traffic to be forwarded to Next-Best Egress Interface. This traffic to be forwarded to Next-Best Egress Interface. This time
time is Tr-rr2 also called TR2-traffic-on is Tr-rr2, also called TR2-traffic-on.
Convergence = TR2-traffic-on - BGPpeer-down Convergence = TR2-traffic-on - BGPpeer-down
H. Stop the offered load and wait for the queues to drain and H. Stop the offered load and wait for the queues to drain and
Restart the data flow. restart the data flow.
I. Bring up BGP adjacency on the Emulator over the Best Egress I. Bring up BGP adjacency on the emulator over the Best Egress
Interface. This time is BGP-adj-up also called BGPpeer-up Interface. This time is BGP-adj-up, also called BGPpeer-up.
J. Measure the convergence time taken for the traffic to be J. Measure the convergence time taken for the traffic to be rerouted
rerouted to Best Interface. This time is Tr-rr1 also called to the Best Egress Interface. This time is Tr-rr1, also called
TR1-traffic-on TR1-traffic-on.
Convergence = TR1-traffic-on - BGPpeer-up Convergence = TR1-traffic-on - BGPpeer-up
5.4. BGP Hard Reset Test Cases 5.4. BGP Hard Reset Test Cases
5.4.1. BGP Non-Recovering Hard Reset Event on DUT 5.4.1. BGP Non-Recovering Hard Reset Event on DUT
Objective: Objective:
This test measures the route convergence time due to Hard Reset on This test measures the route convergence time due to a hard reset
the DUT. on the DUT.
Reference Test Setup: Reference Test Setup:
This test uses the setup as shown in figure 1. This test uses the setup as shown in Figure 1.
Procedure: Procedure:
A. The requirement for this test case is that the Hard Reset A. The requirement for this test case is that the hard reset event
Event should be non-recovering and should affect only the should be non-recovering and should affect only the adjacency
adjacency between DUT and Emulator on the Best Egress between the DUT and the emulator on the Best Egress Interface.
Interface.
B. All variables affecting SHOULD be set to basic-test values. B. All variables affecting the test SHOULD be set to basic-test
values.
C. Establish 2 BGP adjacencies from DUT to Emulator, one over the C. Establish two BGP adjacencies from the DUT to the emulator: one
Best Egress Interface and the other using the Next-Best Egress over the Best Egress Interface and the other using the Next-Best
Interface. Egress Interface.
D. Advertise the same route, routeA over both the adjacencies and D. Advertise the same route, routeA, over both adjacencies with
make Best Egress Interface to be the preferred next hop. preferences so that the Best Egress Interface for the preferred
next hop is (Emp1) interface.
E. To ensure adjacency establishment, wait for 3 KeepAlives from E. To ensure adjacency establishment, wait for three keepalives to
the DUT or a configurable delay before proceeding with the be received from the DUT or a configurable delay before
rest of the test. proceeding with the rest of the test.
F. Start the traffic from the Emulator towards the DUT targeted F. Start the traffic from the emulator towards the DUT targeted at a
at a specific route (e.g routeA). Initially traffic would be specific route (e.g., routeA). Initially, traffic would be
observed on the Best Egress interface. observed on the Best Egress Interface.
G. Trigger the Hard Reset event of Best Egress Interface on DUT. G. Trigger the hard reset event of the Best Egress Interface on the
This time is called time-reset DUT. This time is called time reset.
H. This event is detected and traffic is forwarded to the Next- H. This event is detected and traffic is forwarded to the Next-Best
Best Egress Interface. This tim e called time-traffic flow. Egress Interface. This time is called time-traffic flow.
I. Measure the Convergence Time for the event to be detected and I. Measure the convergence time for the event to be detected and
traffic to be forwarded to Next-Best Egress Interface. traffic to be forwarded to Next-Best Egress Interface.
Time of convergence = time-traffic flow - time-reset Time of convergence = time-traffic flow - time-reset
J. Stop the offered load and wait for the queues to drain and J. Stop the offered load and wait for the queues to drain and
Restart. restart.
K. It is recommended that the test be repeated with varying K. It is recommended that the test be repeated with a varying number
number of routes and route mixtures or with number of routes & of routes and route mixtures or with a number of routes and route
route mixtures closer to what is deployed in operational mixtures closer to what is deployed in operational networks.
networks.
L. When varying number of routes are used, convergence Time is L. When varying number of routes are used, convergence time is
measured using the Loss Derived method [IGPData]. measured using the Loss-Derived method [RFC6412].
M. Convergence Time in this scenario is influenced by Failure M. Convergence time in this scenario is influenced by failure
detection time on Tester, BGP Keep Alive Time and routing, detection time on the Tester, BGP keepalive time and routing, and
forwarding table update time. forwarding table update time.
5.5. BGP Soft Reset 5.5. BGP Soft Reset
Objective: Objective:
This test measures the route convergence time taken by an This test measures the route convergence time taken by an
implementation to service a BGP Route Refresh message and implementation to service a BGP Route Refresh message and
advertise a route. advertise a route.
Reference Test Setup: Reference Test Setup:
This test uses the setup as shown in figure 2. This test uses the setup as shown in Figure 2.
Procedure: Procedure:
A. The BGP implementation on DUT & Helper Node needs to support A. The BGP implementation on the DUT and Helper Node needs to
BGP Route Refresh Capability [RFC2918]. support BGP Route Refresh Capability [RFC2918].
B. All devices MUST be synchronized using NTP or some local B. All devices MUST be synchronized using NTP or some local
reference clock. reference clock.
C. All variables affecting Convergence like authentication, C. All variables affecting convergence, like authentication,
policies, timers should be set to basic-test defaults. policies, and timers, should be set to basic-test defaults.
D. DUT and Helper Node are configured in the same Autonomous D. The DUT and the Helper Node are configured in the same AS,
System whereas Emulator is configured under a different whereas the emulator is configured under a different AS.
Autonomous System.
E. Establish BGP adjacency between DUT and Emulator. E. Establish BGP adjacency between the DUT and the emulator.
F. Establish BGP adjacency between DUT and Helper Node. F. Establish BGP adjacency between the DUT and the Helper Node.
G. To ensure adjacency establishment, wait for 3 KeepAlives from G. To ensure adjacency establishment, wait for three keepalives to
the DUT or a configurable delay before proceeding with the be received from the DUT or a configurable delay before
rest of the test. proceeding with the rest of the test.
H. Configure a policy under BGP on Helper Node to deny routes H. Configure a policy under the BGP on the Helper Node to deny
received from DUT. routes received from the DUT.
I. Advertise routeA from the Emulator to the DUT. I. Advertise routeA from the emulator to the DUT.
J. The DUT will try to advertise the route to Helper Node will be J. The DUT will try to advertise the route to the Helper Node; it
denied. will be denied.
K. Wait for 3 KeepAlives. K. Wait for three keepalives.
L. Start the traffic from the Emulator towards the Helper Node L. Start the traffic from the emulator towards the Helper Node
targeted at a specific route say routeA. Initially no traffic targeted at a specific route, say routeA. Initially, no traffic
would be observed on the Egress interface, as routeA is not would be observed on the egress interface, as routeA is not
present. present.
M. Remove the policy on Helper Node and issue a Route Refresh M. Remove the policy on the Helper Node and issue a route refresh
request towards DUT. Note the timestamp of this event. This request towards the DUT. Note the timestamp of this event. This
is the RefreshTime. is the RefreshTime.
N. Record the time when the traffic targeted towards routeA is N. Record the time when the traffic targeted towards routeA is
received on the Egress Interface. This is RecTime. received on the egress interface. This is RecTime.
O. The following equation represents the Route Refresh O. The following equation represents the Route Refresh Convergence
Convergence Time per route. Time per route.
Route Refresh Convergence Time = (RecTime - RefreshTime) Route Refresh Convergence Time = (RecTime - RefreshTime)
5.6. BGP Route Withdrawal Convergence Time 5.6. BGP Route Withdrawal Convergence Time
Objective: Objective:
This test measures the route convergence time taken by an This test measures the route convergence time taken by an
implementation to service a BGP Withdraw message and advertise the implementation to service a BGP withdraw message and advertise the
withdraw. withdraw.
Reference Test Setup: Reference Test Setup:
This test uses the setup as shown in figure 2. This test uses the setup as shown in Figure 2.
Procedure: Procedure:
A. This test consists of 2 steps to determine the Total Withdraw A. This test consists of two steps to determine the Total Withdraw
Processing Time. Processing Time.
B. Step 1: B. Step 1:
(1) All devices MUST be synchronized using NTP or some local (1) All devices MUST be synchronized using NTP or some local
reference clock. reference clock.
(2) All variables should be set to basic-test parameters. (2) All variables should be set to basic-test parameters.
(3) DUT and Helper Node are configured in the same (3) The DUT and Helper Node are configured in the same AS,
Autonomous System whereas Emulator is configured under a whereas the emulator is configured under a different AS.
different Autonomous System.
(4) Establish BGP adjacency between DUT and Emulator. (4) Establish BGP adjacency between the DUT and the emulator.
(5) To ensure adjacency establishment, wait for 3 KeepAlives (5) To ensure adjacency establishment, wait for three
from the DUT or a configurable delay before proceeding keepalives to be received from the DUT or a configurable
with the rest of the test. delay before proceeding with the rest of the test.
(6) Start the traffic from the Emulator towards the DUT (6) Start the traffic from the emulator towards the DUT
targeted at a specific route (e.g. routeA). Initially targeted at a specific route (e.g., routeA). Initially, no
no traffic would be observed on the Egress interface as traffic would be observed on the egress interface as routeA
the routeA is not present on DUT. is not present on the DUT.
(7) Advertise routeA from the Emulator to the DUT. (7) Advertise routeA from the emulator to the DUT.
(8) The traffic targeted towards routeA is received on the (8) The traffic targeted towards routeA is received on the
Egress Interface. egress interface.
(9) Now the Tester sends request to withdraw routeA to DUT, (9) Now the Tester sends a request to withdraw routeA to the
TRx(Awith) also called WdrawTime1(Rt-A). DUT. TRx(Awith) is also called WdrawTime1(Rt-A).
(10) Record the time when no traffic is observed as (10) Record the time when no traffic is observed as determined
determined by the Emulator. This is the by the emulator. This is the RouteRemoveTime1(Rt-A).
RouteRemoveTime1(Rt-A).
(11) The difference between the RouteRemoveTime1 and (11) The difference between the RouteRemoveTime1 and WdrawTime1
WdrawTime1 is the WdrawConvTime1 is the WdrawConvTime1.
WdrawConvTime1(Rt-A) = RouteRemoveTime1(Rt-A) - WdrawConvTime1(Rt-A) = RouteRemoveTime1(Rt-A) -
WdrawTime1(Rt-A) WdrawTime1(Rt-A)
C. Step 2: C. Step 2:
(1) Continuing from Step 1, re-advertise routeA back to DUT (1) Continuing from Step 1, re-advertise routeA back to the DUT
from Tester. from the Tester.
(2) The DUT will try to advertise the routeA to Helper Node (2) The DUT will try to advertise routeA to the Helper Node
(This assumes there exists a session between DUT and (this assumes there exists a session between the DUT and
helper node). Helper Node).
(3) Start the traffic from the Emulator towards the Helper (3) Start the traffic from the emulator towards the Helper Node
Node targeted at a specific route (e.g. routeA). Traffic targeted at a specific route (e.g., routeA). Traffic would
would be observed on the Egress interface after routeA is be observed on the egress interface after routeA is received
received by the Helper Node by the Helper Node.
WATime=time traffic first flows WATime=time traffic first flows
(4) Now the Tester sends a request to withdraw routeA to DUT. (4) Now the Tester sends a request to withdraw routeA to DUT.
This is the WdrawTime2(Rt-A) This is the WdrawTime2(Rt-A).
WAWtime-TRx(Rt-A) = WdrawTime2(Rt-A) WAWtime-TRx(Rt-A) = WdrawTime2(Rt-A)
(5) DUT processes the withdraw and sends it to Helper Node. (5) DUT processes the withdraw and sends it to the Helper Node.
(6) Record the time when no traffic is observed as determined (6) Record the time when no traffic is observed as determined by
by the Emulator. This is the emulator. This is:
TR-WAW(DUT,RouteA) = RouteRemoveTime2(Rt-A) TR-WAW(DUT,RouteA) = RouteRemoveTime2(Rt-A)
(7) Total withdraw processing time is (7) Total Withdraw Processing Time is:
TotalWdrawTime(Rt-A) = ((RouteRemoveTime2(Rt-A) - TotalWdrawTime(Rt-A) = ((RouteRemoveTime2(Rt-A) -
WdrawTime2(Rt-A)) - WdrawConvTime1(Rt-A)) WdrawTime2(Rt-A)) - WdrawConvTime1(Rt-A))
5.7. BGP Path Attribute Change Convergence Time 5.7. BGP Path Attribute Change Convergence Time
Objective: Objective:
This test measures the convergence time taken by an implementation This test measures the convergence time taken by an implementation
to service a BGP Path Attribute Change. to service a BGP Path Attribute Change.
Reference Test Setup: Reference Test Setup:
This test uses the setup as shown in figure 1. This test uses the setup as shown in Figure 1.
Procedure: Procedure:
A. This test only applies to Well-Known Mandatory Attributes like A. This test only applies to Well-Known Mandatory Attributes like
Origin, AS Path, Next Hop. origin, AS path, and next hop.
B. In each iteration of test only one of these mandatory B. In each iteration of the test, only one of these mandatory
attributes need to be varied whereas the others remain the attributes need to be varied whereas the others remain the same.
same.
C. All devices MUST be synchronized using NTP or some local C. All devices MUST be synchronized using NTP or some local
reference clock. reference clock.
D. All variables should be set to basic-test parameters. D. All variables should be set to basic-test parameters.
E. Advertise the route, routeA over the Best Egress Interface E. Advertise the same route, routeA, over both adjacencies with
only, making it the preferred named Tbest. preferences so that the Best Egress Interface for the preferred
next hop is (Emp1) interface.
F. To ensure adjacency establishment, wait for 3 KeepAlives from F. To ensure adjacency establishment, wait for three keepalives to
the DUT or a configurable delay before proceeding with the be received from the DUT or a configurable delay before
rest of the test. proceeding with the rest of the test.
G. Start the traffic from the Emulator towards the DUT targeted G. Start the traffic from the emulator towards the DUT targeted at
at the specific route (e.g. routeA). Initially traffic would the specific route (e.g., routeA). Initially, traffic would be
be observed on the Best Egress interface. observed on the Best Egress Interface.
H. Now advertise the same route routeA on the Next-Best Egress H. Now advertise the same route, routeA, on the Next-Best Egress
Interface but by varying one of the well-known mandatory Interface but by varying one of the well-known mandatory
attributes to have a preferred value over that interface. We attributes to have a preferred value over that interface. We
call this Tbetter. The other values need to be same as what call this Tbetter. The other values need to be the same as what
was advertised on the Best-Egress adjacency was advertised on the Best-Egress adjacency.
TRx(Path-Change(Rt-A)) = Path Change Event Time(Rt-A) TRx(Path-Change(Rt-A)) = Path Change Event Time(Rt-A)
I. Measure the Convergence Time for the event to be detected and I. Measure the convergence time for the event to be detected and
traffic to be forwarded to Next-Best Egress Interface traffic to be forwarded to Next-Best Egress Interface.
DUT(Path-Change, Rt-A) = Path-switch time(Rt-A) DUT(Path-Change, Rt-A) = Path-switch time(Rt-A)
Convergence = Path-switch time(Rt-A) - Path Change Event Convergence = Path-switch time(Rt-A) - Path Change Event
Time(Rt-A) Time(Rt-A)
J. Stop the offered load and wait for the queues to drain and J. Stop the offered load and wait for the queues to drain and
Restart. restart.
K. Repeat the test for various attributes. K. Repeat the test for various attributes.
5.8. BGP Graceful Restart Convergence Time 5.8. BGP Graceful Restart Convergence Time
Objective: Objective:
This test measures the route convergence time taken by an This test measures the route convergence time taken by an
implementation during a Graceful Restart Event as detailed in the implementation during a Graceful Restart Event as detailed in the
Terminology document [RFC4098]. terminology document [RFC4098].
Reference Test Setup: Reference Test Setup:
This test uses the setup as shown in figure 4. This test uses the setup as shown in Figure 4.
Procedure: Procedure:
A. It measures the time taken by an implementation to service a A. It measures the time taken by an implementation to service a BGP
BGP Graceful Restart Event and advertise a route. Graceful Restart Event and advertise a route.
B. The Helper Nodes are the same model as DUT and run the same B. The Helper Nodes are the same model as the DUT and run the same
BGP implementation as DUT. BGP implementation as the DUT.
C. The BGP implementation on DUT & Helper Node needs to support C. The BGP implementation on the DUT and Helper Node needs to
BGP Graceful Restart Mechanism [RFC4724]. support the BGP Graceful Restart Mechanism [RFC4724].
D. All devices MUST be synchronized using NTP or some local D. All devices MUST be synchronized using NTP or some local
reference clock. reference clock.
E. All variables are set to basic-test values. E. All variables are set to basic-test values.
F. DUT and Helper Node-1(HLP1) are configured in the same F. The DUT and Helper Node 1 (HLP1) are configured in the same AS,
Autonomous System whereas Emulator and Helper Node-2(HLP2) are whereas the emulator and Helper Node 2 (HLP2) are configured
configured under different Autonomous Systems. under different ASs.
G. Establish BGP adjacency between DUT and Helper Nodes. G. Establish BGP adjacency between the DUT and Helper Nodes.
H. Establish BGP adjacency between Helper Node-2 and Emulator. H. Establish BGP adjacency between the Helper Node 2 and the
emulator.
I. To ensure adjacency establishment, wait for 3 KeepAlives from I. To ensure adjacency establishment, wait for three keepalives to
the DUT or a configurable delay before proceeding with the be received from the DUT or a configurable delay before
rest of the test. proceeding with the rest of the test.
J. Configure a policy under BGP on Helper Node-1 to deny routes J. Configure a policy under the BGP on Helper Node 1 to deny routes
received from DUT. received from the DUT.
K. Advertise routeA from the Emulator to Helper Node-2. K. Advertise routeA from the emulator to Helper Node 2.
L. Helper Node-2 advertises the route to DUT and DUT will try to L. Helper Node 2 advertises the route to the DUT and the DUT will
advertise the route to Helper Node-1 which will be denied. try to advertise the route to Helper Node 1, which will be
denied.
M. Wait for 3 KeepAlives. M. Wait for three keepalives.
N. Start the traffic from the Emulator towards the Helper Node-1 N. Start the traffic from the emulator towards the Helper Node 1
targeted at the specific route (e.g. routeA). Initially no targeted at the specific route (e.g., routeA). Initially, no
traffic would be observed on the Egress interface as the traffic would be observed on the egress interface as routeA is
routeA is not present. not present.
O. Perform a Graceful Restart Trigger Event on DUT and note the O. Perform a Graceful Restart Trigger Event on the DUT and note the
time. This is the GREventTime. time. This is the GREventTime.
P. Remove the policy on Helper Node-1. P. Remove the policy on Helper Node 1.
Q. Record the time when the traffic targeted towards routeA is Q. Record the time when the traffic targeted towards routeA is
received on the Egress Interface received on the egress interface.
TRr(DUT, routeA). This is also called RecTime(Rt-A) This is TRr(DUT, routeA), also called RecTime(Rt-A).
R. The following equation represents the Graceful Restart R. The following equation represents the Graceful Restart
Convergence Time Convergence Time.
Graceful Restart Convergence Time(Rt-A) = ((RecTime(Rt-A) - Graceful Restart Convergence Time(Rt-A) = ((RecTime(Rt-A) -
GREventTime) - RIB-IN) GREventTime) - RIB-IN)
S. It is assumed in this test case that after a Switchover is S. It is assumed in this test case that after a switchover is
triggered on the DUT, it will not have any cycles to process triggered on the DUT, it will not have any cycles to process the
BGP Refresh messages. The reason for this assumption is that BGP Refresh messages. The reason for this assumption is that
there is a narrow window of time where after switchover when there is a narrow window of time where after switchover, when we
we remove the policy from Helper Node-1, implementations might remove the policy from Helper Node 1, implementations might
generate Route-Refresh automatically and this request might be generate Route Refresh automatically and this request might be
serviced before the DUT actually switches over and serviced before the DUT actually switches over and re-establishes
reestablishes BGP adjacencies with the peers. BGP adjacencies with the peers.
6. Reporting Format 6. Reporting Format
For each test case, it is recommended that the reporting tables below For each test case, it is recommended that the reporting tables below
are completed and all time values SHOULD be reported with resolution are completed, and all time values SHOULD be reported with resolution
as specified in [RFC4098]. as specified in [RFC4098].
Parameter Units Parameter Units or Description
=========================== ==========================
Test case Test case number
Test case Test case number Test topology 1, 2, 3, or 4
Test topology 1,2,3 or 4 Parallel links Number of parallel links
Parallel links Number of parallel links Interface type Gigabit Ethernet (GigE),
Packet over SONET (POS), ATM, other
Interface type GigE, POS, ATM, other Convergence Event Hard reset, soft reset, link
failure, or other defined
Convergence Event Hard reset, Soft reset, link eBGP sessions Number of eBGP sessions
failure, or other defined
eBGP sessions Number of eBGP sessions iBGP sessions Number of iBGP sessions
iBGP sessions Number of iBGP sessions eBGP neighbor Number of eBGP neighbors
eBGP neighbor Number of eBGP neighbors iBGP neighbor Number of iBGP neighbors
iBGP neighbor Number of iBGP neighbors
Routes per peer Number of routes Routes per peer Number of routes
Total unique routes Number of routes Total unique routes Number of routes
Total non-unique routes Number of routes Total non-unique routes Number of routes
IGP configured ISIS, OSPF, static, or other IGP configured IS-IS, OSPF, static, or other
Route Mixture Description of Route mixture Route mixture Description of route mixture
Route Packing Number of routes in an update Route packing Number of routes included in an update
Policy configured Yes, No Policy configured Yes, No
SIDR Origin Authentication Yes, No SIDR origin authentication Yes, No
[RFC7115] [RFC7115]
bgp-sec [BGPSec] Yes, No bgp-sec [BGPsec] Yes, No
Packet size offered Bytes
to the DUT
Packet size offered to the DUT Bytes Offered load Packets per second
Offered load Packets per second Packet sampling interval Seconds
on Tester
Packet sampling interval on Seconds Forwarding delay threshold Seconds
tester
Forwarding delay threshold Seconds Timer values configured on DUT
Timer Values configured on DUT Interface failure Seconds
Interface failure indication Seconds indication delay
delay Hold time Seconds
Hold time Seconds MinRouteAdvertisementInterval Seconds
MinRouteAdvertisementInterval Seconds (MRAI)
(MRAI) MinASOriginationInterval Seconds
MinASOriginationInterval Seconds (MAOI)
(MAOI) Keepalive time Seconds
Keepalive Time Seconds ConnectRetry Seconds
ConnectRetry Seconds
TCP Parameters for DUT and tester TCP parameters for DUT and Tester
MSS Bytes Maximum Segment Size (MSS) Bytes
Slow start threshold Bytes Slow start threshold Bytes
Maximum window size Bytes Maximum window size Bytes
Test Details: Test Details:
a. If the Offered Load matches a subset of routes, describe how this a. If the Offered Load matches a subset of routes, describe how this
subset is selected. subset is selected.
b. Describe how the Convergence Event is applied, does it cause b. Describe how the convergence event is applied; does it cause
instantaneous traffic loss or not. instantaneous traffic loss or not?
c. If there is any policy configured, describe the configured c. If there is any policy configured, describe the configured
policy. policy.
Complete the table below for the initial Convergence Event and the Complete the table below for the initial convergence event and the
reversion Convergence Event reversion convergence event.
Parameter Unit
Convergence Event Initial or reversion
Traffic Forwarding Metrics
Total number of packets Number of packets
offered to DUT
Total number of packets Number of packets
forwarded by DUT
Connectivity Packet Loss Number of packets
Convergence Packet Loss Number of packets
Out-of-order packets Number of packets
Duplicate packets Number of packets
Convergence Benchmarks
Rate-derived Method[RFC Parameter Unit
6412]: =========================== ==========================
First route convergence Seconds Convergence Event Initial or reversion
time
Full convergence time Seconds
Loss-derived Method [RFC Traffic Forwarding Metrics
6412]: Total number of packets Number of packets
Loss-derived convergence Seconds offered to the DUT
time Total number of packets Number of packets
forwarded by the DUT
Connectivity packet loss Number of packets
Convergence packet loss Number of packets
Out-of-order packets Number of packets
Duplicate packets Number of packets
Route-Specific Loss-Derived Convergence Benchmarks
Method:
Minimum R-S convergence Seconds
time
Maximum R-S convergence Seconds
time
Median R-S convergence Seconds
time
Average R-S convergence Seconds Rate-Derived Method [RFC6412]:
time First route convergence Seconds
time
Full convergence time Seconds
Loss of Connectivity Benchmarks Loss-Derived Method [RFC6412]:
Loss-Derived convergence Seconds
time
Loss-derived Method: Route-Specific (R-S) Loss-Derived
Loss-derived loss of Seconds Method:
connectivity period Minimum R-S convergence Seconds
time
Maximum R-S convergence Seconds
time
Median R-S convergence Seconds
time
Average R-S convergence Seconds
time
Route-Specific loss-derived Loss of Connectivity (LoC) Benchmarks
Method:
Minimum LoC period [n] Array of seconds
Minimum Route LoC period Seconds
Maximum Route LoC period Seconds
Median Route LoC period Seconds
Average Route LoC period Seconds
7. IANA Considerations Loss-Derived Method:
Loss-Derived loss of Seconds
connectivity period
This draft does not require any new allocations by IANA. Route-Specific Loss-Derived
Method:
Minimum LoC period [n] Array of seconds
Minimum Route LoC period Seconds
Maximum Route LoC period Seconds
Median Route LoC period Seconds
Average Route LoC period Seconds
8. Security Considerations 7. Security Considerations
Benchmarking activities as described in this memo are limited to Benchmarking activities as described in this memo are limited to
technology characterization using controlled stimuli in a laboratory technology characterization using controlled stimuli in a laboratory
environment, with dedicated address space and the constraints environment, with dedicated address space and the constraints
specified in the sections above. specified in the sections above.
The benchmarking network topology will be an independent test setup The benchmarking network topology is an independent test setup and
and MUST NOT be connected to devices that may forward the test MUST NOT be connected to devices that may forward the test traffic
traffic into a production network, or misroute traffic to the test into a production network or misroute traffic to the test management
management network. network.
Further, benchmarking is performed on a "black-box" basis, relying Further, benchmarking is performed on a "black-box" basis, relying
solely on measurements observable external to the DUT/SUT. solely on measurements observable and external to the DUT/SUT.
Special capabilities SHOULD NOT exist in the DUT/SUT specifically for Special capabilities SHOULD NOT exist in the DUT/SUT specifically for
benchmarking purposes. Any implications for network security arising benchmarking purposes. Any implications for network security arising
from the DUT/SUT SHOULD be identical in the lab and in production from the DUT/SUT SHOULD be identical in the lab and in production
networks. networks.
9. Acknowledgements 8. References
We would like to thank Anil Tandon, Arvind Pandey, Mohan Nanduri, Jay
Karthik, Eric Brendel for their input and discussions on various
sections in the document. We also like to acknowledge Will Liu,
Semion Lisyansky, Faisal Shah for their review and feedback to the
document.
10. References
10.1. Normative References 8.1. Normative References
[I-D.ietf-sidr-bgpsec-protocol] [IEEE.802.11]
Lepinski, M., "BGPSEC Protocol Specification", IEEE, "IEEE Standard for Information technology --
draft-ietf-sidr-bgpsec-protocol-09 (work in progress), Telecommunications and information exchange between
July 2014. systems Local and metropolitan area networks -- Specific
requirements Part 11: Wireless LAN Medium Access Control
(MAC) and Physical Layer (PHY) Specifications",
IEEE 802.11-2012, DOI 10.1109/ieeestd.2012.6178212, April
2012, <http://ieeexplore.ieee.org/servlet/
opac?punumber=6178209>.
[RFC2119] Bradner, S., "Key words for use in RFCs to Indicate [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate
Requirement Levels", BCP 14, RFC 2119, March 1997. Requirement Levels", BCP 14, RFC 2119,
DOI 10.17487/RFC2119, March 1997,
<http://www.rfc-editor.org/info/rfc2119>.
[RFC2918] Chen, E., "Route Refresh Capability for BGP-4", RFC 2918, [RFC2918] Chen, E., "Route Refresh Capability for BGP-4", RFC 2918,
September 2000. DOI 10.17487/RFC2918, September 2000,
<http://www.rfc-editor.org/info/rfc2918>.
[RFC4098] Berkowitz, H., Davies, E., Hares, S., Krishnaswamy, P., [RFC4098] Berkowitz, H., Davies, E., Ed., Hares, S., Krishnaswamy,
and M. Lepp, "Terminology for Benchmarking BGP Device P., and M. Lepp, "Terminology for Benchmarking BGP Device
Convergence in the Control Plane", RFC 4098, June 2005. Convergence in the Control Plane", RFC 4098,
DOI 10.17487/RFC4098, June 2005,
<http://www.rfc-editor.org/info/rfc4098>.
[RFC4271] Rekhter, Y., Li, T., and S. Hares, "A Border Gateway [RFC4271] Rekhter, Y., Ed., Li, T., Ed., and S. Hares, Ed., "A
Protocol 4 (BGP-4)", RFC 4271, January 2006. Border Gateway Protocol 4 (BGP-4)", RFC 4271,
DOI 10.17487/RFC4271, January 2006,
<http://www.rfc-editor.org/info/rfc4271>.
[RFC6412] Poretsky, S., Imhoff, B., and K. Michielsen, "Terminology [RFC6412] Poretsky, S., Imhoff, B., and K. Michielsen, "Terminology
for Benchmarking Link-State IGP Data-Plane Route for Benchmarking Link-State IGP Data-Plane Route
Convergence", RFC 6412, November 2011. Convergence", RFC 6412, DOI 10.17487/RFC6412, November
2011, <http://www.rfc-editor.org/info/rfc6412>.
[RFC7115] Bush, R., "Origin Validation Operation Based on the 8.2. Informative References
Resource Public Key Infrastructure (RPKI)", BCP 185,
RFC 7115, January 2014.
10.2. Informative References [BGPsec] Lepinski, M. and K. Sriram, "BGPsec Protocol
Specification", Work in Progress, draft-ietf-sidr-bgpsec-
protocol-15, March 2016.
[RFC1242] Bradner, S., "Benchmarking terminology for network [RFC1242] Bradner, S., "Benchmarking Terminology for Network
interconnection devices", RFC 1242, July 1991. Interconnection Devices", RFC 1242, DOI 10.17487/RFC1242,
July 1991, <http://www.rfc-editor.org/info/rfc1242>.
[RFC1983] Malkin, G., "Internet Users' Glossary", RFC 1983, [RFC1983] Malkin, G., Ed., "Internet Users' Glossary", FYI 18,
August 1996. RFC 1983, DOI 10.17487/RFC1983, August 1996,
<http://www.rfc-editor.org/info/rfc1983>.
[RFC2285] Mandeville, R., "Benchmarking Terminology for LAN [RFC2285] Mandeville, R., "Benchmarking Terminology for LAN
Switching Devices", RFC 2285, February 1998. Switching Devices", RFC 2285, DOI 10.17487/RFC2285,
February 1998, <http://www.rfc-editor.org/info/rfc2285>.
[RFC2545] Marques, P. and F. Dupont, "Use of BGP-4 Multiprotocol [RFC2545] Marques, P. and F. Dupont, "Use of BGP-4 Multiprotocol
Extensions for IPv6 Inter-Domain Routing", RFC 2545, Extensions for IPv6 Inter-Domain Routing", RFC 2545,
March 1999. DOI 10.17487/RFC2545, March 1999,
<http://www.rfc-editor.org/info/rfc2545>.
[RFC4724] Sangli, S., Chen, E., Fernando, R., Scudder, J., and Y. [RFC4724] Sangli, S., Chen, E., Fernando, R., Scudder, J., and Y.
Rekhter, "Graceful Restart Mechanism for BGP", RFC 4724, Rekhter, "Graceful Restart Mechanism for BGP", RFC 4724,
January 2007. DOI 10.17487/RFC4724, January 2007,
<http://www.rfc-editor.org/info/rfc4724>.
[RFC4760] Bates, T., Chandra, R., Katz, D., and Y. Rekhter, [RFC4760] Bates, T., Chandra, R., Katz, D., and Y. Rekhter,
"Multiprotocol Extensions for BGP-4", RFC 4760, "Multiprotocol Extensions for BGP-4", RFC 4760,
January 2007. DOI 10.17487/RFC4760, January 2007,
<http://www.rfc-editor.org/info/rfc4760>.
[RFC5925] Touch, J., Mankin, A., and R. Bonica, "The TCP [RFC5925] Touch, J., Mankin, A., and R. Bonica, "The TCP
Authentication Option", RFC 5925, June 2010. Authentication Option", RFC 5925, DOI 10.17487/RFC5925,
June 2010, <http://www.rfc-editor.org/info/rfc5925>.
[RFC6414] Poretsky, S., Papneja, R., Karthik, J., and S. Vapiwala,
"Benchmarking Terminology for Protection Performance",
RFC 6414, DOI 10.17487/RFC6414, November 2011,
<http://www.rfc-editor.org/info/rfc6414>.
[RFC7115] Bush, R., "Origin Validation Operation Based on the
Resource Public Key Infrastructure (RPKI)", BCP 185,
RFC 7115, DOI 10.17487/RFC7115, January 2014,
<http://www.rfc-editor.org/info/rfc7115>.
Acknowledgements
We would like to thank Anil Tandon, Arvind Pandey, Mohan Nanduri, Jay
Karthik, and Eric Brendel for their input and discussions on various
sections in the document. We also like to acknowledge Will Liu,
Hubert Gee, Semion Lisyansky, and Faisal Shah for their review and
feedback on the document.
Authors' Addresses Authors' Addresses
Rajiv Papneja Rajiv Papneja
Huawei Technologies Huawei Technologies
Email: rajiv.papneja@huawei.com Email: rajiv.papneja@huawei.com
Bhavani Parise Bhavani Parise
Cisco Systems Skyport Systems
Email: bhavani@cisco.com Email: bparise@skyportsystems.com
Susan Hares Susan Hares
Huawei Technologies Huawei Technologies
Email: shares@ndzh.com Email: shares@ndzh.com
Dean Lee Dean Lee
IXIA IXIA
Email: dlee@ixiacom.com Email: dlee@ixiacom.com
 End of changes. 345 change blocks. 
791 lines changed or deleted 814 lines changed or added

This html diff was produced by rfcdiff 1.45. The latest version is available from http://tools.ietf.org/tools/rfcdiff/