draft-ietf-bmwg-evpntest-03.txt   draft-ietf-bmwg-evpntest-04.txt 
Internet Engineering Task Force S. Jacob, Ed. Internet Engineering Task Force S. Jacob, Ed.
Internet-Draft K. Tiruveedhula Internet-Draft K. Tiruveedhula
Intended status: Standards Track Juniper Networks Intended status: Informational Juniper Networks
Expires: February 22, 2020 August 21, 2019 Expires: June 20, 2020 December 18, 2019
Benchmarking Methodology for EVPN and PBB-EVPN Benchmarking Methodology for EVPN and PBB-EVPN
draft-ietf-bmwg-evpntest-03 draft-ietf-bmwg-evpntest-04
Abstract Abstract
This document defines methodologies for benchmarking EVPN and PBB- This document defines methodologies for benchmarking EVPN and PBB-
EVPN performance. EVPN is defined in RFC 7432, and is being deployed EVPN performance.EVPN is defined in RFC 7432, and is being deployed
in Service Provider networks. Specifically this document defines the in Service Provider networks.Specifically, this document defines
methodologies for benchmarking EVPN/PBB-EVPN convergence, data plane the methodologies for benchmarking EVPN/PBB-EVPN convergence, data
performance, and control plane performance. plane and control plane performance.
Status of This Memo Status of This Memo
This Internet-Draft is submitted in full conformance with the This Internet-Draft is submitted in full conformance with the
provisions of BCP 78 and BCP 79. provisions of BCP 78 and BCP 79.
Internet-Drafts are working documents of the Internet Engineering Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF). Note that other groups may also distribute Task Force (IETF). Note that other groups may also distribute
working documents as Internet-Drafts. The list of current Internet- working documents as Internet-Drafts. The list of current Internet-
Drafts is at https://datatracker.ietf.org/drafts/current/. Drafts is at https://datatracker.ietf.org/drafts/current/.
Internet-Drafts are draft documents valid for a maximum of six months Internet-Drafts are draft documents valid for a maximum of six months
and may be updated, replaced, or obsoleted by other documents at any and may be updated, replaced, or obsoleted by other documents at any
time. It is inappropriate to use Internet-Drafts as reference time. It is inappropriate to use Internet-Drafts as reference
material or to cite them other than as "work in progress." material or to cite them other than as "work in progress."
This Internet-Draft will expire on February 22, 2020. This Internet-Draft will expire on June 20, 2020.
Copyright Notice Copyright Notice
Copyright (c) 2019 IETF Trust and the persons identified as the Copyright (c) 2019 IETF Trust and the persons identified as the
document authors. All rights reserved. document authors. All rights reserved.
This document is subject to BCP 78 and the IETF Trust's Legal This document is subject to BCP 78 and the IETF Trust's Legal
Provisions Relating to IETF Documents Provisions Relating to IETF Documents
(https://trustee.ietf.org/license-info) in effect on the date of (https://trustee.ietf.org/license-info) in effect on the date of
publication of this document. Please review these documents publication of this document. Please review these documents
skipping to change at page 2, line 10 skipping to change at page 2, line 10
to this document. Code Components extracted from this document must to this document. Code Components extracted from this document must
include Simplified BSD License text as described in Section 4.e of include Simplified BSD License text as described in Section 4.e of
the Trust Legal Provisions and are provided without warranty as the Trust Legal Provisions and are provided without warranty as
described in the Simplified BSD License. described in the Simplified BSD License.
Table of Contents Table of Contents
1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 2 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 2
1.1. Requirements Language . . . . . . . . . . . . . . . . . . 3 1.1. Requirements Language . . . . . . . . . . . . . . . . . . 3
1.2. Terminologies . . . . . . . . . . . . . . . . . . . . . . 3 1.2. Terminologies . . . . . . . . . . . . . . . . . . . . . . 3
2. Test Topology . . . . . . . . . . . . . . . . . . . . . . . . 3 2. Test Topology . . . . . . . . . . . . . . . . . . . . . . . . 4
3. Test Cases for EVPN Benchmarking . . . . . . . . . . . . . . 6 3. Test Cases for EVPN Benchmarking . . . . . . . . . . . . . . 7
3.1. Local MAC Learning . . . . . . . . . . . . . . . . . . . 6 3.1. Local MAC Learning . . . . . . . . . . . . . . . . . . . 7
3.2. Remote MAC Learning . . . . . . . . . . . . . . . . . . . 7 3.2. Remote MAC Learning . . . . . . . . . . . . . . . . . . . 8
3.3. MAC Flush due to local link failure and Relearning . . . 7 3.3. MAC Flush due to local link failure and Relearning . . . 8
3.4. MAC Flush due to remote link failure. . . . . . . . . . . 8 3.4. MAC Flush due to remote link failure. . . . . . . . . . . 9
3.5. MAC Aging . . . . . . . . . . . . . . . . . . . . . . . . 8 3.5. MAC Aging . . . . . . . . . . . . . . . . . . . . . . . . 9
3.6. Remote Mac Aging . . . . . . . . . . . . . . . . . . . . 9 3.6. Remote Mac Aging . . . . . . . . . . . . . . . . . . . . 10
3.7. Local and Remote MAC Learning . . . . . . . . . . . . . . 9 3.7. Local and Remote MAC Learning . . . . . . . . . . . . . . 10
3.8. High Availability. . . . . . . . . . . . . . . . . . . . 10 3.8. High Availability. . . . . . . . . . . . . . . . . . . . 11
3.9. ARP/ND Scale . . . . . . . . . . . . . . . . . . . . . . 11 3.9. ARP/ND Scale . . . . . . . . . . . . . . . . . . . . . . 12
3.10. Scaling of Services . . . . . . . . . . . . . . . . . . . 11 3.10. Scaling of Services . . . . . . . . . . . . . . . . . . . 12
3.11. Scale Convergence . . . . . . . . . . . . . . . . . . . . 12 3.11. Scale Convergence . . . . . . . . . . . . . . . . . . . . 13
3.12. SOAK Test. . . . . . . . . . . . . . . . . . . . . . . . 12 3.12. SOAK Test. . . . . . . . . . . . . . . . . . . . . . . . 14
4. Test Cases for PBB-EVPN Benchmarking . . . . . . . . . . . . 13 4. Test Cases for PBB-EVPN Benchmarking . . . . . . . . . . . . 14
4.1. Local MAC Learning . . . . . . . . . . . . . . . . . . . 13 4.1. Local MAC Learning . . . . . . . . . . . . . . . . . . . 14
4.2. Remote Mac Learning . . . . . . . . . . . . . . . . . . . 13 4.2. Remote Mac Learning . . . . . . . . . . . . . . . . . . . 15
4.3. MAC Flush due to link failure . . . . . . . . . . . . . . 14 4.3. MAC Flush due to link failure . . . . . . . . . . . . . . 15
4.4. MAC Flush due to remote Failure . . . . . . . . . . . . . 14 4.4. MAC Flush due to remote Failure . . . . . . . . . . . . . 16
4.5. MAC aging . . . . . . . . . . . . . . . . . . . . . . . . 15 4.5. MAC aging . . . . . . . . . . . . . . . . . . . . . . . . 17
4.6. Remote MAC Aging. . . . . . . . . . . . . . . . . . . . . 16 4.6. Remote MAC Aging. . . . . . . . . . . . . . . . . . . . . 17
4.7. Local and Remote MAC Learning . . . . . . . . . . . . . . 16 4.7. Local and Remote MAC Learning . . . . . . . . . . . . . . 18
4.8. High Availability . . . . . . . . . . . . . . . . . . . . 17 4.8. High Availability . . . . . . . . . . . . . . . . . . . . 18
4.9. Scale . . . . . . . . . . . . . . . . . . . . . . . . . . 17 4.9. Scale . . . . . . . . . . . . . . . . . . . . . . . . . . 19
4.10. Scale Convergence . . . . . . . . . . . . . . . . . . . . 18 4.10. Scale Convergence . . . . . . . . . . . . . . . . . . . . 19
4.11. Soak Test . . . . . . . . . . . . . . . . . . . . . . . . 18 4.11. Soak Test . . . . . . . . . . . . . . . . . . . . . . . . 20
5. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 19 5. Acknowledgments . . . . . . . . . . . . . . . . . . . . . . 21
6. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 19 6. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 21
7. Security Considerations . . . . . . . . . . . . . . . . . . . 19 7. Security Considerations . . . . . . . . . . . . . . . . . . . 21
8. References . . . . . . . . . . . . . . . . . . . . . . . . . 19 8. References . . . . . . . . . . . . . . . . . . . . . . . . . 21
8.1. Normative References . . . . . . . . . . . . . . . . . . 19 8.1. Normative References . . . . . . . . . . . . . . . . . . 21
8.2. Informative References . . . . . . . . . . . . . . . . . 20 8.2. Informative References . . . . . . . . . . . . . . . . . 22
Appendix A. Appendix . . . . . . . . . . . . . . . . . . . . . . 20 Appendix A. Appendix . . . . . . . . . . . . . . . . . . . . . . 22
Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 20 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 22
1. Introduction 1. Introduction
EVPN is defined in RFC 7432, and describes BGP MPLS- based Ethernet EVPN is defined in RFC 7432, and describes BGP MPLS- based Ethernet
VPNs (EVPN). PBB-EVPN is defined in RFC 7623, discusses how Ethernet VPNs (EVPN).PBB-EVPN is defined in RFC 7623, discusses how Ethernet
Provider backbone Bridging can be combined with EVPNs to provide a Provider backbone Bridging can be combined with EVPNs to provide a
new/combined solution. This draft defines methodologies that can be new/combined solution.This draft defines methodologies that can be
used to benchmark both RFC 7432 and RFC 7623 solutions. Further, used to benchmark both RFC 7432 and RFC 7623 solutions.Further,
this draft provides methodologies for benchmarking the performance of this draft provides methodologies for benchmarking the performance of
EVPN data and control planes, MAC learning, MAC flushing, MAC ageing, EVPN data and control planes, MAC learning, MAC flushing, MAC aging,
convergence, high availability, and scale. convergence, high availability, and scale.
1.1. Requirements Language 1.1. Requirements Language
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
"SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
document are to be interpreted as described in RFC 2119 [RFC2119]. document are to be interpreted as described in RFC 8174 [RFC8174].
1.2. Terminologies 1.2. Terminologies
MHPE Multi homed Provide Edge router. All-Active Redundancy Mode: When all PEs attached to an Ethernet
segment are allowed to forward known unicast traffic to/from that
RR Route Reflector. Ethernet segment for a given VLAN, then the Ethernet segment is
defined to be operating in All-Active redundancy mode.
P Provider Router. AA All Active mode
CE Customer Router/Devices/Switch. CE Customer Router/Devices/Switch.
DF Designated Forwarder
DUT Device under test.
Ethernet Segment (ES): When a customer site (device or network) is
connected to one or more PEs via a set of Ethernet links, then that
set of links is referred to as an 'Ethernet segment'.
EVI: An EVPN instance spanning the Provider Edge (PE) devices
participating in that EVPN.
Ethernet Segment Identifier (ESI): A unique non-zero identifier that
identifies an Ethernet segment is called an 'Ethernet Segment
Identifier'.
Ethernet Tag: An Ethernet tag identifies a particular broadcast
domain, e.g., a VLAN. An EVPN instance consists of one or more
broadcast domains.
Interface Physical interface of a router/switch.
IRB Integrated routing and bridging interface
MAC Media Access Control addresses on a PE.
MHPE2 Multi homed Provider Edge router 2. MHPE2 Multi homed Provider Edge router 2.
MHPE1 Multi homed Provider Edge router 1. MHPE1 Multi homed Provider Edge router 1.
SHPE3 Single homed Provider Edge Router 3. SHPE3 Single homed Provider Edge Router 3.
AA EVPN Terminologies AA All-Active. PE: Provider Edge device.
SA EVPN Terminologies SA Single-Active. P Provider Router.
RT Router Tester. RR Route Reflector.
Sub Interface Each physical Interfaces is subdivided in to Logical RT Traffic Generator.
units.
EVI EVPN Instances which will be running on sub interface or physical Sub Interface Each physical Interfaces is subdivided into Logical
port of the provider Edge routers. units.
DF Designated Forwarder. SA Single Active
ESI Ethernet Segment Identifier. Single-Active Redundancy Mode: When only a single PE, among all the
PEs attached to an Ethernet segment, is allowed to forward traffic
to/from that Ethernet segment for a given VLAN, then the Ethernet
segment is defined to be operating in Single-Active redundancy mode.
2. Test Topology 2. Test Topology
EVPN/PBB-EVPN Services running on SHPE3, MHPE1 and MHPE2 in Single EVPN/PBB-EVPN Services running on SHPE3, MHPE1 and MHPE2 in Single
Active Mode: Active Mode:
| [Traffic Generator ] Router Tester traffic sender/receiver of layer 2 traffic with multiple vlan. | Traffic Generator acts as a sender/receiver of layer 2 traffic with multiple vlan.
+----------+ +----------+
| | | |
| SHPE3 | | SHPE3 |
| | | |
+----------+ +----------+
| |
|Core link |Core link
+----------+ +----------+
| | | |
| RR | | RR |
| | Route Reflector/Core router | | Route Reflector /Provider router
+----------+-------------| +----------+-------------|
| | | |
| Core links | | Core links |
+----------+ +-----------+ +----------+ +-----------+
| | | MHPE2 | | | | MHPE2 |
| DUT | | | | DUT | | |
| MHPE1 | | | | MHPE1 | | |
+----------+ +-----------+ +----------+ +-----------+
| PE-CE link | | PE-CE link |
+----------+------------ +----------+------------
| | | |
| CE | | CE |
| layer2 | | layer2 |
|bridge | |bridge |
+----------+------------ [Traffic Generator](Router Tester sender/reciever of layer 2 traffic with multiple vlan) +----------+------------ Traffic Generator acts as a sender/receiver of layer 2 traffic with multiple vlan.
Topology 1 Topology 1
Test Setup Test Setup
Figure 1 Figure 1
+-----------------+---------------------+---------------------+---------------------+----------------------+-----------------------+ +-----------------+---------------------+---------------------+---------------------+----------------------+-----------------------+
| | | | | | | | | | | | | |
| | | | | | | | | | | | | |
skipping to change at page 5, line 44 skipping to change at page 6, line 44
+-----------------+---------------------+---------------------+--------------------------------------------+-----------------------+ +-----------------+---------------------+---------------------+--------------------------------------------+-----------------------+
| |
++ ++
Table showing Traffic pattern for various test Table showing Traffic pattern for various test
Figure 2 Figure 2
Test Setup Configurations: Test Setup Configurations:
There are five routers in the Test setup. SHPE3, RR/P, MHPE1 and There are five routers in the Test setup.SHPE3, RR/P, MHPE1 and
MHPE2 emulating a service provider network. CE is a customer device MHPE2 emulating a service provider network. CE is a customer device
connected to MHPE1 and MHPE2, it is configured with bridge domains in connected to MHPE1 and MHPE2, it is configured with bridge domains in
multiple vlans. The router tester is connected to CE and SHPE3.The multiple vlans.The traffic generator is connected to CE and
MHPE1 acts as DUT.The RT will be used as sender and receiver of SHPE3.The MHPE1 acts as DUT.The traffic generator will be used as
traffic.The measurement will be taken in DUT. sender and receiver of traffic.The measurement will be taken in DUT.
All routers except CE is configured with OSPF/IS-IS,LDP,MPLS,BGP with All routers except CE are configured with OSPF/IS-IS,LDP,MPLS,BGP
EVPN address family. with EVPN address family.
All routers except CE must have IBGP configured with RR acting as All routers except CE are configured with Interior border gateway
route reflector. protocol,RR acting as route reflector.
MHPE1,MHPE2,SHPE3 must be configured with "N" EVPN/PBB-EVPN instances MHPE1,MHPE2,SHPE3 must be configured with "N" EVPN/PBB-EVPN instances
depends up on the cases. depends up on the cases.
MHPE1 and MHEPE2 must be configured with ESI per vlan or ESI on IFD. MHPE1 and MHEPE2 must be configured with ESI per vlan or ESI on
interface.
MHPE1 and MHEPE2 are running Single Active mode of EVPN. MHPE1 and MHEPE2 are running Single Active mode of EVPN.
CE is acting as bridge configured with vlans that is configured on CE is acting as bridge configured with multiple vlans,the same vlans
MHPE1,MHPE2,SHPE3. are configured on MHPE1,MHPE2,SHPE3.
Depends up on the test traffic will be flowing uni directional or bi Depending up on the test scenarios the traffic generators will be
directional depends on the test performed. used to generate uni directional or bi directional flows.
The above configuration will be serving as the base configuration for The above configuration will be serving as the base configuration for
all test cases. all test cases.
3. Test Cases for EVPN Benchmarking 3. Test Cases for EVPN Benchmarking
3.1. Local MAC Learning 3.1. Local MAC Learning
Objective: Objective:
To Record the time taken to learn the MAC address locally in DUT. To Record the time taken to learn the MAC address locally in DUT.
Topology : Topology 1 Topology : Topology 1
Procedure: Procedure:
The data plane MAC learning can be measured using the parameters The data plane MAC learning can be measured using the parameters
defined in RFC 2889 section 5.8. Send "X" unicast frames from CE to defined in RFC 2889 section 5.8.Send "X" unicast frames from CE to
MHPE1(DUT) working in SA mode with "X" different source and MHPE1(DUT) working in SA mode.Traffic generator connected to CE must
destination address from RT. The DUT must learn these "X" macs in send frames with "X" different source and destination MAC address.
data plane. The DUT must learn these "X" macs in data plane.
Measurement : Measurement :
Measure the time taken to learn "X" MACs in DUT evpn mac table. The Measure the time taken to learn "X" MACs locally in DUT evpn mac
data plane measurement is taken by considering DUT as black box the table. The data plane measurement is taken by considering DUT as
range of X MAC is known from RT and the same must be learned in DUT, black box. The range of MACs are known from traffic generator,the
the time taken to learn "X" macs is measured. same must be learned in DUT, the time taken to learn "X" macs is
measured.
Repeat these test and plot the data. The test is repeated for "N" The test is repeated for "N" times and the values are collected.The
times and the values are collected. The mac learning time is mac learning rate is calculated by averaging the values obtained from
calculated by averaging the values obtained from "N" samples. "N" samples.
Mac learning in sec = (T1+T2+..Tn/N) Mac learning rate = (T1+T2+..Tn)/N
3.2. Remote MAC Learning 3.2. Remote MAC Learning
Objective: Objective:
To Record the time taken to learn the remote macs. To Record the time taken to learn the remote macs.
Topology : Topology 1 Topology : Topology 1
Procedure: Procedure:
Send X frames with X different SA and DA to SHPE3 from RT. SHPE3 Send X frames with X different source and destination mac addresses
will advertise these locally learned macs to MHPE1 and MHPE2 via to SHPE3 from traffic generator.SHPE3 will advertise these locally
control plane.Measure the time taken to learn these X MACs from learned macs to MHPE1 and MHPE2 via control plane.Measure the time
remote peer in DUT EVPN MAC address table.The DUT and MHPE2 are taken to learn these X MACs from remote peer in DUT EVPN MAC address
running SA mode. table.The DUT and MHPE2 are running SA mode.
Measurement : Measurement :
Measure the time taken by the DUT to learn the "X" MACs in the data Measure the time taken by the DUT to learn the "X" MACs in the data
plane.Repeat these test and plot the data.The test is repeated for plane.The test is repeated for "N" times and the values are
"N" times and the values are collected.The mac learning time is collected. The remote mac learning rate is calculated by averaging
calculated by averaging the values obtained from "N" samples. the values obtained from "N" samples.
Mac learning in sec = (T1+T2+..Tn/N) Remote mac learning rate = (T1+T2+..Tn)/N
3.3. MAC Flush due to local link failure and Relearning 3.3. MAC Flush due to local link failure and Relearning
Objective: Objective:
To record the time taken to flush the mac learned locally and the To record the time taken to flush the locally learned mac and the
time taken to relearn the same amount of macs. time taken to relearn the same amount of macs.
Topology : Topology 1 Topology : Topology 1
Procedure: Procedure:
Send X frames with X different SA and DA to DUT from CE using traffic Send X frames with X different source and destination mac addresses
generator. Wait till the MHPE1 learns all X MAC address. Then fail to DUT from CE using traffic generator. Wait till the MHPE1 learns
the MHPE1 CE link and measure the time taken to flush these X MACs all X MAC addresses. Then fail the MHPE1 CE link and measure the
from the EVPN MAC table. Bring up the link which was made Down(the time taken to flush these X MACs from the EVPN MAC table. Bring up
link between MHPE1 and CE).Measure time taken to relearn it. The DUT the link which was made Down(the link between MHPE1 and CE).Measure
and MHPE2 are running SA mode. time taken to relearn it.The DUT and MHPE2 are running SA mode.
Measurement : Measurement :
Measure the time taken for flushing these X MAC address. Measure the Measure the time taken for flushing these X MAC addresses.Measure
time taken to relearn the X MACs in DUT. Repeat these test and plot the time taken to relearn the X MACs in DUT.The test is repeated for
the data. The test is repeated for "N" times and the values are "N" times and the values are collected.The flush and the relearning
collected. The flush and the relearning time is calculated by time is calculated by averaging the values obtained by "N" samples.
averaging the values obtained by "N" samples.
Flush time for X Macs in sec = (T1+T2+..Tn/N) Flush rate = (T1+T2+..Tn)/N
Relearning time for X macs in sec = (T1+T2+..Tn/N) Relearning rate = (T1+T2+..Tn)/N
3.4. MAC Flush due to remote link failure. 3.4. MAC Flush due to remote link failure.
Objective: Objective:
To record the time taken to flush the remote mac learned in DUT To record the time taken to flush the remote mac learned in DUT
during remote link failure. during remote link failure.
Topology : Topology 1 Topology : Topology 1
Procedure: Procedure:
Send X frames with X different SA and DA to DUT from SHPE3 using Send X frames with X different source and destination mac addresses
traffic generator. Bring down the link between SHPE3 and traffic to DUT from SHPE3 using traffic generator. Bring down the link
generator. Then measure the time taken to flush the DUT EVPN MAC between SHPE3 and traffic generator.Then measure the time taken to
table. The DUT and MHPE2 are running SA mode. flush the DUT EVPN MAC table.The DUT and MHPE2 are running SA mode.
Measurement : Measurement :
Measure the time taken to flush X remote MACs from EVPN MAC table of Measure the time taken to flush X remote MACs from EVPN MAC table of
DUT. Repeat these test and plot the data. The test is repeated for the DUT.The test is repeated for "N" times and the values are
"N" times and the values are collected. The flush rate is calculated collected.The flush rate is calculated by averaging the values
averaging the values obtained by "N" samples. obtained by "N" samples.
Flush time for X Macs in sec = (T1+T2+..Tn/N) Flush rate = (T1+T2+..Tn)/N
3.5. MAC Aging 3.5. MAC Aging
Objective: Objective:
To measure the mac aging time. To measure the mac aging time.
Topology : Topology 1 Topology : Topology 1
Procedure: Procedure:
Send X frames with X different SA and DA to DUT from CE using traffic Send X frames with X different source and destination mac addresses
generator. Wait till X MAC address are learned. Then stop the to DUT from CE using traffic generator.Wait till X MAC addresses
traffic. Record the time taken to flush X MACS from DUT EVPN MAC are learned. Then stop the traffic.Record the time taken to flush
table due to aging. The DUT and MHPE2 are running SA mode X MACS from DUT EVPN MAC table due to aging.The DUT and MHPE2 are
running SA mode.
Measurement : Measurement :
Measure the time taken to flush X MAC address due to aging. Repeat Measure the time taken to flush X MAC addresses due to aging.The
these test and plot the data. The test is repeated for "N" times and test is repeated for "N" times and the values are collected.The
the values are collected. The aging is calculated averaging the aging is calculated averaging the values obtained by "N" samples.
values obtained by "N" samples.
Aging time for X Macs in sec = (T1+T2+..Tn/N) Aging time for X Macs in sec = (T1+T2+..Tn)/N
3.6. Remote Mac Aging 3.6. Remote Mac Aging
Objective: Objective:
To measure the remote mac aging time. To measure the remote mac aging time.
Topology : Topology 1 Topology : Topology 1
Procedure: Procedure:
Send X frames with X different SA and DA to DUT from SHPE3 using Send X frames with X different source and destination mac addresses
traffic generator. Stop the traffic at remote PE SHPE3.Due to MAC to DUT from SHPE3 using traffic generator. Stop the traffic at
aging SHPE3 will withdraw its routes from DUT and MHPE2. Measure the remote PE SHPE3.Due to MAC aging SHPE3 will withdraw its routes from
time taken to remove these MACs from DUT EVPN MAC table. DUT and DUT and MHPE2.Measure the time taken to remove these MACs from DUT
MHPE2 are running in SA mode EVPN MAC table.DUT and MHPE2 are running in SA mode
Measurement : Measurement :
Measure the time taken to flush X remote MACs learned in DUT EVPN MAC Measure the time taken to flush X remote MACs learned in DUT EVPN MAC
table due to aging. Repeat these test and plot the data. The test table due to aging.The test is repeated for "N" times and the values
is repeated for "N" times and the values are collected. The aging is are collected.The aging is calculated by averaging the values
calculated by averaging the values obtained by "N" samples. obtained by "N" samples.
Aging time for X Macs in sec = (T1+T2+..Tn/N) Aging time for X Macs in sec = (T1+T2+..Tn)/N
3.7. Local and Remote MAC Learning 3.7. Local and Remote MAC Learning
Objective: Objective:
To record the time taken to learn both local and remote macs. To record the time taken to learn both local and remote macs.
Topology : Topology 1 Topology : Topology 1
Procedure: Procedure:
Send X frames with X different SA and DA to DUT from SHPE3 using Send X frames with X different source and destination mac addresses
traffic generator. Send X frames with different SA and DA from to DUT from SHPE3 using traffic generator.Send X frames with
traffic generator connected to CE. The SA and DA of flows must be different source and destination mac addresses from traffic generator
complimentary to have unicast flows. Measure the time taken by the connected to CE.The source and destination addresses of flows must
DUT to learn 2X in EVPN MAC. DUT and MHPE2 are running in SA mode. be complimentary to have unicast flows.Measure the time taken by
the DUT to learn 2X in EVPN MAC.DUT and MHPE2 are running in SA
mode.
Measurement : Measurement :
Measure the time taken to learn 2X MAC address in DUT EVPN MAC table. Measure the time taken to learn 2X MAC addresses in DUT EVPN MAC
Repeat these test and plot the data. The test is repeated for "N" table. The test is repeated for "N" times and the values are
times and the values are collected. The mac learning time is collected. The mac learning time is calculated by averaging the
calculated by averaging the values obtained by "N" samples. values obtained by "N" samples.
Time to learn 2X Macs in sec = (T1+T2+..Tn/N) Mac learning rate = (T1+T2+..Tn)/N
3.8. High Availability. 3.8. High Availability.
Objective: Objective:
To record traffic loss during routing engine fail over. To record traffic loss during routing engine fail over.
Topology : Topology 1 Topology : Topology 1
Procedure: Procedure:
Send X frames from CE to DUT from traffic generator withX different Send X frames from CE to DUT from traffic generator with X different
SA and DA. Send X frames from traffic generator to SHPE3 with X source and destination mac addresses.Send X frames from traffic
different SA and DA so that 2X MAC address will be learned in DUT. generator to SHPE3 with X different source and destination mac
There is a bi directional traffic flow with X pps in each direction. addresses, so that 2X MAC address will be learned in the DUT.There
Then do a routing engine fail-over. is a bi directional traffic flow with X pps in each direction.Then
do a routing engine fail-over.
Measurement : Measurement :
There should be 0 traffic loss which is the ideal case, No change in The expectation of the test is 0 traffic loss with no change in the
the DF role. DUT should not withdraw any routes.Repeat the test "N" DF role.DUT should not withdraw any routes.But in cases where the
times and plot the data.The packet loss is calculated by averaging DUT is not property synchronized between master and standby,due to
the values obtained from "N" samples. that packet loss are observed.In that scenario the packet loss is
measured.The test is repeated for "N" times and the values are
collected.The packet loss is calculated by averaging the values
obtained by "N" samples.
Packet loss in sec = (T1+T2+..Tn/N) Packet loss in sec with 2X mac addresses = (T1+T2+..Tn)/N
3.9. ARP/ND Scale 3.9. ARP/ND Scale
These tests are conducted to Record the scaling parameter of ARP/ND To Record the DUT scaling limit of ARP/ND.
of the DUT.
Objective: Objective:
To Record the ARP/ND scale of the DUT. To Record the ARP/ND scale of the DUT.
Topology : Topology 1 Topology : Topology 1
Procedure: Procedure:
Send X arp/icmpv6 request from RT to DUT with different sender ip/ Send X arp/neighbor discovery(ND) from the traffic generator to DUT
ipv6 address to the same target gateway ip address. Measure whether with different sender ip/ipv6,mac addresses to the target IRB address
X MAC+IPv4 address/MAC+IPv6 address of the hosts are learned in DUT. configured in EVPN instance. The EVPN instance learns the mac+ip and
mac+ipv6 addresses from these request and advertise as type 2 mac+ip/
mac+ipv6 route to remote provide edge routers which have same EVPN
configurations.The value of X must be increased at a incremental
value of 5% of X, till the limit is reached.The limit is where the
DUT cant learn any more type 2 mac+ip/mac+ipv6.The test must be
separately conducted for arp and ND.
Measurement : Measurement :
The DUT must learn X MAC+IPV4/MAC+IPv6 and it must advertise the X Measure the scale limit of type 2 mac+ip/mac+ipv6 route which DUT can
MAC+IPV4/MAC+IPV6 to the remote router. learn.The test is repeated for "N" times and the values are
collected.The scale limit is calculated by averaging the values
obtained by "N" samples for both mac+ip and mac+ipv6.
DUT scale limit for mac+ip = (v1+v2+..vn)/N
DUT scale limit for mac+ipv6 = (v1+v2+..vn)/N
3.10. Scaling of Services 3.10. Scaling of Services
Objective: Objective:
To measure the scale limit of DUT for EVPN.This is to measure the This test is to measure the scale of EVPN instances that a DUT can
performance of DUT in scaling to "X" EVPN instances. hold.
Topology : Topology 1 Topology : Topology 1
Procedure: Procedure:
The DUT,MHPE2 and SHPE3 are scaled to "N" EVI.Clear BGP neighbors of The DUT,MHPE2 and SHPE3 are scaled to "N" EVI.Clear BGP neighbors of
the DUT. Once adjacency is established in the DUT. Measure the the DUT.Once adjacency is established in the DUT, check the
routes received from MHPE2 and SHPE3 for "N" EVI in the DUT. routes received from MHPE2 and SHPE3 for "N" EVI in the DUT.Then
increment the scale of N by 5% of N till the limit is reached.The
limit is where the DUT cant learn any EVPN routes from peers.
Measurement : Measurement :
There should not be any loss of route types 1,2,3 and 4 in DUT. DUT There should not be any loss of route types 1,2,3 and 4 in DUT.DUT
must relearn all type 1,2,3 and 4 from remote routers. The DUT must must relearn all type 1,2,3 and 4 from remote routers. The DUT must
be subjected to various values of N to find the optimal scale limit be subjected to various values of N to find the optimal scale limit.
The scope of the test is find out the maximum evpn instance that a
DUT can hold.
3.11. Scale Convergence 3.11. Scale Convergence
Objective: Objective:
To measure the convergence time of DUT when the DUT is scaled with To measure the convergence time of DUT when the DUT is scaled with
EVPN instance along with traffic. EVPN instance along with traffic.
Topology : Topology 1 Topology : Topology 1
Procedure: Procedure:
Scale N EVIs in DUT,SHPE3 and MHPE2.Send F frames to DUT from CE Scale N EVIs in DUT,SHPE3 and MHPE2.Send F frames to DUT from CE
using traffic generator with X different SA and DA for N EVI's. Send using traffic generator with X different source and destination mac
F frames from traffic generator to SHPE3 with X different SA and DA. addresses for N EVI's.Send F frames from traffic generator to SHPE3
There will be 2X number of MAC address will be learned in DUT EVPN with X different source and destination mac addresses.There will be
MAC table. There is a bi directional traffic flow with F pps in each 2X number of MAC addresses will be learned in DUT EVPN MAC table.
direction. Then clear the BGP neighbors in the DUT. Once the There is a bi directional traffic flow with F pps in each direction.
adjacency is restored in DUT. Measure the time taken to learn 2X MAC Then clear the BGP neighbors in the DUT.Once the adjacency is
address in DUT MAC table. restored in DUT.Measure the time taken to learn 2X MAC address in
DUT MAC table.
Measurement : Measurement :
The DUT must learn 2X MAC address. Measure the time taken to learn The DUT must learn 2X MAC addresses. Measure the time taken to learn
2X MAC in DUT. Repeat these test and plot the data.The test is 2X MAC in DUT.The test is repeated for "N" times and the values are
repeated for "N" times and the values are collected.The convergence collected. The convergence time is calculated by averaging the
time is calculated by averaging the values obtained by "N" samples. values obtained by "N" samples.
Convergence time in sec = (T1+T2+..Tn/N) Time taken to learn 2X macs in DUT = (T1+T2+..Tn)/N
3.12. SOAK Test. 3.12. SOAK Test.
Objective: Objective:
This test is carried out to measure the stability of the DUT in a This test is carried out to measure the stability of the DUT in a
scaled environment with traffic over a period of time "T'". In each scaled environment with traffic over a period of time "T'". In each
interval "t1" the DUT CPU usage, memory usage are measured. The DUT interval "t1" the DUT CPU usage, memory usage are measured.The DUT
is checked for any crashes during this time period. is checked for any crashes during this time period.
Topology : Topology 1 Topology : Topology 1
Procedure: Procedure:
Scale N EVI's in DUT,SHPE3 and MHPE2.Send F frames to DUT from CE Scale N EVI's in DUT,SHPE3 and MHPE2.Send F frames to DUT from CE
using traffic generator with different X SA and DA for N EVI's. Send using traffic generator with different X source and destination mac
F frames from traffic generator to SHPE3 with X different SA and DA. addresses for N EVI's.Send F frames from traffic generator to SHPE3
There will be 2X number of MAC address will be learned in DUT EVPN with X different source and destination mac addresses.There will be
MAC table. There is a bi directional traffic flow with F pps in each 2X number of MAC addresses will be learned in DUT EVPN MAC table.
direction. The DUT must run with traffic for 24 hours, every hour There is a bi directional traffic flow with F pps in each direction.
check for memory leak, CPU usage and crash. The DUT must run with traffic for 24 hours, every hour check for
memory leak, CPU usage and crash.
Measurement : Measurement :
Take the hourly reading of CPU, process memory. There should not be Take the hourly reading of CPU, process memory. There should not be
any leak, crashes, CPU spikes. any leak, crashes, CPU spikes. Th CPU spike is determined as the CPU
usage which shoots at 40 to 50 percent of the average usage.The
average value vary from device to device. Memory leak is determined
by increase usage of the memory for EVPN process. The expectation is
under steady state the memory usage for EVPN process should not
increase.
4. Test Cases for PBB-EVPN Benchmarking 4. Test Cases for PBB-EVPN Benchmarking
4.1. Local MAC Learning 4.1. Local MAC Learning
Objective: Objective:
To Record the time taken to learn the MAC address locally. To Record the time taken to learn the MAC address locally.
Topology : Topology 1 Topology : Topology 1
Procedure: Procedure:
Send "X" unicast frames from CE to MHPE1(DUT) working in SA mode with Send "X" unicast frames from CE to MHPE1(DUT) working in SA mode with
"X" different source and destination address from RT. The DUT must "X" different source and destination mac addresses from traffic
learn "X" macs in data plane. generator.The DUT must learn "X" macs in data plane.
Measurement : Measurement :
Measure the time taken by the DUT to learn the "X" MACs in the data Measure the time taken by the DUT to learn the "X" MACs in the data
plane. The data plane measurement is taken by considering DUT as plane. The data plane measurement is taken by considering DUT as
black box the range of "X" MAC is known from RT and the same must be black box the range of "X" MAC is known from traffic generator and
learned in DUT, the time taken to learn "X" MAC is measured. Repeat the same must be learned in DUT, the time taken to learn "X" MAC is
these test and plot the data. The test is repeated for "N" times and measured.The test is repeated for "N" times and the values are
the values are collected. The mac learning time is calculated by collected.The mac learning time is calculated by averaging the
averaging the values obtained from "N" samples. values obtained from "N" samples.
Mac learning in sec = (T1+T2+..Tn/N) Mac learning rate = (T1+T2+..Tn)/N
4.2. Remote Mac Learning 4.2. Remote Mac Learning
Objective: Objective:
To Record the time taken to learn the remote macs. To Record the time taken to learn the remote macs.
Topology : Topology 1 Topology : Topology 1
Procedure: Procedure:
Send X frames with X different SA and DA to SHPE3 from RT.These macs Send X frames with X different source and destination mac addresses
will be flooded to MHPE1 and MHPE2 by SHPE3.The DUT and MHPE2 are to SHPE3 from traffic generator.These macs will be flooded to MHPE1
running SA mode. and MHPE2 by SHPE3.The DUT and MHPE2 are running SA mode.
Measurement : Measurement :
Measure the time taken to learn X mac address in DUT mac table. Measure the time taken to learn X mac addresses in DUT mac table.
Repeat these test and plot the data.The test is repeated for "N" The test is repeated for "N" times and the values are collected.The
times and the values are collected.The mac learning time is mac learning time is calculated by averaging the values obtained by
calculated by averaging the values obtained by "N" samples. "N" samples.
Mac learning in sec = (T1+T2+..Tn/N) Remote Mac learning rate = (T1+T2+..Tn)/N
4.3. MAC Flush due to link failure 4.3. MAC Flush due to link failure
Objective: Objective:
To record the time taken to flush the mac learned locally and the To record the time taken to flush the locally learned mac and the
time taken to relearn the same amount of macs. time taken to relearn the same amount of macs.
Topology : Topology 1 Topology : Topology 1
Procedure: Procedure:
Send X frames with X different SA and DA to DUT from CE using traffic Send X frames with X different source and destination mac addresses
generator. Wait till the MHPE1 learn all X MAC address. Then fail to DUT from CE using traffic generator.Wait till the MHPE1 learn
the MHPE1 CE link and measure the time taken to flush these X MACs all X MAC address.Then fail the MHPE1 CE link and measure the time
from the PBB-EVPN MAC table. Then bring up the link. Measure the taken to flush these X MACs from the PBB-EVPN MAC table.Then bring
time taken to relearn X MACS. The DUT and MHPE2 are running SA mode. up the link.Measure the time taken to relearn X MACS.The DUT and
MHPE2 are running SA mode.
Measurement : Measurement :
Measure the time taken for flushing these X MAC address. Measure the Measure the time taken for flushing these X MAC addresses.Measure
time taken to relearn the X MACs in DUT. Repeat these test and plot the time taken to relearn the X MACs in DUT.The test is repeated for
the data. The test is repeated for "N" times and the values are "N" times and the values are collected. The flush and the relearning
collected. The flush and the relearning time is calculated by time is calculated by averaging the values obtained by "N" samples.
averaging the values obtained by "N" samples.
Flush time for X Macs in sec = (T1+T2+..Tn/N) Flush rate = (T1+T2+..Tn)/N
Relearning time for X macs in sec = (T1+T2+..Tn/N) Relearning rate = (T1+T2+..Tn)/N
4.4. MAC Flush due to remote Failure 4.4. MAC Flush due to remote Failure
Objective: Objective:
To record the time taken to flush the remote mac learned in DUT To record the time taken to flush the remote mac learned in DUT
during remote link failure. during remote link failure.
Topology : Topology 1 Topology : Topology 1
Procedure: Procedure:
Send X frames with X different SA and DA to DUT from SHPE3 using Send X frames with X different source and destination mac addresses
traffic generator. Bring down the link between SHPE3 and traffic to DUT from SHPE3 using traffic generator.Bring down the link
generator. Then measure the time taken to flush the DUT PBB-EVPN MAC between SHPE3 and traffic generator. Then measure the time taken to
address table. The remote MACs will be learned by Data plane, but flush the DUT PBB-EVPN MAC address table.The remote MACs will be
the B-MAC will be learned by control plane. The DUT and MHPE2 are learned by Data plane, but the B-MAC will be learned by control
running SA mode. plane.The DUT and MHPE2 are running SA mode.
Measurement : Measurement :
Measure the time taken to flush X remote MACs from PBB-EVPN MAC table Measure the time taken to flush X remote MACs from PBB-EVPN MAC table
of DUT. Repeat these test and plot the data. The test is repeated of DUT.Repeat these test and plot the data.The test is repeated
for "N" times and the values are collected. The flush rate is for "N" times and the values are collected. The flush rate is
calculated by averaging the values obtained by "N" samples. calculated by averaging the values obtained by "N" samples.
Flush time for X Macs in sec = (T1+T2+..Tn/N) Flush rate = (T1+T2+..Tn)/N
4.5. MAC aging 4.5. MAC aging
Objective: Objective:
To measure the mac aging time. To measure the mac aging time.
Topology : Topology 1 Topology : Topology 1
Procedure: Procedure:
Send X frames with X different SA and DA to DUT from CE using traffic Send X frames with X different source and destination mac addresses
generator. Wait till X MAC address are learned in DUT PBB- EVPN MAC to DUT from CE using traffic generator.Wait till X MAC address are
table. Then stop the traffic. Record the time taken to flush X MAC learned in DUT PBB-EVPN MAC table.Then stop the traffic.Record
entries due to aging. The DUT and MHPE2 running in SA mode the time taken to flush X MAC entries due to aging. The DUT and
MHPE2 running in SA mode
Measurement : Measurement :
Measure the time taken to flush X MAC address due to aging. Repeat Measure the time taken to flush X MAC addresses due to aging.Repeat
these test and plot the data. The test is repeated for "N" times and these test and plot the data.The test is repeated for "N" times and
the values are collected. The aging is calculated by averaging the the values are collected.The aging is calculated by averaging the
values obtained by "N" samples. values obtained by "N" samples.
Aging time for X Macs in sec = (T1+T2+..Tn/N) Aging time for X Macs in sec = (T1+T2+..Tn)/N
4.6. Remote MAC Aging. 4.6. Remote MAC Aging.
Objective: Objective:
To measure the remote mac aging time. To measure the remote mac aging time.
Topology : Topology 1 Topology : Topology 1
Procedure: Procedure:
Send X frames with X different SA and DA to DUT from SHPE3 using Send X frames with X different source and destination mac addresses
traffic generator. Stop the traffic at remote PE(SHPE3).Measure the to DUT from SHPE3 using traffic generator.Stop the traffic at
time taken to remove these remote MACs from DUT PBB-EVPN MAC table. remote PE(SHPE3).Measure the time taken to remove these remote MACs
The DUT and MHPE2 are running in SA mode. from DUT PBB-EVPN MAC table.The DUT and MHPE2 are running in SA
mode.
Measurement : Measurement :
Measure the time taken to flush the X remote MACs from DUT PBB-EVPN Measure the time taken to flush the X remote MACs from DUT PBB-EVPN
MAC table due to aging Repeat these test and plot the data. The test MAC table due to aging .The test is repeated for "N" times and the
is repeated for "N" times and the values are collected. The aging is values are collected. The aging is calculated by averaging the
calculated by averaging the values obtained by "N" samples. values obtained by "N" samples.
Aging time for X Macs in sec = (T1+T2+..Tn/N) Aging time for X Macs in sec = (T1+T2+..Tn)/N
4.7. Local and Remote MAC Learning 4.7. Local and Remote MAC Learning
Objective: Objective:
To record the time taken to learn both local and remote macs. To record the time taken to learn both local and remote macs.
Topology : Topology 1 Topology : Topology 1
Procedure: Procedure:
Send X frames with X different SA and DA to DUT from SHPE3 using Send X frames with X different source and destination mac addresses
traffic generator. Send X frames with different SA and DA from to DUT from SHPE3 using traffic generator.Send X frames with
traffic generator connected to CE. The SA and DA of flows must be different source and destination mac addresses from traffic generator
complimentary to have unicast flows. Measure the time taken by the connected to CE.The source and destination mac addresses of flows
DUT to learn 2X in MAC table. DUT and MHPE2 are running in SA mode. must be complimentary to have unicast flows.Measure the time taken
by the DUT to learn 2X in MAC table.DUT and MHPE2 are running in SA
mode.
Measurement : Measurement :
Measure the time taken to learn 2X MAC address table in DUT PBB-EVPN Measure the time taken to learn 2X MAC addresses table in DUT PBB-
MAC table. Repeat these test and plot the data. The test is EVPN MAC table.The test is repeated for "N" times and the values
repeated for "N" times and the values are collected. The mac are collected.The mac learning time is calculated by averaging
learning time is calculated by averaging the values obtained by "N" the values obtained by "N" samples.
samples.
Time to learn 2X Macs in sec = (T1+T2+..Tn/N) Mac learning rate = (T1+T2+..Tn)/N
4.8. High Availability 4.8. High Availability
Objective: Objective:
To record traffic loss during routing engine failover. To record traffic loss during routing engine failover.
Topology : Topology 1 Topology : Topology 1
Procedure: Procedure:
Send X frames to DUT with X different SA and DA from CE using the Send X frames to DUT with X different source and destination mac
traffic generator. Send X frames from traffic generator to SHPE3 addresses from CE using the traffic generator.Send X frames from
with X different SA and DA so that 2X MAC address will be Learned in traffic generator to SHPE3 with X different source and destination
DUT. There is a bi directional traffic flow with X pps in each mac addresses, so that 2X MAC address will be Learned in DUT. There
direction. Then do a routing engine fail-over. is a bi directional traffic flow with X pps in each direction. Then
do a routing engine fail-over.
Measurement : Measurement :
There should be 0 traffic loss which is the ideal case, No change in The expectation of the test is 0 traffic loss with no change in the
the DF role. DUT should not withdraw any routes.Repeat the test "N" DF role.DUT should not withdraw any routes.But in cases where the
times and plot the data.The packet loss is calculated by averaging DUT is not property synchronized between master and standby,due to
the values obtained from "N" samples. that packet loss are observed. In that scenario the packet loss is
measured.The test is repeated for "N" times and the values are
collected.The packet loss is calculated by averaging the values
obtained by "N" samples.
Packet loss in sec = (T1+T2+..Tn/N) Packet loss in sec with 2X mac addresses = (T1+T2+..Tn)/N
4.9. Scale 4.9. Scale
Objective: Objective:
To measure the scale limit of DUT for PBB-EVPN. To measure the scale limit of DUT for PBB-EVPN.
Topology : Topology 1 Topology : Topology 1
Procedure: Procedure:
The DUT,MHPE2 and SHPE3 are scaled to "N" PBB-EVPN instances. Clear The DUT,MHPE2 and SHPE3 are scaled to "N" PBB-EVN instances.Clear BGP
BGP neighbors in the DUT Once adjacency is established in DUT, check neighbors of the DUT. Once adjacency is established in the DUT.
routes received from SHPE3 and MHPE2. check the routes received from MHPE2 and SHPE3 for "N" PBB-EVPN
instances in the DUT.Then increment the scale of N by 5% of N till
the limit is reached.The limit is where the DUT cant learn any EVPN
routes from peers.
Measurement : Measurement :
There should not be any loss of route types 2,3 and 4 in DUT. The There should not be any loss of route types 2,3 and 4 in DUT.DUT
DUT must relearn all type 2,3 and 4 routes from remote routers. The must relearn all type 2,3 and 4 from remote routers.The DUT must be
DUT must be subjected to various values of N to find the optimal subjected to various values of N to find the optimal scale limit.
scale limit. The scope of the test is find out the maximum evpn instance that a
DUT can hold.
4.10. Scale Convergence 4.10. Scale Convergence
Objective: Objective:
To measure the convergence time of DUT when the DUT is scaled with To measure the convergence time of DUT when the DUT is scaled with
EVPN instance along with traffic. EVPN instance along with traffic.
Topology : Topology 1 Topology : Topology 1
Procedure: Procedure:
Scale N PBB-EVIs in DUT,SHPE3 and MHPE2.Send F frames to DUT from CE Scale N PBB-EVPN instances in DUT,SHPE3 and MHPE2.Send F frames to
using traffic generator with X different SA and DA for N PBB-EVI's. DUT from CE using traffic generator with X different source and
Send F frames from traffic generator to SHPE3 with X different SA and destination mac addresses for N PBB-EVPN instances. Send F frames
DA. There will be 2X number of MAC address will be learned in DUT from traffic generator to SHPE3 with X different source and
PBB-EVPN MAC table. There is a bi directional traffic flow with F destination mac addresses.There will be 2X MAC addresses will be
pps in each direction. Then clear the BGP neighbors in the DUT. learned in DUT PBB-EVPN MAC table.There is a bi directional traffic
Once the adjacency is restored in DUT. Measure the time taken to flow with F pps in each direction.Then clear the BGP neighbors in
learn 2X MAC address in DUT MAC table. the DUT. Once the adjacency is restored in DUT. Measure the time
taken to learn 2X MAC address in DUT MAC table.
Measurement : Measurement :
The DUT must learn 2X MAC address. Measure the time taken to learn The DUT must learn 2X MAC addresses.Measure the time taken to learn
2X MAC in DUT. Repeat these test and plot the data.The test is 2X MAC in DUT.The test is repeated for "N" times and the values are
repeated for "N" times and the values are collected.The convergence collected.The convergence time is calculated by averaging the values
time is calculated by averaging the values obtained by "N" samples. obtained by "N" samples.
Convergence time in sec = (T1+T2+..Tn/N) Convergence time for 2X in sec = (T1+T2+..Tn)/N
4.11. Soak Test 4.11. Soak Test
Objective: Objective:
To measure the stability of the DUT in a scaled environment with To measure the stability of the DUT in a scaled environment with
traffic. traffic.
Topology : Topology 1 Topology : Topology 1
Procedure: Procedure:
Scale N PBB-EVI's in DUT,SHPE3 and MHPE2.Send F frames to DUT from CE Scale N PBB-EVPN instances in DUT,SHPE3 and MHPE2.Send F frames to
using traffic generator with X different SA and DA for N EVI's. Send DUT from CE using traffic generator with X different source and
F frames from traffic generator to SHPE3 with X different SA and DA. destination mac addresses for N PBB-EVPN instances. Send F frames
There will be 2X number of MAC address will be learned in DUT PBB- from traffic generator to SHPE3 with X different source and
EVPN MAC table. There is a bi directional traffic flow with F pps in destination mac addresses.There will be 2X MAC addresses will be
Each direction. The DUT must run with traffic for 24 hours, every learned in DUT PBB- EVPN MAC table.There is a bi directional
hour check the memory leak, crashes. traffic flow with F pps in Each direction.The DUT must run with
traffic for 24 hours, every hour check the memory leak, crashes.
Measurement : Measurement :
Take the hourly reading of CPU process, memory usages. There should Take the hourly reading of CPU, process memory. There should not be
not be any memory leak, crashes,CPU spikes. any leak, crashes, CPU spikes.The CPU spike is determined as the CPU
usage which shoots at 40 to 50 percent of the average usage.The
average value vary from device to device.Memory leak is determined
by increase usage of the memory for PBB-EVPN process.The
expectation is under steady state the memory usage for PBB-EVPN process
should not increase.
5. Acknowledgements 5. Acknowledgments
We would like to thank Fioccola Giuseppe of Telecom Italia reviewing We would like to thank Fioccola Giuseppe of Telecom Italia reviewing
our draft and commenting it. We would like to thank Sarah Banks for our draft and commenting it.We would like to thank Sarah Banks for
guiding and mentoring us. guiding and mentoring us.
6. IANA Considerations 6. IANA Considerations
This memo includes no request to IANA. This memo includes no request to IANA.
7. Security Considerations 7. Security Considerations
There is no additional consideration from RFC 6192. The benchmarking tests described in this document are limited to the
performance characterization of controllers in a lab environment with
isolated networks. The benchmarking network topology will be an
independent test setup and MUST NOT be connected to devices that may
forward the test traffic into a production network or misroute
traffic to the test management network. Further, benchmarking is
performed on a "black-box" basis, relying solely on measurements
observable external to the controller. Special capabilities SHOULD
NOT exist in the controller specifically for benchmarking purposes.
Any implications for network security arising from the controller
SHOULD be identical in the lab and in production networks.
8. References 8. References
8.1. Normative References 8.1. Normative References
[RFC2119] Bradner, S., "Key words for use in RFCs to Indicate [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate
Requirement Levels", BCP 14, RFC 2119, Requirement Levels", BCP 14, RFC 2119,
DOI 10.17487/RFC2119, March 1997, DOI 10.17487/RFC2119, March 1997,
<https://www.rfc-editor.org/info/rfc2119>. <https://www.rfc-editor.org/info/rfc2119>.
 End of changes. 106 change blocks. 
302 lines changed or deleted 385 lines changed or added

This html diff was produced by rfcdiff 1.47. The latest version is available from http://tools.ietf.org/tools/rfcdiff/