draft-ietf-bmwg-evpntest-04.txt   draft-ietf-bmwg-evpntest-05.txt 
Internet Engineering Task Force S. Jacob, Ed. Internet Engineering Task Force S. Jacob, Ed.
Internet-Draft K. Tiruveedhula Internet-Draft K. Tiruveedhula
Intended status: Informational Juniper Networks Intended status: Informational Juniper Networks
Expires: June 20, 2020 December 18, 2019 Expires: September 12, 2020 March 11, 2020
Benchmarking Methodology for EVPN and PBB-EVPN Benchmarking Methodology for EVPN and PBB-EVPN
draft-ietf-bmwg-evpntest-04 draft-ietf-bmwg-evpntest-05
Abstract Abstract
This document defines methodologies for benchmarking EVPN and PBB- This document defines methodologies for benchmarking EVPN and PBB-
EVPN performance.EVPN is defined in RFC 7432, and is being deployed EVPN performance.EVPN is defined in RFC 7432, and is being deployed
in Service Provider networks.Specifically, this document defines in Service Provider networks.Specifically, this document defines
the methodologies for benchmarking EVPN/PBB-EVPN convergence, data the methodologies for benchmarking EVPN/PBB-EVPN convergence, data
plane and control plane performance. plane performance, and control plane performance.
Status of This Memo Status of This Memo
This Internet-Draft is submitted in full conformance with the This Internet-Draft is submitted in full conformance with the
provisions of BCP 78 and BCP 79. provisions of BCP 78 and BCP 79.
Internet-Drafts are working documents of the Internet Engineering Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF). Note that other groups may also distribute Task Force (IETF). Note that other groups may also distribute
working documents as Internet-Drafts. The list of current Internet- working documents as Internet-Drafts. The list of current Internet-
Drafts is at https://datatracker.ietf.org/drafts/current/. Drafts is at https://datatracker.ietf.org/drafts/current/.
Internet-Drafts are draft documents valid for a maximum of six months Internet-Drafts are draft documents valid for a maximum of six months
and may be updated, replaced, or obsoleted by other documents at any and may be updated, replaced, or obsoleted by other documents at any
time. It is inappropriate to use Internet-Drafts as reference time. It is inappropriate to use Internet-Drafts as reference
material or to cite them other than as "work in progress." material or to cite them other than as "work in progress."
This Internet-Draft will expire on June 20, 2020. This Internet-Draft will expire on September 12, 2020.
Copyright Notice Copyright Notice
Copyright (c) 2019 IETF Trust and the persons identified as the Copyright (c) 2020 IETF Trust and the persons identified as the
document authors. All rights reserved. document authors. All rights reserved.
This document is subject to BCP 78 and the IETF Trust's Legal This document is subject to BCP 78 and the IETF Trust's Legal
Provisions Relating to IETF Documents Provisions Relating to IETF Documents
(https://trustee.ietf.org/license-info) in effect on the date of (https://trustee.ietf.org/license-info) in effect on the date of
publication of this document. Please review these documents publication of this document. Please review these documents
carefully, as they describe your rights and restrictions with respect carefully, as they describe your rights and restrictions with respect
to this document. Code Components extracted from this document must to this document. Code Components extracted from this document must
include Simplified BSD License text as described in Section 4.e of include Simplified BSD License text as described in Section 4.e of
the Trust Legal Provisions and are provided without warranty as the Trust Legal Provisions and are provided without warranty as
described in the Simplified BSD License. described in the Simplified BSD License.
Table of Contents Table of Contents
1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 2 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 2
1.1. Requirements Language . . . . . . . . . . . . . . . . . . 3 1.1. Requirements Language . . . . . . . . . . . . . . . . . . 3
1.2. Terminologies . . . . . . . . . . . . . . . . . . . . . . 3 1.2. Terminologies . . . . . . . . . . . . . . . . . . . . . . 3
2. Test Topology . . . . . . . . . . . . . . . . . . . . . . . . 4 2. Test Topology . . . . . . . . . . . . . . . . . . . . . . . . 4
3. Test Cases for EVPN Benchmarking . . . . . . . . . . . . . . 7 3. Test Cases for EVPN Benchmarking . . . . . . . . . . . . . . 7
3.1. Local MAC Learning . . . . . . . . . . . . . . . . . . . 7 3.1. Data Plane MAC Learning . . . . . . . . . . . . . . . . . 7
3.2. Remote MAC Learning . . . . . . . . . . . . . . . . . . . 8 3.2. Control Plane MAC Learning . . . . . . . . . . . . . . . 8
3.3. MAC Flush due to local link failure and Relearning . . . 8 3.3. MAC Flush-Local Link Failure and Relearning . . . . . . . 9
3.4. MAC Flush due to remote link failure. . . . . . . . . . . 9 3.4. MAC Flush-Remote Link Failure and Relearning. . . . . . . 10
3.5. MAC Aging . . . . . . . . . . . . . . . . . . . . . . . . 9 3.5. MAC Aging . . . . . . . . . . . . . . . . . . . . . . . . 11
3.6. Remote Mac Aging . . . . . . . . . . . . . . . . . . . . 10 3.6. Remote MAC Aging . . . . . . . . . . . . . . . . . . . . 12
3.7. Local and Remote MAC Learning . . . . . . . . . . . . . . 10 3.7. Control and Data plane MAC Learning . . . . . . . . . . . 12
3.8. High Availability. . . . . . . . . . . . . . . . . . . . 11 3.8. High Availability. . . . . . . . . . . . . . . . . . . . 13
3.9. ARP/ND Scale . . . . . . . . . . . . . . . . . . . . . . 12 3.9. ARP/ND Scale . . . . . . . . . . . . . . . . . . . . . . 14
3.10. Scaling of Services . . . . . . . . . . . . . . . . . . . 12 3.10. Scaling of Services . . . . . . . . . . . . . . . . . . . 15
3.11. Scale Convergence . . . . . . . . . . . . . . . . . . . . 13 3.11. Scale Convergence . . . . . . . . . . . . . . . . . . . . 16
3.12. SOAK Test. . . . . . . . . . . . . . . . . . . . . . . . 14 3.12. SOAK Test. . . . . . . . . . . . . . . . . . . . . . . . 17
4. Test Cases for PBB-EVPN Benchmarking . . . . . . . . . . . . 14 4. Test Cases for PBB-EVPN Benchmarking . . . . . . . . . . . . 18
4.1. Local MAC Learning . . . . . . . . . . . . . . . . . . . 14 4.1. Data Plane Local MAC Learning . . . . . . . . . . . . . . 18
4.2. Remote Mac Learning . . . . . . . . . . . . . . . . . . . 15 4.2. Data Plane Remote MAC Learning . . . . . . . . . . . . . 18
4.3. MAC Flush due to link failure . . . . . . . . . . . . . . 15 4.3. MAC Flush-Local Link Failure . . . . . . . . . . . . . . 19
4.4. MAC Flush due to remote Failure . . . . . . . . . . . . . 16 4.4. MAC Flush-Remote Link Failure . . . . . . . . . . . . . . 20
4.5. MAC aging . . . . . . . . . . . . . . . . . . . . . . . . 17 4.5. MAC Aging . . . . . . . . . . . . . . . . . . . . . . . . 21
4.6. Remote MAC Aging. . . . . . . . . . . . . . . . . . . . . 17 4.6. Remote MAC Aging. . . . . . . . . . . . . . . . . . . . . 22
4.7. Local and Remote MAC Learning . . . . . . . . . . . . . . 18 4.7. Local and Remote MAC Learning . . . . . . . . . . . . . . 23
4.8. High Availability . . . . . . . . . . . . . . . . . . . . 18 4.8. High Availability . . . . . . . . . . . . . . . . . . . . 23
4.9. Scale . . . . . . . . . . . . . . . . . . . . . . . . . . 19 4.9. Scale . . . . . . . . . . . . . . . . . . . . . . . . . . 24
4.10. Scale Convergence . . . . . . . . . . . . . . . . . . . . 19 4.10. Scale Convergence . . . . . . . . . . . . . . . . . . . . 25
4.11. Soak Test . . . . . . . . . . . . . . . . . . . . . . . . 20 4.11. Soak Test . . . . . . . . . . . . . . . . . . . . . . . . 26
5. Acknowledgments . . . . . . . . . . . . . . . . . . . . . . 21 5. Acknowledgments . . . . . . . . . . . . . . . . . . . . . . 27
6. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 21 6. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 27
7. Security Considerations . . . . . . . . . . . . . . . . . . . 21 7. Security Considerations . . . . . . . . . . . . . . . . . . . 27
8. References . . . . . . . . . . . . . . . . . . . . . . . . . 21 8. References . . . . . . . . . . . . . . . . . . . . . . . . . 27
8.1. Normative References . . . . . . . . . . . . . . . . . . 21 8.1. Normative References . . . . . . . . . . . . . . . . . . 27
8.2. Informative References . . . . . . . . . . . . . . . . . 22 8.2. Informative References . . . . . . . . . . . . . . . . . 28
Appendix A. Appendix . . . . . . . . . . . . . . . . . . . . . . 22 Appendix A. Appendix . . . . . . . . . . . . . . . . . . . . . . 28
Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 22 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 28
1. Introduction 1. Introduction
EVPN is defined in RFC 7432, and describes BGP MPLS- based Ethernet EVPN is defined in RFC 7432, and describes BGP MPLS based Ethernet
VPNs (EVPN).PBB-EVPN is defined in RFC 7623, discusses how Ethernet VPNs (EVPN). PBB-EVPN is defined in RFC 7623, discusses how Ethernet
Provider backbone Bridging can be combined with EVPNs to provide a Provider backbone Bridging can be combined with EVPNs to provide a
new/combined solution.This draft defines methodologies that can be new/combined solution. This draft defines methodologies that can be
used to benchmark both RFC 7432 and RFC 7623 solutions.Further, used to benchmark both RFC 7432 and RFC 7623 solutions.Further,
this draft provides methodologies for benchmarking the performance of this draft provides methodologies for benchmarking the performance of
EVPN data and control planes, MAC learning, MAC flushing, MAC aging, EVPN data and control planes, MAC learning, MAC flushing, MAC aging,
convergence, high availability, and scale. convergence, high availability, and scale.
1.1. Requirements Language 1.1. Requirements Language
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
"SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
document are to be interpreted as described in RFC 8174 [RFC8174]. document are to be interpreted as described in RFC 8174 [RFC8174].
1.2. Terminologies 1.2. Terminologies
All-Active Redundancy Mode: When all PEs attached to an Ethernet All-Active Redundancy Mode: When all PEs attached to an Ethernet
segment are allowed to forward known unicast traffic to/from that segment are allowed to forward known unicast traffic to/from that
Ethernet segment for a given VLAN, then the Ethernet segment is Ethernet segment for a given VLAN, then the Ethernet segment is
defined to be operating in All-Active redundancy mode. defined to be operating in All-Active redundancy mode.
AA All Active mode AA: All Active mode
CE Customer Router/Devices/Switch. CE: Customer Router/Devices/Switch.
DF Designated Forwarder DF: Designated Forwarder
DUT Device under test. DUT: Device under test.
Ethernet Segment (ES): When a customer site (device or network) is Ethernet Segment (ES): When a customer site (device or network) is
connected to one or more PEs via a set of Ethernet links, then that connected to one or more PEs via a set of Ethernet links, then that
set of links is referred to as an 'Ethernet segment'. set of links is referred to as an 'Ethernet segment'.
EVI: An EVPN instance spanning the Provider Edge (PE) devices EVI: An EVPN instance spanning the Provider Edge (PE) devices
participating in that EVPN. participating in that EVPN.
Ethernet Segment Identifier (ESI): A unique non-zero identifier that Ethernet Segment Identifier (ESI): A unique non-zero identifier that
identifies an Ethernet segment is called an 'Ethernet Segment identifies an Ethernet segment is called an 'Ethernet Segment
Identifier'. Identifier'.
Ethernet Tag: An Ethernet tag identifies a particular broadcast Ethernet Tag: An Ethernet tag identifies a particular broadcast
domain, e.g., a VLAN. An EVPN instance consists of one or more domain, e.g., a VLAN. An EVPN instance consists of one or more
broadcast domains. broadcast domains.
Interface Physical interface of a router/switch. Interface: Physical interface of a router/switch.
IRB Integrated routing and bridging interface IRB: Integrated routing and bridging interface
MAC Media Access Control addresses on a PE. MAC: Media Access Control addresses on a PE.
MHPE2 Multi homed Provider Edge router 2. MHPE2: Multi homed Provider Edge router 2.
MHPE1 Multi homed Provider Edge router 1. MHPE1: Multi homed Provider Edge router 1.
SHPE3 Single homed Provider Edge Router 3. SHPE3: Single homed Provider Edge Router 3.
PE: Provider Edge device. PE: Provider Edge device.
P Provider Router. P: Provider Router.
RR Route Reflector. RR: Route Reflector.
RT Traffic Generator. RT: Traffic Generator.
Sub Interface Each physical Interfaces is subdivided into Logical Sub Interface: Each physical Interfaces is subdivided into Logical
units. units.
SA Single Active SA: Single Active
Single-Active Redundancy Mode: When only a single PE, among all the Single-Active Redundancy Mode: When only a single PE, among all the
PEs attached to an Ethernet segment, is allowed to forward traffic PEs attached to an Ethernet segment, is allowed to forward traffic
to/from that Ethernet segment for a given VLAN, then the Ethernet to/from that Ethernet segment for a given VLAN, then the Ethernet
segment is defined to be operating in Single-Active redundancy mode. segment is defined to be operating in Single-Active redundancy mode.
2. Test Topology 2. Test Topology
EVPN/PBB-EVPN Services running on SHPE3, MHPE1 and MHPE2 in Single There are five routers in the Test setup.SHPE3, RR/P, MHPE1 and
Active Mode: MHPE2 emulating a service provider network.CE is a customer device
connected to MHPE1 and MHPE2, it is configured with bridge domains in
multiple vlans. The traffic generator is connected to CE and
SHPE3.The MHPE1 acts as DUT.The traffic generator will be used as
sender and receiver of traffic.The DUT will be the reference point
for all the test cases. MHPE1 and MHPE2 are mulihome routers
connected to CE running single active mode.The traffic generator
will be generating traffic at 10% of the line rate.
| Traffic Generator acts as a sender/receiver of layer 2 traffic with multiple vlan. +----------------+ +-------------------------+
+----------+ | | | |
| | | | |Traffic Generator sender/|
| SHPE3 | | SHPE3 | |receiver of layer 2 traffic|
| | | +-----------------+ with multiple Vlans |
+----------+ | | +-------------------------+
| +---------+------+
|Core link | Core Link
+----------+ |
| | +--------+-----+
| RR | | |
| | Route Reflector /Provider router | RR/P |
+----------+-------------| | | Core link
| | | +----------------+
| Core links | +--+-----------+ |
+----------+ +-----------+ | |
| | | MHPE2 | | core link |
| DUT | | | | |
| MHPE1 | | | +-------------+---+ ++------------------+
+----------+ +-----------+ | | | |
| PE-CE link | | | | MHPE2 |
+----------+------------ |MHPE1(DUT) | | |
| | | | | |
| CE | | | | |
| layer2 | +-----------------+------+ +-----+-------------------+
|bridge | | |
+----------+------------ Traffic Generator acts as a sender/receiver of layer 2 traffic with multiple vlan. PE-CE link | | PE-CE link
| |
| |
| |
| |
+-----+----------+----+ +----------------------------+
| CE/Layer 2 bridge +-----------| Traffic Generator sender/ |
| | |receiver of layer 2 traffic|
| | | with multiple Vlans |
| | +----------------------------+
| |
+---------------------+
Topology 1 Topology 1
Test Setup Test Setup
Figure 1 Figure 1
+-----------------+---------------------+---------------------+---------------------+----------------------+-----------------------+ +-----------------+---------------------+---------------------+---------------------+----------------------+-----------------------+
| | | | | | | | | | | | | |
| | | | | | | | | | | | | |
| | | | | | | | | | | | | |
| | | | | | | | | | | | | |
| Mode | | | |Receiver | | | Mode | | | |Receiver | |
| | Test |Traffic Direction |Sender | | | | | Test |Traffic Direction |Sender | | |
| | | | | | | | | | | | | |
| | | | | | | | | | | | | |
| | | | | | | | | | | | | |
+----------------------------------------------------------------------------------------------------------------------------------+ +----------------------------------------------------------------------------------------------------------------------------------+
| | | | | | | | | | | | | |
| | | | | SHPE3 | | | | | | | SHPE3 | |
|Single Active | Local Mac | |CE | |Layer 2 traffic | |Single Active | Local MAC | |CE | |Layer 2 traffic |
| | Learning | Uni | | | | | | Learning | Uni | | | |
| | | | | | multiple MAC | | | | | | | multiple MAC |
| | | | | | | | | | | | | |
+-----------------------------------------------------------------------------------------------------------------------------------+ +-----------------------------------------------------------------------------------------------------------------------------------+
| | | | | | | | | | | | | |
|Single Active | Remote MAC | | | CE |Layer 2 traffic | |Single Active | Remote MAC | | | CE |Layer 2 traffic |
| | Learning | uni | SHPE3 | | | | | Learning | uni | SHPE3 | | |
| | | | | |multiple MAC | | | | | | |multiple MAC |
| | | | | | ++ | | | | | | ++
+----------------------------------------------------------------------------------------------------------------------------------+ +----------------------------------------------------------------------------------------------------------------------------------+
| | | | | | | | | | | | | |
|Single Active | Scale Convergence | Bi | | CE/SHPE3 | | |Single Active | Scale Convergence | Bi | | CE/SHPE3 | |
| | | | CE/SHPE3 | |Layer 2 traffic | | | | | CE/SHPE3 | |Layer 2 traffic |
| | Local& Remote | | | |multiple mac& vlans | | | Local& Remote | | | |multiple MAC& vlans |
| | Learning | | | | | | | Learning | | | | |
+-----------------+---------------------+---------------------+--------------------------------------------+-----------------------+ +-----------------+---------------------+---------------------+--------------------------------------------+-----------------------+
| |
++ ++
Table showing Traffic pattern for various test Table showing the traffic directions of various EVPN/PBB-EVPN
benchmarking test cases. Depends on the test scenario the traffic
can be uni/bi directional generated by the traffic generator.
Figure 2 Figure 2
Test Setup Configurations: Test Setup Configurations:
There are five routers in the Test setup.SHPE3, RR/P, MHPE1 and SHPE3 is configured with Interior Gateway protocols like OPSF or IS-
MHPE2 emulating a service provider network. CE is a customer device IS for underlay, LDP for MPLS support,Interior Border Gateway with
connected to MHPE1 and MHPE2, it is configured with bridge domains in EVPN address family for overlay support.This router must be
multiple vlans.The traffic generator is connected to CE and configured with N EVPN/PBB-EVPN instances for testing.Traffic
SHPE3.The MHPE1 acts as DUT.The traffic generator will be used as generator is connected to this router for sending and receiving
sender and receiver of traffic.The measurement will be taken in DUT. traffic.
All routers except CE are configured with OSPF/IS-IS,LDP,MPLS,BGP
with EVPN address family.
All routers except CE are configured with Interior border gateway
protocol,RR acting as route reflector.
MHPE1,MHPE2,SHPE3 must be configured with "N" EVPN/PBB-EVPN instances RR is configured with Interior Gateway protocols like OPSF or IS-IS
depends up on the cases. for underlay, LDP for MPLS support,Interior Border Gateway with EVPN
address family for overlay support.This router acts as a provider
router and as a route reflector.
MHPE1 and MHEPE2 must be configured with ESI per vlan or ESI on MHPE1 is configured with Interior Gateway protocols like OPSF or IS-
interface. IS for underlay, LDP for MPLS support,Interior Border Gateway with
EVPN address family for overlay support.This router must be
configured with N EVPN/PBB-EVPN instances for testing.This router
is configured with ESI per vlan or ESI per interface.It is
functioning as multi homing PE working on Single Active EVPN
mode.This router serves as the DUT and it is connected to CE.MHPE1
is acting as DUT for all the test cases.
MHPE1 and MHEPE2 are running Single Active mode of EVPN. MHPE2 is configured with Interior Gateway protocols like OPSF or IS-
IS for underlay, LDP for MPLS support,Interior Border Gateway with
EVPN address family for overlay support.This router must be
configured with N EVPN/PBB-EVPN instances for testing.This router
is configured with ESI per vlan or ESI per interface.It is
functioning as multi homing PE working on Single Active EVPN mode.
It is connected to CE.
CE is acting as bridge configured with multiple vlans,the same vlans CE is acting as bridge configured with multiple vlans,the same vlans
are configured on MHPE1,MHPE2,SHPE3. are configured on MHPE1,MHPE2,SHPE3.Traffic generator is connected
to CE. the traffic generator acts as sender or receiver of traffic.
Depending up on the test scenarios the traffic generators will be Depending up on the test scenarios the traffic generators will be
used to generate uni directional or bi directional flows. used to generate uni directional or bi directional flows.
The above configuration will be serving as the base configuration for The above configuration will be serving as the base configuration for
all test cases. all test cases.
3. Test Cases for EVPN Benchmarking 3. Test Cases for EVPN Benchmarking
3.1. Local MAC Learning 3.1. Data Plane MAC Learning
Objective: Objective:
To Record the time taken to learn the MAC address locally in DUT. Measure the time taken to learn the Data Plane MAC in DUT.
Topology : Topology 1 Topology : Topology 1
Procedure: Procedure:
The data plane MAC learning can be measured using the parameters The data plane MAC learning can be measured using the parameters
defined in RFC 2889 section 5.8.Send "X" unicast frames from CE to defined in RFC 2889 section 5.8.
MHPE1(DUT) working in SA mode.Traffic generator connected to CE must
send frames with "X" different source and destination MAC address. Confirm the DUT is up and running with EVPN.
The DUT must learn these "X" macs in data plane.
Traffic generator connected to CE must send frames with "X" different
source and destination MAC address for one vlan, the same vlan must
be present in all the devices except RR.
Send "X" unicast frames from CE to MHPE1(DUT) for one EVPN instance
working in SA mode.
The DUT will learn these "X" MAC in data plane.
Measurement : Measurement :
Measure the time taken to learn "X" MACs locally in DUT evpn mac Measure the time taken to learn "X" MAC locally in DUT evpn MAC
table. The data plane measurement is taken by considering DUT as table.The data plane measurement is taken by considering DUT as
black box. The range of MACs are known from traffic generator,the black box. The range of MAC are known from traffic generator,the
same must be learned in DUT, the time taken to learn "X" macs is same must be learned in DUT, the time taken to learn "X" MAC is
measured. measured.The measurement is carried out using external server which
polls the DUT using automated scripts.
The test is repeated for "N" times and the values are collected.The The test is repeated for "N" times and the values are collected.The
mac learning rate is calculated by averaging the values obtained from MAC learning rate is calculated by averaging the values obtained from
"N" samples. "N" samples. "N" is an arbitrary number to get a sufficient sample.
The time measured for each sample is denoted by T1,T2...Tn.
Mac learning rate = (T1+T2+..Tn)/N MAC learning rate = (T1+T2+..Tn)/N
3.2. Remote MAC Learning 3.2. Control Plane MAC Learning
Objective: Objective:
To Record the time taken to learn the remote macs. Measure the time taken to learn the control plane MAC.
Topology : Topology 1 Topology : Topology 1
Procedure: Procedure:
Send X frames with X different source and destination mac addresses Confirm the DUT is up and running with EVPN.
to SHPE3 from traffic generator.SHPE3 will advertise these locally
learned macs to MHPE1 and MHPE2 via control plane.Measure the time Traffic generator connected to SHPE3 must send frames with "X"
taken to learn these X MACs from remote peer in DUT EVPN MAC address different source and destination MAC address for one vlan, the same
table.The DUT and MHPE2 are running SA mode. vlan must be present in all the devices except RR.
Ensure the frames must be destined to one EVPN instance.
The DUT will learn these "X" MAC in control plane.
Measurement : Measurement :
Measure the time taken by the DUT to learn the "X" MACs in the data Measure the time taken by the DUT to learn the "X" MAC in the data
plane.The test is repeated for "N" times and the values are plane.The test is repeated for "N" times and the values are
collected. The remote mac learning rate is calculated by averaging collected. The remote MAC learning rate is calculated by averaging
the values obtained from "N" samples. the values obtained from "N" samples. "N" is an arbitrary number to
get a sufficient sample. The time measured for each sample is
denoted by T1,T2...Tn.The measurement is carried out using external
server which polls the DUT using automated scripts.
Remote mac learning rate = (T1+T2+..Tn)/N MAC learning rate = (T1+T2+..Tn)/N
3.3. MAC Flush due to local link failure and Relearning 3.3. MAC Flush-Local Link Failure and Relearning
Objective: Objective:
To record the time taken to flush the locally learned mac and the Measure the time taken to flush the Data Plane MAC and the time taken
time taken to relearn the same amount of macs. to relearn the same amount of MAC.
Topology : Topology 1 Topology : Topology 1
Procedure: Procedure:
Send X frames with X different source and destination mac addresses Confirm the DUT is up and running with EVPN.
to DUT from CE using traffic generator. Wait till the MHPE1 learns
all X MAC addresses. Then fail the MHPE1 CE link and measure the Send X frames with X different source and destination MAC addresses
time taken to flush these X MACs from the EVPN MAC table. Bring up to DUT from CE using traffic generator for one vlan.
the link which was made Down(the link between MHPE1 and CE).Measure
time taken to relearn it.The DUT and MHPE2 are running SA mode. Ensure the DUT learns all X MAC addresses in data plane.
Fail the DUT-CE link and measure the time taken to flush these X MAC
from the EVPN MAC table.
Bring up the link which was made Down(the link between DUT and
CE).Measure time taken by the DUT to relearn these "X" MAC.
The DUT and MHPE2 are running SA mode.
Measurement : Measurement :
Measure the time taken for flushing these X MAC addresses.Measure Measure the time taken for flushing these X MAC addresses. Measure
the time taken to relearn the X MACs in DUT.The test is repeated for the time taken to relearn these X MAC in DUT.The test is repeated for
"N" times and the values are collected.The flush and the relearning "N" times and the values are collected. The flush and the relearning
time is calculated by averaging the values obtained by "N" samples. time is calculated by averaging the values obtained by "N"
samples."N" is an arbitrary number to get a sufficient sample.The
time measured for each sample is denoted by T1,T2...Tn.The
measurement is carried out using external server which polls the DUT
using automated scripts.
Flush rate = (T1+T2+..Tn)/N Flush rate = (T1+T2+..Tn)/N
Relearning rate = (T1+T2+..Tn)/N Relearning rate = (T1+T2+..Tn)/N
3.4. MAC Flush due to remote link failure. 3.4. MAC Flush-Remote Link Failure and Relearning.
Objective: Objective:
To record the time taken to flush the remote mac learned in DUT Measure the time taken to flush the Control plane MAC learned in DUT
during remote link failure. during remote link failure and the time taken to relearn.
Topology : Topology 1 Topology : Topology 1
Procedure: Procedure:
Send X frames with X different source and destination mac addresses Confirm the DUT is up and running with EVPN.
to DUT from SHPE3 using traffic generator. Bring down the link
between SHPE3 and traffic generator.Then measure the time taken to Send X frames with X different source and destination MAC addresses
flush the DUT EVPN MAC table.The DUT and MHPE2 are running SA mode. to DUT from SHPE3 using traffic generator for one vlan.
Bring down the link between SHPE3 and traffic generator.
SHPE3 will withdraw the routes from DUT due to link failure.
Measure the time taken to flush the DUT EVPN MAC table. The DUT and
MHPE2 are running SA mode.
Bring up the link which was made Down(the link between SHPE3 and
traffic generator).
Measure time taken by the DUT to relearn these "X" MAC from control
plane.
Measurement : Measurement :
Measure the time taken to flush X remote MACs from EVPN MAC table of Measure the time taken to flush X remote MAC from EVPN MAC table of
the DUT.The test is repeated for "N" times and the values are the DUT.Measure the time taken to relearn these X MAC in DUT.The
collected.The flush rate is calculated by averaging the values test is repeated for "N" times and the values are collected.The
obtained by "N" samples. flush rate is calculated by averaging the values obtained by "N"
samples. "N" is an arbitrary number to get a sufficient sample.The
time measured for each sample is denoted by T1,T2...Tn.The
measurement is carried out using external server which polls the DUT
using automated scripts.
Flush rate = (T1+T2+..Tn)/N Flush rate = (T1+T2+..Tn)/N
Relearning rate = (T1+T2+..Tn)/N
3.5. MAC Aging 3.5. MAC Aging
Objective: Objective:
To measure the mac aging time. To measure the MAC aging time.
Topology : Topology 1 Topology : Topology 1
Procedure: Procedure:
Send X frames with X different source and destination mac addresses Confirm the DUT is up and running with EVPN.
to DUT from CE using traffic generator.Wait till X MAC addresses
are learned. Then stop the traffic.Record the time taken to flush Send X frames with X different source and destination MAC addresses
X MACS from DUT EVPN MAC table due to aging.The DUT and MHPE2 are to DUT from CE using traffic generator for one vlan.
running SA mode.
Ensure these X MAC addresses are learned in DUT.
Then stop the traffic.
Ensure the DUT and other devices in the test are using the default
timers for aging.
Measure the time taken to flush X MAC from DUT EVPN MAC table due to
aging.
The DUT and MHPE2 are running SA mode.
Measurement : Measurement :
Measure the time taken to flush X MAC addresses due to aging.The Measure the time taken to flush X MAC addresses due to aging. The
test is repeated for "N" times and the values are collected.The test is repeated for "N" times and the values are collected. The
aging is calculated averaging the values obtained by "N" samples. aging is calculated averaging the values obtained by "N" samples.
"N" is an arbitrary number to get a sufficient sample.The time
measured for each sample is denoted by T1,T2...Tn. The measurement
is carried out using external server which polls the DUT using
automated scripts.
Aging time for X Macs in sec = (T1+T2+..Tn)/N Aging time for X MAC in sec = (T1+T2+..Tn)/N
3.6. Remote Mac Aging 3.6. Remote MAC Aging
Objective: Objective:
To measure the remote mac aging time. Measure the control plane learned MAC aging time.
Topology : Topology 1 Topology : Topology 1
Procedure: Procedure:
Send X frames with X different source and destination mac addresses Confirm the DUT is up and running with EVPN.
to DUT from SHPE3 using traffic generator. Stop the traffic at
remote PE SHPE3.Due to MAC aging SHPE3 will withdraw its routes from Send X frames with X different source and destination MAC addresses
DUT and MHPE2.Measure the time taken to remove these MACs from DUT to DUT from SHPE3 using traffic generator for one vlan.
EVPN MAC table.DUT and MHPE2 are running in SA mode
Ensure these X MAC addresses are learned in DUT via control plane.
Then stop the traffic.
Ensure the DUT and other devices in the test are using the default
timers for aging.
Measure the time taken to flush X MAC from DUT EVPN MAC table due to
aging.
The DUT and MHPE2 are running SA mode.
Measurement : Measurement :
Measure the time taken to flush X remote MACs learned in DUT EVPN MAC Measure the time taken to flush X remote MAC learned in DUT EVPN MAC
table due to aging.The test is repeated for "N" times and the values table due to aging.The test is repeated for "N" times and the values
are collected.The aging is calculated by averaging the values are collected.The aging is calculated by averaging the values
obtained by "N" samples. obtained by "N" samples."N" is an arbitrary number to get a
sufficient sample. The time measured for each sample is denoted by
T1,T2...Tn.The measurement is carried out using external server
which polls the DUT using automated scripts.
Aging time for X Macs in sec = (T1+T2+..Tn)/N Aging time for X MAC in sec = (T1+T2+..Tn)/N
3.7. Local and Remote MAC Learning 3.7. Control and Data plane MAC Learning
Objective: Objective:
To record the time taken to learn both local and remote macs. To record the time taken to learn both local and remote MAC.
Topology : Topology 1 Topology : Topology 1
Procedure: Procedure:
Send X frames with X different source and destination mac addresses Confirm the DUT is up and running with EVPN.
to DUT from SHPE3 using traffic generator.Send X frames with
different source and destination mac addresses from traffic generator Send X frames with X different source and destination MAC addresses
connected to CE.The source and destination addresses of flows must to DUT from SHPE3 using traffic generator for one vlan.
be complimentary to have unicast flows.Measure the time taken by
the DUT to learn 2X in EVPN MAC.DUT and MHPE2 are running in SA Send X frames with different source and destination MAC addresses
mode. from traffic generator connected to CE for one vlan.
The source and destination addresses of flows must be complimentary
to have unicast flows.
Measure the time taken by the DUT to learn 2X in EVPN MAC table.
DUT and MHPE2 are running in SA mode.
Measurement : Measurement :
Measure the time taken to learn 2X MAC addresses in DUT EVPN MAC Measure the time taken to learn 2X MAC addresses in DUT EVPN MAC
table. The test is repeated for "N" times and the values are table.The test is repeated for "N" times and the values are
collected. The mac learning time is calculated by averaging the collected. The MAC learning time is calculated by averaging the
values obtained by "N" samples. values obtained by "N" samples."N" is an arbitrary number to get a
sufficient sample.The time measured for each sample is denoted by
T1,T2...Tn. The measurement is carried out using external server
which polls the DUT using automated scripts
Mac learning rate = (T1+T2+..Tn)/N MAC learning rate = (T1+T2+..Tn)/N
3.8. High Availability. 3.8. High Availability.
Objective: Objective:
To record traffic loss during routing engine fail over. Measure traffic loss during routing engine fail over.
Topology : Topology 1 Topology : Topology 1
Procedure: Procedure:
Confirm the DUT is up and running with EVPN.
Send X frames from CE to DUT from traffic generator with X different Send X frames from CE to DUT from traffic generator with X different
source and destination mac addresses.Send X frames from traffic source and destination MAC addresses.
generator to SHPE3 with X different source and destination mac
addresses, so that 2X MAC address will be learned in the DUT.There Send X frames from traffic generator to SHPE3 with X different source
is a bi directional traffic flow with X pps in each direction.Then and destination MAC addresses, so that 2X MAC address will be learned
do a routing engine fail-over. in the DUT.
There is a bi directional traffic flow with X pps in each direction.
Ensure the DUT learn 2X MAC.
Then do a routing engine fail-over.
Measurement : Measurement :
The expectation of the test is 0 traffic loss with no change in the The expectation of the test is 0 traffic loss with no change in the
DF role.DUT should not withdraw any routes.But in cases where the DF role. DUT should not withdraw any routes.But in cases where the
DUT is not property synchronized between master and standby,due to DUT is not property synchronized between master and standby,due to
that packet loss are observed.In that scenario the packet loss is that packet loss are observed. In that scenario the packet loss is
measured.The test is repeated for "N" times and the values are measured.The test is repeated for "N" times and the values are
collected.The packet loss is calculated by averaging the values collected.The packet loss is calculated by averaging the values
obtained by "N" samples. obtained by "N" samples."N" is an arbitrary number to get a
sufficient sample. The time measured for each sample is denoted by
T1,T2...Tn. The measurement is carried out using external server
which polls the DUT using automated scripts to ensure the DUT learned
2X MAC. The packet drop is measured using traffic generator.
Packet loss in sec with 2X mac addresses = (T1+T2+..Tn)/N Packet loss in sec with 2X MAC addresses = (T1+T2+..Tn)/N
3.9. ARP/ND Scale 3.9. ARP/ND Scale
To Record the DUT scaling limit of ARP/ND. Measure the DUT scaling limit of ARP/ND.
Objective: Objective:
To Record the ARP/ND scale of the DUT. Measure the ARP/ND scale of the DUT.
Topology : Topology 1 Topology : Topology 1
Procedure: Procedure:
Confirm the DUT is up and running with EVPN.
Send X arp/neighbor discovery(ND) from the traffic generator to DUT Send X arp/neighbor discovery(ND) from the traffic generator to DUT
with different sender ip/ipv6,mac addresses to the target IRB address with different sender ip/ipv6,MAC addresses to the target IRB address
configured in EVPN instance. The EVPN instance learns the mac+ip and configured in EVPN instance.
mac+ipv6 addresses from these request and advertise as type 2 mac+ip/
mac+ipv6 route to remote provide edge routers which have same EVPN The EVPN instance learns the MAC+ip and MAC+ipv6 addresses from these
configurations.The value of X must be increased at a incremental request and advertise as type 2 MAC+ip/MAC+ipv6 route to remote
value of 5% of X, till the limit is reached.The limit is where the provide edge routers which have same EVPN configurations.
DUT cant learn any more type 2 mac+ip/mac+ipv6.The test must be
separately conducted for arp and ND. The value of X must be increased at a incremental value of 5% of X,
till the limit is reached.The limit is where the DUT cant learn any
more type 2 MAC+ip/MAC+ipv6.The test must be separately conducted for
arp and ND.
Measurement : Measurement :
Measure the scale limit of type 2 mac+ip/mac+ipv6 route which DUT can Measure the scale limit of type 2 MAC+ip/MAC+ipv6 route which DUT can
learn.The test is repeated for "N" times and the values are learn.The test is repeated for "N" times and the values are
collected.The scale limit is calculated by averaging the values collected.The scale limit is calculated by averaging the values
obtained by "N" samples for both mac+ip and mac+ipv6. obtained by "N" samples for both MAC+ip and MAC+ipv6."N" is an
arbitrary number to get a sufficient sample.The scale value
obtained by each sample be v1,v2..vn.The measurement is carried out
using external server which polls the DUT using automated scripts to
find the scale limit of MAC+ipv4/MAC+ipv6.
DUT scale limit for mac+ip = (v1+v2+..vn)/N Scale limit for MAC+ip = (v1+v2+..vn)/N
DUT scale limit for mac+ipv6 = (v1+v2+..vn)/N Scale limit for MAC+ipv6 = (v1+v2+..vn)/N
3.10. Scaling of Services 3.10. Scaling of Services
Objective: Objective:
This test is to measure the scale of EVPN instances that a DUT can Measure the scale of EVPN instances that a DUT can hold.
hold.
Topology : Topology 1 Topology : Topology 1
Procedure: Procedure:
The DUT,MHPE2 and SHPE3 are scaled to "N" EVI.Clear BGP neighbors of Confirm the DUT is up and running with EVPN.
the DUT.Once adjacency is established in the DUT, check the
routes received from MHPE2 and SHPE3 for "N" EVI in the DUT.Then The DUT,MHPE2 and SHPE3 are scaled to "N" EVI.
increment the scale of N by 5% of N till the limit is reached.The
limit is where the DUT cant learn any EVPN routes from peers. Ensure routes received from MHPE2 and SHPE3 for "N" EVI in the DUT.
Then increment the scale of N by 5% of N till the limit is reached.
The limit is where the DUT cant learn any EVPN routes from its peers.
Measurement : Measurement :
There should not be any loss of route types 1,2,3 and 4 in DUT.DUT There should not be any loss of route types 1,2,3 and 4 in DUT. DUT
must relearn all type 1,2,3 and 4 from remote routers. The DUT must must relearn all type 1,2,3 and 4 from remote routers.The DUT must
be subjected to various values of N to find the optimal scale limit. be subjected to various values of N to find the optimal scale limit.
The scope of the test is find out the maximum evpn instance that a The scope of the test is find out the maximum evpn instance that a
DUT can hold. DUT can hold.The measurement is carried out using external server
which polls the DUT using automated scripts to find the scale limit
of EVPN instances.
3.11. Scale Convergence 3.11. Scale Convergence
Objective: Objective:
To measure the convergence time of DUT when the DUT is scaled with Measure the convergence time of DUT when the DUT is scaled with EVPN
EVPN instance along with traffic. instance along with traffic.
Topology : Topology 1 Topology : Topology 1
Procedure: Procedure:
Scale N EVIs in DUT,SHPE3 and MHPE2.Send F frames to DUT from CE Confirm the DUT is up and running with EVPN.
using traffic generator with X different source and destination mac
addresses for N EVI's.Send F frames from traffic generator to SHPE3 Scale N EVIs in DUT,SHPE3 and MHPE2.
with X different source and destination mac addresses.There will be
2X number of MAC addresses will be learned in DUT EVPN MAC table. Send F frames to DUT from CE using traffic generator with X different
source and destination MAC addresses for N EVI's.
Send F frames from traffic generator to SHPE3 with X different source
and destination MAC addresses.
There will be 2X number of MAC addresses will be learned in DUT EVPN
MAC table.
There is a bi directional traffic flow with F pps in each direction. There is a bi directional traffic flow with F pps in each direction.
Then clear the BGP neighbors in the DUT.Once the adjacency is
restored in DUT.Measure the time taken to learn 2X MAC address in Then clear the BGP neighbors in the DUT.
DUT MAC table.
Once the BGP session is in established state in DUT.
Measure the time taken to learn 2X MAC address in DUT MAC table.
Measurement : Measurement :
The DUT must learn 2X MAC addresses. Measure the time taken to learn The DUT must learn 2X MAC addresses. Measure the time taken to learn
2X MAC in DUT.The test is repeated for "N" times and the values are 2X MAC in DUT.The test is repeated for "N" times and the values are
collected. The convergence time is calculated by averaging the collected. The convergence time is calculated by averaging the
values obtained by "N" samples. values obtained by "N" samples."N" is an arbitrary number to get a
sufficient sample.The time measured for each sample is denoted by
T1,T2...Tn.The measurement is carried out using external server which
polls the DUT using automated scripts.
Time taken to learn 2X macs in DUT = (T1+T2+..Tn)/N Time taken to learn 2X MAC in DUT = (T1+T2+..Tn)/N
3.12. SOAK Test. 3.12. SOAK Test.
Objective: Objective:
This test is carried out to measure the stability of the DUT in a This test is carried out to measure the stability of the DUT in a
scaled environment with traffic over a period of time "T'". In each scaled environment with traffic over a period of time "T'". In each
interval "t1" the DUT CPU usage, memory usage are measured.The DUT interval "t1" the DUT CPU usage, memory usage are measured.The DUT
is checked for any crashes during this time period. is checked for any crashes during this time period.
Topology : Topology 1 Topology : Topology 1
Procedure: Procedure:
Confirm the DUT is up and running with EVPN.
Scale N EVI's in DUT,SHPE3 and MHPE2.Send F frames to DUT from CE Scale N EVI's in DUT,SHPE3 and MHPE2.Send F frames to DUT from CE
using traffic generator with different X source and destination mac using traffic generator with different X source and destination MAC
addresses for N EVI's.Send F frames from traffic generator to SHPE3 addresses for N EVI's.
with X different source and destination mac addresses.There will be
2X number of MAC addresses will be learned in DUT EVPN MAC table. Send F frames from traffic generator to SHPE3 with X different source
and destination MAC addresses.
There will be 2X number of MAC addresses will be learned in DUT EVPN
MAC table.
There is a bi directional traffic flow with F pps in each direction. There is a bi directional traffic flow with F pps in each direction.
The DUT must run with traffic for 24 hours, every hour check for
memory leak, CPU usage and crash. The DUT must run with traffic for 24 hours.
Every hour check for memory leak in EVPN process, CPU usage and
crashes in DUT.
Measurement : Measurement :
Take the hourly reading of CPU, process memory. There should not be Take the hourly reading of CPU, process memory. There should not be
any leak, crashes, CPU spikes. Th CPU spike is determined as the CPU any leak, crashes, CPU spikes. Th CPU spike is determined as the CPU
usage which shoots at 40 to 50 percent of the average usage.The usage which shoots at 40 to 50 percent of the average usage. The
average value vary from device to device. Memory leak is determined average value vary from device to device. Memory leak is determined
by increase usage of the memory for EVPN process. The expectation is by increase usage of the memory for EVPN process. The expectation is
under steady state the memory usage for EVPN process should not under steady state the memory usage for EVPN process should not
increase. increase. The measurement is carried out using external server which
polls the DUT using automated scripts which captures the CPU usage
and process memory.
4. Test Cases for PBB-EVPN Benchmarking 4. Test Cases for PBB-EVPN Benchmarking
4.1. Local MAC Learning 4.1. Data Plane Local MAC Learning
Objective: Objective:
To Record the time taken to learn the MAC address locally. Measure the time taken to learn the Data Plane MAC in DUT.
Topology : Topology 1 Topology : Topology 1
Procedure: Procedure:
Send "X" unicast frames from CE to MHPE1(DUT) working in SA mode with Confirm the DUT is up and running with PBB-EVPN.
"X" different source and destination mac addresses from traffic
generator.The DUT must learn "X" macs in data plane. Traffic generator connected to CE must send frames with "X" different
source and destination MAC address for one vlan, the same vlan must
be present in all the devices except RR.
Send "X" unicast frames from CE to MHPE1(DUT) for one PBB-EVPN
instance working in SA mode.
The DUT will learn these "X" MAC in data plane.
Measurement : Measurement :
Measure the time taken by the DUT to learn the "X" MACs in the data Measure the time taken to learn "X" MAC locally in DUT PBB-EVPN MAC
plane. The data plane measurement is taken by considering DUT as table. The data plane measurement is taken by considering DUT as
black box the range of "X" MAC is known from traffic generator and black box. The range of MAC are known from traffic generator,the
the same must be learned in DUT, the time taken to learn "X" MAC is same must be learned in DUT, the time taken to learn "X" MAC is
measured.The test is repeated for "N" times and the values are measured.The measurement is carried out using external server which
collected.The mac learning time is calculated by averaging the polls the DUT using automated scripts.
values obtained from "N" samples.
Mac learning rate = (T1+T2+..Tn)/N The test is repeated for "N" times and the values are collected.The
MAC learning rate is calculated by averaging the values obtained from
"N" samples. "N" is an arbitrary number to get a sufficient sample.
The time measured for each sample is denoted by T1,T2...Tn.
4.2. Remote Mac Learning MAC learning rate = (T1+T2+..Tn)/N
4.2. Data Plane Remote MAC Learning
Objective: Objective:
To Record the time taken to learn the remote macs. To Record the time taken to learn the remote MAC.
Topology : Topology 1 Topology : Topology 1
Procedure: Procedure:
Send X frames with X different source and destination mac addresses Confirm the DUT is up and running with PBB-EVPN.
to SHPE3 from traffic generator.These macs will be flooded to MHPE1
and MHPE2 by SHPE3.The DUT and MHPE2 are running SA mode. Traffic generator connected to SHPE3 must send frames with "X"
different source and destination MAC address for one vlan, the same
vlan must be present in all the devices except RR.
Ensure the frames must be destined to one PBB-EVPN instance.
The DUT will learn these "X" MAC in data plane.
Measurement : Measurement :
Measure the time taken to learn X mac addresses in DUT mac table. Measure the time taken by the DUT to learn the "X" MAC in the data
The test is repeated for "N" times and the values are collected.The plane.The test is repeated for "N" times and the values are
mac learning time is calculated by averaging the values obtained by collected. The remote MAC learning rate is calculated by averaging
"N" samples. the values obtained from "N" samples. "N" is an arbitrary number to
get a sufficient sample. The time measured for each sample is
denoted by T1,T2...Tn.The measurement is carried out using external
server which polls the DUT using automated scripts.
Remote Mac learning rate = (T1+T2+..Tn)/N MAC learning rate = (T1+T2+..Tn)/N
4.3. MAC Flush due to link failure 4.3. MAC Flush-Local Link Failure
Objective: Objective:
To record the time taken to flush the locally learned mac and the Measure the time taken to flush the locally learned MAC and the time
time taken to relearn the same amount of macs. taken to relearn the same amount of MAC.
Topology : Topology 1 Topology : Topology 1
Procedure: Procedure:
Send X frames with X different source and destination mac addresses Confirm the DUT is up and running with PBB-EVPN.
to DUT from CE using traffic generator.Wait till the MHPE1 learn
all X MAC address.Then fail the MHPE1 CE link and measure the time Send X frames with X different source and destination MAC addresses
taken to flush these X MACs from the PBB-EVPN MAC table.Then bring to DUT from CE using traffic generator for one vlan.
up the link.Measure the time taken to relearn X MACS.The DUT and
MHPE2 are running SA mode. Ensure the DUT learns all X MAC addresses in data plane.
Fail the DUT-CE link and measure the time taken to flush these X MAC
from the PBB-EVPN MAC table.
Bring up the link which was made Down(the link between DUT and
CE).Measure time taken by the DUT to relearn these "X" MAC.
The DUT and MHPE2 are running SA mode.
Measurement : Measurement :
Measure the time taken for flushing these X MAC addresses.Measure Measure the time taken for flushing these X MAC addresses. Measure
the time taken to relearn the X MACs in DUT.The test is repeated for the time taken to relearn these X MAC in DUT.The test is repeated for
"N" times and the values are collected. The flush and the relearning "N" times and the values are collected. The flush and the relearning
time is calculated by averaging the values obtained by "N" samples. time is calculated by averaging the values obtained by "N"
samples."N" is an arbitrary number to get a sufficient sample. The
time measured for each sample is denoted by T1,T2...Tn. The
measurement is carried out using external server which polls the DUT
using automated scripts.
Flush rate = (T1+T2+..Tn)/N Flush rate = (T1+T2+..Tn)/N
Relearning rate = (T1+T2+..Tn)/N Relearning rate = (T1+T2+..Tn)/N
4.4. MAC Flush due to remote Failure 4.4. MAC Flush-Remote Link Failure
Objective: Objective:
To record the time taken to flush the remote mac learned in DUT Measure the time taken to flush the remote MAC learned in DUT due to
during remote link failure. remote link failure and relearning it.
Topology : Topology 1 Topology : Topology 1
Procedure: Procedure:
Send X frames with X different source and destination mac addresses confirm the DUT is up and running with PBB-EVPN.
to DUT from SHPE3 using traffic generator.Bring down the link
between SHPE3 and traffic generator. Then measure the time taken to Send X frames with X different source and destination MAC addresses
flush the DUT PBB-EVPN MAC address table.The remote MACs will be to DUT from SHPE3 using traffic generator for one vlan.
learned by Data plane, but the B-MAC will be learned by control
plane.The DUT and MHPE2 are running SA mode. Bring down the link between SHPE3 and traffic generator.
Measure the time taken to flush the DUT PBB-EVPN MAC table. The DUT
and MHPE2 are running SA mode.
Bring up the link which was made Down(the link between SHPE3 and
traffic generator).
Measure time taken by the DUT to relearn these "X" MAC
Measurement : Measurement :
Measure the time taken to flush X remote MACs from PBB-EVPN MAC table Measure the time taken to flush X remote MAC from PBB-EVPN MAC table
of DUT.Repeat these test and plot the data.The test is repeated of the DUT.Measure the time taken to relearn these X MAC in DUT. The
for "N" times and the values are collected. The flush rate is test is repeated for "N" times and the values are collected. The
calculated by averaging the values obtained by "N" samples. flush rate is calculated by averaging the values obtained by "N"
samples. "N" is an arbitrary number to get a sufficient sample. The
time measured for each sample is denoted by T1,T2...Tn.The
measurement is carried out using external server which polls the DUT
using automated scripts.
Flush rate = (T1+T2+..Tn)/N Flush rate = (T1+T2+..Tn)/N
4.5. MAC aging Relearning rate = (T1+T2+..Tn)/N
4.5. MAC Aging
Objective: Objective:
To measure the mac aging time. Measure the MAC aging time.
Topology : Topology 1 Topology : Topology 1
Procedure: Procedure:
Send X frames with X different source and destination mac addresses Confirm the DUT is up and running with PBB-EVPN.
to DUT from CE using traffic generator.Wait till X MAC address are
learned in DUT PBB-EVPN MAC table.Then stop the traffic.Record Send X frames with X different source and destination MAC addresses
the time taken to flush X MAC entries due to aging. The DUT and to DUT from CE using traffic generator for one vlan.
MHPE2 running in SA mode
Ensure these X MAC addresses are learned in DUT.
Then stop the traffic.
Ensure the DUT and other devices in the test are using the default
timers for aging.
Measure the time taken to flush X MAC from DUT PBB-EVPN MAC table due
to aging.
The DUT and MHPE2 are running SA mode.
Measurement : Measurement :
Measure the time taken to flush X MAC addresses due to aging.Repeat Measure the time taken to flush X MAC addresses due to aging. The
these test and plot the data.The test is repeated for "N" times and test is repeated for "N" times and the values are collected. The
the values are collected.The aging is calculated by averaging the aging is calculated averaging the values obtained by "N" samples.
values obtained by "N" samples. "N" is an arbitrary number to get a sufficient sample. The time
measured for each sample is denoted by T1,T2...Tn. The measurement
is carried out using external server which polls the DUT using
automated scripts.
Aging time for X Macs in sec = (T1+T2+..Tn)/N Aging time for X MAC in sec = (T1+T2+..Tn)/N
4.6. Remote MAC Aging. 4.6. Remote MAC Aging.
Objective: Objective:
To measure the remote mac aging time. Measure the remote MAC aging time.
Topology : Topology 1 Topology : Topology 1
Procedure: Procedure:
Send X frames with X different source and destination mac addresses Confirm the DUT is up and running with PBB-EVPN.
to DUT from SHPE3 using traffic generator.Stop the traffic at
remote PE(SHPE3).Measure the time taken to remove these remote MACs Send X frames with X different source and destination MAC addresses
from DUT PBB-EVPN MAC table.The DUT and MHPE2 are running in SA to DUT from SHPE3 using traffic generator for one vlan.
mode.
Ensure these X MAC addresses are learned in DUT.
Then stop the traffic.
Ensure the DUT and other devices in the test are using the default
timers for aging.
Measure the time taken to flush X MAC from DUT PBB-EVPN MAC table due
to aging.
The DUT and MHPE2 are running SA mode.
Measurement : Measurement :
Measure the time taken to flush the X remote MACs from DUT PBB-EVPN Measure the time taken to flush X remote MAC learned in DUT EVPN MAC
MAC table due to aging .The test is repeated for "N" times and the table due to aging.The test is repeated for "N" times and the values
values are collected. The aging is calculated by averaging the are collected. The aging is calculated by averaging the values
values obtained by "N" samples. obtained by "N" samples."N" is an arbitrary number to get a
sufficient sample. The time measured for each sample is denoted by
T1,T2...Tn. The measurement is carried out using external server
which polls the DUT using automated scripts.
Aging time for X Macs in sec = (T1+T2+..Tn)/N Aging time for X MAC in sec = (T1+T2+..Tn)/N
4.7. Local and Remote MAC Learning 4.7. Local and Remote MAC Learning
Objective: Objective:
To record the time taken to learn both local and remote macs. Measure the time taken to learn both local and remote MAC.
Topology : Topology 1 Topology : Topology 1
Procedure: Procedure:
Send X frames with X different source and destination mac addresses Confirm the DUT is up and running with PBB-EVPN.
to DUT from SHPE3 using traffic generator.Send X frames with
different source and destination mac addresses from traffic generator Send X frames with X different source and destination MAC addresses
connected to CE.The source and destination mac addresses of flows to DUT from SHPE3 using traffic generator for one vlan.
must be complimentary to have unicast flows.Measure the time taken
by the DUT to learn 2X in MAC table.DUT and MHPE2 are running in SA Send X frames with different source and destination MAC addresses
mode. from traffic generator connected to CE for one vlan.
The source and destination addresses of flows must be complimentary
to have unicast flows.
Measure the time taken by the DUT to learn 2X in PBB-EVPN MAC table.
DUT and MHPE2 are running in SA mode.
Measurement : Measurement :
Measure the time taken to learn 2X MAC addresses table in DUT PBB- Measure the time taken to learn 2X MAC addresses in DUT PBB-EVPN MAC
EVPN MAC table.The test is repeated for "N" times and the values table. The test is repeated for "N" times and the values are
are collected.The mac learning time is calculated by averaging collected. The MAC learning time is calculated by averaging the
the values obtained by "N" samples. values obtained by "N" samples."N" is an arbitrary number to get a
sufficient sample. The time measured for each sample is denoted by
T1,T2...Tn. The measurement is carried out using external server
which polls the DUT using automated scripts
Mac learning rate = (T1+T2+..Tn)/N MAC learning rate = (T1+T2+..Tn)/N
4.8. High Availability 4.8. High Availability
Objective: Objective:
To record traffic loss during routing engine failover. Measure traffic loss during routing engine failover.
Topology : Topology 1 Topology : Topology 1
Procedure: Procedure:
Send X frames to DUT with X different source and destination mac Confirm the DUT is up and running with PBB-EVPN.
addresses from CE using the traffic generator.Send X frames from
traffic generator to SHPE3 with X different source and destination Send X frames from CE to DUT from traffic generator with X different
mac addresses, so that 2X MAC address will be Learned in DUT. There source and destination MAC addresses.
is a bi directional traffic flow with X pps in each direction. Then
do a routing engine fail-over. Send X frames from traffic generator to SHPE3 with X different source
and destination MAC addresses, so that 2X MAC address will be learned
in the DUT.
There is a bi directional traffic flow with X pps in each direction.
Ensure the DUT learn 2X MAC.
Then do a routing engine fail-over.
Measurement : Measurement :
The expectation of the test is 0 traffic loss with no change in the The expectation of the test is 0 traffic loss with no change in the
DF role.DUT should not withdraw any routes.But in cases where the DF role. DUT should not withdraw any routes.But in cases where the
DUT is not property synchronized between master and standby,due to DUT is not property synchronized between master and standby,due to
that packet loss are observed. In that scenario the packet loss is that packet loss are observed. In that scenario the packet loss is
measured.The test is repeated for "N" times and the values are measured.The test is repeated for "N" times and the values are
collected.The packet loss is calculated by averaging the values collected.The packet loss is calculated by averaging the values
obtained by "N" samples. obtained by "N" samples."N" is an arbitrary number to get a
sufficient sample. The time measured for each sample is denoted by
T1,T2...Tn. The measurement is carried out using external server
which polls the DUT using automated scripts to ensure the DUT learned
2X MAC. The packet drop is measured using traffic generator.
Packet loss in sec with 2X mac addresses = (T1+T2+..Tn)/N Packet loss in sec with 2X MAC addresses = (T1+T2+..Tn)/N
4.9. Scale 4.9. Scale
Objective: Objective:
To measure the scale limit of DUT for PBB-EVPN. Measure the scale limit of DUT for PBB-EVPN.
Topology : Topology 1 Topology : Topology 1
Procedure: Procedure:
The DUT,MHPE2 and SHPE3 are scaled to "N" PBB-EVN instances.Clear BGP Confirm the DUT is up and running with PBB-EVPN.
neighbors of the DUT. Once adjacency is established in the DUT.
check the routes received from MHPE2 and SHPE3 for "N" PBB-EVPN The DUT,MHPE2 and SHPE3 are scaled to "N" PBB-EVI.
instances in the DUT.Then increment the scale of N by 5% of N till
the limit is reached.The limit is where the DUT cant learn any EVPN Ensure routes received from MHPE2 and SHPE3 for "N" PBB-EVI in the
routes from peers. DUT.
Then increment the scale of N by 5% of N till the limit is reached.
The limit is where the DUT cant learn any EVPN routes from its peers.
Measurement : Measurement :
There should not be any loss of route types 2,3 and 4 in DUT.DUT There should not be any loss of route types 2,3 and 4 in DUT. DUT
must relearn all type 2,3 and 4 from remote routers.The DUT must be must relearn all type 2,3 and 4 from remote routers. The DUT must be
subjected to various values of N to find the optimal scale limit. subjected to various values of N to find the optimal scale limit.
The scope of the test is find out the maximum evpn instance that a The scope of the test is find out the maximum evpn instance that a
DUT can hold. DUT can hold.The measurement is carried out using external server
which polls the DUT using automated scripts to find the scale limit
of PBB-EVPN instances.
4.10. Scale Convergence 4.10. Scale Convergence
Objective: Objective:
To measure the convergence time of DUT when the DUT is scaled with To measure the convergence time of DUT when the DUT is scaled with
EVPN instance along with traffic. EVPN instance along with traffic.
Topology : Topology 1 Topology : Topology 1
Procedure: Procedure:
Scale N PBB-EVPN instances in DUT,SHPE3 and MHPE2.Send F frames to Confirm the DUT is up and running with PBB-EVPN.
DUT from CE using traffic generator with X different source and
destination mac addresses for N PBB-EVPN instances. Send F frames Scale N PBB-EVIs in DUT,SHPE3 and MHPE2.
from traffic generator to SHPE3 with X different source and
destination mac addresses.There will be 2X MAC addresses will be Send F frames to DUT from CE using traffic generator with X different
learned in DUT PBB-EVPN MAC table.There is a bi directional traffic source and destination MAC addresses for N PBB-EVI's.
flow with F pps in each direction.Then clear the BGP neighbors in
the DUT. Once the adjacency is restored in DUT. Measure the time Send F frames from traffic generator to SHPE3 with X different source
taken to learn 2X MAC address in DUT MAC table. and destination MAC addresses.
There will be 2X number of MAC addresses will be learned in DUT PBB-
EVPN MAC table.
There is a bi directional traffic flow with F pps in each direction.
Then clear the BGP neighbors in the DUT.
Once the BGP session is in established state in DUT.
Measure the time taken to learn 2X MAC address in DUT MAC table.
Measurement : Measurement :
The DUT must learn 2X MAC addresses.Measure the time taken to learn The DUT must learn 2X MAC addresses. Measure the time taken to learn
2X MAC in DUT.The test is repeated for "N" times and the values are 2X MAC in DUT.The test is repeated for "N" times and the values are
collected.The convergence time is calculated by averaging the values collected. The convergence time is calculated by averaging the
obtained by "N" samples. values obtained by "N" samples. "N" is an arbitrary number to get a
sufficient sample.The time measured for each sample is denoted by
T1,T2...Tn.The measurement is carried out using external server which
polls the DUT using automated scripts.
Convergence time for 2X in sec = (T1+T2+..Tn)/N Time taken to learn 2X MAC in DUT = (T1+T2+..Tn)/N
4.11. Soak Test 4.11. Soak Test
Objective: Objective:
To measure the stability of the DUT in a scaled environment with To measure the stability of the DUT in a scaled environment with
traffic. traffic.
Topology : Topology 1 Topology : Topology 1
Procedure: Procedure:
Scale N PBB-EVPN instances in DUT,SHPE3 and MHPE2.Send F frames to Confirm the DUT is up and running with PBB-EVPN.
DUT from CE using traffic generator with X different source and
destination mac addresses for N PBB-EVPN instances. Send F frames Scale N PBB-EVI's in DUT,SHPE3 and MHPE2.Send F frames to DUT from CE
from traffic generator to SHPE3 with X different source and using traffic generator with different X source and destination MAC
destination mac addresses.There will be 2X MAC addresses will be addresses for N EVI's.
learned in DUT PBB- EVPN MAC table.There is a bi directional
traffic flow with F pps in Each direction.The DUT must run with Send F frames from traffic generator to SHPE3 with X different source
traffic for 24 hours, every hour check the memory leak, crashes. and destination MAC addresses.
There will be 2X number of MAC addresses will be learned in DUT PBB-
EVPN MAC table.
There is a bi directional traffic flow with F pps in each direction.
The DUT must run with traffic for 24 hours.
Every hour check for memory leak in PBB-EVPN process,CPU usage and
crashes in DUT.
Measurement : Measurement :
Take the hourly reading of CPU, process memory. There should not be Take the hourly reading of CPU, process memory. There should not be
any leak, crashes, CPU spikes.The CPU spike is determined as the CPU any leak, crashes, CPU spikes. Th CPU spike is determined as the CPU
usage which shoots at 40 to 50 percent of the average usage.The usage which shoots at 40 to 50 percent of the average usage. The
average value vary from device to device.Memory leak is determined average value vary from device to device. Memory leak is determined
by increase usage of the memory for PBB-EVPN process.The by increase usage of the memory for PBB-EVPN process. The
expectation is under steady state the memory usage for PBB-EVPN process expectation is under steady state the memory usage for PBB-EVPN
should not increase. process should not increase. The measurement is carried out using
external server which polls the DUT using automated scripts which
captures the CPU usage and process memory.
5. Acknowledgments 5. Acknowledgements
We would like to thank Fioccola Giuseppe of Telecom Italia reviewing We would like to thank Fioccola Giuseppe of Telecom Italia reviewing
our draft and commenting it.We would like to thank Sarah Banks for our draft and commenting it. We would like to thank Sarah Banks for
guiding and mentoring us. guiding and mentoring us.
6. IANA Considerations 6. IANA Considerations
This memo includes no request to IANA. This memo includes no request to IANA.
7. Security Considerations 7. Security Considerations
The benchmarking tests described in this document are limited to the The benchmarking tests described in this document are limited to the
performance characterization of controllers in a lab environment with performance characterization of controllers in a lab environment with
 End of changes. 155 change blocks. 
379 lines changed or deleted 676 lines changed or added

This html diff was produced by rfcdiff 1.47. The latest version is available from http://tools.ietf.org/tools/rfcdiff/