Internet Engineering Task Force                            S. Jacob, Ed.
Internet-Draft                                           K. Tiruveedhula
Intended status: Informational                          Juniper Networks
Expires: June 20, September 12, 2020                               March 11, 2020                                 December 18, 2019

             Benchmarking Methodology for EVPN and PBB-EVPN
                      draft-ietf-bmwg-evpntest-04
                      draft-ietf-bmwg-evpntest-05

Abstract

   This document defines methodologies for benchmarking EVPN and PBB-
   EVPN performance.EVPN is defined in RFC 7432, and is being deployed
   in Service Provider networks.Specifically, this document defines
   the methodologies for benchmarking EVPN/PBB-EVPN convergence, data
   plane performance, and control plane performance.

Status of This Memo

   This Internet-Draft is submitted in full conformance with the
   provisions of BCP 78 and BCP 79.

   Internet-Drafts are working documents of the Internet Engineering
   Task Force (IETF).  Note that other groups may also distribute
   working documents as Internet-Drafts.  The list of current Internet-
   Drafts is at https://datatracker.ietf.org/drafts/current/.

   Internet-Drafts are draft documents valid for a maximum of six months
   and may be updated, replaced, or obsoleted by other documents at any
   time.  It is inappropriate to use Internet-Drafts as reference
   material or to cite them other than as "work in progress."

   This Internet-Draft will expire on June 20, September 12, 2020.

Copyright Notice

   Copyright (c) 2019 2020 IETF Trust and the persons identified as the
   document authors. All rights reserved.

   This document is subject to BCP 78 and the IETF Trust's Legal
   Provisions Relating to IETF Documents
   (https://trustee.ietf.org/license-info) in effect on the date of
   publication of this document. Please review these documents
   carefully, as they describe your rights and restrictions with respect
   to this document. Code Components extracted from this document must
   include Simplified BSD License text as described in Section 4.e of
   the Trust Legal Provisions and are provided without warranty as
   described in the Simplified BSD License.

Table of Contents

   1.  Introduction  . . . . . . . . . . . . . . . . . . . . . . . .   2
     1.1.  Requirements Language . . . . . . . . . . . . . . . . . .   3
     1.2.  Terminologies . . . . . . . . . . . . . . . . . . . . . .   3
   2.  Test Topology . . . . . . . . . . . . . . . . . . . . . . . .   4
   3.  Test Cases for EVPN  Benchmarking . . . . . . . . . . . . . .   7
     3.1.  Local  Data Plane MAC Learning . . . . . . . . . . . . . . . . . . .   7
     3.2.  Remote  Control Plane MAC Learning  . . . . . . . . . . . . . . . . . . .   8
     3.3.  MAC Flush due to local link failure Flush-Local Link Failure and Relearning . . .   8
     3.4.  MAC Flush due to remote link failure. . . . .   9
     3.4.  MAC Flush-Remote Link Failure and Relearning. . . . . . .   9  10
     3.5.  MAC Aging . . . . . . . . . . . . . . . . . . . . . . . .   9  11
     3.6.  Remote Mac MAC Aging  . . . . . . . . . . . . . . . . . . . .  10  12
     3.7.  Local  Control and Remote Data plane MAC Learning . . . . . . . . . . . . . .  10  12
     3.8.  High Availability.  . . . . . . . . . . . . . . . . . . .  11  13
     3.9.  ARP/ND Scale  . . . . . . . . . . . . . . . . . . . . . .  12  14
     3.10. Scaling of Services . . . . . . . . . . . . . . . . . . .  12  15
     3.11. Scale Convergence . . . . . . . . . . . . . . . . . . . .  13  16
     3.12. SOAK Test.  . . . . . . . . . . . . . . . . . . . . . . .  14  17
   4.  Test Cases for PBB-EVPN  Benchmarking . . . . . . . . . . . .  14  18
     4.1.  Data Plane Local MAC Learning . . . . . . . . . . . . . . . . . . .  14  18
     4.2.  Data Plane Remote Mac MAC Learning  . . . . . . . . . . . . . . . . . . .  15  18
     4.3.  MAC Flush due to link failure Flush-Local Link Failure  . . . . . . . . . . . . . .  15  19
     4.4.  MAC Flush due to remote Flush-Remote Link Failure . . . . . . . . . . . . .  16 .  20
     4.5.  MAC aging Aging . . . . . . . . . . . . . . . . . . . . . . . .  17  21
     4.6.  Remote MAC Aging. . . . . . . . . . . . . . . . . . . . .  17  22
     4.7.  Local and Remote MAC Learning . . . . . . . . . . . . . .  18  23
     4.8.  High Availability . . . . . . . . . . . . . . . . . . . .  18  23
     4.9.  Scale . . . . . . . . . . . . . . . . . . . . . . . . . .  19  24
     4.10. Scale Convergence . . . . . . . . . . . . . . . . . . . .  19  25
     4.11. Soak Test . . . . . . . . . . . . . . . . . . . . . . . .  20  26
   5.  Acknowledgments   . . . . . . . . . . . . . . . . . . . . . .   21  27
   6.  IANA Considerations . . . . . . . . . . . . . . . . . . . . .  21  27
   7.  Security Considerations . . . . . . . . . . . . . . . . . . .  21  27
   8.  References  . . . . . . . . . . . . . . . . . . . . . . . . .  21  27
     8.1.  Normative References  . . . . . . . . . . . . . . . . . .  21  27
     8.2.  Informative References  . . . . . . . . . . . . . . . . .  22  28
   Appendix A.  Appendix . . . . . . . . . . . . . . . . . . . . . .  22  28
   Authors' Addresses  . . . . . . . . . . . . . . . . . . . . . . .  22  28

1.  Introduction

   EVPN is defined in RFC 7432, and describes BGP MPLS- MPLS based Ethernet
   VPNs (EVPN).PBB-EVPN (EVPN).  PBB-EVPN is defined in RFC 7623, discusses how Ethernet
   Provider backbone Bridging can be combined with EVPNs to provide a
   new/combined solution.This solution.  This draft defines methodologies that can be
   used to benchmark both RFC 7432 and RFC 7623 solutions.Further,
   this draft provides methodologies for benchmarking the performance of
   EVPN data and control planes, MAC learning, MAC flushing, MAC aging,
   convergence, high availability, and scale.

1.1.  Requirements Language

   The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
   "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
   document are to be interpreted as described in RFC 8174 [RFC8174].

1.2.  Terminologies

   All-Active Redundancy Mode: When all PEs attached to an Ethernet
   segment are allowed to forward known unicast traffic to/from that
   Ethernet segment for a given VLAN, then the Ethernet segment is
   defined to be operating in All-Active redundancy mode.

   AA

   AA: All Active mode

   CE

   CE: Customer Router/Devices/Switch.

   DF

   DF: Designated Forwarder

   DUT

   DUT: Device under test.

   Ethernet Segment (ES): When a customer site (device or network) is
   connected to one or more PEs via a set of Ethernet links, then that
   set of links is referred to as an 'Ethernet segment'.

   EVI: An EVPN instance spanning the Provider Edge (PE) devices
   participating in that EVPN.

   Ethernet Segment Identifier (ESI): A unique non-zero identifier that
   identifies an Ethernet segment is called an 'Ethernet Segment
   Identifier'.

   Ethernet Tag: An Ethernet tag identifies a particular broadcast
   domain, e.g., a VLAN.  An EVPN instance consists of one or more
   broadcast domains.

   Interface

   Interface: Physical interface of a router/switch.

   IRB

   IRB: Integrated routing and bridging interface

   MAC

   MAC: Media Access Control addresses on a PE.

   MHPE2

   MHPE2: Multi homed Provider Edge router 2.

   MHPE1

   MHPE1: Multi homed Provider Edge router 1.

   SHPE3

   SHPE3: Single homed Provider Edge Router 3.

   PE: Provider Edge device.

   P

   P: Provider Router.

   RR

   RR: Route Reflector.

   RT

   RT: Traffic Generator.

   Sub Interface Interface: Each physical Interfaces is subdivided into Logical
   units.

   SA

   SA: Single Active

   Single-Active Redundancy Mode: When only a single PE, among all the
   PEs attached to an Ethernet segment, is allowed to forward traffic
   to/from that Ethernet segment for a given VLAN, then the Ethernet
   segment is defined to be operating in Single-Active redundancy mode.

2.  Test Topology

   EVPN/PBB-EVPN Services running on SHPE3,

   There are five routers in the Test setup.SHPE3, RR/P, MHPE1 and
   MHPE2 emulating a service provider network.CE is a customer device
   connected to MHPE1 and MHPE2, it is configured with bridge domains in Single
   Active Mode:

         | Traffic Generator
   multiple vlans.  The traffic generator is connected to CE and
   SHPE3.The MHPE1 acts as a sender/receiver DUT.The traffic generator will be used as
   sender and receiver of traffic.The DUT will be the reference point
   for all the test cases.  MHPE1 and MHPE2 are mulihome routers
   connected to CE running single active mode.The traffic generator
   will be generating traffic at 10% of the line rate.

             +----------------+                +-------------------------+
             |                |                |                         |
             |                |                |Traffic Generator sender/|
             |  SHPE3         |                |receiver of layer 2 traffic traffic|
             |                +-----------------+  with multiple vlan.
+----------+ Vlans   |
             |                |  SHPE3                +-------------------------+
             +---------+------+
                       |  Core Link
                       |
              +--------+-----+
              |              |
              |   RR/P       |
+----------+
              |
    |Core              |        Core link
+----------+
              |              +----------------+
              +--+-----------+                |
                 |                            |  RR
                 |   core link                |
                 | Route Reflector /Provider router
+----------+-------------|                            |
   +-------------+---+                       ++------------------+
   |                 |                       |     Core links                   |
+----------+       +-----------+
   |                 |                       |   MHPE2           |
   |MHPE1(DUT)       |   DUT                       |                   |
   |                 |  MHPE1                       |                   |
   |
+----------+       +-----------+                 |                       |                   |
   +-----------------+------+          +-----+-------------------+
                            |          |
           PE-CE link       |
+----------+------------          | PE-CE link
                            |          |  CE
                            |          |  layer2
                            |
|bridge          |
+----------+------------ Traffic
                            |          |
                      +-----+----------+----+           +----------------------------+
                      | CE/Layer 2 bridge   +-----------| Traffic Generator acts as a sender/receiver  sender/ |
                      |                     |           |receiver  of layer 2 traffic traffic|
                      |                     |           | with multiple vlan. Vlans        |
                      |                     |           +----------------------------+
                      |                     |
                      +---------------------+

Topology 1

                                Test Setup

                                 Figure 1

+-----------------+---------------------+---------------------+---------------------+----------------------+-----------------------+
|                 |                     |                     |                     |                      |                       |
|                 |                     |                     |                     |                      |                       |
|                 |                     |                     |                     |                      |                       |
|                 |                     |                     |                     |                      |                       |
| Mode            |                     |                     |                     |Receiver              |                       |
|                 |  Test               |Traffic Direction    |Sender               |                      |                       |
|                 |                     |                     |                     |                      |                       |
|                 |                     |                     |                     |                      |                       |
|                 |                     |                     |                     |                      |                       |
+----------------------------------------------------------------------------------------------------------------------------------+
|                 |                     |                     |                     |                      |                       |
|                 |                     |                     |                     |      SHPE3           |                       |
|Single Active    |  Local Mac MAC          |                     |CE                   |                      |Layer 2 traffic        |
|                 | Learning            | Uni                 |                     |                      |                       |
|                 |                     |                     |                     |                      | multiple MAC          |
|                 |                     |                     |                     |                      |                       |
+-----------------------------------------------------------------------------------------------------------------------------------+
|                 |                     |                     |                     |                      |                       |
|Single Active    | Remote MAC          |                     |                     |         CE           |Layer 2 traffic        |
|                 | Learning            | uni                 | SHPE3               |                      |                       |
|                 |                     |                     |                     |                      |multiple MAC           |
|                 |                     |                     |                     |                      |                      ++
+----------------------------------------------------------------------------------------------------------------------------------+
|                 |                     |                     |                     |                      |                       |
|Single Active    | Scale Convergence   |   Bi                |                     |  CE/SHPE3            |                       |
|                 |                     |                     |   CE/SHPE3          |                      |Layer 2 traffic        |
|                 | Local& Remote       |                     |                     |                      |multiple mac& MAC& vlans    |
|                 | Learning            |                     |                     |                      |                       |
+-----------------+---------------------+---------------------+--------------------------------------------+-----------------------+
                                                                                    |
                                                                                   ++

       Table showing Traffic pattern for the traffic directions of various EVPN/PBB-EVPN
    benchmarking test cases.  Depends on the test scenario the traffic
       can be uni/bi directional generated by the traffic generator.

                                 Figure 2

   Test Setup Configurations:

   There are five routers in the Test setup.SHPE3, RR/P, MHPE1 and
   MHPE2 emulating a service provider network. CE is a customer device
   connected to MHPE1 and MHPE2, it

   SHPE3 is configured with bridge domains in
   multiple vlans.The traffic Interior Gateway protocols like OPSF or IS-
   IS for underlay, LDP for MPLS support,Interior Border Gateway with
   EVPN address family for overlay support.This router must be
   configured with N EVPN/PBB-EVPN instances for testing.Traffic
   generator is connected to CE and
   SHPE3.The MHPE1 acts as DUT.The traffic generator will be used as
   sender this router for sending and receiver of traffic.The measurement will be taken in DUT.

   All routers except CE are receiving
   traffic.

   RR is configured with OSPF/IS-IS,LDP,MPLS,BGP Interior Gateway protocols like OPSF or IS-IS
   for underlay, LDP for MPLS support,Interior Border Gateway with EVPN
   address family.

   All routers except CE are configured with Interior border gateway
   protocol,RR acting family for overlay support.This router acts as a provider
   router and as a route reflector.

   MHPE1,MHPE2,SHPE3

   MHPE1 is configured with Interior Gateway protocols like OPSF or IS-
   IS for underlay, LDP for MPLS support,Interior Border Gateway with
   EVPN address family for overlay support.This router must be
   configured with "N" N EVPN/PBB-EVPN instances
   depends up for testing.This router
   is configured with ESI per vlan or ESI per interface.It is
   functioning as multi homing PE working on Single Active EVPN
   mode.This router serves as the cases.

   MHPE1 DUT and MHEPE2 it is connected to CE.MHPE1
   is acting as DUT for all the test cases.

   MHPE2 is configured with Interior Gateway protocols like OPSF or IS-
   IS for underlay, LDP for MPLS support,Interior Border Gateway with
   EVPN address family for overlay support.This router must be
   configured with N EVPN/PBB-EVPN instances for testing.This router
   is configured with ESI per vlan or ESI per interface.It is
   functioning as multi homing PE working on
   interface.

   MHPE1 and MHEPE2 are running Single Active mode of EVPN. EVPN mode.
   It is connected to CE.

   CE is acting as bridge configured with multiple vlans,the same vlans
   are configured on MHPE1,MHPE2,SHPE3.

   Depending up on the test scenarios MHPE1,MHPE2,SHPE3.Traffic generator is connected
   to CE. the traffic generators generator acts as sender or receiver of traffic.

   Depending up on the test scenarios the traffic generators will be
   used to generate uni directional or bi directional flows.

   The above configuration will be serving as the base configuration for
   all test cases.

3.  Test Cases for EVPN Benchmarking

3.1.  Local  Data Plane MAC Learning

   Objective:

   To Record

   Measure the time taken to learn the Data Plane MAC address locally in DUT.

   Topology : Topology 1

   Procedure:

   The data plane MAC learning can be measured using the parameters
   defined in RFC 2889 section 5.8.Send "X" unicast frames from CE to
   MHPE1(DUT) working in SA mode.Traffic 5.8.

   Confirm the DUT is up and running with EVPN.

   Traffic generator connected to CE must send frames with "X" different
   source and destination MAC address. address for one vlan, the same vlan must
   be present in all the devices except RR.

   Send "X" unicast frames from CE to MHPE1(DUT) for one EVPN instance
   working in SA mode.

   The DUT must will learn these "X" macs MAC in data plane.

   Measurement :

   Measure the time taken to learn "X" MACs MAC locally in DUT evpn mac
   table.  The MAC
   table.The data plane measurement is taken by considering DUT as
   black box. The range of MACs MAC are known from traffic generator,the
   same must be learned in DUT, the time taken to learn "X" macs MAC is
   measured.The measurement is
   measured. carried out using external server which
   polls the DUT using automated scripts.

   The test is repeated for "N" times and the values are collected.The
   mac
   MAC learning rate is calculated by averaging the values obtained from
   "N" samples.

   Mac  "N" is an arbitrary number to get a sufficient sample.
   The time measured for each sample is denoted by T1,T2...Tn.

   MAC learning rate = (T1+T2+..Tn)/N

3.2.  Remote  Control Plane MAC Learning

   Objective:

   To Record

   Measure the time taken to learn the remote macs. control plane MAC.

   Topology : Topology 1

   Procedure:

   Send X

   Confirm the DUT is up and running with EVPN.

   Traffic generator connected to SHPE3 must send frames with X "X"
   different source and destination mac addresses
   to SHPE3 from traffic generator.SHPE3 will advertise these locally
   learned macs to MHPE1 and MHPE2 via control plane.Measure MAC address for one vlan, the time
   taken same
   vlan must be present in all the devices except RR.

   Ensure the frames must be destined to one EVPN instance.

   The DUT will learn these X MACs from remote peer in DUT EVPN "X" MAC address
   table.The DUT and MHPE2 are running SA mode. in control plane.

   Measurement :

   Measure the time taken by the DUT to learn the "X" MACs MAC in the data
   plane.The test is repeated for "N" times and the values are
   collected.  The remote mac MAC learning rate is calculated by averaging
   the values obtained from "N" samples.

   Remote mac  "N" is an arbitrary number to
   get a sufficient sample.  The time measured for each sample is
   denoted by T1,T2...Tn.The measurement is carried out using external
   server which polls the DUT using automated scripts.

   MAC learning rate = (T1+T2+..Tn)/N

3.3.  MAC Flush due to local link failure Flush-Local Link Failure and Relearning

   Objective:

   To record

   Measure the time taken to flush the locally learned mac Data Plane MAC and the time taken
   to relearn the same amount of macs. MAC.

   Topology : Topology 1

   Procedure:

   Confirm the DUT is up and running with EVPN.

   Send X frames with X different source and destination mac MAC addresses
   to DUT from CE using traffic generator. Wait till generator for one vlan.

   Ensure the MHPE1 DUT learns all X MAC addresses.  Then fail the MHPE1 CE addresses in data plane.

   Fail the DUT-CE link and measure the time taken to flush these X MACs MAC
   from the EVPN MAC table.

   Bring up the link which was made Down(the link between MHPE1 DUT and
   CE).Measure time taken by the DUT to relearn it.The these "X" MAC.

   The DUT and MHPE2 are running SA mode.

   Measurement :

   Measure the time taken for flushing these X MAC addresses.Measure addresses.  Measure
   the time taken to relearn the these X MACs MAC in DUT.The test is repeated for
   "N" times and the values are collected.The collected.  The flush and the relearning
   time is calculated by averaging the values obtained by "N" samples.
   samples."N" is an arbitrary number to get a sufficient sample.The
   time measured for each sample is denoted by T1,T2...Tn.The
   measurement is carried out using external server which polls the DUT
   using automated scripts.

   Flush rate = (T1+T2+..Tn)/N

   Relearning rate = (T1+T2+..Tn)/N

3.4.  MAC Flush due to remote link failure. Flush-Remote Link Failure and Relearning.

   Objective:

   To record

   Measure the time taken to flush the remote mac Control plane MAC learned in DUT
   during remote link failure. failure and the time taken to relearn.

   Topology : Topology 1

   Procedure:

   Confirm the DUT is up and running with EVPN.

   Send X frames with X different source and destination mac MAC addresses
   to DUT from SHPE3 using traffic generator. generator for one vlan.

   Bring down the link between SHPE3 and traffic generator.Then measure generator.

   SHPE3 will withdraw the routes from DUT due to link failure.

   Measure the time taken to flush the DUT EVPN MAC table.The table.  The DUT and
   MHPE2 are running SA mode.

   Bring up the link which was made Down(the link between SHPE3 and
   traffic generator).

   Measure time taken by the DUT to relearn these "X" MAC from control
   plane.

   Measurement :

   Measure the time taken to flush X remote MACs MAC from EVPN MAC table of
   the DUT.Measure the time taken to relearn these X MAC in DUT.The
   test is repeated for "N" times and the values are collected.The
   flush rate is calculated by averaging the values obtained by "N"
   samples.  "N" is an arbitrary number to get a sufficient sample.The
   time measured for each sample is denoted by T1,T2...Tn.The
   measurement is carried out using external server which polls the DUT
   using automated scripts.

   Flush rate = (T1+T2+..Tn)/N

   Relearning rate = (T1+T2+..Tn)/N

3.5.  MAC Aging

   Objective:

   To measure the mac MAC aging time.

   Topology : Topology 1

   Procedure:

   Confirm the DUT is up and running with EVPN.

   Send X frames with X different source and destination mac MAC addresses
   to DUT from CE using traffic generator.Wait till generator for one vlan.

   Ensure these X MAC addresses are learned. learned in DUT.

   Then stop the traffic.Record traffic.

   Ensure the DUT and other devices in the test are using the default
   timers for aging.

   Measure the time taken to flush X MACS MAC from DUT EVPN MAC table due to aging.The
   aging.

   The DUT and MHPE2 are running SA mode.

   Measurement :

   Measure the time taken to flush X MAC addresses due to aging.The aging.  The
   test is repeated for "N" times and the values are collected.The collected.  The
   aging is calculated averaging the values obtained by "N" samples.
   "N" is an arbitrary number to get a sufficient sample.The time
   measured for each sample is denoted by T1,T2...Tn.  The measurement
   is carried out using external server which polls the DUT using
   automated scripts.

   Aging time for X Macs MAC in sec = (T1+T2+..Tn)/N

3.6.  Remote Mac MAC Aging

   Objective:

   To measure

   Measure the remote mac control plane learned MAC aging time.

   Topology : Topology 1

   Procedure:

   Confirm the DUT is up and running with EVPN.

   Send X frames with X different source and destination mac MAC addresses
   to DUT from SHPE3 using traffic generator.  Stop the traffic at
   remote PE SHPE3.Due to MAC aging SHPE3 will withdraw its routes from generator for one vlan.

   Ensure these X MAC addresses are learned in DUT via control plane.

   Then stop the traffic.

   Ensure the DUT and MHPE2.Measure other devices in the test are using the default
   timers for aging.

   Measure the time taken to remove these MACs flush X MAC from DUT EVPN MAC table.DUT table due to
   aging.

   The DUT and MHPE2 are running in SA mode mode.

   Measurement :

   Measure the time taken to flush X remote MACs MAC learned in DUT EVPN MAC
   table due to aging.The test is repeated for "N" times and the values
   are collected.The aging is calculated by averaging the values
   obtained by "N" samples. samples."N" is an arbitrary number to get a
   sufficient sample. The time measured for each sample is denoted by
   T1,T2...Tn.The measurement is carried out using external server
   which polls the DUT using automated scripts.

   Aging time for X Macs MAC in sec = (T1+T2+..Tn)/N

3.7.  Local  Control and Remote Data plane MAC Learning

   Objective:

   To record the time taken to learn both local and remote macs. MAC.

   Topology : Topology 1
   Procedure:

   Confirm the DUT is up and running with EVPN.

   Send X frames with X different source and destination mac MAC addresses
   to DUT from SHPE3 using traffic generator.Send generator for one vlan.

   Send X frames with different source and destination mac MAC addresses
   from traffic generator connected to CE.The CE for one vlan.

   The source and destination addresses of flows must be complimentary
   to have unicast flows.Measure flows.

   Measure the time taken by the DUT to learn 2X in EVPN MAC.DUT MAC table.

   DUT and MHPE2 are running in SA mode.

   Measurement :

   Measure the time taken to learn 2X MAC addresses in DUT EVPN MAC
   table.  The
   table.The test is repeated for "N" times and the values are
   collected. The mac MAC learning time is calculated by averaging the
   values obtained by "N" samples.

   Mac samples."N" is an arbitrary number to get a
   sufficient sample.The time measured for each sample is denoted by
   T1,T2...Tn.  The measurement is carried out using external server
   which polls the DUT using automated scripts

   MAC learning rate = (T1+T2+..Tn)/N

3.8.  High Availability.

   Objective:

   To record

   Measure traffic loss during routing engine fail over.

   Topology : Topology 1

   Procedure:

   Confirm the DUT is up and running with EVPN.

   Send X frames from CE to DUT from traffic generator with X different
   source and destination mac addresses.Send MAC addresses.

   Send X frames from traffic generator to SHPE3 with X different source
   and destination mac MAC addresses, so that 2X MAC address will be learned
   in the DUT.There DUT.

   There is a bi directional traffic flow with X pps in each direction.Then direction.

   Ensure the DUT learn 2X MAC.

   Then do a routing engine fail-over.

   Measurement :

   The expectation of the test is 0 traffic loss with no change in the
   DF role.DUT role.  DUT should not withdraw any routes.But in cases where the
   DUT is not property synchronized between master and standby,due to
   that packet loss are observed.In observed. In that scenario the packet loss is
   measured.The test is repeated for "N" times and the values are
   collected.The packet loss is calculated by averaging the values
   obtained by "N" samples. samples."N" is an arbitrary number to get a
   sufficient sample. The time measured for each sample is denoted by
   T1,T2...Tn.  The measurement is carried out using external server
   which polls the DUT using automated scripts to ensure the DUT learned
   2X MAC.  The packet drop is measured using traffic generator.

   Packet loss in sec with 2X mac MAC addresses = (T1+T2+..Tn)/N

3.9.  ARP/ND Scale

   To Record

   Measure the DUT scaling limit of ARP/ND.

   Objective:

   To Record

   Measure the ARP/ND scale of the DUT.

   Topology : Topology 1

   Procedure:

   Confirm the DUT is up and running with EVPN.

   Send X arp/neighbor discovery(ND) from the traffic generator to DUT
   with different sender ip/ipv6,mac ip/ipv6,MAC addresses to the target IRB address
   configured in EVPN instance.

   The EVPN instance learns the mac+ip MAC+ip and
   mac+ipv6 MAC+ipv6 addresses from these
   request and advertise as type 2 mac+ip/
   mac+ipv6 MAC+ip/MAC+ipv6 route to remote
   provide edge routers which have same EVPN
   configurations.The configurations.

   The value of X must be increased at a incremental value of 5% of X,
   till the limit is reached.The limit is where the DUT cant learn any
   more type 2 mac+ip/mac+ipv6.The MAC+ip/MAC+ipv6.The test must be separately conducted for
   arp and ND.

   Measurement :

   Measure the scale limit of type 2 mac+ip/mac+ipv6 MAC+ip/MAC+ipv6 route which DUT can
   learn.The test is repeated for "N" times and the values are
   collected.The scale limit is calculated by averaging the values
   obtained by "N" samples for both mac+ip MAC+ip and mac+ipv6. MAC+ipv6."N" is an
   arbitrary number to get a sufficient sample.The scale value
   obtained by each sample be v1,v2..vn.The measurement is carried out
   using external server which polls the DUT using automated scripts to
   find the scale limit of MAC+ipv4/MAC+ipv6.

   Scale limit for mac+ip MAC+ip = (v1+v2+..vn)/N

   DUT scale

   Scale limit for mac+ipv6 MAC+ipv6 = (v1+v2+..vn)/N

3.10.  Scaling of Services

   Objective:

   This test is to measure

   Measure the scale of EVPN instances that a DUT can hold.

   Topology : Topology 1

   Procedure:

   Confirm the DUT is up and running with EVPN.

   The DUT,MHPE2 and SHPE3 are scaled to "N" EVI.Clear BGP neighbors of
   the DUT.Once adjacency is established in the DUT, check the EVI.

   Ensure routes received from MHPE2 and SHPE3 for "N" EVI in the DUT.Then DUT.

   Then increment the scale of N by 5% of N till the limit is reached.The reached.

   The limit is where the DUT cant learn any EVPN routes from its peers.

   Measurement :

   There should not be any loss of route types 1,2,3 and 4 in DUT.DUT DUT.  DUT
   must relearn all type 1,2,3 and 4 from remote routers. The routers.The DUT must
   be subjected to various values of N to find the optimal scale limit.
   The scope of the test is find out the maximum evpn instance that a
   DUT can hold. hold.The measurement is carried out using external server
   which polls the DUT using automated scripts to find the scale limit
   of EVPN instances.

3.11.  Scale Convergence

   Objective:

   To measure

   Measure the convergence time of DUT when the DUT is scaled with EVPN
   instance along with traffic.

   Topology : Topology 1

   Procedure:

   Confirm the DUT is up and running with EVPN.

   Scale N EVIs in DUT,SHPE3 and MHPE2.Send MHPE2.

   Send F frames to DUT from CE using traffic generator with X different
   source and destination mac MAC addresses for N EVI's.Send EVI's.

   Send F frames from traffic generator to SHPE3 with X different source
   and destination mac addresses.There MAC addresses.

   There will be 2X number of MAC addresses will be learned in DUT EVPN
   MAC table.

   There is a bi directional traffic flow with F pps in each direction.

   Then clear the BGP neighbors in the DUT.Once DUT.

   Once the adjacency BGP session is
   restored in DUT.Measure established state in DUT.

   Measure the time taken to learn 2X MAC address in DUT MAC table.

   Measurement :

   The DUT must learn 2X MAC addresses.  Measure the time taken to learn
   2X MAC in DUT.The test is repeated for "N" times and the values are
   collected.  The convergence time is calculated by averaging the
   values obtained by "N" samples. samples."N" is an arbitrary number to get a
   sufficient sample.The time measured for each sample is denoted by
   T1,T2...Tn.The measurement is carried out using external server which
   polls the DUT using automated scripts.

   Time taken to learn 2X macs MAC in DUT = (T1+T2+..Tn)/N

3.12.  SOAK Test.

   Objective:

   This test is carried out to measure the stability of the DUT in a
   scaled environment with traffic over a period of time "T'". In each
   interval "t1" the DUT CPU usage, memory usage are measured.The DUT
   is checked for any crashes during this time period.

   Topology : Topology 1

   Procedure:

   Confirm the DUT is up and running with EVPN.

   Scale N EVI's in DUT,SHPE3 and MHPE2.Send F frames to DUT from CE
   using traffic generator with different X source and destination mac MAC
   addresses for N EVI's.Send EVI's.

   Send F frames from traffic generator to SHPE3 with X different source
   and destination mac addresses.There MAC addresses.

   There will be 2X number of MAC addresses will be learned in DUT EVPN
   MAC table.

   There is a bi directional traffic flow with F pps in each direction.

   The DUT must run with traffic for 24 hours, every hours.

   Every hour check for memory leak, leak in EVPN process, CPU usage and crash.
   crashes in DUT.

   Measurement :

   Take the hourly reading of CPU, process memory.  There should not be
   any leak, crashes, CPU spikes.  Th CPU spike is determined as the CPU
   usage which shoots at 40 to 50 percent of the average usage.The usage.  The
   average value vary from device to device. Memory leak is determined
   by increase usage of the memory for EVPN process. The expectation is
   under steady state the memory usage for EVPN process should not
   increase.  The measurement is carried out using external server which
   polls the DUT using automated scripts which captures the CPU usage
   and process memory.

4.  Test Cases for PBB-EVPN Benchmarking

4.1.  Data Plane Local MAC Learning

   Objective:

   To Record

   Measure the time taken to learn the Data Plane MAC address locally. in DUT.

   Topology : Topology 1

   Procedure:

   Confirm the DUT is up and running with PBB-EVPN.

   Traffic generator connected to CE must send frames with "X" different
   source and destination MAC address for one vlan, the same vlan must
   be present in all the devices except RR.

   Send "X" unicast frames from CE to MHPE1(DUT) for one PBB-EVPN
   instance working in SA mode with
   "X" different source and destination mac addresses from traffic
   generator.The mode.

   The DUT must will learn these "X" macs MAC in data plane.

   Measurement :

   Measure the time taken by the DUT to learn the "X" MACs MAC locally in the data
   plane. DUT PBB-EVPN MAC
   table.  The data plane measurement is taken by considering DUT as
   black box the box.  The range of "X" MAC is are known from traffic generator and
   the generator,the
   same must be learned in DUT, the time taken to learn "X" MAC is
   measured.The measurement is carried out using external server which
   polls the DUT using automated scripts.

   The test is repeated for "N" times and the values are collected.The mac
   MAC learning time rate is calculated by averaging the values obtained from
   "N" samples.

   Mac  "N" is an arbitrary number to get a sufficient sample.
   The time measured for each sample is denoted by T1,T2...Tn.

   MAC learning rate = (T1+T2+..Tn)/N

4.2.  Data Plane Remote Mac MAC Learning

   Objective:

   To Record the time taken to learn the remote macs. MAC.

   Topology : Topology 1
   Procedure:

   Send X

   Confirm the DUT is up and running with PBB-EVPN.

   Traffic generator connected to SHPE3 must send frames with X "X"
   different source and destination mac addresses
   to SHPE3 from traffic generator.These macs will MAC address for one vlan, the same
   vlan must be flooded present in all the devices except RR.

   Ensure the frames must be destined to MHPE1
   and MHPE2 by SHPE3.The one PBB-EVPN instance.

   The DUT and MHPE2 are running SA mode. will learn these "X" MAC in data plane.

   Measurement :

   Measure the time taken by the DUT to learn X mac addresses the "X" MAC in DUT mac table.
   The the data
   plane.The test is repeated for "N" times and the values are collected.The
   mac
   collected.  The remote MAC learning time rate is calculated by averaging
   the values obtained by from "N" samples.

   Remote Mac  "N" is an arbitrary number to
   get a sufficient sample.  The time measured for each sample is
   denoted by T1,T2...Tn.The measurement is carried out using external
   server which polls the DUT using automated scripts.

   MAC learning rate = (T1+T2+..Tn)/N

4.3.  MAC Flush due to link failure Flush-Local Link Failure

   Objective:

   To record

   Measure the time taken to flush the locally learned mac MAC and the time
   taken to relearn the same amount of macs. MAC.

   Topology : Topology 1

   Procedure:

   Confirm the DUT is up and running with PBB-EVPN.

   Send X frames with X different source and destination mac MAC addresses
   to DUT from CE using traffic generator.Wait till generator for one vlan.

   Ensure the MHPE1 learn DUT learns all X MAC address.Then fail addresses in data plane.

   Fail the MHPE1 CE DUT-CE link and measure the time taken to flush these X MACs MAC
   from the PBB-EVPN MAC table.Then bring table.

   Bring up the link.Measure the link which was made Down(the link between DUT and
   CE).Measure time taken by the DUT to relearn X MACS.The these "X" MAC.

   The DUT and MHPE2 are running SA mode.

   Measurement :

   Measure the time taken for flushing these X MAC addresses.Measure addresses.  Measure
   the time taken to relearn the these X MACs MAC in DUT.The test is repeated for
   "N" times and the values are collected.  The flush and the relearning
   time is calculated by averaging the values obtained by "N" samples.
   samples."N" is an arbitrary number to get a sufficient sample.  The
   time measured for each sample is denoted by T1,T2...Tn.  The
   measurement is carried out using external server which polls the DUT
   using automated scripts.

   Flush rate = (T1+T2+..Tn)/N

   Relearning rate = (T1+T2+..Tn)/N

4.4.  MAC Flush due to remote Flush-Remote Link Failure

   Objective:

   To record

   Measure the time taken to flush the remote mac MAC learned in DUT
   during due to
   remote link failure. failure and relearning it.

   Topology : Topology 1

   Procedure:

   confirm the DUT is up and running with PBB-EVPN.

   Send X frames with X different source and destination mac MAC addresses
   to DUT from SHPE3 using traffic generator.Bring generator for one vlan.

   Bring down the link between SHPE3 and traffic generator.  Then measure

   Measure the time taken to flush the DUT PBB-EVPN MAC address table.The remote MACs will be
   learned by Data plane, but the B-MAC will be learned by control
   plane.The table.  The DUT
   and MHPE2 are running SA mode.

   Bring up the link which was made Down(the link between SHPE3 and
   traffic generator).

   Measure time taken by the DUT to relearn these "X" MAC

   Measurement :

   Measure the time taken to flush X remote MACs MAC from PBB-EVPN MAC table
   of DUT.Repeat these test and plot the data.The DUT.Measure the time taken to relearn these X MAC in DUT. The
   test is repeated for "N" times and the values are collected.  The
   flush rate is
   calculated calculated by averaging the values obtained by "N"
   samples.  "N" is an arbitrary number to get a sufficient sample.  The
   time measured for each sample is denoted by averaging T1,T2...Tn.The
   measurement is carried out using external server which polls the values obtained by "N" samples. DUT
   using automated scripts.

   Flush rate = (T1+T2+..Tn)/N

   Relearning rate = (T1+T2+..Tn)/N

4.5.  MAC aging Aging

   Objective:

   To measure

   Measure the mac MAC aging time.

   Topology : Topology 1

   Procedure:

   Confirm the DUT is up and running with PBB-EVPN.

   Send X frames with X different source and destination mac MAC addresses
   to DUT from CE using traffic generator.Wait till generator for one vlan.

   Ensure these X MAC address addresses are learned in DUT PBB-EVPN MAC table.Then DUT.

   Then stop the traffic.Record traffic.

   Ensure the DUT and other devices in the test are using the default
   timers for aging.

   Measure the time taken to flush X MAC entries from DUT PBB-EVPN MAC table due
   to aging.

   The DUT and MHPE2 are running in SA mode mode.

   Measurement :

   Measure the time taken to flush X MAC addresses due to aging.Repeat
   these test and plot the data.The aging.  The
   test is repeated for "N" times and the values are collected.The collected.  The
   aging is calculated by averaging the values obtained by "N" samples.
   "N" is an arbitrary number to get a sufficient sample.  The time
   measured for each sample is denoted by T1,T2...Tn.  The measurement
   is carried out using external server which polls the DUT using
   automated scripts.

   Aging time for X Macs MAC in sec = (T1+T2+..Tn)/N

4.6.  Remote MAC Aging.

   Objective:

   To measure

   Measure the remote mac MAC aging time.

   Topology : Topology 1

   Procedure:

   Confirm the DUT is up and running with PBB-EVPN.

   Send X frames with X different source and destination mac MAC addresses
   to DUT from SHPE3 using traffic generator.Stop generator for one vlan.

   Ensure these X MAC addresses are learned in DUT.

   Then stop the traffic at
   remote PE(SHPE3).Measure traffic.

   Ensure the DUT and other devices in the test are using the default
   timers for aging.

   Measure the time taken to remove these remote MACs flush X MAC from DUT PBB-EVPN MAC table.The table due
   to aging.

   The DUT and MHPE2 are running in SA mode.

   Measurement :

   Measure the time taken to flush the X remote MACs from MAC learned in DUT PBB-EVPN EVPN MAC
   table due to aging .The aging.The test is repeated for "N" times and the values
   are collected.  The aging is calculated by averaging the values
   obtained by "N" samples. samples."N" is an arbitrary number to get a
   sufficient sample.  The time measured for each sample is denoted by
   T1,T2...Tn.  The measurement is carried out using external server
   which polls the DUT using automated scripts.

   Aging time for X Macs MAC in sec = (T1+T2+..Tn)/N

4.7.  Local and Remote MAC Learning

   Objective:

   To record

   Measure the time taken to learn both local and remote macs. MAC.

   Topology : Topology 1

   Procedure:

   Confirm the DUT is up and running with PBB-EVPN.

   Send X frames with X different source and destination mac MAC addresses
   to DUT from SHPE3 using traffic generator.Send generator for one vlan.

   Send X frames with different source and destination mac MAC addresses
   from traffic generator connected to CE.The CE for one vlan.

   The source and destination mac addresses of flows must be complimentary
   to have unicast flows.Measure flows.

   Measure the time taken by the DUT to learn 2X in PBB-EVPN MAC table.DUT table.

   DUT and MHPE2 are running in SA mode.

   Measurement :

   Measure the time taken to learn 2X MAC addresses table in DUT PBB-
   EVPN PBB-EVPN MAC table.The
   table.  The test is repeated for "N" times and the values are collected.The mac
   collected.  The MAC learning time is calculated by averaging the
   values obtained by "N" samples.

   Mac samples."N" is an arbitrary number to get a
   sufficient sample.  The time measured for each sample is denoted by
   T1,T2...Tn.  The measurement is carried out using external server
   which polls the DUT using automated scripts

   MAC learning rate = (T1+T2+..Tn)/N

4.8.  High Availability

   Objective:

   To record

   Measure traffic loss during routing engine failover.

   Topology : Topology 1

   Procedure:

   Confirm the DUT is up and running with PBB-EVPN.

   Send X frames from CE to DUT from traffic generator with X different
   source and destination mac
   addresses from CE using the traffic generator.Send MAC addresses.

   Send X frames from traffic generator to SHPE3 with X different source
   and destination
   mac MAC addresses, so that 2X MAC address will be Learned learned
   in the DUT.

   There is a bi directional traffic flow with X pps in each direction.

   Ensure the DUT learn 2X MAC.

   Then do a routing engine fail-over.

   Measurement :

   The expectation of the test is 0 traffic loss with no change in the
   DF role.DUT role.  DUT should not withdraw any routes.But in cases where the
   DUT is not property synchronized between master and standby,due to
   that packet loss are observed.  In that scenario the packet loss is
   measured.The test is repeated for "N" times and the values are
   collected.The packet loss is calculated by averaging the values
   obtained by "N" samples. samples."N" is an arbitrary number to get a
   sufficient sample.  The time measured for each sample is denoted by
   T1,T2...Tn.  The measurement is carried out using external server
   which polls the DUT using automated scripts to ensure the DUT learned
   2X MAC.  The packet drop is measured using traffic generator.

   Packet loss in sec with 2X mac MAC addresses = (T1+T2+..Tn)/N

4.9.  Scale

   Objective:

   To measure

   Measure the scale limit of DUT for PBB-EVPN.

   Topology : Topology 1

   Procedure:

   Confirm the DUT is up and running with PBB-EVPN.

   The DUT,MHPE2 and SHPE3 are scaled to "N" PBB-EVN instances.Clear BGP
   neighbors of the DUT.  Once adjacency is established in the DUT.
   check the PBB-EVI.

   Ensure routes received from MHPE2 and SHPE3 for "N" PBB-EVPN
   instances PBB-EVI in the DUT.Then
   DUT.

   Then increment the scale of N by 5% of N till the limit is reached.The reached.

   The limit is where the DUT cant learn any EVPN routes from its peers.

   Measurement :

   There should not be any loss of route types 2,3 and 4 in DUT.DUT DUT.  DUT
   must relearn all type 2,3 and 4 from remote routers.The routers.  The DUT must be
   subjected to various values of N to find the optimal scale limit.
   The scope of the test is find out the maximum evpn instance that a
   DUT can hold. hold.The measurement is carried out using external server
   which polls the DUT using automated scripts to find the scale limit
   of PBB-EVPN instances.

4.10.  Scale Convergence

   Objective:

   To measure the convergence time of DUT when the DUT is scaled with
   EVPN instance along with traffic.

   Topology : Topology 1

   Procedure:

   Confirm the DUT is up and running with PBB-EVPN.

   Scale N PBB-EVPN instances PBB-EVIs in DUT,SHPE3 and MHPE2.Send MHPE2.

   Send F frames to DUT from CE using traffic generator with X different
   source and destination mac MAC addresses for N PBB-EVPN instances. PBB-EVI's.

   Send F frames from traffic generator to SHPE3 with X different source
   and destination mac addresses.There MAC addresses.

   There will be 2X number of MAC addresses will be learned in DUT PBB-EVPN PBB-
   EVPN MAC table.There table.

   There is a bi directional traffic flow with F pps in each direction.Then direction.

   Then clear the BGP neighbors in the DUT.

   Once the adjacency BGP session is restored in established state in DUT.

   Measure the time taken to learn 2X MAC address in DUT MAC table.

   Measurement :

   The DUT must learn 2X MAC addresses.Measure addresses.  Measure the time taken to learn
   2X MAC in DUT.The test is repeated for "N" times and the values are
   collected.The
   collected.  The convergence time is calculated by averaging the
   values obtained by "N" samples.

   Convergence  "N" is an arbitrary number to get a
   sufficient sample.The time measured for each sample is denoted by
   T1,T2...Tn.The measurement is carried out using external server which
   polls the DUT using automated scripts.

   Time taken to learn 2X MAC in sec DUT = (T1+T2+..Tn)/N

4.11.  Soak Test

   Objective:

   To measure the stability of the DUT in a scaled environment with
   traffic.

   Topology : Topology 1

   Procedure:

   Confirm the DUT is up and running with PBB-EVPN.

   Scale N PBB-EVPN instances PBB-EVI's in DUT,SHPE3 and MHPE2.Send F frames to DUT from CE
   using traffic generator with X different X source and destination mac MAC
   addresses for N PBB-EVPN instances. EVI's.

   Send F frames from traffic generator to SHPE3 with X different source
   and destination mac addresses.There MAC addresses.

   There will be 2X number of MAC addresses will be learned in DUT PBB-
   EVPN MAC table.There table.

   There is a bi directional traffic flow with F pps in Each direction.The each direction.

   The DUT must run with traffic for 24 hours, every hours.

   Every hour check the for memory leak, crashes. leak in PBB-EVPN process,CPU usage and
   crashes in DUT.

   Measurement :

   Take the hourly reading of CPU, process memory.  There should not be
   any leak, crashes, CPU spikes.The spikes.  Th CPU spike is determined as the CPU
   usage which shoots at 40 to 50 percent of the average usage.The usage.  The
   average value vary from device to device.Memory device.  Memory leak is determined
   by increase usage of the memory for PBB-EVPN process.The process.  The
   expectation is under steady state the memory usage for PBB-EVPN
   process should not increase.  The measurement is carried out using
   external server which polls the DUT using automated scripts which
   captures the CPU usage and process memory.

5.  Acknowledgments  Acknowledgements

   We would like to thank Fioccola Giuseppe of Telecom Italia reviewing
   our draft and commenting it.We it.  We would like to thank Sarah Banks for
   guiding and mentoring us.

6.  IANA Considerations

   This memo includes no request to IANA.

7.  Security Considerations

   The benchmarking tests described in this document are limited to the
   performance characterization of controllers in a lab environment with
   isolated networks.  The benchmarking network topology will be an
   independent test setup and MUST NOT be connected to devices that may
   forward the test traffic into a production network or misroute
   traffic to the test management network.  Further, benchmarking is
   performed on a "black-box" basis, relying solely on measurements
   observable external to the controller.  Special capabilities SHOULD
   NOT exist in the controller specifically for benchmarking purposes.
   Any implications for network security arising from the controller
   SHOULD be identical in the lab and in production networks.

8.  References

8.1.  Normative References

   [RFC2119]  Bradner, S., "Key words for use in RFCs to Indicate
              Requirement Levels", BCP 14, RFC 2119,
              DOI 10.17487/RFC2119, March 1997,
              <https://www.rfc-editor.org/info/rfc2119>.

   [RFC2544]  Bradner, S. and J. McQuaid, "Benchmarking Methodology for
              Network Interconnect Devices", RFC 2544,
              DOI 10.17487/RFC2544, March 1999,
              <https://www.rfc-editor.org/info/rfc2544>.

   [RFC2899]  Ginoza, S., "Request for Comments Summary RFC Numbers
              2800-2899", RFC 2899, DOI 10.17487/RFC2899, May 2001,
              <https://www.rfc-editor.org/info/rfc2899>.

8.2.  Informative References

   [RFC7432]  Sajassi, A., Ed., Aggarwal, R., Bitar, N., Isaac, A.,
              Uttaro, J., Drake, J., and W. Henderickx, "BGP MPLS-Based
              Ethernet VPN", RFC 7432, DOI 10.17487/RFC7432, February
              2015, <https://www.rfc-editor.org/info/rfc7432>.

   [RFC7623]  Sajassi, A., Ed., Salam, S., Bitar, N., Isaac, A., and W.
              Henderickx, "Provider Backbone Bridging Combined with
              Ethernet VPN (PBB-EVPN)", RFC 7623, DOI 10.17487/RFC7623,
              September 2015, <https://www.rfc-editor.org/info/rfc7623>.

Appendix A.  Appendix

Authors' Addresses

   Sudhin Jacob (editor)
   Juniper Networks
   Bangalore
   India

   Phone: +91 8061212543
   Email: sjacob@juniper.net

   Kishore Tiruveedhula
   Juniper Networks
   10 Technology Park Dr
   Westford, MA  01886
   USA

   Phone: +1 9785898861
   Email: kishoret@juniper.net