Internet Engineering Task Force                            S. Jacob, Ed.
Internet-Draft                                           K. Tiruveedhula
Intended status: Standards Track                        Juniper Networks
Expires: November 3, 2019                                    May 2, February 22, 2020                               August 21, 2019

             Benchmarking Methodology for EVPN and PBB-EVPN
                      draft-ietf-bmwg-evpntest-02
                      draft-ietf-bmwg-evpntest-03

Abstract

   This document defines methodologies for benchmarking EVPN and PBB-
   EVPN performance.  EVPN is defined in RFC 7432, and is being deployed
   in Service Provider networks.  Specifically this document defines the
   methodologies for benchmarking EVPN/PBB-EVPN convergence, data plane
   performance, and control plane performance.

Status of This Memo

   This Internet-Draft is submitted in full conformance with the
   provisions of BCP 78 and BCP 79.

   Internet-Drafts are working documents of the Internet Engineering
   Task Force (IETF).  Note that other groups may also distribute
   working documents as Internet-Drafts.  The list of current Internet-
   Drafts is at https://datatracker.ietf.org/drafts/current/.

   Internet-Drafts are draft documents valid for a maximum of six months
   and may be updated, replaced, or obsoleted by other documents at any
   time.  It is inappropriate to use Internet-Drafts as reference
   material or to cite them other than as "work in progress."

   This Internet-Draft will expire on November 3, 2019. February 22, 2020.

Copyright Notice

   Copyright (c) 2019 IETF Trust and the persons identified as the
   document authors.  All rights reserved.

   This document is subject to BCP 78 and the IETF Trust's Legal
   Provisions Relating to IETF Documents
   (https://trustee.ietf.org/license-info) in effect on the date of
   publication of this document.  Please review these documents
   carefully, as they describe your rights and restrictions with respect
   to this document.  Code Components extracted from this document must
   include Simplified BSD License text as described in Section 4.e of
   the Trust Legal Provisions and are provided without warranty as
   described in the Simplified BSD License.

Table of Contents

   1.  Introduction  . . . . . . . . . . . . . . . . . . . . . . . .   3   2
     1.1.  Requirements Language . . . . . . . . . . . . . . . . . .   3
     1.2.  Terminologies . . . . . . . . . . . . . . . . . . . . . .   3
   2.  Test Topology . . . . . . . . . . . . . . . . . . . . . . . .   4   3
   3.  Test Cases for EVPN  Benchmarking . . . . . . . . . . . . . .   6
     3.1.  Local MAC Learning  . . . . . . . . . . .   7
     3.1.  How long it takes to learn local mac address in EVPN . .   7
     3.2.  How long it takes to learn local mac address in PBB EVPN    8
     3.3.  How long it takes to learn the remote macs . . . . . .   6
     3.2.  Remote MAC Learning .   8
     3.4.  PBB-EVPN How long it takes to learn the mac from remote
           peer . . . . . . . . . . . . . . . . . .   7
     3.3.  MAC Flush due to local link failure and Relearning  . . .   7
     3.4.  MAC Flush due to remote link failure. . . . . .   9
     3.5.  How long it takes to flush the local macs due to CE link
           flap and measure the relearning rate of MACs . . . . .   8
     3.5.  MAC Aging .   9
     3.6.  PBB-EVPN  how long it takes to flush the local macs and
           measure the relearning rate of macs during PE-CE  link
           flap . . . . . . . . . . . . . . . . . . . . . . .   8
     3.6.  Remote Mac Aging  . . .  10
     3.7.  How long it takes to flush the remote macs, due to remote
           link failure. . . . . . . . . . . . . . . . . .   9
     3.7.  Local and Remote MAC Learning . . . . .  11
     3.8.  PBB-EVPN How long it takes to flush the remote macs due
           to remote link failure . . . . . . . . .   9
     3.8.  High Availability.  . . . . . . . .  11
     3.9.  To measure the MAC aging time. . . . . . . . . . . .  10
     3.9.  ARP/ND Scale  . .  12
     3.10. PBB-EVPN To measure the MAC aging time. . . . . . . . . .  12
     3.11. How long it takes to age out the remote macs . . . . . .  13
     3.12. PBB-EVPN How long it takes to age out the remote macs. .  13
     3.13. How long it takes to learn both local and remote macs. .  14
     3.14. PBB-EVPN How long it takes to learn both local and remote
           macs . . .  11
     3.10. Scaling of Services . . . . . . . . . . . . . . . . . . .  11
     3.11. Scale Convergence . . . .  14
   4.  High Availability . . . . . . . . . . . . . . . .  12
     3.12. SOAK Test.  . . . . . .  15
     4.1.  To Record the whether there is traffic loss due to
           routing engine failover for redundancy test. . . . . . .  15
     4.2.  PBB-EVPN To Record the whether there is traffic loss due
           to routing engine failover for redundancy test . . . . .  16
   5.  ARP/ND Scale . . . . . .  12
   4.  Test Cases for PBB-EVPN  Benchmarking . . . . . . . . . . . .  13
     4.1.  Local MAC Learning  . . . . . .  16
     5.1.  To find ARP/ND scale . . . . . . . . . . . . .  13
     4.2.  Remote Mac Learning . . . . .  16
   6.  Scale . . . . . . . . . . . . . .  13
     4.3.  MAC Flush due to link failure . . . . . . . . . . . . . .  17
     6.1.  To Measure the scale limit of DUT with trigger (Scale
           without traffic)  14
     4.4.  MAC Flush due to remote Failure . . . . . . . . . . . . .  14
     4.5.  MAC aging . . . . . . .  17
     6.2.  PBB-EVPN To measure the scale limit with trigger. . . . .  17
     6.3.  To measure the convergence time of DUT with scale and
           traffic. . . . . . . . . . . . . .  15
     4.6.  Remote MAC Aging. . . . . . . . . . . .  18
     6.4.  .PBB-EVPN To measure the convergence time of DUT with
           scale . . . . . . . . .  16
     4.7.  Local and traffic. Remote MAC Learning . . . . . . . . . . . . . .  16
     4.8.  High Availability . . . . .  18
   7.  SOAK Test . . . . . . . . . . . . . . .  17
     4.9.  Scale . . . . . . . . . . .  19
     7.1.  To Measure the stability of the DUT with scale and
           traffic. . . . . . . . . . . . . . . .  17
     4.10. Scale Convergence . . . . . . . . .  19
     7.2.  PBB-EVPN to measure the stability of DUT with scale and
           traffic. . . . . . . . . . . .  18
     4.11. Soak Test . . . . . . . . . . . . .  20

   8. . . . . . . . . . . .  18
   5.  Acknowledgements  . . . . . . . . . . . . . . . . . . . . . .  20
   9.  19
   6.  IANA Considerations . . . . . . . . . . . . . . . . . . . . .  20
   10.  19
   7.  Security Considerations . . . . . . . . . . . . . . . . . . .  20
   11.  19
   8.  References  . . . . . . . . . . . . . . . . . . . . . . . . .  20
     11.1.  19
     8.1.  Normative References  . . . . . . . . . . . . . . . . . .  21
     11.2.  19
     8.2.  Informative References  . . . . . . . . . . . . . . . . .  21  20
   Appendix A.  Appendix . . . . . . . . . . . . . . . . . . . . . .  21  20
   Authors' Addresses  . . . . . . . . . . . . . . . . . . . . . . .  21  20

1.  Introduction

   EVPN is defined in RFC 7432, and describes BGP MPLS- based Ethernet
   VPNs (EVPN).  PBB-EVPN is defined in RFC 7623, discusses how Ethernet
   Provider backbone Bridging can be combined with EVPNs to provide a
   new/combined solution.  This draft defines methodologies that can be
   used to benchmark both RFC 7432 and RFC 7623 solutions.  Further,
   this draft provides methodologies for benchmarking the performance of
   EVPN data and control planes, MAC learning, MAC flushing, MAC ageing,
   convergence, high availability, and scale.

1.1.  Requirements Language

   The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
   "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
   document are to be interpreted as described in RFC 2119 [RFC2119].

1.2.  Terminologies

   MHPE Multi homed Provide Edge router.

   RR Route Reflector.

   P Provider Router.

   CE Customer Router/Devices/Switch.

   MHPE2 Multi homed Provider Edge router 2.

   MHPE1 Multi homed Provider Edge router 1.

   SHPE3 Single homed Provider Edge Router 3.

   AA EVPN Terminologies AA All-Active.

   SA EVPN Terminologies SA Single-Active.

   RT Router Tester.

   Sub Interface Each physical Interfaces is subdivided in to Logical
   units.

   EVI EVPN Instances which will be running on sub interface or physical
   port of the provider Edge routers.

   DF Designated Forwarder.

   ESI Ethernet Segment Identifier.

2.  Test Topology

   EVPN/PBB-EVPN Services running on SHPE3, MHPE1 and MHPE2 in Single
   Active Mode:

                             Topology Diagram

         | [Traffic Generator ] Router Tester traffic receiver for sender/receiver of layer 2 traffic from CE with multiple vlan.
+----------+
|          |
|  SHPE3   |
|  SHPE3          |
+----------+
    |
    |Core link
+----------+
|          |
|  RR      |
|          | Route Reflector/Core router
+----------+-------------|
   |                     |
   |     Core links      |
+----------+       +-----------+
|          |       |    MHPE2  |
|   DUT    |       |           |
|  MHPE1   |       |           |
+----------+       +-----------+
     |    PE-CE link    |
+----------+------------
|          |
|  CE      |
|  layer2  |
|bridge    |
+----------+------------ [Traffic Generator](Router Tester sending sender/reciever of layer 2 traffic with different VLAN ) multiple vlan)

Topology 1

                                Test Setup

                                 Figure 1

+-----------------+---------------------+---------------------+---------------------+----------------------+-----------------------+
| [Traffic Generator ] Router Tester sending layer 2 traffic.
+----------+                 |                     |                     |  SHPE3                     |                      |  SHPE3                       |
+----------+
|
    |Core link
+----------+                 |                     |                     |  RR                     |                      |                       | Route Reflector/Core router
+----------+-------------|
|                 |                     |     Core links                     |
+----------+       +-----------+                     |                      |                       |    MHPE2
|                 |   DUT                     |                     |                     |                      |  MHPE1                       |
| Mode            |
+----------+       +-----------+                     |    PE-CE link                     |
+----------+------------                     |Receiver              |                       |
|  CE                 |  Test               |Traffic Direction    |Sender               |  layer2                      |
|bridge                       |
+----------+------------ [Traffic Generator](Router Tester receiver for layer 2 traffic with different vlans.)

Topology 2
| [Traffic Generator ] Router Tester  sending layer 2 bi directional traffic sender/receiver
+----------+                 |                     |                     |  SHPE3                     |                      |  SHPE3                       |
+----------+
|
    |Core link
+----------+                 |                     |                     |  RR                     |                      |                       | Route Reflector/Core router
+----------+-------------|
|                 |                     |     Core links                     |
+----------+       +-----------+                     |                      |                       |    MHPE2
+----------------------------------------------------------------------------------------------------------------------------------+
|                 |   DUT                     |                     |                     |                      |  MHPE1                       |
|                 |
+----------+       +-----------+                     |    PE-CE link                     |
+----------+------------                     |      SHPE3           |                       |
|Single Active    |  Local Mac          |                     |CE                   |                      |Layer 2 traffic        |
|                 | Learning            | Uni                 |                     |                      |                       |
|                 |                     |                     |                     |                      | multiple MAC          |
|                 |                     |                     |                     |                      |                       |
+-----------------------------------------------------------------------------------------------------------------------------------+
|                 |                     |                     |                     |                      |                       |
|Single Active    | Remote MAC          |                     |                     |         CE           |Layer 2 traffic        |
|  layer2                 |
|bridge Learning            |
+----------+------------ [Traffic Generator](Router Tester sending bi directional layer uni                 | SHPE3               |                      |                       |
|                 |                     |                     |                     |                      |multiple MAC           |
|                 |                     |                     |                     |                      |                      ++
+----------------------------------------------------------------------------------------------------------------------------------+
|                 |                     |                     |                     |                      |                       |
|Single Active    | Scale Convergence   |   Bi                |                     |  CE/SHPE3            |                       |
|                 |                     |                     |   CE/SHPE3          |                      |Layer 2 traffic with different VLAN sender/receiver)

Topology 3

                             Topology Diagram        |
|                 | Local& Remote       |                     |                     |                      |multiple mac& vlans    |
|                 | Learning            |                     |                     |                      |                       |
+-----------------+---------------------+---------------------+--------------------------------------------+-----------------------+
                                                                                    |
                                                                                   ++

              Table showing Traffic pattern for various test

                                 Figure 1 2

   Test Setup Configurations:

   There are five routers in the topology. Test setup.  SHPE3, RR/P, MHPE1 and
   MHPE2 emulating a service provider network.  CE is a customer device
   connected to MHPE1 and MHPE2, it is configured with bridge domains in
   different
   multiple vlans.  The router tester is connected to CE and SHPE3.The
   MHPE1 acts as DUT.The RT will act be used as sender and receiver.The receiver of
   traffic.The measurement will be taken in DUT.

   All routers except CE is configured with OSPF/IS-IS,LDP,MPLS,BGP with
   EVPN address family.

   All routers except CE must have IBGP configured with RR acting as
   route reflector.

   MHPE1,MHPE2,SHPE3 must be configured with "N" EVPN/PBB-EVPN instances
   depends up on the cases.

   MHPE1 and MHEPE2 must be configured with ESI per vlan or ESI on IFD.

   MHPE1 and MHEPE2 are running Single Active mode of EVPN.

   CE is acting as bridge configured with vlans that is configured on
   MHPE1,MHPE2,SHPE3.

   Depends up on the test traffic will be flowing uni directional or bi
   directional depends on the topology mentioned above. test performed.

   The above configuration will serve be serving as the base configuration for
   all the test cases.

3.  Test Cases

   The following tests are conducted to measure for EVPN Benchmarking

3.1.  Local MAC Learning

   Objective:

   To Record the time taken to learn the "X" number of MAC's locally in EVI . The data plane learning of MAC will happen address locally from connected interface.  The control plane
   learning of MAC is through BGP advertisements from the remote
   PE(SHPE3).  The control plane learning of "X" MAC. in DUT.

   Topology : Topology 1

   Procedure:

   The data plane MAC learning can be measured using the parameters
   defined in RFC 2889 section 5.8.

3.1.  How long it takes to learn local mac address in EVPN

   Objective:

   To Record the time taken to learn the MAC address locally in DUT.

   Topology : Topology 1

   Procedure:  Send "X" unicast frames from CE to
   MHPE1(DUT) working in SA mode with "X" different source and
   destination address from RT.  The DUT must learn these "X" macs in
   data plane.

   Measurement :

   Measure the time taken to learn "X" MACs in DUT evpn mac table.  The
   data plane measurement is taken by considering DUT as black box the
   range of X MAC is known from RT and the same must be learned in DUT,
   the time taken to learn "X" macs is measured.

   Repeat these test and plot the data.  The test is repeated for "N"
   times and the values are collected.  The mac learning time is
   calculated by averaging the values obtained from "N" samples.

   Mac learning in sec = (T1+T2+..Tn/N)

3.2.  How long it takes to learn local mac address in PBB EVPN  Remote MAC Learning

   Objective:

   To Record the time taken to learn the MAC address locally. remote macs.

   Topology : Topology 1

   Procedure:

   Send "X" unicast X frames from CE to MHPE1(DUT) working in SA mode with
   "X" X different source SA and destination address DA to SHPE3 from RT.  The DUT must
   learn "X"  SHPE3
   will advertise these locally learned macs to MHPE1 and MHPE2 via
   control plane.Measure the time taken to learn these X MACs from
   remote peer in data plane. DUT EVPN MAC address table.The DUT and MHPE2 are
   running SA mode.

   Measurement :

   Measure the time taken by the DUT to learn the "X" MACs in the data
   plane.  The data plane measurement is taken by considering DUT as
   black box the range of "X" MAC is known from RT and the same must be
   learned in DUT, the time taken to learn "X" MAC is measured.  Repeat
   plane.Repeat these test and plot the data.  The data.The test is repeated for
   "N" times and the values are collected.  The collected.The mac learning time is
   calculated by averaging the values obtained from "N" samples.

   Mac learning in sec = (T1+T2+..Tn/N)

3.3.  How long it takes  MAC Flush due to learn the remote macs local link failure and Relearning

   Objective:

   To Record record the time taken to learn flush the remote mac learned locally and the
   time taken to relearn the same amount of macs.

   Topology : Topology 2 1

   Procedure:

   Send X frames with X different SA and DA to SHPE3 DUT from RT.  SHPE3
   will advertise these locally learned macs to CE using traffic
   generator.  Wait till the MHPE1 learns all X MAC address.  Then fail
   the MHPE1 CE link and MHPE2 via
   control plane.Measure measure the time taken to learn flush these X MACs
   from
   remote peer in DUT the EVPN MAC address table.The table.  Bring up the link which was made Down(the
   link between MHPE1 and CE).Measure time taken to relearn it.  The DUT
   and MHPE2 are running SA mode.

   Measurement :

   Measure the time taken by for flushing these X MAC address.  Measure the DUT
   time taken to learn relearn the "X" X MACs in the data
   plane.Repeat DUT.  Repeat these test and plot
   the data.The data.  The test is repeated for "N" times and the values are collected.The mac learning
   collected.  The flush and the relearning time is calculated by
   averaging the values obtained from by "N" samples.

   Mac learning

   Flush time for X Macs in sec = (T1+T2+..Tn/N)

   Relearning time for X macs in sec = (T1+T2+..Tn/N)

3.4.  PBB-EVPN How long it takes  MAC Flush due to learn the mac from remote peer link failure.

   Objective:

   To Record record the time taken to learn flush the remote macs.

   Topology : Topology 2

   Procedure: mac learned in DUT
   during remote link failure.

   Topology : Topology 1

   Procedure:

   Send X frames with X different SA and DA to SHPE3 DUT from RT.These macs
   will be flooded to MHPE1 SHPE3 using
   traffic generator.  Bring down the link between SHPE3 and MHPE2 by SHPE3.The traffic
   generator.  Then measure the time taken to flush the DUT EVPN MAC
   table.  The DUT and MHPE2 are running SA mode.

   Measurement :

   Measure the time taken to learn flush X mac address in DUT mac table. remote MACs from EVPN MAC table of
   DUT.  Repeat these test and plot the data.The data.  The test is repeated for
   "N" times and the values are collected.The mac learning time collected.  The flush rate is calculated by
   averaging the values obtained by "N" samples.

   Mac learning

   Flush time for X Macs in sec = (T1+T2+..Tn/N)

3.5.  How long it takes to flush the local macs due to CE link flap and
      measure the relearning rate of MACs  MAC Aging

   Objective:

   To record the time taken to flush measure the mac learned locally and the
   time taken to relearn the same amount of macs. aging time.

   Topology : Topology 1
   Procedure:

   Send X frames with X different SA and DA to DUT from CE using traffic
   generator.  Wait till the MHPE1 learns all X MAC address. address are learned.  Then fail stop the MHPE1 CE link and measure
   traffic.  Record the time taken to flush these X MACs MACS from the DUT EVPN MAC table.  Bring up the link which was made Down(the
   link between MHPE1 and CE).Measure time taken
   table due to relearn it. aging.  The DUT and MHPE2 are running SA mode. mode

   Measurement :

   Measure the time taken for flushing these to flush X MAC address.  Measure the
   time taken address due to relearn the X MACs in DUT. aging.  Repeat
   these test and plot the data.  The test is repeated for "N" times and
   the values are collected.  The flush and the relearning time aging is calculated by averaging the
   values obtained by "N" samples.

   Flush

   Aging time for X Macs in sec = (T1+T2+..Tn/N) Relearning time for X
   macs in sec = (T1+T2+..Tn/N)

3.6.  PBB-EVPN how long it takes to flush the local macs and measure the
      relearning rate of macs during PE-CE link flap  Remote Mac Aging

   Objective:

   To record the time taken to flush measure the remote mac learned locally and the
   time taken to relearn the same amount of macs. aging time.

   Topology : Topology 1

   Procedure:

   Send X frames with X different SA and DA to DUT from CE SHPE3 using
   traffic generator.  Wait till  Stop the MHPE1 learn all X traffic at remote PE SHPE3.Due to MAC address.  Then fail
   the MHPE1 CE link
   aging SHPE3 will withdraw its routes from DUT and measure MHPE2.  Measure the
   time taken to flush remove these X MACs from the PBB-EVPN DUT EVPN MAC table.  Then bring up the link.  Measure the
   time taken to relearn X MACS.  The  DUT and
   MHPE2 are running in SA mode. mode

   Measurement :

   Measure the time taken for flushing these X MAC address.  Measure the
   time taken to relearn the flush X remote MACs learned in DUT. DUT EVPN MAC
   table due to aging.  Repeat these test and plot the data.  The test
   is repeated for "N" times and the values are collected.  The flush and the relearning time aging is
   calculated by averaging the values obtained by "N" samples.

   Flush

   Aging time for X Macs in sec = (T1+T2+..Tn/N) Relearning time for X
   macs in sec = (T1+T2+..Tn/N)

3.7.  How long it takes to flush the remote macs, due to remote link
      failure.  Local and Remote MAC Learning

   Objective:

   To record the time taken to flush the remote mac learned in DUT
   during learn both local and remote link failure. macs.

   Topology : Topology 2 1

   Procedure:

   Send X frames with X different SA and DA to DUT from SHPE3 using
   traffic generator.  Bring down the link between SHPE3  Send X frames with different SA and DA from
   traffic
   generator.  Then measure generator connected to CE.  The SA and DA of flows must be
   complimentary to have unicast flows.  Measure the time taken to flush by the
   DUT to learn 2X in EVPN MAC
   table.  The MAC.  DUT and MHPE2 are running in SA mode.

   Measurement :

   Measure the time taken to flush X remote MACs from learn 2X MAC address in DUT EVPN MAC table of
   DUT. table.
   Repeat these test and plot the data.  The test is repeated for "N"
   times and the values are collected.  The flush rate mac learning time is
   calculated by averaging the values obtained by "N" samples.

   Flush time for X

   Time to learn 2X Macs in sec = (T1+T2+..Tn/N)

3.8.  PBB-EVPN How long it takes to flush the remote macs due to remote
      link failure  High Availability.

   Objective:

   To record the time taken to flush the remote mac learned in DUT traffic loss during remote link failure. routing engine fail over.

   Topology : Topology 2 1

   Procedure:

   Send X frames from CE to DUT from traffic generator withX different
   SA and DA.  Send X frames from traffic generator to SHPE3 with X
   different SA and DA to DUT from SHPE3 using so that 2X MAC address will be learned in DUT.
   There is a bi directional traffic flow with X pps in each direction.
   Then do a routing engine fail-over.

   Measurement :

   There should be 0 traffic loss which is the ideal case, No change in
   the DF role.  DUT should not withdraw any routes.Repeat the test "N"
   times and plot the data.The packet loss is calculated by averaging
   the values obtained from "N" samples.

   Packet loss in sec = (T1+T2+..Tn/N)

3.9.  ARP/ND Scale

   These tests are conducted to Record the scaling parameter of ARP/ND
   of the DUT.

   Objective:

   To Record the ARP/ND scale of the DUT.

   Topology : Topology 1

   Procedure:

   Send X arp/icmpv6 request from RT to DUT with different sender ip/
   ipv6 address to the same target gateway ip address.  Measure whether
   X MAC+IPv4 address/MAC+IPv6 address of the hosts are learned in DUT.

   Measurement :

   The DUT must learn X MAC+IPV4/MAC+IPv6 and it must advertise the X
   MAC+IPV4/MAC+IPV6 to the remote router.

3.10.  Scaling of Services

   Objective:

   To measure the scale limit of DUT for EVPN.This is to measure the
   performance of DUT in scaling to "X" EVPN instances.

   Topology : Topology 1

   Procedure:

   The DUT,MHPE2 and SHPE3 are scaled to "N" EVI.Clear BGP neighbors of
   the DUT.  Once adjacency is established in the DUT.  Measure the
   routes received from MHPE2 and SHPE3 for "N" EVI in the DUT.

   Measurement :

   There should not be any loss of route types 1,2,3 and 4 in DUT.  DUT
   must relearn all type 1,2,3 and 4 from remote routers.  The DUT must
   be subjected to various values of N to find the optimal scale limit

3.11.  Scale Convergence

   Objective:

   To measure the convergence time of DUT when the DUT is scaled with
   EVPN instance along with traffic.

   Topology : Topology 1

   Procedure:

   Scale N EVIs in DUT,SHPE3 and MHPE2.Send F frames to DUT from CE
   using traffic generator with X different SA and DA for N EVI's.  Send
   F frames from traffic generator to SHPE3 with X different SA and DA.
   There will be 2X number of MAC address will be learned in DUT EVPN
   MAC table.  There is a bi directional traffic flow with F pps in each
   direction.  Then clear the BGP neighbors in the DUT.  Once the
   adjacency is restored in DUT.  Measure the time taken to learn 2X MAC
   address in DUT MAC table.

   Measurement :

   The DUT must learn 2X MAC address.  Measure the time taken to learn
   2X MAC in DUT.  Repeat these test and plot the data.The test is
   repeated for "N" times and the values are collected.The convergence
   time is calculated by averaging the values obtained by "N" samples.

   Convergence time in sec = (T1+T2+..Tn/N)

3.12.  SOAK Test.

   Objective:

   This test is carried out to measure the stability of the DUT in a
   scaled environment with traffic generator.  Bring down over a period of time "T'".  In each
   interval "t1" the link between DUT CPU usage, memory usage are measured.  The DUT
   is checked for any crashes during this time period.

   Topology : Topology 1

   Procedure:

   Scale N EVI's in DUT,SHPE3 and MHPE2.Send F frames to DUT from CE
   using traffic generator with different X SA and DA for N EVI's.  Send
   F frames from traffic generator to SHPE3 with X different SA and DA.
   There will be 2X number of MAC address will be learned in DUT EVPN
   MAC table.  There is a bi directional traffic flow with F pps in each
   direction.  The DUT must run with traffic for 24 hours, every hour
   check for memory leak, CPU usage and crash.

   Measurement :

   Take the hourly reading of CPU, process memory.  There should not be
   any leak, crashes, CPU spikes.

4.  Test Cases for PBB-EVPN Benchmarking

4.1.  Local MAC Learning

   Objective:

   To Record the time taken to learn the MAC address locally.

   Topology : Topology 1

   Procedure:

   Send "X" unicast frames from CE to MHPE1(DUT) working in SA mode with
   "X" different source and traffic
   generator.  Then measure destination address from RT.  The DUT must
   learn "X" macs in data plane.

   Measurement :

   Measure the time taken to flush by the DUT PBB-EVPN MAC
   address table.  The remote to learn the "X" MACs will be learned by Data plane, but in the B-MAC will be learned by control data
   plane.  The data plane measurement is taken by considering DUT as
   black box the range of "X" MAC is known from RT and MHPE2 are
   running SA mode.

   Measurement :

   Measure the same must be
   learned in DUT, the time taken to flush X remote MACs from PBB-EVPN learn "X" MAC table
   of DUT. is measured.  Repeat
   these test and plot the data.  The test is repeated for "N" times and
   the values are collected.  The flush rate mac learning time is calculated by
   averaging the values obtained by from "N" samples.

   Flush time for X Macs

   Mac learning in sec = (T1+T2+..Tn/N)

3.9.  To measure the MAC aging time.

4.2.  Remote Mac Learning

   Objective:

   To measure Record the mac aging time. time taken to learn the remote macs.

   Topology : Topology 1

   Procedure:

   Send X frames with X different SA and DA to DUT from CE using traffic
   generator.  Wait till X MAC address are learned.  Then stop the
   traffic.  Record the time taken to flush X MACS SHPE3 from DUT EVPN MAC
   table due RT.These macs
   will be flooded to aging.  The MHPE1 and MHPE2 by SHPE3.The DUT and MHPE2 are
   running SA mode mode.

   Measurement :

   Measure the time taken to flush learn X MAC mac address due to aging. in DUT mac table.
   Repeat these test and plot the data.  The data.The test is repeated for "N"
   times and the values are collected.  The aging collected.The mac learning time is
   calculated by averaging the values obtained by "N" samples.

   Aging time for X Macs

   Mac learning in sec = (T1+T2+..Tn/N)

3.10.  PBB-EVPN To measure the

4.3.  MAC aging time. Flush due to link failure

   Objective:

   To measure record the time taken to flush the mac aging time. learned locally and the
   time taken to relearn the same amount of macs.

   Topology : Topology 1

   Procedure:

   Send X frames with X different SA and DA to DUT from CE using traffic
   generator.  Wait till the MHPE1 learn all X MAC address are learned in DUT PBB- EVPN MAC
   table. address.  Then stop fail
   the traffic.  Record MHPE1 CE link and measure the time taken to flush these X MACs
   from the PBB-EVPN MAC
   entries due table.  Then bring up the link.  Measure the
   time taken to aging. relearn X MACS.  The DUT and MHPE2 are running in SA mode mode.

   Measurement :

   Measure the time taken to flush for flushing these X MAC address due address.  Measure the
   time taken to aging. relearn the X MACs in DUT.  Repeat these test and plot
   the data.  The test is repeated for "N" times and the values are
   collected.  The aging flush and the relearning time is calculated by
   averaging the values obtained by "N" samples.

   Aging

   Flush time for X Macs in sec = (T1+T2+..Tn/N)

3.11.  How long it takes

   Relearning time for X macs in sec = (T1+T2+..Tn/N)

4.4.  MAC Flush due to age out the remote macs Failure

   Objective:

   To measure record the time taken to flush the remote mac aging time. learned in DUT
   during remote link failure.

   Topology : Topology 2 1

   Procedure:

   Send X frames with X different SA and DA to DUT from SHPE3 using
   traffic generator.  Stop  Bring down the traffic at remote PE SHPE3.Due to MAC
   aging link between SHPE3 will withdraw its routes from DUT and MHPE2.  Measure traffic
   generator.  Then measure the time taken to remove these MACs from flush the DUT EVPN PBB-EVPN MAC
   address table.  The remote MACs will be learned by Data plane, but
   the B-MAC will be learned by control plane.  The DUT and MHPE2 are
   running in SA mode mode.

   Measurement :

   Measure the time taken to flush X remote MACs learned in DUT EVPN from PBB-EVPN MAC table due to aging.
   of DUT.  Repeat these test and plot the data.  The test is repeated
   for "N" times and the values are collected.  The aging flush rate is
   calculated by averaging the values obtained by "N" samples.

   Aging

   Flush time for X Macs in sec = (T1+T2+..Tn/N)

3.12.  PBB-EVPN How long it takes to age out the remote macs.

4.5.  MAC aging

   Objective:

   To measure the remote mac aging time.

   Topology : Topology 2 1

   Procedure:

   Send X frames with X different SA and DA to DUT from SHPE3 CE using traffic
   generator.  Stop  Wait till X MAC address are learned in DUT PBB- EVPN MAC
   table.  Then stop the traffic at remote PE(SHPE3).Measure traffic.  Record the time taken to remove these remote MACs from DUT PBB-EVPN flush X MAC table.
   entries due to aging.  The DUT and MHPE2 are running in SA mode. mode

   Measurement :

   Measure the time taken to flush the X remote MACs from DUT PBB-EVPN MAC table address due to aging aging.  Repeat
   these test and plot the data.  The test is repeated for "N" times and
   the values are collected.  The aging is calculated by averaging the
   values obtained by "N" samples.

   Aging time for X Macs in sec = (T1+T2+..Tn/N)

3.13.  How long it takes to learn both local and remote macs.

4.6.  Remote MAC Aging.

   Objective:

   To record measure the time taken to learn both local and remote macs. mac aging time.

   Topology : Topology 3 1

   Procedure:

   Send X frames with X different SA and DA to DUT from SHPE3 using
   traffic generator.  Send X frames with different SA and DA from  Stop the traffic generator connected to CE.  The SA and DA of flows must be
   complimentary to have unicast flows.  Measure at remote PE(SHPE3).Measure the
   time taken by the
   DUT to learn 2X in EVPN MAC. remove these remote MACs from DUT PBB-EVPN MAC table.
   The DUT and MHPE2 are running in SA mode.

   Measurement :

   Measure the time taken to learn 2X MAC address in flush the X remote MACs from DUT EVPN PBB-EVPN
   MAC table. table due to aging Repeat these test and plot the data.  The test
   is repeated for "N" times and the values are collected.  The mac learning time aging is
   calculated by averaging the values obtained by "N" samples.

   Time to learn 2X

   Aging time for X Macs in sec = (T1+T2+..Tn/N)

3.14.  PBB-EVPN How long it takes to learn both local

4.7.  Local and remote macs Remote MAC Learning

   Objective:

   To record the time taken to learn both local and remote macs.

   Topology : Topology 3 1

   Procedure:

   Send X frames with X different SA and DA to DUT from SHPE3 using
   traffic generator.  Send X frames with different SA and DA from
   traffic generator connected to CE.  The SA and DA of flows must be
   complimentary to have unicast flows.  Measure the time taken by the
   DUT to learn 2X in MAC table.  DUT and MHPE2 are running in SA mode.

   Measurement :

   Measure the time taken to learn 2X MAC address table in DUT PBB-EVPN
   MAC table.  Repeat these test and plot the data.  The test is
   repeated for "N" times and the values are collected.  The mac
   learning time is calculated by averaging the values obtained by "N"
   samples.

   Time to learn 2X Macs in sec = (T1+T2+..Tn/N)

4.  High Availability

4.1.  To Record the whether there is traffic loss due to routing engine
      failover for redundancy test.

   Objective:

   To record traffic loss during routing engine failover.

   Topology : Topology 3

   Procedure:

   Send X frames from CE to DUT from traffic generator withX different
   SA and DA.  Send X frames from traffic generator to SHPE3 with X
   different SA and DA so that 2X MAC address will be learned in DUT.
   There is a bi directional traffic flow with X pps in each direction.
   Then do a routing engine fail-over.

   Measurement :

   There should be 0 traffic loss which is the ideal case, No change in
   the DF role.  DUT should not withdraw any routes.Repeat the test "N"
   times and plot the data.The packet loss values are collected.  The mac
   learning time is calculated by averaging the values obtained from by "N"
   samples.

   Packet loss

   Time to learn 2X Macs in sec = (T1+T2+..Tn/N)

4.2.  PBB-EVPN To Record the whether there is traffic loss due to
      routing engine failover for redundancy test

4.8.  High Availability

   Objective:

   To record traffic loss during routing engine failover.

   Topology : Topology 3 1

   Procedure:

   Send X frames to DUT with X different SA and DA from CE using the
   traffic generator.  Send X frames from traffic generator to SHPE3
   with X different SA and DA so that 2X MAC address will be Learned in
   DUT.  There is a bi directional traffic flow with X pps in each
   direction.  Then do a routing engine fail-over.

   Measurement :

   There should be 0 traffic loss which is the ideal case, No change in
   the DF role.  DUT should not withdraw any routes.Repeat the test "N"
   times and plot the data.The packet loss is calculated by averaging
   the values obtained from "N" samples.

   Packet loss in sec = (T1+T2+..Tn/N)

5.  ARP/ND Scale

   These tests are conducted to Record the scaling parameter of ARP/ND
   of the DUT.

5.1.  To find ARP/ND scale

   Objective:

   To Record the ARP/ND scale of the DUT.

   Topology : Topology 1

   Procedure:

   Send X arp/icmpv6 request from RT to DUT with different sender ip/
   ipv6 address to the same target gateway ip address.  Measure whether
   X MAC+IPv4 address/MAC+IPv6 address of the hosts are learned in DUT.

   Measurement :

   The DUT must learn X MAC+IPV4/MAC+IPv6 and it must advertise the X
   MAC+IPV4/MAC+IPV6 to the remote router.

6.  Scale

   This is to measure the performance of DUT in scaling to "X" EVPN
   instances.  The measured parameters are CPU usage, memory
   leak,crashes.

6.1.  To Measure the scale limit of DUT with trigger (Scale without
      traffic)

   Objective:

   To measure the scale limit of DUT for EVPN.

   Topology : Topology 3

   Procedure:

   The DUT,MHPE2 and SHPE3 are scaled to "N" EVI.Clear BGP neighbors of
   the DUT.  Once adjacency is established in the DUT.  Measure the
   routes received from MHPE2 and SHPE3 for "N" EVI in the DUT.

   Measurement :

   There should not be any loss of route types 1,2,3 and 4 in DUT.
   the DF role.  DUT
   must relearn all type 1,2,3 should not withdraw any routes.Repeat the test "N"
   times and 4 from remote routers.  The DUT must
   be subjected to various values of N to find plot the optimal scale limit

6.2.  PBB-EVPN To measure data.The packet loss is calculated by averaging
   the scale limit with trigger. values obtained from "N" samples.

   Packet loss in sec = (T1+T2+..Tn/N)

4.9.  Scale

   Objective:

   To measure the scale limit of DUT for PBB-EVPN.

   Topology : Topology 3 1

   Procedure:

   The DUT,MHPE2 and SHPE3 are scaled to "N" PBB-EVPN instances.  Clear
   BGP neighbors in the DUT Once adjacency is established in DUT, check
   routes received from SHPE3 and MHPE2.

   Measurement :

   There should not be any loss of route types 2,3 and 4 in DUT.  The
   DUT must relearn all type 2,3 and 4 routes from remote routers.  The
   DUT must be subjected to various values of N to find the optimal
   scale limit.

6.3.  To measure the convergence time of DUT with scale and traffic.

   Objective:

   To measure the convergence time of DUT when the DUT is scaled with
   EVPN instance along with traffic.

   Topology : Topology 3

   Procedure:

   Scale N EVIs in DUT,SHPE3 and MHPE2.Send F frames to DUT from CE
   using traffic generator with X different SA and DA for N EVI's.  Send
   F frames from traffic generator to SHPE3 with X different SA and DA.
   There will be 2X number of MAC address will be learned in DUT EVPN
   MAC table.  There is a bi directional traffic flow with F pps in each
   direction.  Then clear the BGP neighbors in the DUT.  Once the
   adjacency is restored in DUT.  Measure the time taken to learn 2X MAC
   address in DUT MAC table.

   Measurement :

   The DUT must learn 2X MAC address.  Measure the time taken to learn
   2X MAC in DUT.  Repeat these test and plot the data.The test is
   repeated for "N" times  The
   DUT must relearn all type 2,3 and the values are collected.The convergence
   time is calculated by averaging the 4 routes from remote routers.  The
   DUT must be subjected to various values obtained by "N" samples.

   Convergence time in sec = (T1+T2+..Tn/N)

6.4.  .PBB-EVPN To measure the convergence time of DUT with N to find the optimal
   scale and
      traffic. limit.

4.10.  Scale Convergence

   Objective:

   To measure the convergence time of DUT when the DUT is scaled with
   PBB-EVPN
   EVPN instance along with traffic.

   Topology : Topology 3 1

   Procedure:

   Scale N PBB-EVI's PBB-EVIs in DUT,SHPE3 and MHPE2.Send F frames to DUT from CE
   using traffic generator with X different SA and DA for N EVI's. PBB-EVI's.
   Send F frames from traffic generator to SHPE3 with X different SA and
   DA.  There will be 2X number of MAC address will be learned in DUT PBB-
   EVPN
   PBB-EVPN MAC table.  There is a bi directional traffic flow with F
   pps in each direction.  Then clear the BGP neighbors in the DUT.
   Once the adjacency is restored in DUT.  Measure the time taken to
   learn 2X MAC address in DUT PBB-MAC MAC table.

   Measurement :

   The DUT must learn 2X MAC address.  Measure the time taken to learn
   2X MAC in DUT.  Repeat these test and plot the data.  The data.The test is
   repeated for "N" times and the values are collected.  The collected.The convergence
   time is calculated by averaging the values obtained by "N" samples.

   Convergence time in sec = (T1+T2+..Tn/N)

7.  SOAK

4.11.  Soak Test

   This is measuring the performance of DUT running with scaled
   configuration with traffic over a peroid of time "T'".  In each
   interval "t1" the parameters measured are CPU usage, memory usage,
   crashes.

7.1.  To Measure the stability of the DUT with scale and traffic.

   Objective:

   To measure the stability of the DUT in a scaled environment with
   traffic.

   Topology : Topology 3

   Procedure:

   Scale N EVI's in DUT,SHPE3 and MHPE2.Send F frames to DUT from CE
   using traffic generator with different X SA and DA for N EVI's.  Send
   F frames from traffic generator to SHPE3 with X different SA and DA.
   There will be 2X number of MAC address will be learned in DUT EVPN
   MAC table.  There is a bi directional traffic flow with F pps in each
   direction.  The DUT must run with traffic for 24 hours, every hour
   check for memory leak, crash.

   Measurement :

   Take the hourly reading of CPU, process memory.  There should not be
   any leak, crashes, CPU spikes.

7.2.  PBB-EVPN to measure the stability of DUT with scale and traffic.

   Objective:

   To measure the stability of the DUT in a scaled environment with
   traffic.

   Topology : Topology 3 1

   Procedure:

   Scale N PBB-EVI's in DUT,SHPE3 and MHPE2.Send F frames to DUT from CE
   using traffic generator with X different SA and DA for N EVI's.  Send
   F frames from traffic generator to SHPE3 with X different SA and DA.
   There will be 2X number of MAC address will be learned in DUT PBB-
   EVPN MAC table.  There is a bi directional traffic flow with F pps in
   Each direction.  The DUT must run with traffic for 24 hours, every
   hour check the memory leak, crashes.

   Measurement :

   Take the hourly reading of CPU process, memory usages.  There should
   not be any memory leak, crashes,CPU spikes.

8.

5.  Acknowledgements

   We would like to thank Fioccola Giuseppe of Telecom Italia reviewing
   our draft and commenting it.  We would like to thank Sarah Banks for
   guiding and mentoring us.

9.

6.  IANA Considerations

   This memo includes no request to IANA.

10.

7.  Security Considerations

   There is no additional consideration from RFC 6192.

11.

8.  References
11.1.

8.1.  Normative References

   [RFC2119]  Bradner, S., "Key words for use in RFCs to Indicate
              Requirement Levels", BCP 14, RFC 2119,
              DOI 10.17487/RFC2119, March 1997,
              <https://www.rfc-editor.org/info/rfc2119>.

   [RFC2544]  Bradner, S. and J. McQuaid, "Benchmarking Methodology for
              Network Interconnect Devices", RFC 2544,
              DOI 10.17487/RFC2544, March 1999,
              <https://www.rfc-editor.org/info/rfc2544>.

   [RFC2899]  Ginoza, S., "Request for Comments Summary RFC Numbers
              2800-2899", RFC 2899, DOI 10.17487/RFC2899, May 2001,
              <https://www.rfc-editor.org/info/rfc2899>.

11.2.

8.2.  Informative References

   [RFC7432]  Sajassi, A., Ed., Aggarwal, R., Bitar, N., Isaac, A.,
              Uttaro, J., Drake, J., and W. Henderickx, "BGP MPLS-Based
              Ethernet VPN", RFC 7432, DOI 10.17487/RFC7432, February
              2015, <https://www.rfc-editor.org/info/rfc7432>.

   [RFC7623]  Sajassi, A., Ed., Salam, S., Bitar, N., Isaac, A., and W.
              Henderickx, "Provider Backbone Bridging Combined with
              Ethernet VPN (PBB-EVPN)", RFC 7623, DOI 10.17487/RFC7623,
              September 2015, <https://www.rfc-editor.org/info/rfc7623>.

Appendix A.  Appendix

Authors' Addresses

   Sudhin Jacob (editor)
   Juniper Networks
   Bangalore
   India

   Phone: +91 8061212543
   Email: sjacob@juniper.net

   Kishore Tiruveedhula
   Juniper Networks
   10 Technology Park Dr
   Westford, MA  01886
   USA

   Phone: +1 9785898861
   Email: kishoret@juniper.net