Internet Engineering Task Force                            S. Jacob, Ed.
Internet-Draft                                           K. Tiruveedhula
Intended status: Standards Track Informational                          Juniper Networks
Expires: February 22, June 20, 2020                               August 21,                                 December 18, 2019

             Benchmarking Methodology for EVPN and PBB-EVPN
                      draft-ietf-bmwg-evpntest-03
                      draft-ietf-bmwg-evpntest-04

Abstract

   This document defines methodologies for benchmarking EVPN and PBB-
   EVPN performance.  EVPN performance.EVPN is defined in RFC 7432, and is being deployed
   in Service Provider networks.  Specifically networks.Specifically, this document defines
   the methodologies for benchmarking EVPN/PBB-EVPN convergence, data
   plane
   performance, and control plane performance.

Status of This Memo

   This Internet-Draft is submitted in full conformance with the
   provisions of BCP 78 and BCP 79.

   Internet-Drafts are working documents of the Internet Engineering
   Task Force (IETF).  Note that other groups may also distribute
   working documents as Internet-Drafts.  The list of current Internet-
   Drafts is at https://datatracker.ietf.org/drafts/current/.

   Internet-Drafts are draft documents valid for a maximum of six months
   and may be updated, replaced, or obsoleted by other documents at any
   time.  It is inappropriate to use Internet-Drafts as reference
   material or to cite them other than as "work in progress."

   This Internet-Draft will expire on February 22, June 20, 2020.

Copyright Notice

   Copyright (c) 2019 IETF Trust and the persons identified as the
   document authors.  All rights reserved.

   This document is subject to BCP 78 and the IETF Trust's Legal
   Provisions Relating to IETF Documents
   (https://trustee.ietf.org/license-info) in effect on the date of
   publication of this document.  Please review these documents
   carefully, as they describe your rights and restrictions with respect
   to this document.  Code Components extracted from this document must
   include Simplified BSD License text as described in Section 4.e of
   the Trust Legal Provisions and are provided without warranty as
   described in the Simplified BSD License.

Table of Contents

   1.  Introduction  . . . . . . . . . . . . . . . . . . . . . . . .   2
     1.1.  Requirements Language . . . . . . . . . . . . . . . . . .   3
     1.2.  Terminologies . . . . . . . . . . . . . . . . . . . . . .   3
   2.  Test Topology . . . . . . . . . . . . . . . . . . . . . . . .   3   4
   3.  Test Cases for EVPN  Benchmarking . . . . . . . . . . . . . .   6   7
     3.1.  Local MAC Learning  . . . . . . . . . . . . . . . . . . .   6   7
     3.2.  Remote MAC Learning . . . . . . . . . . . . . . . . . . .   7   8
     3.3.  MAC Flush due to local link failure and Relearning  . . .   7   8
     3.4.  MAC Flush due to remote link failure. . . . . . . . . . .   8   9
     3.5.  MAC Aging . . . . . . . . . . . . . . . . . . . . . . . .   8   9
     3.6.  Remote Mac Aging  . . . . . . . . . . . . . . . . . . . .   9  10
     3.7.  Local and Remote MAC Learning . . . . . . . . . . . . . .   9  10
     3.8.  High Availability.  . . . . . . . . . . . . . . . . . . .  10  11
     3.9.  ARP/ND Scale  . . . . . . . . . . . . . . . . . . . . . .  11  12
     3.10. Scaling of Services . . . . . . . . . . . . . . . . . . .  11  12
     3.11. Scale Convergence . . . . . . . . . . . . . . . . . . . .  12  13
     3.12. SOAK Test.  . . . . . . . . . . . . . . . . . . . . . . .  12  14
   4.  Test Cases for PBB-EVPN  Benchmarking . . . . . . . . . . . .  13  14
     4.1.  Local MAC Learning  . . . . . . . . . . . . . . . . . . .  13  14
     4.2.  Remote Mac Learning . . . . . . . . . . . . . . . . . . .  13  15
     4.3.  MAC Flush due to link failure . . . . . . . . . . . . . .  14  15
     4.4.  MAC Flush due to remote Failure . . . . . . . . . . . . .  14  16
     4.5.  MAC aging . . . . . . . . . . . . . . . . . . . . . . . .  15  17
     4.6.  Remote MAC Aging. . . . . . . . . . . . . . . . . . . . .  16  17
     4.7.  Local and Remote MAC Learning . . . . . . . . . . . . . .  16  18
     4.8.  High Availability . . . . . . . . . . . . . . . . . . . .  17  18
     4.9.  Scale . . . . . . . . . . . . . . . . . . . . . . . . . .  17  19
     4.10. Scale Convergence . . . . . . . . . . . . . . . . . . . .  18  19
     4.11. Soak Test . . . . . . . . . . . . . . . . . . . . . . . .  18  20
   5.  Acknowledgements  Acknowledgments  . . . . . . . . . . . . . . . . . . . . . .  19   21
   6.  IANA Considerations . . . . . . . . . . . . . . . . . . . . .  19  21
   7.  Security Considerations . . . . . . . . . . . . . . . . . . .  19  21
   8.  References  . . . . . . . . . . . . . . . . . . . . . . . . .  19  21
     8.1.  Normative References  . . . . . . . . . . . . . . . . . .  19  21
     8.2.  Informative References  . . . . . . . . . . . . . . . . .  20  22
   Appendix A.  Appendix . . . . . . . . . . . . . . . . . . . . . .  20  22
   Authors' Addresses  . . . . . . . . . . . . . . . . . . . . . . .  20  22

1.  Introduction

   EVPN is defined in RFC 7432, and describes BGP MPLS- based Ethernet
   VPNs (EVPN).  PBB-EVPN (EVPN).PBB-EVPN is defined in RFC 7623, discusses how Ethernet
   Provider backbone Bridging can be combined with EVPNs to provide a
   new/combined solution.  This solution.This draft defines methodologies that can be
   used to benchmark both RFC 7432 and RFC 7623 solutions.  Further, solutions.Further,
   this draft provides methodologies for benchmarking the performance of
   EVPN data and control planes, MAC learning, MAC flushing, MAC ageing, aging,
   convergence, high availability, and scale.

1.1.  Requirements Language

   The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
   "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
   document are to be interpreted as described in RFC 2119 [RFC2119]. 8174 [RFC8174].

1.2.  Terminologies

   MHPE Multi homed Provide Edge router.

   RR Route Reflector.

   P Provider Router.

   All-Active Redundancy Mode: When all PEs attached to an Ethernet
   segment are allowed to forward known unicast traffic to/from that
   Ethernet segment for a given VLAN, then the Ethernet segment is
   defined to be operating in All-Active redundancy mode.

   AA All Active mode

   CE Customer Router/Devices/Switch.

   DF Designated Forwarder

   DUT Device under test.

   Ethernet Segment (ES): When a customer site (device or network) is
   connected to one or more PEs via a set of Ethernet links, then that
   set of links is referred to as an 'Ethernet segment'.

   EVI: An EVPN instance spanning the Provider Edge (PE) devices
   participating in that EVPN.

   Ethernet Segment Identifier (ESI): A unique non-zero identifier that
   identifies an Ethernet segment is called an 'Ethernet Segment
   Identifier'.

   Ethernet Tag: An Ethernet tag identifies a particular broadcast
   domain, e.g., a VLAN.  An EVPN instance consists of one or more
   broadcast domains.

   Interface Physical interface of a router/switch.

   IRB Integrated routing and bridging interface

   MAC Media Access Control addresses on a PE.

   MHPE2 Multi homed Provider Edge router 2.

   MHPE1 Multi homed Provider Edge router 1.

   SHPE3 Single homed Provider Edge Router 3.

   AA EVPN Terminologies AA All-Active.

   SA EVPN Terminologies SA Single-Active.

   PE: Provider Edge device.

   P Provider Router.

   RR Route Reflector.

   RT Router Tester. Traffic Generator.

   Sub Interface Each physical Interfaces is subdivided in to into Logical
   units.

   EVI EVPN Instances which will be running on sub interface or physical
   port of

   SA Single Active

   Single-Active Redundancy Mode: When only a single PE, among all the provider Edge routers.

   DF Designated Forwarder.

   ESI
   PEs attached to an Ethernet Segment Identifier. segment, is allowed to forward traffic
   to/from that Ethernet segment for a given VLAN, then the Ethernet
   segment is defined to be operating in Single-Active redundancy mode.

2.  Test Topology

   EVPN/PBB-EVPN Services running on SHPE3, MHPE1 and MHPE2 in Single
   Active Mode:

         | [Traffic Traffic Generator ] Router Tester traffic acts as a sender/receiver of layer 2 traffic with multiple vlan.
+----------+
|          |
|  SHPE3   |
|          |
+----------+
    |
    |Core link
+----------+
|          |
|  RR      |
|          | Route Reflector/Core Reflector /Provider router
+----------+-------------|
   |                     |
   |     Core links      |
+----------+       +-----------+
|          |       |    MHPE2  |
|   DUT    |       |           |
|  MHPE1   |       |           |
+----------+       +-----------+
     |    PE-CE link    |
+----------+------------
|          |
|  CE      |
|  layer2  |
|bridge    |
+----------+------------ [Traffic Generator](Router Tester sender/reciever Traffic Generator acts as a sender/receiver of layer 2 traffic with multiple vlan) vlan.

Topology 1

                                Test Setup

                                 Figure 1

+-----------------+---------------------+---------------------+---------------------+----------------------+-----------------------+
|                 |                     |                     |                     |                      |                       |
|                 |                     |                     |                     |                      |                       |
|                 |                     |                     |                     |                      |                       |
|                 |                     |                     |                     |                      |                       |
| Mode            |                     |                     |                     |Receiver              |                       |
|                 |  Test               |Traffic Direction    |Sender               |                      |                       |
|                 |                     |                     |                     |                      |                       |
|                 |                     |                     |                     |                      |                       |
|                 |                     |                     |                     |                      |                       |
+----------------------------------------------------------------------------------------------------------------------------------+
|                 |                     |                     |                     |                      |                       |
|                 |                     |                     |                     |      SHPE3           |                       |
|Single Active    |  Local Mac          |                     |CE                   |                      |Layer 2 traffic        |
|                 | Learning            | Uni                 |                     |                      |                       |
|                 |                     |                     |                     |                      | multiple MAC          |
|                 |                     |                     |                     |                      |                       |
+-----------------------------------------------------------------------------------------------------------------------------------+
|                 |                     |                     |                     |                      |                       |
|Single Active    | Remote MAC          |                     |                     |         CE           |Layer 2 traffic        |
|                 | Learning            | uni                 | SHPE3               |                      |                       |
|                 |                     |                     |                     |                      |multiple MAC           |
|                 |                     |                     |                     |                      |                      ++
+----------------------------------------------------------------------------------------------------------------------------------+
|                 |                     |                     |                     |                      |                       |
|Single Active    | Scale Convergence   |   Bi                |                     |  CE/SHPE3            |                       |
|                 |                     |                     |   CE/SHPE3          |                      |Layer 2 traffic        |
|                 | Local& Remote       |                     |                     |                      |multiple mac& vlans    |
|                 | Learning            |                     |                     |                      |                       |
+-----------------+---------------------+---------------------+--------------------------------------------+-----------------------+
                                                                                    |
                                                                                   ++

              Table showing Traffic pattern for various test

                                 Figure 2

   Test Setup Configurations:

   There are five routers in the Test setup.  SHPE3, setup.SHPE3, RR/P, MHPE1 and
   MHPE2 emulating a service provider network. CE is a customer device
   connected to MHPE1 and MHPE2, it is configured with bridge domains in
   multiple vlans.  The router tester vlans.The traffic generator is connected to CE and
   SHPE3.The MHPE1 acts as DUT.The RT traffic generator will be used as
   sender and receiver of traffic.The measurement will be taken in DUT.

   All routers except CE is are configured with OSPF/IS-IS,LDP,MPLS,BGP
   with EVPN address family.

   All routers except CE must have IBGP are configured with RR Interior border gateway
   protocol,RR acting as route reflector.

   MHPE1,MHPE2,SHPE3 must be configured with "N" EVPN/PBB-EVPN instances
   depends up on the cases.

   MHPE1 and MHEPE2 must be configured with ESI per vlan or ESI on IFD.
   interface.

   MHPE1 and MHEPE2 are running Single Active mode of EVPN.

   CE is acting as bridge configured with multiple vlans,the same vlans that is
   are configured on MHPE1,MHPE2,SHPE3.

   Depends

   Depending up on the test scenarios the traffic generators will be flowing
   used to generate uni directional or bi directional depends on the test performed. flows.

   The above configuration will be serving as the base configuration for
   all test cases.

3.  Test Cases for EVPN Benchmarking

3.1.  Local MAC Learning

   Objective:

   To Record the time taken to learn the MAC address locally in DUT.

   Topology : Topology 1

   Procedure:

   The data plane MAC learning can be measured using the parameters
   defined in RFC 2889 section 5.8.  Send 5.8.Send "X" unicast frames from CE to
   MHPE1(DUT) working in SA mode with mode.Traffic generator connected to CE must
   send frames with "X" different source and destination address from RT. MAC address.
   The DUT must learn these "X" macs in data plane.

   Measurement :

   Measure the time taken to learn "X" MACs locally in DUT evpn mac
   table.  The data plane measurement is taken by considering DUT as
   black box the box.  The range of X MAC is MACs are known from RT and the traffic generator,the
   same must be learned in DUT, the time taken to learn "X" macs is
   measured.

   Repeat these test and plot the data.

   The test is repeated for "N" times and the values are collected.  The collected.The
   mac learning time rate is calculated by averaging the values obtained from
   "N" samples.

   Mac learning in sec rate = (T1+T2+..Tn/N) (T1+T2+..Tn)/N

3.2.  Remote MAC Learning

   Objective:

   To Record the time taken to learn the remote macs.

   Topology : Topology 1

   Procedure:

   Send X frames with X different SA source and DA destination mac addresses
   to SHPE3 from RT.  SHPE3 traffic generator.SHPE3 will advertise these locally
   learned macs to MHPE1 and MHPE2 via control plane.Measure the time
   taken to learn these X MACs from remote peer in DUT EVPN MAC address
   table.The DUT and MHPE2 are running SA mode.

   Measurement :

   Measure the time taken by the DUT to learn the "X" MACs in the data
   plane.Repeat these test and plot the data.The
   plane.The test is repeated for "N" times and the values are collected.The
   collected.  The remote mac learning time rate is calculated by averaging
   the values obtained from "N" samples.

   Mac

   Remote mac learning in sec rate = (T1+T2+..Tn/N) (T1+T2+..Tn)/N

3.3.  MAC Flush due to local link failure and Relearning

   Objective:

   To record the time taken to flush the mac learned locally learned mac and the
   time taken to relearn the same amount of macs.

   Topology : Topology 1

   Procedure:

   Send X frames with X different SA source and DA destination mac addresses
   to DUT from CE using traffic generator. Wait till the MHPE1 learns
   all X MAC address. addresses.  Then fail the MHPE1 CE link and measure the
   time taken to flush these X MACs from the EVPN MAC table.  Bring up
   the link which was made Down(the link between MHPE1 and CE).Measure
   time taken to relearn it.  The it.The DUT and MHPE2 are running SA mode.

   Measurement :

   Measure the time taken for flushing these X MAC address.  Measure addresses.Measure
   the time taken to relearn the X MACs in DUT.  Repeat these test and plot
   the data.  The DUT.The test is repeated for
   "N" times and the values are
   collected.  The collected.The flush and the relearning
   time is calculated by averaging the values obtained by "N" samples.

   Flush time for X Macs in sec rate = (T1+T2+..Tn/N) (T1+T2+..Tn)/N

   Relearning time for X macs in sec rate = (T1+T2+..Tn/N) (T1+T2+..Tn)/N

3.4.  MAC Flush due to remote link failure.

   Objective:

   To record the time taken to flush the remote mac learned in DUT
   during remote link failure.

   Topology : Topology 1

   Procedure:

   Send X frames with X different SA source and DA destination mac addresses
   to DUT from SHPE3 using traffic generator.  Bring down the link
   between SHPE3 and traffic
   generator.  Then generator.Then measure the time taken to
   flush the DUT EVPN MAC
   table.  The table.The DUT and MHPE2 are running SA mode.

   Measurement :

   Measure the time taken to flush X remote MACs from EVPN MAC table of
   DUT.  Repeat these test and plot
   the data.  The DUT.The test is repeated for "N" times and the values are collected.  The
   collected.The flush rate is calculated by averaging the values
   obtained by "N" samples.

   Flush time for X Macs in sec rate = (T1+T2+..Tn/N) (T1+T2+..Tn)/N

3.5.  MAC Aging

   Objective:

   To measure the mac aging time.

   Topology : Topology 1
   Procedure:

   Send X frames with X different SA source and DA destination mac addresses
   to DUT from CE using traffic
   generator.  Wait generator.Wait till X MAC address addresses
   are learned.  Then stop the
   traffic.  Record traffic.Record the time taken to flush
   X MACS from DUT EVPN MAC table due to aging.  The aging.The DUT and MHPE2 are
   running SA mode mode.

   Measurement :

   Measure the time taken to flush X MAC address addresses due to aging.  Repeat
   these test and plot the data.  The aging.The
   test is repeated for "N" times and the values are collected.  The collected.The
   aging is calculated averaging the values obtained by "N" samples.

   Aging time for X Macs in sec = (T1+T2+..Tn/N) (T1+T2+..Tn)/N

3.6.  Remote Mac Aging

   Objective:

   To measure the remote mac aging time.

   Topology : Topology 1

   Procedure:

   Send X frames with X different SA source and DA destination mac addresses
   to DUT from SHPE3 using traffic generator.  Stop the traffic at
   remote PE SHPE3.Due to MAC aging SHPE3 will withdraw its routes from
   DUT and MHPE2.  Measure MHPE2.Measure the time taken to remove these MACs from DUT
   EVPN MAC table.  DUT table.DUT and MHPE2 are running in SA mode

   Measurement :

   Measure the time taken to flush X remote MACs learned in DUT EVPN MAC
   table due to aging.  Repeat these test and plot the data.  The aging.The test is repeated for "N" times and the values
   are collected.  The collected.The aging is calculated by averaging the values
   obtained by "N" samples.

   Aging time for X Macs in sec = (T1+T2+..Tn/N) (T1+T2+..Tn)/N

3.7.  Local and Remote MAC Learning

   Objective:

   To record the time taken to learn both local and remote macs.

   Topology : Topology 1

   Procedure:

   Send X frames with X different SA source and DA destination mac addresses
   to DUT from SHPE3 using traffic generator.  Send generator.Send X frames with
   different SA source and DA destination mac addresses from traffic generator
   connected to CE.  The SA CE.The source and DA destination addresses of flows must
   be complimentary to have unicast flows.  Measure flows.Measure the time taken by
   the DUT to learn 2X in EVPN MAC.  DUT MAC.DUT and MHPE2 are running in SA
   mode.

   Measurement :

   Measure the time taken to learn 2X MAC address addresses in DUT EVPN MAC
   table.
   Repeat these test and plot the data.  The test is repeated for "N" times and the values are
   collected.  The mac learning time is calculated by averaging the
   values obtained by "N" samples.

   Time to learn 2X Macs in sec

   Mac learning rate = (T1+T2+..Tn/N) (T1+T2+..Tn)/N

3.8.  High Availability.

   Objective:

   To record traffic loss during routing engine fail over.

   Topology : Topology 1

   Procedure:

   Send X frames from CE to DUT from traffic generator withX with X different
   SA
   source and DA.  Send destination mac addresses.Send X frames from traffic
   generator to SHPE3 with X different SA source and DA destination mac
   addresses, so that 2X MAC address will be learned in DUT.
   There the DUT.There
   is a bi directional traffic flow with X pps in each direction.
   Then direction.Then
   do a routing engine fail-over.

   Measurement :

   There should be

   The expectation of the test is 0 traffic loss which is the ideal case, No with no change in the
   DF role.  DUT role.DUT should not withdraw any routes.Repeat routes.But in cases where the
   DUT is not property synchronized between master and standby,due to
   that packet loss are observed.In that scenario the packet loss is
   measured.The test is repeated for "N" times and plot the data.The values are
   collected.The packet loss is calculated by averaging the values
   obtained from by "N" samples.

   Packet loss in sec = (T1+T2+..Tn/N) with 2X mac addresses = (T1+T2+..Tn)/N

3.9.  ARP/ND Scale

   These tests are conducted to

   To Record the DUT scaling parameter of ARP/ND limit of the DUT. ARP/ND.

   Objective:

   To Record the ARP/ND scale of the DUT.

   Topology : Topology 1

   Procedure:

   Send X arp/icmpv6 request arp/neighbor discovery(ND) from RT the traffic generator to DUT
   with different sender ip/
   ipv6 address ip/ipv6,mac addresses to the same target gateway ip address.  Measure whether
   X MAC+IPv4 address/MAC+IPv6 IRB address of the hosts are learned
   configured in DUT.

   Measurement : EVPN instance.  The DUT must learn X MAC+IPV4/MAC+IPv6 EVPN instance learns the mac+ip and
   mac+ipv6 addresses from these request and it must advertise the X
   MAC+IPV4/MAC+IPV6 as type 2 mac+ip/
   mac+ipv6 route to the remote router.

3.10.  Scaling provide edge routers which have same EVPN
   configurations.The value of Services

   Objective:

   To measure X must be increased at a incremental
   value of 5% of X, till the limit is reached.The limit is where the
   DUT cant learn any more type 2 mac+ip/mac+ipv6.The test must be
   separately conducted for arp and ND.

   Measurement :

   Measure the scale limit of type 2 mac+ip/mac+ipv6 route which DUT can
   learn.The test is repeated for "N" times and the values are
   collected.The scale limit is calculated by averaging the values
   obtained by "N" samples for both mac+ip and mac+ipv6.

   DUT scale limit for mac+ip = (v1+v2+..vn)/N

   DUT scale limit for EVPN.This mac+ipv6 = (v1+v2+..vn)/N

3.10.  Scaling of Services

   Objective:

   This test is to measure the
   performance scale of DUT in scaling to "X" EVPN instances. instances that a DUT can
   hold.

   Topology : Topology 1
   Procedure:

   The DUT,MHPE2 and SHPE3 are scaled to "N" EVI.Clear BGP neighbors of
   the DUT.  Once DUT.Once adjacency is established in the DUT.  Measure DUT, check the
   routes received from MHPE2 and SHPE3 for "N" EVI in the DUT. DUT.Then
   increment the scale of N by 5% of N till the limit is reached.The
   limit is where the DUT cant learn any EVPN routes from peers.

   Measurement :

   There should not be any loss of route types 1,2,3 and 4 in DUT.  DUT DUT.DUT
   must relearn all type 1,2,3 and 4 from remote routers. The DUT must
   be subjected to various values of N to find the optimal scale limit limit.
   The scope of the test is find out the maximum evpn instance that a
   DUT can hold.

3.11.  Scale Convergence

   Objective:

   To measure the convergence time of DUT when the DUT is scaled with
   EVPN instance along with traffic.

   Topology : Topology 1

   Procedure:

   Scale N EVIs in DUT,SHPE3 and MHPE2.Send F frames to DUT from CE
   using traffic generator with X different SA source and DA destination mac
   addresses for N EVI's.  Send EVI's.Send F frames from traffic generator to SHPE3
   with X different SA source and DA.
   There destination mac addresses.There will be
   2X number of MAC address addresses will be learned in DUT EVPN MAC table.
   There is a bi directional traffic flow with F pps in each direction.
   Then clear the BGP neighbors in the DUT.  Once DUT.Once the adjacency is
   restored in DUT.  Measure DUT.Measure the time taken to learn 2X MAC address in
   DUT MAC table.

   Measurement :

   The DUT must learn 2X MAC address. addresses.  Measure the time taken to learn
   2X MAC in DUT.  Repeat these test and plot the data.The DUT.The test is repeated for "N" times and the values are collected.The
   collected.  The convergence time is calculated by averaging the
   values obtained by "N" samples.

   Convergence time

   Time taken to learn 2X macs in sec DUT = (T1+T2+..Tn/N) (T1+T2+..Tn)/N

3.12.  SOAK Test.

   Objective:

   This test is carried out to measure the stability of the DUT in a
   scaled environment with traffic over a period of time "T'". In each
   interval "t1" the DUT CPU usage, memory usage are measured.  The measured.The DUT
   is checked for any crashes during this time period.

   Topology : Topology 1

   Procedure:

   Scale N EVI's in DUT,SHPE3 and MHPE2.Send F frames to DUT from CE
   using traffic generator with different X SA source and DA destination mac
   addresses for N EVI's.  Send EVI's.Send F frames from traffic generator to SHPE3
   with X different SA source and DA.
   There destination mac addresses.There will be
   2X number of MAC address addresses will be learned in DUT EVPN MAC table.
   There is a bi directional traffic flow with F pps in each direction.
   The DUT must run with traffic for 24 hours, every hour check for
   memory leak, CPU usage and crash.

   Measurement :

   Take the hourly reading of CPU, process memory.  There should not be
   any leak, crashes, CPU spikes.

4.  Test Cases for PBB-EVPN Th CPU spike is determined as the CPU
   usage which shoots at 40 to 50 percent of the average usage.The
   average value vary from device to device. Memory leak is determined
   by increase usage of the memory for EVPN process. The expectation is
   under steady state the memory usage for EVPN process should not
   increase.

4.  Test Cases for PBB-EVPN Benchmarking

4.1.  Local MAC Learning

   Objective:

   To Record the time taken to learn the MAC address locally.

   Topology : Topology 1

   Procedure:

   Send "X" unicast frames from CE to MHPE1(DUT) working in SA mode with
   "X" different source and destination address mac addresses from RT.  The traffic
   generator.The DUT must learn "X" macs in data plane.

   Measurement :

   Measure the time taken by the DUT to learn the "X" MACs in the data
   plane.  The data plane measurement is taken by considering DUT as
   black box the range of "X" MAC is known from RT traffic generator and
   the same must be learned in DUT, the time taken to learn "X" MAC is measured.  Repeat
   these test and plot the data.  The
   measured.The test is repeated for "N" times and the values are collected.  The
   collected.The mac learning time is calculated by averaging the
   values obtained from "N" samples.

   Mac learning in sec rate = (T1+T2+..Tn/N) (T1+T2+..Tn)/N

4.2.  Remote Mac Learning

   Objective:

   To Record the time taken to learn the remote macs.

   Topology : Topology 1

   Procedure:

   Send X frames with X different SA source and DA destination mac addresses
   to SHPE3 from RT.These traffic generator.These macs will be flooded to MHPE1
   and MHPE2 by SHPE3.The DUT and MHPE2 are running SA mode.

   Measurement :

   Measure the time taken to learn X mac address addresses in DUT mac table.
   Repeat these test and plot the data.The
   The test is repeated for "N" times and the values are collected.The
   mac learning time is calculated by averaging the values obtained by
   "N" samples.

   Remote Mac learning in sec rate = (T1+T2+..Tn/N) (T1+T2+..Tn)/N

4.3.  MAC Flush due to link failure

   Objective:

   To record the time taken to flush the mac learned locally learned mac and the
   time taken to relearn the same amount of macs.

   Topology : Topology 1
   Procedure:

   Send X frames with X different SA source and DA destination mac addresses
   to DUT from CE using traffic
   generator.  Wait generator.Wait till the MHPE1 learn
   all X MAC address.  Then address.Then fail the MHPE1 CE link and measure the time
   taken to flush these X MACs from the PBB-EVPN MAC table.  Then table.Then bring
   up the link.  Measure link.Measure the time taken to relearn X MACS.  The MACS.The DUT and
   MHPE2 are running SA mode.

   Measurement :

   Measure the time taken for flushing these X MAC address.  Measure addresses.Measure
   the time taken to relearn the X MACs in DUT.  Repeat these test and plot
   the data.  The DUT.The test is repeated for
   "N" times and the values are collected.  The flush and the relearning
   time is calculated by averaging the values obtained by "N" samples.

   Flush time for X Macs in sec rate = (T1+T2+..Tn/N) (T1+T2+..Tn)/N

   Relearning time for X macs in sec rate = (T1+T2+..Tn/N) (T1+T2+..Tn)/N

4.4.  MAC Flush due to remote Failure

   Objective:

   To record the time taken to flush the remote mac learned in DUT
   during remote link failure.

   Topology : Topology 1

   Procedure:

   Send X frames with X different SA source and DA destination mac addresses
   to DUT from SHPE3 using traffic generator.  Bring generator.Bring down the link
   between SHPE3 and traffic generator.  Then measure the time taken to
   flush the DUT PBB-EVPN MAC address table.  The table.The remote MACs will be
   learned by Data plane, but the B-MAC will be learned by control plane.  The
   plane.The DUT and MHPE2 are running SA mode.

   Measurement :

   Measure the time taken to flush X remote MACs from PBB-EVPN MAC table
   of DUT.  Repeat DUT.Repeat these test and plot the data.  The data.The test is repeated
   for "N" times and the values are collected. The flush rate is
   calculated by averaging the values obtained by "N" samples.

   Flush time for X Macs in sec rate = (T1+T2+..Tn/N) (T1+T2+..Tn)/N

4.5.  MAC aging

   Objective:

   To measure the mac aging time.

   Topology : Topology 1

   Procedure:

   Send X frames with X different SA source and DA destination mac addresses
   to DUT from CE using traffic
   generator.  Wait generator.Wait till X MAC address are
   learned in DUT PBB- EVPN PBB-EVPN MAC
   table.  Then table.Then stop the traffic.  Record traffic.Record
   the time taken to flush X MAC entries due to aging. The DUT and
   MHPE2 running in SA mode

   Measurement :

   Measure the time taken to flush X MAC address addresses due to aging.  Repeat aging.Repeat
   these test and plot the data.  The data.The test is repeated for "N" times and
   the values are collected.  The collected.The aging is calculated by averaging the
   values obtained by "N" samples.

   Aging time for X Macs in sec = (T1+T2+..Tn/N) (T1+T2+..Tn)/N

4.6.  Remote MAC Aging.

   Objective:

   To measure the remote mac aging time.

   Topology : Topology 1

   Procedure:

   Send X frames with X different SA source and DA destination mac addresses
   to DUT from SHPE3 using traffic generator.  Stop generator.Stop the traffic at
   remote PE(SHPE3).Measure the time taken to remove these remote MACs
   from DUT PBB-EVPN MAC table.
   The table.The DUT and MHPE2 are running in SA
   mode.

   Measurement :

   Measure the time taken to flush the X remote MACs from DUT PBB-EVPN
   MAC table due to aging Repeat these test and plot the data.  The .The test is repeated for "N" times and the
   values are collected.  The aging is calculated by averaging the
   values obtained by "N" samples.

   Aging time for X Macs in sec = (T1+T2+..Tn/N) (T1+T2+..Tn)/N

4.7.  Local and Remote MAC Learning

   Objective:

   To record the time taken to learn both local and remote macs.

   Topology : Topology 1

   Procedure:

   Send X frames with X different SA source and DA destination mac addresses
   to DUT from SHPE3 using traffic generator.  Send generator.Send X frames with
   different SA source and DA destination mac addresses from traffic generator
   connected to CE.  The SA CE.The source and DA destination mac addresses of flows
   must be complimentary to have unicast flows.  Measure flows.Measure the time taken
   by the DUT to learn 2X in MAC table.  DUT table.DUT and MHPE2 are running in SA
   mode.

   Measurement :

   Measure the time taken to learn 2X MAC address addresses table in DUT PBB-EVPN PBB-
   EVPN MAC table.  Repeat these test and plot the data.  The table.The test is repeated for "N" times and the values
   are collected.  The collected.The mac learning time is calculated by averaging
   the values obtained by "N" samples.

   Time to learn 2X Macs in sec

   Mac learning rate = (T1+T2+..Tn/N) (T1+T2+..Tn)/N

4.8.  High Availability

   Objective:

   To record traffic loss during routing engine failover.

   Topology : Topology 1

   Procedure:

   Send X frames to DUT with X different SA source and DA destination mac
   addresses from CE using the traffic generator.  Send generator.Send X frames from
   traffic generator to SHPE3 with X different SA source and DA destination
   mac addresses, so that 2X MAC address will be Learned in DUT.  There
   is a bi directional traffic flow with X pps in each direction.  Then
   do a routing engine fail-over.

   Measurement :

   There should be

   The expectation of the test is 0 traffic loss which is the ideal case, No with no change in the
   DF role.  DUT role.DUT should not withdraw any routes.Repeat routes.But in cases where the
   DUT is not property synchronized between master and standby,due to
   that packet loss are observed.  In that scenario the packet loss is
   measured.The test is repeated for "N" times and plot the data.The values are
   collected.The packet loss is calculated by averaging the values
   obtained from by "N" samples.

   Packet loss in sec with 2X mac addresses = (T1+T2+..Tn/N) (T1+T2+..Tn)/N

4.9.  Scale

   Objective:

   To measure the scale limit of DUT for PBB-EVPN.

   Topology : Topology 1

   Procedure:

   The DUT,MHPE2 and SHPE3 are scaled to "N" PBB-EVPN instances.  Clear PBB-EVN instances.Clear BGP
   neighbors in of the DUT DUT.  Once adjacency is established in DUT, the DUT.
   check the routes received from SHPE3 MHPE2 and MHPE2. SHPE3 for "N" PBB-EVPN
   instances in the DUT.Then increment the scale of N by 5% of N till
   the limit is reached.The limit is where the DUT cant learn any EVPN
   routes from peers.

   Measurement :

   There should not be any loss of route types 2,3 and 4 in DUT.  The
   DUT DUT.DUT
   must relearn all type 2,3 and 4 routes from remote routers.  The routers.The DUT must be
   subjected to various values of N to find the optimal scale limit.
   The scope of the test is find out the maximum evpn instance that a
   DUT can hold.

4.10.  Scale Convergence

   Objective:

   To measure the convergence time of DUT when the DUT is scaled with
   EVPN instance along with traffic.

   Topology : Topology 1

   Procedure:

   Scale N PBB-EVIs PBB-EVPN instances in DUT,SHPE3 and MHPE2.Send F frames to
   DUT from CE using traffic generator with X different SA source and DA
   destination mac addresses for N PBB-EVI's. PBB-EVPN instances.  Send F frames
   from traffic generator to SHPE3 with X different SA source and
   DA.  There
   destination mac addresses.There will be 2X number of MAC address addresses will be
   learned in DUT PBB-EVPN MAC table.  There table.There is a bi directional traffic
   flow with F pps in each direction.  Then direction.Then clear the BGP neighbors in
   the DUT.  Once the adjacency is restored in DUT.  Measure the time
   taken to learn 2X MAC address in DUT MAC table.

   Measurement :

   The DUT must learn 2X MAC address.  Measure addresses.Measure the time taken to learn
   2X MAC in DUT.  Repeat these test and plot the data.The DUT.The test is repeated for "N" times and the values are
   collected.The convergence time is calculated by averaging the values
   obtained by "N" samples.

   Convergence time for 2X in sec = (T1+T2+..Tn/N) (T1+T2+..Tn)/N

4.11.  Soak Test

   Objective:

   To measure the stability of the DUT in a scaled environment with
   traffic.

   Topology : Topology 1

   Procedure:

   Scale N PBB-EVI's PBB-EVPN instances in DUT,SHPE3 and MHPE2.Send F frames to
   DUT from CE using traffic generator with X different SA source and DA
   destination mac addresses for N EVI's. PBB-EVPN instances.  Send F frames
   from traffic generator to SHPE3 with X different SA source and DA.
   There
   destination mac addresses.There will be 2X number of MAC address addresses will be
   learned in DUT PBB- EVPN MAC table.  There table.There is a bi directional
   traffic flow with F pps in Each direction.  The direction.The DUT must run with
   traffic for 24 hours, every hour check the memory leak, crashes.

   Measurement :

   Take the hourly reading of CPU process, memory usages. CPU, process memory.  There should not be
   any memory leak, crashes,CPU spikes. crashes, CPU spikes.The CPU spike is determined as the CPU
   usage which shoots at 40 to 50 percent of the average usage.The
   average value vary from device to device.Memory leak is determined
   by increase usage of the memory for PBB-EVPN process.The
   expectation is under steady state the memory usage for PBB-EVPN process
   should not increase.

5.  Acknowledgements  Acknowledgments

   We would like to thank Fioccola Giuseppe of Telecom Italia reviewing
   our draft and commenting it.  We it.We would like to thank Sarah Banks for
   guiding and mentoring us.

6.  IANA Considerations

   This memo includes no request to IANA.

7.  Security Considerations

   There

   The benchmarking tests described in this document are limited to the
   performance characterization of controllers in a lab environment with
   isolated networks.  The benchmarking network topology will be an
   independent test setup and MUST NOT be connected to devices that may
   forward the test traffic into a production network or misroute
   traffic to the test management network.  Further, benchmarking is no additional consideration
   performed on a "black-box" basis, relying solely on measurements
   observable external to the controller.  Special capabilities SHOULD
   NOT exist in the controller specifically for benchmarking purposes.
   Any implications for network security arising from RFC 6192. the controller
   SHOULD be identical in the lab and in production networks.

8.  References

8.1.  Normative References

   [RFC2119]  Bradner, S., "Key words for use in RFCs to Indicate
              Requirement Levels", BCP 14, RFC 2119,
              DOI 10.17487/RFC2119, March 1997,
              <https://www.rfc-editor.org/info/rfc2119>.

   [RFC2544]  Bradner, S. and J. McQuaid, "Benchmarking Methodology for
              Network Interconnect Devices", RFC 2544,
              DOI 10.17487/RFC2544, March 1999,
              <https://www.rfc-editor.org/info/rfc2544>.

   [RFC2899]  Ginoza, S., "Request for Comments Summary RFC Numbers
              2800-2899", RFC 2899, DOI 10.17487/RFC2899, May 2001,
              <https://www.rfc-editor.org/info/rfc2899>.

8.2.  Informative References

   [RFC7432]  Sajassi, A., Ed., Aggarwal, R., Bitar, N., Isaac, A.,
              Uttaro, J., Drake, J., and W. Henderickx, "BGP MPLS-Based
              Ethernet VPN", RFC 7432, DOI 10.17487/RFC7432, February
              2015, <https://www.rfc-editor.org/info/rfc7432>.

   [RFC7623]  Sajassi, A., Ed., Salam, S., Bitar, N., Isaac, A., and W.
              Henderickx, "Provider Backbone Bridging Combined with
              Ethernet VPN (PBB-EVPN)", RFC 7623, DOI 10.17487/RFC7623,
              September 2015, <https://www.rfc-editor.org/info/rfc7623>.

Appendix A.  Appendix

Authors' Addresses

   Sudhin Jacob (editor)
   Juniper Networks
   Bangalore
   India

   Phone: +91 8061212543
   Email: sjacob@juniper.net

   Kishore Tiruveedhula
   Juniper Networks
   10 Technology Park Dr
   Westford, MA  01886
   USA

   Phone: +1 9785898861
   Email: kishoret@juniper.net