draft-ietf-bmwg-sdn-controller-benchmark-meth-08.txt   draft-ietf-bmwg-sdn-controller-benchmark-meth-09.txt 
Internet-Draft Bhuvaneswaran Vengainathan Internet-Draft Bhuvaneswaran Vengainathan
Network Working Group Anton Basil Network Working Group Anton Basil
Intended Status: Informational Veryx Technologies Intended Status: Informational Veryx Technologies
Expires: August 25, 2018 Mark Tassinari Expires: November 25, 2018 Mark Tassinari
Hewlett-Packard Hewlett-Packard
Vishwas Manral Vishwas Manral
Nano Sec Nano Sec
Sarah Banks Sarah Banks
VSS Monitoring VSS Monitoring
February 25, 2018 May 25, 2018
Benchmarking Methodology for SDN Controller Performance Benchmarking Methodology for SDN Controller Performance
draft-ietf-bmwg-sdn-controller-benchmark-meth-08 draft-ietf-bmwg-sdn-controller-benchmark-meth-09
Abstract Abstract
This document defines the methodologies for benchmarking control This document defines methodologies for benchmarking control plane
plane performance of SDN controllers. SDN controller is a core performance of SDN controllers. SDN controller is a core component
component in software-defined networking architecture that controls in software-defined networking architecture that controls the
the network behavior. Terminology related to benchmarking SDN network behavior. SDN controllers have been implemented with many
controllers is described in the companion terminology documentI-D varying designs in order to achieve their intended network
sdn-controller-benchmark-term. SDN controllers have been implemented
with many varying designs in order to achieve their intended network
functionality. Hence, the authors have taken the approach of functionality. Hence, the authors have taken the approach of
considering an SDN controller as a black box, defining the considering an SDN controller as a black box, defining the
methodology in a manner that is agnostic to protocols and network methodology in a manner that is agnostic to protocols and network
services supported by controllers. The intent of this document is to services supported by controllers. The intent of this document is to
provide a standard mechanism to measure the performance of all provide a method to measure the performance of all controller
controller implementations. implementations.
Status of this Memo Status of this Memo
This Internet-Draft is submitted in full conformance with the This Internet-Draft is submitted in full conformance with the
provisions of BCP 78 and BCP 79. provisions of BCP 78 and BCP 79.
Internet-Drafts are working documents of the Internet Engineering Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF). Note that other groups may also distribute Task Force (IETF). Note that other groups may also distribute
working documents as Internet-Drafts. The list of current Internet- working documents as Internet-Drafts. The list of current Internet-
Drafts is at http://datatracker.ietf.org/drafts/current. Drafts is at http://datatracker.ietf.org/drafts/current.
Internet-Drafts are draft documents valid for a maximum of six Internet-Drafts are draft documents valid for a maximum of six
months and may be updated, replaced, or obsoleted by other documents months and may be updated, replaced, or obsoleted by other documents
at any time. It is inappropriate to use Internet-Drafts as reference at any time. It is inappropriate to use Internet-Drafts as reference
material or to cite them other than as "work in progress. material or to cite them other than as "work in progress.
This Internet-Draft will expire on August 25, 2018. This Internet-Draft will expire on November 25, 2018.
Copyright Notice Copyright Notice
Copyright (c) 2018 IETF Trust and the persons identified as the Copyright (c) 2018 IETF Trust and the persons identified as the
document authors. All rights reserved. document authors. All rights reserved.
This document is subject to BCP 78 and the IETF Trust's Legal This document is subject to BCP 78 and the IETF Trust's Legal
Provisions Relating to IETF Documents Provisions Relating to IETF Documents
(http://trustee.ietf.org/license-info) in effect on the date of (http://trustee.ietf.org/license-info) in effect on the date of
publication of this document. Please review these documents publication of this document. Please review these documents
skipping to change at page 2, line 35 skipping to change at page 2, line 35
3.1. Test setup - Controller working in Standalone Mode........5 3.1. Test setup - Controller working in Standalone Mode........5
3.2. Test setup - Controller working in Cluster Mode...........6 3.2. Test setup - Controller working in Cluster Mode...........6
4. Test Considerations............................................7 4. Test Considerations............................................7
4.1. Network Topology..........................................7 4.1. Network Topology..........................................7
4.2. Test Traffic..............................................7 4.2. Test Traffic..............................................7
4.3. Test Emulator Requirements................................7 4.3. Test Emulator Requirements................................7
4.4. Connection Setup..........................................7 4.4. Connection Setup..........................................7
4.5. Measurement Point Specification and Recommendation........8 4.5. Measurement Point Specification and Recommendation........8
4.6. Connectivity Recommendation...............................8 4.6. Connectivity Recommendation...............................8
4.7. Test Repeatability........................................8 4.7. Test Repeatability........................................8
4.8. Test Reporting............................................8
5. Benchmarking Tests.............................................9 5. Benchmarking Tests.............................................9
5.1. Performance...............................................9 5.1. Performance...............................................9
5.1.1. Network Topology Discovery Time......................9 5.1.1. Network Topology Discovery Time......................9
5.1.2. Asynchronous Message Processing Time................11 5.1.2. Asynchronous Message Processing Time................11
5.1.3. Asynchronous Message Processing Rate................13 5.1.3. Asynchronous Message Processing Rate................12
5.1.4. Reactive Path Provisioning Time.....................15 5.1.4. Reactive Path Provisioning Time.....................15
5.1.5. Proactive Path Provisioning Time....................16 5.1.5. Proactive Path Provisioning Time....................16
5.1.6. Reactive Path Provisioning Rate.....................18 5.1.6. Reactive Path Provisioning Rate.....................18
5.1.7. Proactive Path Provisioning Rate....................19 5.1.7. Proactive Path Provisioning Rate....................19
5.1.8. Network Topology Change Detection Time..............21 5.1.8. Network Topology Change Detection Time..............21
5.2. Scalability..............................................23 5.2. Scalability..............................................22
5.2.1. Control Session Capacity............................23 5.2.1. Control Session Capacity............................22
5.2.2. Network Discovery Size..............................23 5.2.2. Network Discovery Size..............................23
5.2.3. Forwarding Table Capacity...........................24 5.2.3. Forwarding Table Capacity...........................24
5.3. Security.................................................26 5.3. Security.................................................26
5.3.1. Exception Handling..................................26 5.3.1. Exception Handling..................................26
5.3.2. Denial of Service Handling..........................27 5.3.2. Denial of Service Handling..........................27
5.4. Reliability..............................................29 5.4. Reliability..............................................29
5.4.1. Controller Failover Time............................29 5.4.1. Controller Failover Time............................29
5.4.2. Network Re-Provisioning Time........................30 5.4.2. Network Re-Provisioning Time........................30
6. References....................................................32 6. References....................................................32
6.1. Normative References.....................................32 6.1. Normative References.....................................32
6.2. Informative References...................................32 6.2. Informative References...................................32
7. IANA Considerations...........................................32 7. IANA Considerations...........................................32
8. Security Considerations.......................................32 8. Security Considerations.......................................32
9. Acknowledgments...............................................33 9. Acknowledgments...............................................33
Appendix A. Example Test Topology................................34 Appendix A Benchmarking Methodology using OpenFlow Controllers..34
A.1. Leaf-Spine Topology......................................34 A.1. Protocol Overview........................................34
Appendix B. Benchmarking Methodology using OpenFlow Controllers..35 A.2. Messages Overview........................................34
B.1. Protocol Overview........................................35 A.3. Connection Overview......................................34
B.2. Messages Overview........................................35 A.4. Performance Benchmarking Tests...........................35
B.3. Connection Overview......................................35 A.4.1. Network Topology Discovery Time.....................35
B.4. Performance Benchmarking Tests...........................36 A.4.2. Asynchronous Message Processing Time................36
B.4.1. Network Topology Discovery Time.....................36 A.4.3. Asynchronous Message Processing Rate................37
B.4.2. Asynchronous Message Processing Time................37 A.4.4. Reactive Path Provisioning Time.....................38
B.4.3. Asynchronous Message Processing Rate................38 A.4.5. Proactive Path Provisioning Time....................39
B.4.4. Reactive Path Provisioning Time.....................39 A.4.6. Reactive Path Provisioning Rate.....................40
B.4.5. Proactive Path Provisioning Time....................40 A.4.7. Proactive Path Provisioning Rate....................41
B.4.6. Reactive Path Provisioning Rate.....................41 A.4.8. Network Topology Change Detection Time..............42
B.4.7. Proactive Path Provisioning Rate....................42 A.5. Scalability..............................................43
B.4.8. Network Topology Change Detection Time..............43 A.5.1. Control Sessions Capacity...........................43
B.5. Scalability..............................................44 A.5.2. Network Discovery Size..............................43
B.5.1. Control Sessions Capacity...........................44 A.5.3. Forwarding Table Capacity...........................44
B.5.2. Network Discovery Size..............................44 A.6. Security.................................................46
B.5.3. Forwarding Table Capacity...........................45 A.6.1. Exception Handling..................................46
B.6. Security.................................................47 A.6.2. Denial of Service Handling..........................47
B.6.1. Exception Handling..................................47 A.7. Reliability..............................................49
B.6.2. Denial of Service Handling..........................48 A.7.1. Controller Failover Time............................49
B.7. Reliability..............................................50 A.7.2. Network Re-Provisioning Time........................50
B.7.1. Controller Failover Time............................50 Authors' Addresses...............................................53
B.7.2. Network Re-Provisioning Time........................51
Authors' Addresses...............................................54
1. Introduction 1. Introduction
This document provides generic methodologies for benchmarking SDN This document provides generic methodologies for benchmarking SDN
controller performance. An SDN controller may support many controller performance. An SDN controller may support many
northbound and southbound protocols, implement a wide range of northbound and southbound protocols, implement a wide range of
applications, and work solely, or as a group to achieve the desired applications, and work solely, or as a group to achieve the desired
functionality. This document considers an SDN controller as a black functionality. This document considers an SDN controller as a black
box, regardless of design and implementation. The tests defined in box, regardless of design and implementation. The tests defined in
the document can be used to benchmark SDN controller for the document can be used to benchmark SDN controller for
performance, scalability, reliability and security independent of performance, scalability, reliability and security independent of
northbound and southbound protocols. These tests can be performed on northbound and southbound protocols. Terminology related to
an SDN controller running as a virtual machine (VM) instance or on a benchmarking SDN controllers is described in the companion
bare metal server. This document is intended for those who want to terminology document [I-D.sdn-controller-benchmark-term]. These
measure the SDN controller performance as well as compare various tests can be performed on an SDN controller running as a virtual
SDN controllers performance. machine (VM) instance or on a bare metal server. This document is
intended for those who want to measure the SDN controller
performance as well as compare various SDN controllers performance.
Conventions used in this document Conventions used in this document
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
"SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and
"OPTIONAL" in this document are to be interpreted as described in "OPTIONAL" in this document are to be interpreted as described in
BCP 14 [RFC2119] [RFC8174] when, and only when, they appear in all BCP 14 [RFC2119] [RFC8174] when, and only when, they appear in all
capitals, as shown here. capitals, as shown here.
2. Scope 2. Scope
skipping to change at page 7, line 9 skipping to change at page 7, line 9
| | | |
| Forwarding Plane Test Emulator | | Forwarding Plane Test Emulator |
+-----------------------------------------------------------+ +-----------------------------------------------------------+
Figure 2 Figure 2
4. Test Considerations 4. Test Considerations
4.1. Network Topology 4.1. Network Topology
The test cases SHOULD use Leaf-Spine topology with at least 1 The test cases SHOULD use Leaf-Spine topology with at least 2
Network Device in the topology for benchmarking. The test traffic Network Devices in the topology for benchmarking. The test traffic
generators TP1 and TP2 SHOULD be connected to the first and the last generators TP1 and TP2 SHOULD be connected to the leaf Network
leaf Network Device. If a test case uses test topology with 1 Device 1 and the leaf Network Device n. To achieve a complete
Network Device, the test traffic generators TP1 and TP2 SHOULD be
connected to the same node. However to achieve a complete
performance characterization of the SDN controller, it is performance characterization of the SDN controller, it is
recommended that the controller be benchmarked for many network recommended that the controller be benchmarked for many network
topologies and a varying number of Network Devices. This document topologies and a varying number of Network Devices. Further, care
includes a sample test topology, defined in Section 10 - Appendix A should be taken to make sure that a loop prevention mechanism is
for reference. Further, care should be taken to make sure that a enabled either in the SDN controller, or in the network when the
loop prevention mechanism is enabled either in the SDN controller, topology contains redundant network paths.
or in the network when the topology contains redundant network
paths.
4.2. Test Traffic 4.2. Test Traffic
Test traffic is used to notify the controller about the asynchronous Test traffic is used to notify the controller about the asynchronous
arrival of new flows. The test cases SHOULD use frame sizes of 128, arrival of new flows. The test cases SHOULD use frame sizes of 128,
512 and 1508 bytes for benchmarking. Tests using jumbo frames are 512 and 1508 bytes for benchmarking. Tests using jumbo frames are
optional. optional.
4.3. Test Emulator Requirements 4.3. Test Emulator Requirements
skipping to change at page 8, line 44 skipping to change at page 8, line 41
The SDN controller in the test setup SHOULD be connected directly The SDN controller in the test setup SHOULD be connected directly
with the forwarding and the management plane test emulators to avoid with the forwarding and the management plane test emulators to avoid
any delays or failure introduced by the intermediate devices during any delays or failure introduced by the intermediate devices during
benchmarking tests. When the controller is implemented as a virtual benchmarking tests. When the controller is implemented as a virtual
machine, details of the physical and logical connectivity MUST be machine, details of the physical and logical connectivity MUST be
reported. reported.
4.7. Test Repeatability 4.7. Test Repeatability
To increase the confidence in measured result, it is recommended To increase the confidence in measured result, it is recommended
that each test SHOULD be repeated a minimum of 10 times. that each test RECOMMENDED be repeated a minimum of 10 times.
Test Reporting 4.8. Test Reporting
Each test has a reporting format that contains some global and Each test has a reporting format that contains some global and
identical reporting components, and some individual components that identical reporting components, and some individual components that
are specific to individual tests. The following test configuration are specific to individual tests. The following test configuration
parameters and controller settings parameters MUST be reflected in parameters and controller settings parameters MUST be reflected in
the test report. the test report.
Test Configuration Parameters: Test Configuration Parameters:
1. Controller name and version 1. Controller name and version
skipping to change at page 10, line 10 skipping to change at page 10, line 8
Objective: Objective:
The time taken by controller(s) to determine the complete network The time taken by controller(s) to determine the complete network
topology, defined as the interval starting with the first discovery topology, defined as the interval starting with the first discovery
message from the controller(s) at its Southbound interface, ending message from the controller(s) at its Southbound interface, ending
with all features of the static topology determined. with all features of the static topology determined.
Reference Test Setup: Reference Test Setup:
The test SHOULD use one of the test setups described in section 3.1 The test SHOULD use one of the test setups described in section 3.1
or section 3.2 of this document in combination with Appendix A. or section 3.2 of this document.
Prerequisite: Prerequisite:
1. The controller MUST support network discovery. 1. The controller MUST support network discovery.
2. Tester should be able to retrieve the discovered topology 2. Tester should be able to retrieve the discovered topology
information either through the controller's management interface, information either through the controller's management interface,
or northbound interface to determine if the discovery was or northbound interface to determine if the discovery was
successful and complete. successful and complete.
3. Ensure that the controller's topology re-discovery timeout has 3. Ensure that the controller's topology re-discovery timeout has
been set to the maximum value to avoid initiation of re-discovery been set to the maximum value to avoid initiation of re-discovery
skipping to change at page 10, line 33 skipping to change at page 10, line 31
Procedure: Procedure:
1. Ensure that the controller is operational, its network 1. Ensure that the controller is operational, its network
applications, northbound and southbound interfaces are up and applications, northbound and southbound interfaces are up and
running. running.
2. Establish the network connections between controller and Network 2. Establish the network connections between controller and Network
Devices. Devices.
3. Record the time for the first discovery message (Tm1) received 3. Record the time for the first discovery message (Tm1) received
from the controller at forwarding plane test emulator interface from the controller at forwarding plane test emulator interface
I1. I1.
4. Query the controller every 3 seconds to obtain the discovered 4. Query the controller every t seconds (RECOMMENDED value for t is
network topology information through the northbound interface or 3) to obtain the discovered network topology information through
the management interface and compare it with the deployed network the northbound interface or the management interface and compare
topology information. it with the deployed network topology information.
5. Stop the trial when the discovered topology information matches 5. Stop the trial when the discovered topology information matches
the deployed network topology, or when the discovered topology the deployed network topology, or when the discovered topology
information return the same details for 3 consecutive queries. information return the same details for 3 consecutive queries.
6. Record the time last discovery message (Tmn) sent to controller 6. Record the time last discovery message (Tmn) sent to controller
from the forwarding plane test emulator interface (I1) when the from the forwarding plane test emulator interface (I1) when the
trial completed successfully. (e.g., the topology matches). trial completed successfully. (e.g., the topology matches).
Measurement: Measurement:
Topology Discovery Time Tr1 = Tmn-Tm1. Topology Discovery Time Tr1 = Tmn-Tm1.
skipping to change at page 11, line 34 skipping to change at page 11, line 30
The time taken by controller(s) to process an asynchronous message, The time taken by controller(s) to process an asynchronous message,
defined as the interval starting with an asynchronous message from a defined as the interval starting with an asynchronous message from a
network device after the discovery of all the devices by the network device after the discovery of all the devices by the
controller(s), ending with a response message from the controller(s) controller(s), ending with a response message from the controller(s)
at its Southbound interface. at its Southbound interface.
Reference Test Setup: Reference Test Setup:
This test SHOULD use one of the test setup described in section 3.1 This test SHOULD use one of the test setup described in section 3.1
or section 3.2 of this document in combination with Appendix A. or section 3.2 of this document.
Prerequisite: Prerequisite:
1. The controller MUST have successfully completed the network 1. The controller MUST have successfully completed the network
topology discovery for the connected Network Devices. topology discovery for the connected Network Devices.
Procedure: Procedure:
1. Generate asynchronous messages from every connected Network 1. Generate asynchronous messages from every connected Network
Device, to the SDN controller, one at a time in series from the Device, to the SDN controller, one at a time in series from the
forwarding plane test emulator for the trial duration. forwarding plane test emulator for the trial duration.
2. Record every request transmit time (T1) and the corresponding 2. Record every request transmit time (T1) and the corresponding
response received time (R1) at the forwarding plane test emulator response received time (R1) at the forwarding plane test emulator
interface (I1) for every successful message exchange. interface (I1) for every successful message exchange.
Measurement: Measurement:
(R1-T1) + (R2-T2)..(Rn-Tn) SUM{Ri} - SUM{Ti}
Asynchronous Message Processing Time Tr1 = ----------------------- Asynchronous Message Processing Time Tr1 = -----------------------
Nrx Nrx
Where Nrx is the total number of successful messages exchanged Where Nrx is the total number of successful messages exchanged
Tr1 + Tr2 + Tr3..Trn Tr1 + Tr2 + Tr3..Trn
Average Asynchronous Message Processing Time = -------------------- Average Asynchronous Message Processing Time = --------------------
Total Trials Total Trials
Asynchronous Message Processing Time Variance (TAMv) = Asynchronous Message Processing Time Variance (TAMv) =
skipping to change at page 12, line 33 skipping to change at page 12, line 27
Where TAMm is the Average Asynchronous Message Processing Time. Where TAMm is the Average Asynchronous Message Processing Time.
Reporting Format: Reporting Format:
The Asynchronous Message Processing Time results MUST be reported in The Asynchronous Message Processing Time results MUST be reported in
the format of a table with a row for each iteration. The last row of the format of a table with a row for each iteration. The last row of
the table indicates the Asynchronous Message Processing Time the table indicates the Asynchronous Message Processing Time
variance and the previous row indicates the average Asynchronous variance and the previous row indicates the average Asynchronous
Message Processing Time. Message Processing Time.
The report should capture the following information in addition to The report SHOULD capture the following information in addition to
the configuration parameters captured in section 5. the configuration parameters captured in section 4.8.
- Successful messages exchanged (Nrx) - Successful messages exchanged (Nrx)
- Percentage of unsuccessful messages exchanged, computed using the - Percentage of unsuccessful messages exchanged, computed using the
formula (1 - Nrx/Ntx) * 100), Where Ntx is the total number of formula (1 - Nrx/Ntx) * 100), Where Ntx is the total number of
messages transmitted to the controller. messages transmitted to the controller.
If this test is repeated with varying number of nodes with same If this test is repeated with varying number of nodes with same
topology, the results SHOULD be reported in the form of a graph. The topology, the results SHOULD be reported in the form of a graph. The
X coordinate SHOULD be the Number of nodes (N), the Y coordinate X coordinate SHOULD be the Number of nodes (N), the Y coordinate
SHOULD be the average Asynchronous Message Processing Time. SHOULD be the average Asynchronous Message Processing Time.
5.1.3. Asynchronous Message Processing Rate 5.1.3. Asynchronous Message Processing Rate
Objective: Objective:
Measure the number of responses to asynchronous messages (such as Measure the number of responses to asynchronous messages (such as
new flow arrival notification message, etc.) for which the new flow arrival notification message, link down, etc.) for which
controller(s) performed processing and replied with a valid and the controller(s) performed processing and replied with a valid and
productive (non-trivial) response message productive (non-trivial) response message
This test will measure two benchmarks on Asynchronous Message This test will measure two benchmarks on Asynchronous Message
Processing Rate using a single procedure. The two benchmarks are Processing Rate using a single procedure. The two benchmarks are
(see section 2.3.1.3 of [I-D.sdn-controller-benchmark-term]): (see section 2.3.1.3 of [I-D.sdn-controller-benchmark-term]):
1. Loss-free Asynchronous Message Processing Rate 1. Loss-free Asynchronous Message Processing Rate
2. Maximum Asynchronous Message Processing Rate 2. Maximum Asynchronous Message Processing Rate
Here two benchmarks are determined through a series of trials where Here two benchmarks are determined through a series of trials where
the number of messages are sent to the controller(s), and the the number of messages are sent to the controller(s), and the
responses from the controller(s) are counted over the trial responses from the controller(s) are counted over the trial
duration. The message response rate and the message loss ratio are duration. The message response rate and the message loss ratio are
calculated for each trial. calculated for each trial.
Reference Test Setup: Reference Test Setup:
The test SHOULD use one of the test setups described in section 3.1 The test SHOULD use one of the test setups described in section 3.1
or section 3.2 of this document in combination with Appendix A. or section 3.2 of this document.
Prerequisite: Prerequisite:
1. The controller(s) MUST have successfully completed the network 1. The controller(s) MUST have successfully completed the network
topology discovery for the connected Network Devices. topology discovery for the connected Network Devices.
2. Choose and record the Trial Duration (Td), the sending rate step- 2. Choose and record the Trial Duration (Td), the sending rate step-
size (STEP), the tolerance on equality for two consecutive trials size (STEP), the tolerance on equality for two consecutive trials
(P%),and the maximum possible message sending rate (Ntx1/Td). (P%),and the maximum possible message sending rate (Ntx1/Td).
Procedure: Procedure:
1. Generate asynchronous messages continuously at the maximum 1. Generate asynchronous messages continuously at the maximum
possible rate on the established connections from all the possible rate on the established connections from all the
emulated/simulated Network Devices for the given trial Duration emulated/simulated Network Devices for the given trial Duration
(Td). (Td).
2. Record the total number of responses received from the controller 2. Record the total number of responses received from the controller
(Nrx1) as well as the number of messages sent (Ntx1) to the (Nrx1) as well as the number of messages sent (Ntx1) to the
controller within the trial duration (Td). controller within the trial duration (Td).
3. Calculate the Asynchronous Message Processing Rate (Tr1) and the
3. Calculate the Asynchronous Message Processing Rate (Tr1) and Message Loss Ratio (Lr1). Ensure that the controller(s) have
the Message Loss Ratio (Lr1). Ensure that the controller(s) have
returned to normal operation. returned to normal operation.
4. Repeat the trial by reducing the asynchronous message sending rate 4. Repeat the trial by reducing the asynchronous message sending rate
used in last trial by the STEP size. used in last trial by the STEP size.
5. Continue repeating the trials and reducing the sending rate until 5. Continue repeating the trials and reducing the sending rate until
both the maximum value of Nrxn and the Nrxn corresponding to zero both the maximum value of Nrxn (number of responses received from
loss ratio have been found. the controller) and the Nrxn corresponding to zero loss ratio have
been found.
6. The trials corresponding to the benchmark levels MUST be repeated 6. The trials corresponding to the benchmark levels MUST be repeated
using the same asynchronous message rates until the responses using the same asynchronous message rates until the responses
received from the controller are equal (+/-P%) for two consecutive received from the controller are equal (+/-P%) for two consecutive
trials. trials.
7. Record the number of responses received from the controller (Nrxn) 7. Record the number of responses received from the controller (Nrxn)
as well as the number of messages sent (Ntxn) to the controller in as well as the number of messages sent (Ntxn) to the controller in
the last trial. the last trial.
Measurement: Measurement:
skipping to change at page 14, line 42 skipping to change at page 14, line 35
Loss-free Asynchronous Message Processing Rate = MAX(Trn) given Loss-free Asynchronous Message Processing Rate = MAX(Trn) given
Lrn=0 Lrn=0
Reporting Format: Reporting Format:
The Asynchronous Message Processing Rate results MUST be reported in The Asynchronous Message Processing Rate results MUST be reported in
the format of a table with a row for each trial. the format of a table with a row for each trial.
The table should report the following information in addition to the The table should report the following information in addition to the
configuration parameters captured in section 5, with columns: configuration parameters captured in section 4.8, with columns:
- Offered rate (Ntxn/Td) - Offered rate (Ntxn/Td)
- Asynchronous Message Processing Rate (Trn) - Asynchronous Message Processing Rate (Trn)
- Loss Ratio (Lr) - Loss Ratio (Lr)
- Benchmark at this iteration (blank for none, Maximum, Loss-Free) - Benchmark at this iteration (blank for none, Maximum, Loss-Free)
The results MAY be presented in the form of a graph. The X axis The results MAY be presented in the form of a graph. The X axis
SHOULD be the Offered rate, and dual Y axes would represent SHOULD be the Offered rate, and dual Y axes would represent
Asynchronous Message Processing Rate and Loss Ratio, respectively. Asynchronous Message Processing Rate and Loss Ratio, respectively.
If this test is repeated with varying number of nodes over same If this test is repeated with varying number of nodes over same
topology, the results SHOULD be reported in the form of a graph. The topology, the results SHOULD be reported in the form of a graph. The
X axis SHOULD be the Number of nodes (N), the Y axis SHOULD be the X axis SHOULD be the Number of nodes (N), the Y axis SHOULD be the
Asynchronous Message Processing Rate. Both the Maximum and the Loss- Asynchronous Message Processing Rate. Both the Maximum and the Loss-
Free Rates should be plotted for each N. Free Rates should be plotted for each N.
skipping to change at page 15, line 28 skipping to change at page 15, line 22
The time taken by the controller to setup a path reactively between The time taken by the controller to setup a path reactively between
source and destination node, defined as the interval starting with source and destination node, defined as the interval starting with
the first flow provisioning request message received by the the first flow provisioning request message received by the
controller(s) at its Southbound interface, ending with the last flow controller(s) at its Southbound interface, ending with the last flow
provisioning response message sent from the controller(s) at its provisioning response message sent from the controller(s) at its
Southbound interface. Southbound interface.
Reference Test Setup: Reference Test Setup:
The test SHOULD use one of the test setups described in section 3.1 The test SHOULD use one of the test setups described in section 3.1
or section 3.2 of this document in combination with Appendix A. The or section 3.2 of this document. The number of Network Devices in
number of Network Devices in the path is a parameter of the test the path is a parameter of the test that may be varied from 2 to
that may be varied from 2 to maximum discovery size in repetitions maximum discovery size in repetitions of this test.
of this test.
Prerequisite: Prerequisite:
1. The controller MUST contain the network topology information for 1. The controller MUST contain the network topology information for
the deployed network topology. the deployed network topology.
2. The controller should have the knowledge about the location of 2. The controller should have the knowledge about the location of
destination endpoint for which the path has to be provisioned. destination endpoint for which the path has to be provisioned.
This can be achieved through dynamic learning or static This can be achieved through dynamic learning or static
provisioning. provisioning.
3. Ensure that the default action for 'flow miss' in Network Device 3. Ensure that the default action for 'flow miss' in Network Device
skipping to change at page 16, line 36 skipping to change at page 16, line 27
Where TRPm is the Average Reactive Path Provisioning Time. Where TRPm is the Average Reactive Path Provisioning Time.
Reporting Format: Reporting Format:
The Reactive Path Provisioning Time results MUST be reported in the The Reactive Path Provisioning Time results MUST be reported in the
format of a table with a row for each iteration. The last row of the format of a table with a row for each iteration. The last row of the
table indicates the Reactive Path Provisioning Time variance and the table indicates the Reactive Path Provisioning Time variance and the
previous row indicates the Average Reactive Path Provisioning Time. previous row indicates the Average Reactive Path Provisioning Time.
The report should capture the following information in addition to The report should capture the following information in addition to
the configuration parameters captured in section 5. the configuration parameters captured in section 4.8.
- Number of Network Devices in the path - Number of Network Devices in the path
5.1.5. Proactive Path Provisioning Time 5.1.5. Proactive Path Provisioning Time
Objective: Objective:
The time taken by the controller to setup a path proactively between The time taken by the controller to setup a path proactively between
source and destination node, defined as the interval starting with source and destination node, defined as the interval starting with
the first proactive flow provisioned in the controller(s) at its the first proactive flow provisioned in the controller(s) at its
Northbound interface, ending with the last flow provisioning Northbound interface, ending with the last flow provisioning
response message sent from the controller(s) at its Southbound response message sent from the controller(s) at its Southbound
interface. interface.
Reference Test Setup: Reference Test Setup:
The test SHOULD use one of the test setups described in section 3.1 The test SHOULD use one of the test setups described in section 3.1
or section 3.2 of this document in combination with Appendix A. or section 3.2 of this document.
Prerequisite: Prerequisite:
1. The controller MUST contain the network topology information for 1. The controller MUST contain the network topology information for
the deployed network topology. the deployed network topology.
2. The controller should have the knowledge about the location of 2. The controller should have the knowledge about the location of
destination endpoint for which the path has to be provisioned. destination endpoint for which the path has to be provisioned.
This can be achieved through dynamic learning or static This can be achieved through dynamic learning or static
provisioning. provisioning.
3. Ensure that the default action for flow miss in Network Device is 3. Ensure that the default action for flow miss in Network Device is
skipping to change at page 18, line 16 skipping to change at page 18, line 8
Reporting Format: Reporting Format:
The Proactive Path Provisioning Time results MUST be reported in the The Proactive Path Provisioning Time results MUST be reported in the
format of a table with a row for each iteration. The last row of the format of a table with a row for each iteration. The last row of the
table indicates the Proactive Path Provisioning Time variance and table indicates the Proactive Path Provisioning Time variance and
the previous row indicates the Average Proactive Path Provisioning the previous row indicates the Average Proactive Path Provisioning
Time. Time.
The report should capture the following information in addition to The report should capture the following information in addition to
the configuration parameters captured in section 5. the configuration parameters captured in section 4.8.
- Number of Network Devices in the path - Number of Network Devices in the path
5.1.6. Reactive Path Provisioning Rate 5.1.6. Reactive Path Provisioning Rate
Objective: Objective:
The maximum number of independent paths a controller can The maximum number of independent paths a controller can
concurrently establish per second between source and destination concurrently establish per second between source and destination
nodes reactively, defined as the number of paths provisioned per nodes reactively, defined as the number of paths provisioned per
second by the controller(s) at its Southbound interface for the flow second by the controller(s) at its Southbound interface for the flow
provisioning requests received for path provisioning at its provisioning requests received for path provisioning at its
Southbound interface between the start of the test and the expiry of Southbound interface between the start of the test and the expiry of
given trial duration. given trial duration.
Reference Test Setup: Reference Test Setup:
The test SHOULD use one of the test setups described in section 3.1 The test SHOULD use one of the test setups described in section 3.1
or section 3.2 of this document in combination with Appendix A. or section 3.2 of this document.
Prerequisite: Prerequisite:
1. The controller MUST contain the network topology information for 1. The controller MUST contain the network topology information for
the deployed network topology. the deployed network topology.
2. The controller should have the knowledge about the location of 2. The controller should have the knowledge about the location of
destination addresses for which the paths have to be provisioned. destination addresses for which the paths have to be provisioned.
This can be achieved through dynamic learning or static This can be achieved through dynamic learning or static
provisioning. provisioning.
3. Ensure that the default action for 'flow miss' in Network Device 3. Ensure that the default action for 'flow miss' in Network Device
skipping to change at page 19, line 36 skipping to change at page 19, line 29
Where RPPm is the Average Reactive Path Provisioning Rate. Where RPPm is the Average Reactive Path Provisioning Rate.
Reporting Format: Reporting Format:
The Reactive Path Provisioning Rate results MUST be reported in the The Reactive Path Provisioning Rate results MUST be reported in the
format of a table with a row for each iteration. The last row of the format of a table with a row for each iteration. The last row of the
table indicates the Reactive Path Provisioning Rate variance and the table indicates the Reactive Path Provisioning Rate variance and the
previous row indicates the Average Reactive Path Provisioning Rate. previous row indicates the Average Reactive Path Provisioning Rate.
The report should capture the following information in addition to The report should capture the following information in addition to
the configuration parameters captured in section 5. the configuration parameters captured in section 4.8.
- Number of Network Devices in the path - Number of Network Devices in the path
- Offered rate - Offered rate
5.1.7. Proactive Path Provisioning Rate 5.1.7. Proactive Path Provisioning Rate
Objective: Objective:
Measure the maximum number of independent paths a controller can Measure the maximum number of independent paths a controller can
concurrently establish per second between source and destination concurrently establish per second between source and destination
nodes proactively, defined as the number of paths provisioned per nodes proactively, defined as the number of paths provisioned per
second by the controller(s) at its Southbound interface for the second by the controller(s) at its Southbound interface for the
paths requested in its Northbound interface between the start of the paths requested in its Northbound interface between the start of the
test and the expiry of given trial duration. The measurement is test and the expiry of given trial duration. The measurement is
based on dataplane observations of successful path activation based on dataplane observations of successful path activation
Reference Test Setup: Reference Test Setup:
The test SHOULD use one of the test setups described in section 3.1 The test SHOULD use one of the test setups described in section 3.1
or section 3.2 of this document in combination with Appendix A. or section 3.2 of this document.
Prerequisite: Prerequisite:
1. The controller MUST contain the network topology information for 1. The controller MUST contain the network topology information for
the deployed network topology. the deployed network topology.
2. The controller should have the knowledge about the location of 2. The controller should have the knowledge about the location of
destination addresses for which the paths have to be provisioned. destination addresses for which the paths have to be provisioned.
This can be achieved through dynamic learning or static This can be achieved through dynamic learning or static
provisioning. provisioning.
skipping to change at page 21, line 19 skipping to change at page 21, line 14
Reporting Format: Reporting Format:
The Proactive Path Provisioning Rate results MUST be reported in the The Proactive Path Provisioning Rate results MUST be reported in the
format of a table with a row for each iteration. The last row of the format of a table with a row for each iteration. The last row of the
table indicates the Proactive Path Provisioning Rate variance and table indicates the Proactive Path Provisioning Rate variance and
the previous row indicates the Average Proactive Path Provisioning the previous row indicates the Average Proactive Path Provisioning
Rate. Rate.
The report should capture the following information in addition to The report should capture the following information in addition to
the configuration parameters captured in section 5. the configuration parameters captured in section 4.8.
- Number of Network Devices in the path - Number of Network Devices in the path
- Offered rate - Offered rate
5.1.8. Network Topology Change Detection Time 5.1.8. Network Topology Change Detection Time
Objective: Objective:
The amount of time required for the controller to detect any changes The amount of time required for the controller to detect any changes
in the network topology, defined as the interval starting with the in the network topology, defined as the interval starting with the
notification message received by the controller(s) at its Southbound notification message received by the controller(s) at its Southbound
interface, ending with the first topology rediscovery messages sent interface, ending with the first topology rediscovery messages sent
from the controller(s) at its Southbound interface. from the controller(s) at its Southbound interface.
Reference Test Setup: Reference Test Setup:
The test SHOULD use one of the test setups described in section 3.1 The test SHOULD use one of the test setups described in section 3.1
or section 3.2 of this document in combination with Appendix A. or section 3.2 of this document.
Prerequisite: Prerequisite:
1. The controller MUST have successfully discovered the network 1. The controller MUST have successfully discovered the network
topology information for the deployed network topology. topology information for the deployed network topology.
2. The periodic network discovery operation should be configured to 2. The periodic network discovery operation should be configured to
twice the Trial duration (Td) value. twice the Trial duration (Td) value.
Procedure: Procedure:
skipping to change at page 23, line 20 skipping to change at page 23, line 10
Measure the maximum number of control sessions the controller can Measure the maximum number of control sessions the controller can
maintain, defined as the number of sessions that the controller can maintain, defined as the number of sessions that the controller can
accept from network devices, starting with the first control accept from network devices, starting with the first control
session, ending with the last control session that the controller(s) session, ending with the last control session that the controller(s)
accepts at its Southbound interface. accepts at its Southbound interface.
Reference Test Setup: Reference Test Setup:
The test SHOULD use one of the test setups described in section 3.1 The test SHOULD use one of the test setups described in section 3.1
or section 3.2 of this document in combination with Appendix A. or section 3.2 of this document.
Procedure: Procedure:
1. Establish control connection with controller from every Network 1. Establish control connection with controller from every Network
Device emulated in the forwarding plane test emulator. Device emulated in the forwarding plane test emulator.
2. Stop the trial when the controller starts dropping the control 2. Stop the trial when the controller starts dropping the control
connections. connections.
3. Record the number of successful connections established with the 3. Record the number of successful connections established with the
controller (CCn) at the forwarding plane test emulator. controller (CCn) at the forwarding plane test emulator.
Measurement: Measurement:
Control Sessions Capacity = CCn. Control Sessions Capacity = CCn.
Reporting Format: Reporting Format:
The Control Session Capacity results MUST be reported in addition to The Control Session Capacity results MUST be reported in addition to
the configuration parameters captured in section 5. the configuration parameters captured in section 4.8.
5.2.2. Network Discovery Size 5.2.2. Network Discovery Size
Objective: Objective:
Measure the network size (number of nodes, links and hosts) that a Measure the network size (number of nodes, links and hosts) that a
controller can discover, defined as the size of a network that the controller can discover, defined as the size of a network that the
controller(s) can discover, starting from a network topology given controller(s) can discover, starting from a network topology given
by the user for discovery, ending with the topology that the by the user for discovery, ending with the topology that the
controller(s) could successfully discover. controller(s) could successfully discover.
Reference Test Setup: Reference Test Setup:
The test SHOULD use one of the test setups described in section 3.1 The test SHOULD use one of the test setups described in section 3.1
or section 3.2 of this document in combination with Appendix A. or section 3.2 of this document.
Prerequisite: Prerequisite:
1. The controller MUST support automatic network discovery. 1. The controller MUST support automatic network discovery.
2. Tester should be able to retrieve the discovered topology 2. Tester should be able to retrieve the discovered topology
information either through controller's management interface or information either through controller's management interface or
northbound interface. northbound interface.
Procedure: Procedure:
1. Establish the network connections between controller and network 1. Establish the network connections between controller and network
nodes. nodes.
2. Query the controller for the discovered network topology 2. Query the controller every t seconds (RECOMMENDED value for t is
information and compare it with the deployed network topology 30) to obtain the discovered network topology information through
information. the northbound interface or the management interface.
3. If the comparison is successful, increase the number of nodes by 1 3. Stop the trial when the discovered network topology information
remains the same as that of last two query responses.
4. Compare the obtained network topology information with the
deployed network topology information.
5. If the comparison is successful, increase the number of nodes by 1
and repeat the trial. and repeat the trial.
If the comparison is unsuccessful, decrease the number of nodes by If the comparison is unsuccessful, decrease the number of nodes by
1 and repeat the trial. 1 and repeat the trial.
4. Continue the trial until the comparison of step 3 is successful. 6. Continue the trial until the comparison of step 5 is successful.
5. Record the number of nodes for the last trial (Ns) where the 7. Record the number of nodes for the last trial run (Ns) where the
topology comparison was successful. topology comparison was successful.
Measurement: Measurement:
Network Discovery Size = Ns. Network Discovery Size = Ns.
Reporting Format: Reporting Format:
The Network Discovery Size results MUST be reported in addition to The Network Discovery Size results MUST be reported in addition to
the configuration parameters captured in section 5. the configuration parameters captured in section 4.8.
5.2.3. Forwarding Table Capacity 5.2.3. Forwarding Table Capacity
Objective: Objective:
Measure the maximum number of flow entries a controller can manage Measure the maximum number of flow entries a controller can manage
in its Forwarding table. in its Forwarding table.
Reference Test Setup: Reference Test Setup:
The test SHOULD use one of the test setups described in section 3.1 The test SHOULD use one of the test setups described in section 3.1
or section 3.2 of this document in combination with Appendix A. or section 3.2 of this document.
Prerequisite: Prerequisite:
1. The controller Forwarding table should be empty. 1. The controller Forwarding table should be empty.
2. Flow Idle time MUST be set to higher or infinite value. 2. Flow Idle time MUST be set to higher or infinite value.
3. The controller MUST have successfully completed network topology 3. The controller MUST have successfully completed network topology
discovery. discovery.
4. Tester should be able to retrieve the forwarding table information 4. Tester should be able to retrieve the forwarding table information
either through controller's management interface or northbound either through controller's management interface or northbound
interface. interface.
Procedure: Procedure:
Reactive Flow Provisioning Mode: Reactive Flow Provisioning Mode:
1. Send bi-directional traffic continuously with unique source and/or 1. Send bi-directional traffic continuously with unique source and
destination addresses from test traffic generators TP1 and TP2 at destination addresses from test traffic generators TP1 and TP2 at
the asynchronous message processing rate of controller. the asynchronous message processing rate of controller.
2. Query the controller at a regular interval (e.g., 5 seconds) for 2. Query the controller at a regular interval (e.g., 5 seconds) for
the number of learned flow entries from its northbound interface. the number of learned flow entries from its northbound interface.
3. Stop the trial when the retrieved value is constant for three 3. Stop the trial when the retrieved value is constant for three
consecutive iterations and record the value received from the last consecutive iterations and record the value received from the last
query (Nrp). query (Nrp).
Proactive Flow Provisioning Mode: Proactive Flow Provisioning Mode:
skipping to change at page 26, line 8 skipping to change at page 26, line 4
Measurement: Measurement:
Proactive Flow Provisioning Mode: Proactive Flow Provisioning Mode:
Max Flow Entries = Total number of flows provisioned (Nrp) Max Flow Entries = Total number of flows provisioned (Nrp)
Reactive Flow Provisioning Mode: Reactive Flow Provisioning Mode:
Max Flow Entries = Total number of learned flow entries (Nrp) Max Flow Entries = Total number of learned flow entries (Nrp)
Forwarding Table Capacity = Max Flow Entries. Forwarding Table Capacity = Max Flow Entries.
Reporting Format: Reporting Format:
The Forwarding Table Capacity results MUST be tabulated with the The Forwarding Table Capacity results MUST be tabulated with the
following information in addition to the configuration parameters following information in addition to the configuration parameters
captured in section 5. captured in section 4.8.
- Provisioning Type (Proactive/Reactive) - Provisioning Type (Proactive/Reactive)
5.3. Security 5.3. Security
5.3.1. Exception Handling 5.3.1. Exception Handling
Objective: Objective:
Determine the effect of handling error packets and notifications on Determine the effect of handling error packets and notifications on
skipping to change at page 26, line 38 skipping to change at page 26, line 33
a. Path Provisioning Rate a. Path Provisioning Rate
b. Path Provisioning Time b. Path Provisioning Time
c. Network Topology Change Detection Time c. Network Topology Change Detection Time
Reference Test Setup: Reference Test Setup:
The test SHOULD use one of the test setups described in section 3.1 The test SHOULD use one of the test setups described in section 3.1
or section 3.2 of this document in combination with Appendix A. or section 3.2 of this document.
Prerequisite: Prerequisite:
1. This test MUST be performed after obtaining the baseline 1. This test MUST be performed after obtaining the baseline
measurement results for the above performance tests. measurement results for the above performance tests.
2. Ensure that the invalid messages are not dropped by the 2. Ensure that the invalid messages are not dropped by the
intermediate devices connecting the controller and Network intermediate devices connecting the controller and Network
Devices. Devices.
Procedure: Procedure:
1. Perform the above listed performance tests and send 1% of messages 1. Perform the above listed performance tests and send 1% of messages
from the Asynchronous Message Processing Rate as invalid messages from the Asynchronous Message Processing Rate as invalid messages
from the connected Network Devices emulated at the forwarding from the connected Network Devices emulated at the forwarding
plane test emulator. plane test emulator.
skipping to change at page 28, line 8 skipping to change at page 28, line 4
Objective: Objective:
Determine the effect of handling DoS attacks on performance and Determine the effect of handling DoS attacks on performance and
scalability tests the impact MUST be measured for the following scalability tests the impact MUST be measured for the following
tests: tests:
a. Path Provisioning Rate a. Path Provisioning Rate
b. Path Provisioning Time b. Path Provisioning Time
c. Network Topology Change Detection Time c. Network Topology Change Detection Time
d. Network Discovery Size d. Network Discovery Size
Reference Test Setup: Reference Test Setup:
The test SHOULD use one of the test setups described in section 3.1 The test SHOULD use one of the test setups described in section 3.1
or section 3.2 of this document in combination with Appendix A. or section 3.2 of this document.
Prerequisite: Prerequisite:
This test MUST be performed after obtaining the baseline measurement This test MUST be performed after obtaining the baseline measurement
results for the above tests. results for the above tests.
Procedure: Procedure:
1. Perform the listed tests and launch a DoS attack towards 1. Perform the listed tests and launch a DoS attack towards
controller while the trial is running. controller while the trial is running.
skipping to change at page 29, line 24 skipping to change at page 29, line 21
The time taken to switch from an active controller to the backup The time taken to switch from an active controller to the backup
controller, when the controllers work in redundancy mode and the controller, when the controllers work in redundancy mode and the
active controller fails, defined as the interval starting with the active controller fails, defined as the interval starting with the
active controller bringing down, ending with the first re-discovery active controller bringing down, ending with the first re-discovery
message received from the new controller at its Southbound message received from the new controller at its Southbound
interface. interface.
Reference Test Setup: Reference Test Setup:
The test SHOULD use the test setup described in section 3.2 of this The test SHOULD use the test setup described in section 3.2 of this
document in combination with Appendix A. document.
Prerequisite: Prerequisite:
1. Master controller election MUST be completed. 1. Master controller election MUST be completed.
2. Nodes are connected to the controller cluster as per the 2. Nodes are connected to the controller cluster as per the
Redundancy Mode (RM). Redundancy Mode (RM).
3. The controller cluster should have successfully completed the 3. The controller cluster should have successfully completed the
network topology discovery. network topology discovery.
4. The Network Device MUST send all new flows to the controller when 4. The Network Device MUST send all new flows to the controller when
it receives from the test traffic generator. it receives from the test traffic generator.
5. Controller should have learned the location of destination (D1) at 5. Controller should have learned the location of destination (D1) at
test traffic generator TP2. TP2.
Procedure: Procedure:
1. Send uni-directional traffic continuously with incremental 1. Send uni-directional traffic continuously with incremental
sequence number and source addresses from test traffic generator sequence number and source addresses from test traffic generator
TP1 at the rate that the controller processes without any drops. TP1 at the rate that the controller processes without any drops.
2. Ensure that there are no packet drops observed at the test traffic 2. Ensure that there are no packet drops observed at TP2.
generator TP2.
3. Bring down the active controller. 3. Bring down the active controller.
4. Stop the trial when a first frame received on TP2 after failover 4. Stop the trial when a first frame received on TP2 after failover
operation. operation.
5. Record the time at which the last valid frame received (T1) at 5. Record the time at which the last valid frame received (T1) at
test traffic generator TP2 before sequence error and the first test traffic generator TP2 before sequence error and the first
valid frame received (T2) after the sequence error at TP2 valid frame received (T2) after the sequence error at TP2
Measurement: Measurement:
Controller Failover Time = (T2 - T1) Controller Failover Time = (T2 - T1)
Packet Loss = Number of missing packet sequences. Packet Loss = Number of missing packet sequences.
skipping to change at page 30, line 43 skipping to change at page 30, line 39
The time taken to re-route the traffic by the Controller, when there The time taken to re-route the traffic by the Controller, when there
is a failure in existing traffic paths, defined as the interval is a failure in existing traffic paths, defined as the interval
starting from the first failure notification message received by the starting from the first failure notification message received by the
controller, ending with the last flow re-provisioning message sent controller, ending with the last flow re-provisioning message sent
by the controller at its Southbound interface. by the controller at its Southbound interface.
Reference Test Setup: Reference Test Setup:
This test SHOULD use one of the test setup described in section 3.1 This test SHOULD use one of the test setup described in section 3.1
or section 3.2 of this document in combination with Appendix A. or section 3.2 of this document.
Prerequisite: Prerequisite:
1. Network with the given number of nodes and redundant paths MUST be 1. Network with the given number of nodes and redundant paths MUST be
deployed. deployed.
2. Ensure that the controller MUST have knowledge about the location 2. Ensure that the controller MUST have knowledge about the location
of test traffic generators TP1 and TP2. of test traffic generators TP1 and TP2.
3. Ensure that the controller does not pre-provision the alternate 3. Ensure that the controller does not pre-provision the alternate
path in the emulated Network Devices at the forwarding plane test path in the emulated Network Devices at the forwarding plane test
emulator. emulator.
Procedure: Procedure:
1. Send bi-directional traffic continuously with unique sequence 1. Send bi-directional traffic continuously with unique sequence
number from TP1 and TP2. number from TP1 and TP2.
2. Bring down a link or switch in the traffic path. 2. Bring down a link or switch in the traffic path.
3. Stop the trial after receiving first frame after network re- 3. Stop the trial after receiving first frame after network re-
skipping to change at page 32, line 4 skipping to change at page 31, line 48
at TP2 at TP2
Reporting Format: Reporting Format:
The Network Re-Provisioning Time results MUST be tabulated with the The Network Re-Provisioning Time results MUST be tabulated with the
following information. following information.
- Number of nodes in the primary path - Number of nodes in the primary path
- Number of nodes in the alternate path - Number of nodes in the alternate path
- Network Re-Provisioning Time
- Network Re-Provisioning Time
- Forward Direction Packet Loss - Forward Direction Packet Loss
- Reverse Direction Packet Loss - Reverse Direction Packet Loss
6. References 6. References
6.1. Normative References 6.1. Normative References
[RFC2119] S. Bradner, "Key words for use in RFCs to Indicate [RFC2119] S. Bradner, "Key words for use in RFCs to Indicate
Requirement Levels", RFC 2119, March 1997. Requirement Levels", RFC 2119, March 1997.
[RFC8174] B. Leiba, "Ambiguity of Uppercase vs Lowercase in RFC [RFC8174] B. Leiba, "Ambiguity of Uppercase vs Lowercase in RFC
2119 Key Words", RFC 8174, May 2017. 2119 Key Words", RFC 8174, May 2017.
[I-D.sdn-controller-benchmark-term] Bhuvaneswaran.V, Anton Basil, [I-D.sdn-controller-benchmark-term] Bhuvaneswaran.V, Anton Basil,
Mark.T, Vishwas Manral, Sarah Banks, "Terminology for Mark.T, Vishwas Manral, Sarah Banks, "Terminology for
Benchmarking SDN Controller Performance", Benchmarking SDN Controller Performance",
draft-ietf-bmwg-sdn-controller-benchmark-term-08 draft-ietf-bmwg-sdn-controller-benchmark-term-10
(Work in progress), February 25, 2018 (Work in progress), May 25, 2018
6.2. Informative References 6.2. Informative References
[OpenFlow Switch Specification] ONF,"OpenFlow Switch Specification" [OpenFlow Switch Specification] ONF,"OpenFlow Switch Specification"
Version 1.4.0 (Wire Protocol 0x05), October 14, 2013. Version 1.4.0 (Wire Protocol 0x05), October 14, 2013.
7. IANA Considerations 7. IANA Considerations
This document does not have any IANA requests. This document does not have any IANA requests.
skipping to change at page 34, line 5 skipping to change at page 34, line 5
The authors would like to thank the following individuals for The authors would like to thank the following individuals for
providing their valuable comments to the earlier versions of this providing their valuable comments to the earlier versions of this
document: Al Morton (AT&T), Sandeep Gangadharan (HP), M. Georgescu document: Al Morton (AT&T), Sandeep Gangadharan (HP), M. Georgescu
(NAIST), Andrew McGregor (Google), Scott Bradner , Jay Karthik (NAIST), Andrew McGregor (Google), Scott Bradner , Jay Karthik
(Cisco), Ramakrishnan (Dell), Khasanov Boris (Huawei), Brian (Cisco), Ramakrishnan (Dell), Khasanov Boris (Huawei), Brian
Castelli (Spirent) Castelli (Spirent)
This document was prepared using 2-Word-v2.0.template.dot. This document was prepared using 2-Word-v2.0.template.dot.
Appendix A. Example Test Topology Appendix A Benchmarking Methodology using OpenFlow Controllers
A.1. Leaf-Spine Topology
+------+ +------+
| SDN | | SDN | (Spine)
| Node |.. | Node |
+------+ +------+
/ \ / \
/ \ / \
l1 / / \ ln
/ / \ \
+--------+ +-------+
| SDN | | SDN |
| Node |.. | Node | (Leaf)
+--------+ +-------+
Appendix B. Benchmarking Methodology using OpenFlow Controllers
This section gives an overview of OpenFlow protocol and provides This section gives an overview of OpenFlow protocol and provides
test methodology to benchmark SDN controllers supporting OpenFlow test methodology to benchmark SDN controllers supporting OpenFlow
southbound protocol. southbound protocol. OpenFlow protocol is used as an example to
illustrate the methodologies defined in this document.
B.1. Protocol Overview A.1. Protocol Overview
OpenFlow is an open standard protocol defined by Open Networking OpenFlow is an open standard protocol defined by Open Networking
Foundation (ONF)[ OpenFlow Switch Specification], used for Foundation (ONF)[ OpenFlow Switch Specification], used for
programming the forwarding plane of network switches or routers via programming the forwarding plane of network switches or routers via
a centralized controller. a centralized controller.
B.2. Messages Overview A.2. Messages Overview
OpenFlow protocol supports three messages types namely controller- OpenFlow protocol supports three messages types namely controller-
to-switch, asynchronous and symmetric. to-switch, asynchronous and symmetric.
Controller-to-switch messages are initiated by the controller and Controller-to-switch messages are initiated by the controller and
used to directly manage or inspect the state of the switch. These used to directly manage or inspect the state of the switch. These
messages allow controllers to query/configure the switch (Features, messages allow controllers to query/configure the switch (Features,
Configuration messages), collect information from switch (Read-State Configuration messages), collect information from switch (Read-State
message), send packets on specified port of switch (Packet-out message), send packets on specified port of switch (Packet-out
message), and modify switch forwarding plane and state (Modify- message), and modify switch forwarding plane and state (Modify-
skipping to change at page 35, line 41 skipping to change at page 34, line 42
Asynchronous messages are generated by the switch without a Asynchronous messages are generated by the switch without a
controller soliciting them. These messages allow switches to update controller soliciting them. These messages allow switches to update
controllers to denote an arrival of new flow (Packet-in), switch controllers to denote an arrival of new flow (Packet-in), switch
state change (Flow-Removed, Port-status) and error (Error). state change (Flow-Removed, Port-status) and error (Error).
Symmetric messages are generated in either direction without Symmetric messages are generated in either direction without
solicitation. These messages allow switches and controllers to set solicitation. These messages allow switches and controllers to set
up connection (Hello), verify for liveness (Echo) and offer up connection (Hello), verify for liveness (Echo) and offer
additional functionalities (Experimenter). additional functionalities (Experimenter).
B.3. Connection Overview A.3. Connection Overview
OpenFlow channel is used to exchange OpenFlow message between an OpenFlow channel is used to exchange OpenFlow message between an
OpenFlow switch and an OpenFlow controller. The OpenFlow channel OpenFlow switch and an OpenFlow controller. The OpenFlow channel
connection can be setup using plain TCP or TLS. By default, a switch connection can be setup using plain TCP or TLS. By default, a switch
establishes single connection with SDN controller. A switch may establishes single connection with SDN controller. A switch may
establish multiple parallel connections to single controller establish multiple parallel connections to single controller
(auxiliary connection) or multiple controllers to handle controller (auxiliary connection) or multiple controllers to handle controller
failures and load balancing. failures and load balancing.
B.4. Performance Benchmarking Tests A.4. Performance Benchmarking Tests
B.4.1. Network Topology Discovery Time A.4.1. Network Topology Discovery Time
Procedure: Procedure:
Network Devices OpenFlow SDN Network Devices OpenFlow SDN
Controller Application Controller Application
| | | | | |
| |<Initialize controller | | |<Initialize controller |
| |app.,NB and SB interfaces> | | |app.,NB and SB interfaces> |
| | | | | |
|<Deploy network with | | |<Deploy network with | |
skipping to change at page 37, line 21 skipping to change at page 36, line 21
Tmn: Time of last LLDP message sent to controller Tmn: Time of last LLDP message sent to controller
Discussion: Discussion:
The Network Topology Discovery Time can be obtained by calculating The Network Topology Discovery Time can be obtained by calculating
the time difference between the first PACKET_OUT with LLDP message the time difference between the first PACKET_OUT with LLDP message
received from the controller (Tm1) and the last PACKET_IN with LLDP received from the controller (Tm1) and the last PACKET_IN with LLDP
message sent to the controller (Tmn) when the comparison is message sent to the controller (Tmn) when the comparison is
successful. successful.
B.4.2. Asynchronous Message Processing Time A.4.2. Asynchronous Message Processing Time
Procedure: Procedure:
Network Devices OpenFlow SDN Network Devices OpenFlow SDN
Controller Application Controller Application
| | | | | |
|PACKET_IN with single | | |PACKET_IN with single | |
|OFP match header | | |OFP match header | |
(T0)|--------------------------->| | (T0)|--------------------------->| |
| | | | | |
skipping to change at page 38, line 20 skipping to change at page 37, line 20
T0,T1, ..Tn are PACKET_IN messages transmit timestamps. T0,T1, ..Tn are PACKET_IN messages transmit timestamps.
R0,R1, ..Rn are PACKET_OUT messages receive timestamps. R0,R1, ..Rn are PACKET_OUT messages receive timestamps.
Nrx : Number of successful PACKET_IN/PACKET_OUT message Nrx : Number of successful PACKET_IN/PACKET_OUT message
exchanges exchanges
Discussion: Discussion:
The Asynchronous Message Processing Time will be obtained by sum of The Asynchronous Message Processing Time will be obtained by sum of
((R0-T0),(R1-T1)..(Rn - Tn))/ Nrx. ((R0-T0),(R1-T1)..(Rn - Tn))/ Nrx.
B.4.3. Asynchronous Message Processing Rate A.4.3. Asynchronous Message Processing Rate
Procedure: Procedure:
Network Devices OpenFlow SDN Network Devices OpenFlow SDN
Controller Application Controller Application
| | | | | |
|PACKET_IN with single OFP | | |PACKET_IN with single OFP | |
|match headers | | |match headers | |
|--------------------------->| | |--------------------------->| |
| | | | | |
skipping to change at page 39, line 25 skipping to change at page 38, line 25
Discussion: Discussion:
This test will measure two benchmarks using single procedure. 1) The This test will measure two benchmarks using single procedure. 1) The
Maximum Asynchronous Message Processing Rate will be obtained by Maximum Asynchronous Message Processing Rate will be obtained by
calculating the maximum PACKET OUTs (Nrxn) received from the calculating the maximum PACKET OUTs (Nrxn) received from the
controller(s) across n trials. 2) The Loss-free Asynchronous Message controller(s) across n trials. 2) The Loss-free Asynchronous Message
Processing Rate will be obtained by calculating the maximum PACKET Processing Rate will be obtained by calculating the maximum PACKET
OUTs received from controller (s) when Loss Ratio equals zero. The OUTs received from controller (s) when Loss Ratio equals zero. The
loss ratio is obtained by 1 - Nrxn/Ntxn loss ratio is obtained by 1 - Nrxn/Ntxn
B.4.4. Reactive Path Provisioning Time A.4.4. Reactive Path Provisioning Time
Procedure: Procedure:
Test Traffic Test Traffic Network Devices OpenFlow Test Traffic Test Traffic Network Devices OpenFlow
Generator TP1 Generator TP2 Controller Generator TP1 Generator TP2 Controller
| | | | | | | |
| |G-ARP (D1) | | | |G-ARP (D1) | |
| |--------------------->| | | |--------------------->| |
| | | | | | | |
| | |PACKET_IN(D1) | | | |PACKET_IN(D1) |
skipping to change at page 40, line 18 skipping to change at page 39, line 18
G-ARP: Gratuitous ARP message. G-ARP: Gratuitous ARP message.
Tsf1: Time of first frame sent from TP1 Tsf1: Time of first frame sent from TP1
Tdf1: Time of first frame received from TP2 Tdf1: Time of first frame received from TP2
Discussion: Discussion:
The Reactive Path Provisioning Time can be obtained by finding the The Reactive Path Provisioning Time can be obtained by finding the
time difference between the transmit and receive time of the traffic time difference between the transmit and receive time of the traffic
(Tsf1-Tdf1). (Tsf1-Tdf1).
B.4.5. Proactive Path Provisioning Time A.4.5. Proactive Path Provisioning Time
Procedure: Procedure:
Test Traffic Test Traffic Network Devices OpenFlow SDN Test Traffic Test Traffic Network Devices OpenFlow SDN
Generator TP1 Generator TP2 Controller Application Generator TP1 Generator TP2 Controller Application
| | | | | | | | | |
| | | | |
| | | | <Install flow|
| | | | for S1,D1> |
| |G-ARP (D1) | | | | |G-ARP (D1) | | |
| |-------------->| | | | |-------------->| | |
| | | | | | | | | |
| | |PACKET_IN(D1) | | | | |PACKET_IN(D1) | |
| | |--------------->| | | | |--------------->| |
| | | | | | | | | |
|Traffic (S1,D1) | | | |Traffic (S1,D1) | | |
Tsf1)|---------------------------->| | | Tsf1)|---------------------------->| | |
| | | | | | | | | |
| | | | <Install flow|
| | | | for S1,D1> |
| | | | |
| | | FLOW_MOD(D1) | | | | | FLOW_MOD(D1) | |
| | |<---------------| | | | |<---------------| |
| | | | | | | | | |
| |Traffic (S1,D1)| | | | |Traffic (S1,D1)| | |
| (Tdf1)|<--------------| | | | (Tdf1)|<--------------| | |
| | | | | | | | | |
Legend: Legend:
G-ARP: Gratuitous ARP message. G-ARP: Gratuitous ARP message.
Tsf1: Time of first frame sent from TP1 Tsf1: Time of first frame sent from TP1
Tdf1: Time of first frame received from TP2 Tdf1: Time of first frame received from TP2
Discussion: Discussion:
The Proactive Path Provisioning Time can be obtained by finding the The Proactive Path Provisioning Time can be obtained by finding the
time difference between the transmit and receive time of the traffic time difference between the transmit and receive time of the traffic
(Tsf1-Tdf1). (Tsf1-Tdf1).
B.4.6. Reactive Path Provisioning Rate A.4.6. Reactive Path Provisioning Rate
Procedure: Procedure:
Test Traffic Test Traffic Network Devices OpenFlow Test Traffic Test Traffic Network Devices OpenFlow
Generator TP1 Generator TP2 Controller Generator TP1 Generator TP2 Controller
| | | | | | | |
| | | | | | | |
| | | | | | | |
| |G-ARP (D1..Dn) | | | |G-ARP (D1..Dn) | |
| |--------------------| | | |--------------------| |
skipping to change at page 42, line 23 skipping to change at page 41, line 23
D1..Dn: Destination Endpoint 1, Destination Endpoint 2 .... D1..Dn: Destination Endpoint 1, Destination Endpoint 2 ....
Destination Endpoint n Destination Endpoint n
S1..Sn: Source Endpoint 1, Source Endpoint 2 .., Source S1..Sn: Source Endpoint 1, Source Endpoint 2 .., Source
Endpoint n Endpoint n
Discussion: Discussion:
The Reactive Path Provisioning Rate can be obtained by finding the The Reactive Path Provisioning Rate can be obtained by finding the
total number of frames received at TP2 after the trial duration. total number of frames received at TP2 after the trial duration.
B.4.7. Proactive Path Provisioning Rate A.4.7. Proactive Path Provisioning Rate
Procedure: Procedure:
Test Traffic Test Traffic Network Devices OpenFlow SDN Test Traffic Test Traffic Network Devices OpenFlow SDN
Generator TP1 Generator TP2 Controller Application Generator TP1 Generator TP2 Controller Application
| | | | | | | | | |
| |G-ARP (D1..Dn) | | | | |G-ARP (D1..Dn) | | |
| |-------------->| | | | |-------------->| | |
| | | | | | | | | |
| | |PACKET_IN(D1.Dn)| | | | |PACKET_IN(D1.Dn)| |
skipping to change at page 43, line 30 skipping to change at page 42, line 30
D1..Dn: Destination Endpoint 1, Destination Endpoint 2 .... D1..Dn: Destination Endpoint 1, Destination Endpoint 2 ....
Destination Endpoint n Destination Endpoint n
S1..Sn: Source Endpoint 1, Source Endpoint 2 .., Source S1..Sn: Source Endpoint 1, Source Endpoint 2 .., Source
Endpoint n Endpoint n
Discussion: Discussion:
The Proactive Path Provisioning Rate can be obtained by finding the The Proactive Path Provisioning Rate can be obtained by finding the
total number of frames received at TP2 after the trial duration total number of frames received at TP2 after the trial duration
B.4.8. Network Topology Change Detection Time A.4.8. Network Topology Change Detection Time
Procedure: Procedure:
Network Devices OpenFlow SDN Network Devices OpenFlow SDN
Controller Application Controller Application
| | | | | |
| | <Bring down a link in | | | <Bring down a link in |
| | switch S1>| | | switch S1>|
| | | | | |
T0 |PORT_STATUS with link down | | T0 |PORT_STATUS with link down | |
skipping to change at page 44, line 13 skipping to change at page 43, line 13
| | PACKET_OUT with LLDP T1>| | | PACKET_OUT with LLDP T1>|
Discussion: Discussion:
The Network Topology Change Detection Time can be obtained by The Network Topology Change Detection Time can be obtained by
finding the difference between the time the OpenFlow switch S1 sends finding the difference between the time the OpenFlow switch S1 sends
the PORT_STATUS message (T0) and the time that the OpenFlow the PORT_STATUS message (T0) and the time that the OpenFlow
controller sends the first topology re-discovery message (T1) to controller sends the first topology re-discovery message (T1) to
OpenFlow switches. OpenFlow switches.
B.5. Scalability A.5. Scalability
B.5.1. Control Sessions Capacity A.5.1. Control Sessions Capacity
Procedure: Procedure:
Network Devices OpenFlow Network Devices OpenFlow
Controller Controller
| | | |
| OFPT_HELLO Exchange for Switch 1 | | OFPT_HELLO Exchange for Switch 1 |
|<------------------------------------->| |<------------------------------------->|
| | | |
| OFPT_HELLO Exchange for Switch 2 | | OFPT_HELLO Exchange for Switch 2 |
skipping to change at page 44, line 38 skipping to change at page 43, line 38
| . | | . |
| . | | . |
| OFPT_HELLO Exchange for Switch n | | OFPT_HELLO Exchange for Switch n |
|X<----------------------------------->X| |X<----------------------------------->X|
| | | |
Discussion: Discussion:
The value of Switch n-1 will provide Control Sessions Capacity. The value of Switch n-1 will provide Control Sessions Capacity.
B.5.2. Network Discovery Size A.5.2. Network Discovery Size
Procedure: Procedure:
Network Devices OpenFlow SDN Network Devices OpenFlow SDN
Controller Application Controller Application
| | | | | |
| | <Deploy network with | | | <Deploy network with |
| |given no. of OF switches N>| | |given no. of OF switches N>|
| | | | | |
| OFPT_HELLO Exchange | | | OFPT_HELLO Exchange | |
skipping to change at page 45, line 47 skipping to change at page 44, line 47
n/w topo: Network Topology n/w topo: Network Topology
OF: OpenFlow OF: OpenFlow
Discussion: Discussion:
The value of N1 provides the Network Discovery Size value. The trial The value of N1 provides the Network Discovery Size value. The trial
duration can be set to the stipulated time within which the user duration can be set to the stipulated time within which the user
expects the controller to complete the discovery process. expects the controller to complete the discovery process.
B.5.3. Forwarding Table Capacity A.5.3. Forwarding Table Capacity
Procedure: Procedure:
Test Traffic Network Devices OpenFlow SDN Test Traffic Network Devices OpenFlow SDN
Generator TP1 Controller Application Generator TP1 Controller Application
| | | | | | | |
| | | | | | | |
|G-ARP (H1..Hn) | | | |G-ARP (H1..Hn) | | |
|----------------->| | | |----------------->| | |
| | | | | | | |
| |PACKET_IN(D1..Dn) | | | |PACKET_IN(D1..Dn) | |
skipping to change at page 47, line 5 skipping to change at page 46, line 5
FWD: Forwarding Table FWD: Forwarding Table
Discussion: Discussion:
Query the controller forwarding table entries for multiple times Query the controller forwarding table entries for multiple times
until the three consecutive queries return the same value. The last until the three consecutive queries return the same value. The last
value retrieved from the controller will provide the Forwarding value retrieved from the controller will provide the Forwarding
Table Capacity value. The query interval is user configurable. The 5 Table Capacity value. The query interval is user configurable. The 5
seconds shown in this example is for representational purpose. seconds shown in this example is for representational purpose.
B.6. Security A.6. Security
B.6.1. Exception Handling A.6.1. Exception Handling
Procedure: Procedure:
Test Traffic Test Traffic Network Devices OpenFlow SDN Test Traffic Test Traffic Network Devices OpenFlow SDN
Generator TP1 Generator TP2 Controller Application Generator TP1 Generator TP2 Controller Application
| | | | | | | | | |
| |G-ARP (D1..Dn) | | | | |G-ARP (D1..Dn) | | |
| |------------------>| | | | |------------------>| | |
| | | | | | | | | |
| | |PACKET_IN(D1..Dn)| | | | |PACKET_IN(D1..Dn)| |
skipping to change at page 48, line 37 skipping to change at page 47, line 37
should be 1% higher than the Path Programming Rate. Rn1 will provide should be 1% higher than the Path Programming Rate. Rn1 will provide
the Path Provisioning Rate of controller at 1% of incorrect frames the Path Provisioning Rate of controller at 1% of incorrect frames
handling and Rn2 will provide the Path Provisioning Rate of handling and Rn2 will provide the Path Provisioning Rate of
controller at 2% of incorrect frames handling. controller at 2% of incorrect frames handling.
The procedure defined above provides test steps to determine the The procedure defined above provides test steps to determine the
effect of handling error packets on Path Programming Rate. Same effect of handling error packets on Path Programming Rate. Same
procedure can be adopted to determine the effects on other procedure can be adopted to determine the effects on other
performance tests listed in this benchmarking tests. performance tests listed in this benchmarking tests.
B.6.2. Denial of Service Handling A.6.2. Denial of Service Handling
Procedure: Procedure:
Test Traffic Test Traffic Network Devic OpenFlow SDN Test Traffic Test Traffic Network Devic OpenFlow SDN
Generator TP1 Generator TP2 Controller Application Generator TP1 Generator TP2 Controller Application
| | | | | | | | | |
| |G-ARP (D1..Dn) | | | | |G-ARP (D1..Dn) | | |
| |------------------>| | | | |------------------>| | |
| | | | | | | | | |
| | |PACKET_IN(D1..Dn)| | | | |PACKET_IN(D1..Dn)| |
skipping to change at page 50, line 5 skipping to change at page 49, line 5
TCP SYN attack should be launched from one of the emulated/simulated TCP SYN attack should be launched from one of the emulated/simulated
OpenFlow Switch. Rn1 provides the Path Programming Rate of OpenFlow Switch. Rn1 provides the Path Programming Rate of
controller uponhandling denial of service attack. controller uponhandling denial of service attack.
The procedure defined above provides test steps to determine the The procedure defined above provides test steps to determine the
effect of handling denial of service on Path Programming Rate. Same effect of handling denial of service on Path Programming Rate. Same
procedure can be adopted to determine the effects on other procedure can be adopted to determine the effects on other
performance tests listed in this benchmarking tests. performance tests listed in this benchmarking tests.
B.7. Reliability A.7. Reliability
B.7.1. Controller Failover Time A.7.1. Controller Failover Time
Procedure: Procedure:
Test Traffic Test Traffic Network Device OpenFlow SDN Test Traffic Test Traffic Network Device OpenFlow SDN
Generator TP1 Generator TP2 Controller Application Generator TP1 Generator TP2 Controller Application
| | | | | | | | | |
| |G-ARP (D1) | | | | |G-ARP (D1) | | |
| |------------>| | | | |------------>| | |
| | | | | | | | | |
| | |PACKET_IN(D1) | | | | |PACKET_IN(D1) | |
skipping to change at page 51, line 32 skipping to change at page 50, line 32
Discussion: Discussion:
The time difference between the last valid frame received before the The time difference between the last valid frame received before the
traffic loss and the first frame received after the traffic loss traffic loss and the first frame received after the traffic loss
will provide the controller failover time. will provide the controller failover time.
If there is no frame loss during controller failover time, the If there is no frame loss during controller failover time, the
controller failover time can be deemed negligible. controller failover time can be deemed negligible.
B.7.2. Network Re-Provisioning Time A.7.2. Network Re-Provisioning Time
Procedure: Procedure:
Test Traffic Test Traffic Network Devices OpenFlow SDN Test Traffic Test Traffic Network Devices OpenFlow SDN
Generator TP1 Generator TP2 Controller Application Generator TP1 Generator TP2 Controller Application
| | | | | | | | | |
| |G-ARP (D1) | | | | |G-ARP (D1) | | |
| |-------------->| | | | |-------------->| | |
| | | | | | | | | |
| | |PACKET_IN(D1) | | | | |PACKET_IN(D1) | |
 End of changes. 86 change blocks. 
163 lines changed or deleted 142 lines changed or added

This html diff was produced by rfcdiff 1.46. The latest version is available from http://tools.ietf.org/tools/rfcdiff/