draft-ietf-bmwg-sdn-controller-benchmark-term-08.txt   draft-ietf-bmwg-sdn-controller-benchmark-term-09.txt 
skipping to change at page 1, line 14 skipping to change at page 1, line 14
Intended Status: Informational Veryx Technologies Intended Status: Informational Veryx Technologies
Expires: August 25, 2018 Mark Tassinari Expires: August 25, 2018 Mark Tassinari
Hewlett-Packard Hewlett-Packard
Vishwas Manral Vishwas Manral
Nano Sec Nano Sec
Sarah Banks Sarah Banks
VSS Monitoring VSS Monitoring
February 25, 2018 February 25, 2018
Terminology for Benchmarking SDN Controller Performance Terminology for Benchmarking SDN Controller Performance
draft-ietf-bmwg-sdn-controller-benchmark-term-08 draft-ietf-bmwg-sdn-controller-benchmark-term-09
Abstract Abstract
This document defines terminology for benchmarking an SDN This document defines terminology for benchmarking an SDN
controller's control plane performance. It extends the terminology controller's control plane performance. It extends the terminology
already defined in RFC 7426 for the purpose of benchmarking SDN already defined in RFC 7426 for the purpose of benchmarking SDN
controllers. The terms provided in this document help to benchmark controllers. The terms provided in this document help to benchmark
SDN controller's performance independent of the controller's SDN controller's performance independent of the controller's
supported protocols and/or network services. A mechanism for supported protocols and/or network services. A mechanism for
benchmarking the performance of SDN controllers is defined in the benchmarking the performance of SDN controllers is defined in the
skipping to change at page 2, line 27 skipping to change at page 2, line 27
Section 4.e of the Trust Legal Provisions and are provided without Section 4.e of the Trust Legal Provisions and are provided without
warranty as described in the Simplified BSD License. warranty as described in the Simplified BSD License.
Table of Contents Table of Contents
1. Introduction...................................................4 1. Introduction...................................................4
2. Term Definitions...............................................4 2. Term Definitions...............................................4
2.1. SDN Terms.................................................4 2.1. SDN Terms.................................................4
2.1.1. Flow.................................................4 2.1.1. Flow.................................................4
2.1.2. Northbound Interface.................................5 2.1.2. Northbound Interface.................................5
2.1.3. Controller Forwarding Table..........................5 2.1.3. Southbound Interface.................................5
2.1.4. Proactive Flow Provisioning Mode.....................5 2.1.4. Controller Forwarding Table..........................5
2.1.5. Reactive Flow Provisioning Mode......................6 2.1.5. Proactive Flow Provisioning Mode.....................6
2.1.6. Path.................................................6 2.1.6. Reactive Flow Provisioning Mode......................6
2.1.7. Standalone Mode......................................7 2.1.7. Path.................................................7
2.1.8. Cluster/Redundancy Mode..............................7 2.1.8. Standalone Mode......................................7
2.1.9. Asynchronous Message.................................7 2.1.9. Cluster/Redundancy Mode..............................7
2.1.10. Test Traffic Generator..............................8 2.1.10. Asynchronous Message................................8
2.1.11. Leaf-Spine Topology.................................8 2.1.11. Test Traffic Generator..............................8
2.1.12. Leaf-Spine Topology.................................9
2.2. Test Configuration/Setup Terms............................9 2.2. Test Configuration/Setup Terms............................9
2.2.1. Number of Network Devices............................9 2.2.1. Number of Network Devices............................9
2.2.2. Trial Repetition.....................................9 2.2.2. Trial Repetition.....................................9
2.2.3. Trial Duration.......................................9 2.2.3. Trial Duration......................................10
2.2.4. Number of Cluster nodes.............................10 2.2.4. Number of Cluster nodes.............................10
2.3. Benchmarking Terms.......................................10 2.3. Benchmarking Terms.......................................10
2.3.1. Performance.........................................10 2.3.1. Performance.........................................11
2.3.1.1. Network Topology Discovery Time................10 2.3.1.1. Network Topology Discovery Time................11
2.3.1.2. Asynchronous Message Processing Time...........11 2.3.1.2. Asynchronous Message Processing Time...........11
2.3.1.3. Asynchronous Message Processing Rate...........11 2.3.1.3. Asynchronous Message Processing Rate...........12
2.3.1.4. Reactive Path Provisioning Time................12 2.3.1.4. Reactive Path Provisioning Time................13
2.3.1.5. Proactive Path Provisioning Time...............13 2.3.1.5. Proactive Path Provisioning Time...............13
2.3.1.6. Reactive Path Provisioning Rate................13 2.3.1.6. Reactive Path Provisioning Rate................14
2.3.1.7. Proactive Path Provisioning Rate...............14 2.3.1.7. Proactive Path Provisioning Rate...............14
2.3.1.8. Network Topology Change Detection Time.........15 2.3.1.8. Network Topology Change Detection Time.........15
2.3.2. Scalability.........................................15 2.3.2. Scalability.........................................16
2.3.2.1. Control Sessions Capacity......................15 2.3.2.1. Control Sessions Capacity......................16
2.3.2.2. Network Discovery Size.........................16 2.3.2.2. Network Discovery Size.........................16
2.3.2.3. Forwarding Table Capacity......................16 2.3.2.3. Forwarding Table Capacity......................17
2.3.3. Security............................................17 2.3.3. Security............................................17
2.3.3.1. Exception Handling.............................17 2.3.3.1. Exception Handling.............................17
2.3.3.2. Denial of Service Handling.....................17 2.3.3.2. Denial of Service Handling.....................18
2.3.4. Reliability.........................................18 2.3.4. Reliability.........................................18
2.3.4.1. Controller Failover Time.......................18 2.3.4.1. Controller Failover Time.......................18
2.3.4.2. Network Re-Provisioning Time...................18 2.3.4.2. Network Re-Provisioning Time...................19
3. Test Setup....................................................19 3. Test Setup....................................................19
3.1. Test setup - Controller working in Standalone Mode.......20 3.1. Test setup - Controller working in Standalone Mode.......20
3.2. Test setup - Controller working in Cluster Mode..........21 3.2. Test setup - Controller working in Cluster Mode..........21
4. Test Coverage.................................................22 4. Test Coverage.................................................22
5. References....................................................23 5. References....................................................23
5.1. Normative References.....................................23 5.1. Normative References.....................................23
5.2. Informative References...................................23 5.2. Informative References...................................23
6. IANA Considerations...........................................23 6. IANA Considerations...........................................23
7. Security Considerations.......................................23 7. Security Considerations.......................................23
8. Acknowledgements..............................................24 8. Acknowledgements..............................................24
skipping to change at page 5, line 11 skipping to change at page 5, line 11
Measurement Units: Measurement Units:
N/A N/A
See Also: See Also:
None None
2.1.2. Northbound Interface 2.1.2. Northbound Interface
Definition: Definition:
The definition of northbound interface is same Service Interface The definition of northbound interface is same the Service Interface
defined in [RFC7426]. defined in [RFC7426].
Discussion: Discussion:
The northbound interface allows SDN applications and orchestration The northbound interface allows SDN applications and orchestration
systems to program and retrieve the network information through the systems to program and retrieve the network information through the
SDN controller. SDN controller.
Measurement Units: Measurement Units:
N/A N/A
See Also: See Also:
None None
2.1.3. Controller Forwarding Table 2.1.3. Southbound Interface
Definition:
The southbound interface is the application programming interface
provided by the SDN controller to interact with the SDN nodes.
Discussion:
Southbound interface enables controller to interact with the SDN
nodes in the network for dynamically defining the traffic forwarding
behaviour.
Measurement Units:
N/A
See Also:
None
2.1.4. Controller Forwarding Table
Definition: Definition:
A controller forwarding table contains flow entries learned in one A controller forwarding table contains flow entries learned in one
of two ways: first, entries could be learned from traffic received of two ways: first, entries could be learned from traffic received
through the data plane, or second, these entries could be statically through the data plane, or second, these entries could be statically
provisioned on the controller and distributed to devices via the provisioned on the controller and distributed to devices via the
southbound interface. southbound interface.
Discussion: Discussion:
The controller forwarding table has an aging mechanism which will be The controller forwarding table has an aging mechanism which will be
applied only for dynamically learned entries. applied only for dynamically learned entries.
Measurement Units: Measurement Units:
N/A N/A
See Also: See Also:
None None
2.1.4. Proactive Flow Provisioning Mode 2.1.5. Proactive Flow Provisioning Mode
Definition: Definition:
Controller programming flows in Network Devices based on the flow Controller programming flows in Network Devices based on the flow
entries provisioned through controller's northbound interface. entries provisioned through controller's northbound interface.
Discussion: Discussion:
Network orchestration systems and SDN applications can define the
Orchestration systems and SDN applications can define the network network forwarding behaviour by programming the controller using
forwarding behaviour by programming the controller using proactive proactive flow provisioning. The controller can then program the
flow provisioning. The controller can then program the Network Network Devices with the pre-provisioned entries.
Devices with the pre-provisioned entries.
Measurement Units: Measurement Units:
N/A N/A
See Also: See Also:
None None
2.1.5. Reactive Flow Provisioning Mode 2.1.6. Reactive Flow Provisioning Mode
Definition: Definition:
Controller programming flows in Network Devices based on the traffic Controller programming flows in Network Devices based on the traffic
received from Network Devices through controller's southbound received from Network Devices through controller's southbound
interface interface
Discussion: Discussion:
The SDN controller dynamically decides the forwarding behaviour The SDN controller dynamically decides the forwarding behaviour
based on the incoming traffic from the Network Devices. The based on the incoming traffic from the Network Devices. The
controller then programs the Network Devices using Reactive Flow controller then programs the Network Devices using Reactive Flow
Provisioning. Provisioning.
Measurement Units: Measurement Units:
N/A N/A
See Also: See Also:
None None
2.1.6. Path 2.1.7. Path
Definition: Definition:
Refer to Section 5 in [RFC2330] Refer to Section 5 in [RFC2330]
Discussion: Discussion:
None None
Measurement Units: Measurement Units:
N/A N/A
See Also: See Also:
None None
2.1.7. Standalone Mode 2.1.8. Standalone Mode
Definition: Definition:
Single controller handling all control plane functionalities without Single controller handling all control plane functionalities without
redundancy, or the ability to provide high availability and/or redundancy, or the ability to provide high availability and/or
automatic failover. automatic failover.
Discussion: Discussion:
In standalone mode, one controller manages one or more network In standalone mode, one controller manages one or more network
domains. domains.
Measurement Units: Measurement Units:
N/A N/A
See Also: See Also:
None None
2.1.8. Cluster/Redundancy Mode 2.1.9. Cluster/Redundancy Mode
Definition: Definition:
A group of 2 or more controllers handling all control plane A group of 2 or more controllers handling all control plane
functionalities. functionalities.
Discussion: Discussion:
In cluster mode, multiple controllers are teamed together for the In cluster mode, multiple controllers are teamed together for the
purpose of load sharing and/or high availability. The controllers in purpose of load sharing and/or high availability. The controllers in
the group may work in active/standby (master/slave) or active/active the group may work in active/standby (master/slave) or active/active
(equal) mode depending on the intended purpose. (equal) mode depending on the intended purpose.
skipping to change at page 7, line 35 skipping to change at page 8, line 4
A group of 2 or more controllers handling all control plane A group of 2 or more controllers handling all control plane
functionalities. functionalities.
Discussion: Discussion:
In cluster mode, multiple controllers are teamed together for the In cluster mode, multiple controllers are teamed together for the
purpose of load sharing and/or high availability. The controllers in purpose of load sharing and/or high availability. The controllers in
the group may work in active/standby (master/slave) or active/active the group may work in active/standby (master/slave) or active/active
(equal) mode depending on the intended purpose. (equal) mode depending on the intended purpose.
Measurement Units: Measurement Units:
N/A N/A
See Also: See Also:
None None
2.1.9. Asynchronous Message 2.1.10. Asynchronous Message
Definition: Definition:
Any message from the Network Device that is generated for network Any message from the Network Device that is generated for network
events. events.
Discussion: Discussion:
Control messages like flow setup request and response message is Control messages like flow setup request and response message is
classified as asynchronous message. The controller has to return a classified as asynchronous message. The controller has to return a
response message. Note that the Network Device will not be in response message. Note that the Network Device will not be in
blocking mode and continues to send/receive other control messages. blocking mode and continues to send/receive other control messages.
Measurement Units: Measurement Units:
N/A N/A
See Also: See Also:
None None
2.1.10. Test Traffic Generator 2.1.11. Test Traffic Generator
Definition: Definition:
Test Traffic Generator is an entity that generates/receives network Test Traffic Generator is an entity that generates/receives network
traffic. traffic.
Discussion: Discussion:
Test Traffic Generator typically connects with Network Devices to Test Traffic Generator typically connects with Network Devices to
send/receive real-time network traffic. send/receive real-time network traffic.
Measurement Units: Measurement Units:
N/A N/A
See Also: See Also:
None None
2.1.11. Leaf-Spine Topology 2.1.12. Leaf-Spine Topology
Definition: Definition:
Leaf-Spine is a two layered network topology, where a series of Leaf-Spine is a two layered network topology, where a series of
leaf switches, form the access layer, are fully meshed to a series leaf switches, form the access layer, are fully meshed to a series
of spine switches that form the backbone layer. of spine switches that form the backbone layer.
Discussion: Discussion:
In Leaf-Spine Topology, every leaf switch is connected to each of In Leaf-Spine Topology, every leaf switch is connected to each of
the spine switches in the topology. the spine switches in the topology.
skipping to change at page 13, line 5 skipping to change at page 13, line 24
See Also: See Also:
None None
2.3.1.4. Reactive Path Provisioning Time 2.3.1.4. Reactive Path Provisioning Time
Definition: Definition:
The time taken by the controller to setup a path reactively between The time taken by the controller to setup a path reactively between
source and destination node, defined as the interval starting with source and destination node, defined as the interval starting with
the first flow provisioning request message received by the the first flow provisioning request message received by the
controller(s), ending with the last flow provisioning response controller(s), ending with the last flow provisioning response
message sent from the controller(s) at it Southbound interface. message sent from the controller(s) at its Southbound interface.
Discussion: Discussion:
As SDN supports agile provisioning, it is important to measure how As SDN supports agile provisioning, it is important to measure how
fast that the controller provisions an end-to-end flow in the fast that the controller provisions an end-to-end flow in the
dataplane. The benchmark is obtained by sending traffic from a dataplane. The benchmark is obtained by sending traffic from a
source endpoint to the destination endpoint, finding the time source endpoint to the destination endpoint, finding the time
difference between the first and the last flow provisioning message difference between the first and the last flow provisioning message
exchanged between the controller and the Network Devices for the exchanged between the controller and the Network Devices for the
traffic path. traffic path.
skipping to change at page 13, line 29 skipping to change at page 13, line 48
See Also: See Also:
None None
2.3.1.5. Proactive Path Provisioning Time 2.3.1.5. Proactive Path Provisioning Time
Definition: Definition:
The time taken by the controller to proactively setup a path between The time taken by the controller to proactively setup a path between
source and destination node, defined as the interval starting with source and destination node, defined as the interval starting with
the first proactive flow provisioned in the controller(s) at its the first proactive flow provisioned in the controller(s) at its
Northbound interface, ending with the last flow provisioning command Northbound interface, ending with the last flow provisioning command
message sent from the controller(s) at it Southbound interface. message sent from the controller(s) at its Southbound interface.
Discussion: Discussion:
For SDN to support pre-provisioning of traffic path from For SDN to support pre-provisioning of traffic path from
application, it is important to measure how fast that the controller application, it is important to measure how fast that the controller
provisions an end-to-end flow in the dataplane. The benchmark is provisions an end-to-end flow in the dataplane. The benchmark is
obtained by provisioning a flow on controller's northbound interface obtained by provisioning a flow on controller's northbound interface
for the traffic to reach from a source to a destination endpoint, for the traffic to reach from a source to a destination endpoint,
finding the time difference between the first and the last flow finding the time difference between the first and the last flow
provisioning message exchanged between the controller and the provisioning message exchanged between the controller and the
Network Devices for the traffic path. Network Devices for the traffic path.
Measurement Units: Measurement Units:
skipping to change at page 16, line 11 skipping to change at page 16, line 33
Measurement Units: Measurement Units:
N/A N/A
See Also: See Also:
None None
2.3.2.2. Network Discovery Size 2.3.2.2. Network Discovery Size
Definition: Definition:
Measure the network size (number of nodes, links and hosts) that a Measure the network size (number of nodes and links) that a
controller can discover, defined as the size of a network that the controller can discover, defined as the size of a network that the
controller(s) can discover, starting from a network topology given controller(s) can discover, starting from a network topology given
by the user for discovery, ending with the topology that the by the user for discovery, ending with the topology that the
controller(s) could successfully discover. controller(s) could successfully discover.
Discussion: Discussion:
For optimal network planning, it is key to measure the maximum For optimal network planning, it is key to measure the maximum
network size that the controller can discover. This benchmark is network size that the controller can discover. This benchmark is
obtained by presenting an initial set of Network Devices for obtained by presenting an initial set of Network Devices for
discovery to the controller. Based on the initial discovery, the discovery to the controller. Based on the initial discovery, the
skipping to change at page 22, line 8 skipping to change at page 22, line 8
| +---------------+ +---------------+ | | +---------------+ +---------------+ |
| | | |
| Forwarding Plane Test Emulator | | Forwarding Plane Test Emulator |
+-----------------------------------------------------------+ +-----------------------------------------------------------+
Figure 2 Figure 2
4. Test Coverage 4. Test Coverage
+ -----------------------------------------------------------------+ + -----------------------------------------------------------------+
| | Speed | Scalability | Reliability | | Lifecycle | Speed | Scalability | Reliability |
+ -----------+-------------------+---------------+-----------------+ + -----------+-------------------+---------------+-----------------+
| | 1. Network Topolo-|1. Network | | | | 1. Network Topolo-|1. Network | |
| | -gy Discovery | Discovery | | | | -gy Discovery | Discovery | |
| | | Size | | | | Time | Size | |
| | | | |
| | 2. Reactive Path | | | | | 2. Reactive Path | | |
| | Provisioning | | | | | Provisioning | | |
| | Time | | | | | Time | | |
| | | | | | | | | |
| | 3. Proactive Path | | | | | 3. Proactive Path | | |
| | Provisioning | | | | | Provisioning | | |
| Setup | Time | | | | Setup | Time | | |
| | | | | | | | | |
| | 4. Reactive Path | | | | | 4. Reactive Path | | |
| | Provisioning | | | | | Provisioning | | |
skipping to change at page 22, line 37 skipping to change at page 22, line 38
| | Rate | | | | | Rate | | |
| | | | | | | | | |
+------------+-------------------+---------------+-----------------+ +------------+-------------------+---------------+-----------------+
| | 1. Maximum |1. Control |1. Network | | | 1. Maximum |1. Control |1. Network |
| | Asynchronous | Sessions | Topology | | | Asynchronous | Sessions | Topology |
| | Message Proces-| Capacity | Change | | | Message Proces-| Capacity | Change |
| | -sing Rate | | Detection Time| | | -sing Rate | | Detection Time|
| | |2. Forwarding | | | | |2. Forwarding | |
| | 2. Loss-Free | Table |2. Exception | | | 2. Loss-Free | Table |2. Exception |
| | Asynchronous | Capacity | Handling | | | Asynchronous | Capacity | Handling |
| | Message Proces-| | Detection Time| | | Message Proces-| | |
| Operational| -sing Rate | | | | Operational| -sing Rate | |3. Denial of |
| | | |3. Denial of | | | | | Service |
| | 3. Asynchronous | | Service | | | 3. Asynchronous | | Handling |
| | Message Proces-| | Handling | | | Message Proces-| | |
| | -sing Time | | | | | -sing Time | |4. Network Re- |
| | | |4. Network Re- |
| | | | Provisioning | | | | | Provisioning |
| | | | Time | | | | | Time |
| | | | | | | | | |
+------------+-------------------+---------------+-----------------+ +------------+-------------------+---------------+-----------------+
| | | | | | | | | |
| Tear Down | | |1. Controller | | Tear Down | | |1. Controller |
| | | | Failover Time | | | | | Failover Time |
+------------+-------------------+---------------+-----------------+ +------------+-------------------+---------------+-----------------+
5. References 5. References
 End of changes. 29 change blocks. 
48 lines changed or deleted 67 lines changed or added

This html diff was produced by rfcdiff 1.46. The latest version is available from http://tools.ietf.org/tools/rfcdiff/