draft-ietf-bmwg-sdn-controller-benchmark-term-00.txt   draft-ietf-bmwg-sdn-controller-benchmark-term-01.txt 
Internet-Draft Bhuvaneswaran Vengainathan Internet-Draft Bhuvaneswaran Vengainathan
Network Working Group Anton Basil Network Working Group Anton Basil
Intended Status: Informational Veryx Technologies Intended Status: Informational Veryx Technologies
Expires: April 18, 2016 Mark Tassinari Expires: September 19, 2016 Mark Tassinari
Hewlett-Packard Hewlett-Packard
Vishwas Manral Vishwas Manral
Ionos Corp Nano Sec
Sarah Banks Sarah Banks
VSS Monitoring VSS Monitoring
October 19, 2015 March 21, 2016
Terminology for Benchmarking SDN Controller Performance Terminology for Benchmarking SDN Controller Performance
draft-ietf-bmwg-sdn-controller-benchmark-term-00 draft-ietf-bmwg-sdn-controller-benchmark-term-01
Abstract Abstract
This document defines terminology for benchmarking an SDN This document defines terminology for benchmarking an SDN
Controller's performance. The terms provided in this document help controller's control plane performance. It extends the terminology
to benchmark SDN controller's performance independent of the already defined in RFC 7426 for the purpose of benchmarking SDN
controller's supported protocols and/or network services. A controllers. The terms provided in this document help to benchmark
mechanism for benchmarking the performance of SDN controllers is SDN controller's performance independent of the controller's
defined in the companion methodology document. These two documents supported protocols and/or network services. A mechanism for
provide a standard mechanism to measure and evaluate the performance benchmarking the performance of SDN controllers is defined in the
of various controller implementations. companion methodology document. These two documents provide a
standard mechanism to measure and evaluate the performance of
various controller implementations.
Status of this Memo Status of this Memo
This Internet-Draft is submitted in full conformance with the This Internet-Draft is submitted in full conformance with the
provisions of BCP 78 and BCP 79. provisions of BCP 78 and BCP 79.
Internet-Drafts are working documents of the Internet Engineering Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF). Note that other groups may also distribute Task Force (IETF). Note that other groups may also distribute
working documents as Internet-Drafts. The list of current Internet- working documents as Internet-Drafts. The list of current Internet-
Drafts is at http://datatracker.ietf.org/drafts/current. Drafts is at http://datatracker.ietf.org/drafts/current.
Internet-Drafts are draft documents valid for a maximum of six Internet-Drafts are draft documents valid for a maximum of six
months and may be updated, replaced, or obsoleted by other documents months and may be updated, replaced, or obsoleted by other documents
at any time. It is inappropriate to use Internet-Drafts as reference at any time. It is inappropriate to use Internet-Drafts as reference
material or to cite them other than as "work in progress. material or to cite them other than as "work in progress.
This Internet-Draft will expire on April 18, 2016. This Internet-Draft will expire on September 19, 2016.
Copyright Notice Copyright Notice
Copyright (c) 2015 IETF Trust and the persons identified as the Copyright (c) 2016 IETF Trust and the persons identified as the
document authors. All rights reserved. document authors. All rights reserved.
This document is subject to BCP 78 and the IETF Trust's Legal This document is subject to BCP 78 and the IETF Trust's Legal
Provisions Relating to IETF Documents Provisions Relating to IETF Documents
(http://trustee.ietf.org/license-info) in effect on the date of (http://trustee.ietf.org/license-info) in effect on the date of
publication of this document. Please review these documents publication of this document. Please review these documents
carefully, as they describe your rights and restrictions with carefully, as they describe your rights and restrictions with
respect to this document. Code Components extracted from this respect to this document. Code Components extracted from this
document must include Simplified BSD License text as described in document must include Simplified BSD License text as described in
Section 4.e of the Trust Legal Provisions and are provided without Section 4.e of the Trust Legal Provisions and are provided without
warranty as described in the Simplified BSD License. warranty as described in the Simplified BSD License.
Table of Contents Table of Contents
1. Introduction ................................................ 3 1. Introduction...................................................3
2. Term Definitions ............................................ 4 2. Term Definitions...............................................4
2.1. SDN Terms .............................................. 4 2.1. SDN Terms.................................................4
2.1.1. SDN Node .......................................... 4 2.1.1. Flow.................................................4
2.1.2. SDN Application ................................... 4 2.1.2. Northbound Interface.................................4
2.1.3. Flow .............................................. 4 2.1.3. Controller Forwarding Table..........................4
2.1.4. Northbound Interface .............................. 5 2.1.4. Proactive Flow Provisioning Mode.....................5
2.1.5. Southbound Interface .............................. 5 2.1.5. Reactive Flow Provisioning Mode......................5
2.1.6. Controller Forwarding Table ....................... 5 2.1.6. Path.................................................6
2.1.7. Proactive Flow Provisioning Mode .................. 6 2.1.7. Standalone Mode......................................6
2.1.8. Reactive Flow Provisioning Mode ................... 6 2.1.8. Cluster/Redundancy Mode..............................6
2.1.9. Path .............................................. 7 2.1.9. Asynchronous Message.................................7
2.1.10. Standalone Mode .................................. 7 2.1.10. Test Traffic Generator..............................7
2.1.11. Cluster/Redundancy Mode .......................... 7 2.2. Test Configuration/Setup Terms............................8
2.1.12. Asynchronous Message ............................. 8 2.2.1. Number of Network Devices............................8
2.1.13. Test Traffic Generator ........................... 8 2.2.2. Test Iterations......................................8
2.2. Test Configuration/Setup Terms ......................... 9 2.2.3. Test Duration........................................8
2.2.1. Number of SDN Nodes ............................... 9 2.2.4. Number of Cluster nodes..............................9
2.2.2. Test Iterations ................................... 9 2.3. Benchmarking Terms........................................9
2.2.3. Test Duration ..................................... 9 2.3.1. Performance..........................................9
2.2.4. Number of Cluster nodes ........................... 10 2.3.1.1. Network Topology Discovery Time.................9
2.3. Benchmarking Terms ..................................... 10 2.3.1.2. Asynchronous Message Processing Time...........10
2.3.1. Performance ....................................... 10 2.3.1.3. Asynchronous Message Processing Rate...........10
2.3.1.1. Network Topology Discovery Time .............. 10 2.3.1.4. Reactive Path Provisioning Time................11
2.3.1.2. Asynchronous Message Processing Time.......... 11 2.3.1.5. Proactive Path Provisioning Time...............11
2.3.1.3. Asynchronous Message Processing Rate.......... 11 2.3.1.6. Reactive Path Provisioning Rate................12
2.3.1.4. Reactive Path Provisioning Time .............. 12 2.3.1.7. Proactive Path Provisioning Rate...............12
2.3.1.5. Proactive Path Provisioning Time ............. 12 2.3.1.8. Network Topology Change Detection Time.........12
2.3.1.6. Reactive Path Provisioning Rate .............. 13 2.3.2. Scalability.........................................13
2.3.1.7. Proactive Path Provisioning Rate ............. 13 2.3.2.1. Control Sessions Capacity......................13
2.3.1.8. Network Topology Change Detection Time ....... 13 2.3.2.2. Network Discovery Size.........................13
2.3.2. Scalability ....................................... 14 2.3.2.3. Forwarding Table Capacity......................14
2.3.2.1. Control Sessions Capacity .................... 14 2.3.3. Security............................................14
2.3.2.2. Network Discovery Size ....................... 14 2.3.3.1. Exception Handling.............................14
2.3.2.3. Forwarding Table Capacity .................... 15 2.3.3.2. Denial of Service Handling.....................15
2.3.3. Security ......................................... 15 2.3.4. Reliability.........................................15
2.3.3.1. Exception Handling ........................... 15 2.3.4.1. Controller Failover Time.......................15
2.3.3.2. Denial of Service Handling ................... 16 2.3.4.2. Network Re-Provisioning Time...................16
2.3.4. Reliability ....................................... 16 3. Test Setup....................................................16
2.3.4.1. Controller Failover Time ..................... 16 3.1. Test setup - Controller working in Standalone Mode.......17
2.3.4.2. Network Re-Provisioning Time ................. 17 3.2. Test setup - Controller working in Cluster Mode..........18
3. Test Coverage .............................................. 17 4. Test Coverage.................................................19
4. References ................................................. 18 5. References....................................................20
4.1. Normative References ................................... 18 5.1. Normative References.....................................20
4.2. Informative References ................................. 19 5.2. Informative References...................................20
5. IANA Considerations ........................................ 19 6. IANA Considerations...........................................20
6. Security Considerations ..................................... 19 7. Security Considerations.......................................20
7. Acknowledgements ........................................... 19 8. Acknowledgements..............................................21
8. Authors' Addresses ......................................... 19 9. Authors' Addresses............................................21
1. Introduction 1. Introduction
Software Defined Networking (SDN) is a networking architecture in Software Defined Networking (SDN) is a networking architecture in
which network control is decoupled from the underlying forwarding which network control is decoupled from the underlying forwarding
function and is placed in a centralized location called the SDN function and is placed in a centralized location called the SDN
controller. The SDN controller abstracts the underlying network and controller. The SDN controller abstracts the underlying network and
offers a global view of the overall network to applications and offers a global view of the overall network to applications and
business logic. Thus, an SDN controller provides the flexibility to business logic. Thus, an SDN controller provides the flexibility to
program, control, and manage network behaviour dynamically through program, control, and manage network behaviour dynamically through
skipping to change at page 4, line 9 skipping to change at page 4, line 9
Conventions used in this document Conventions used in this document
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
"SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
document are to be interpreted as described in RFC 2119. document are to be interpreted as described in RFC 2119.
2. Term Definitions 2. Term Definitions
2.1. SDN Terms 2.1. SDN Terms
2.1.1. SDN Node The terms defined in this section are extensions to the terms
defined in RFC 7426 ''Software-Defined Networking (SDN): Layers and
Definition: Architecture Terminology''. This RFC should be referred before
An SDN node is a simulated/emulated entity that forwards data in a attempting to make use of this document.
software defined environment.
Discussion:
An SDN node can be an emulated/simulated switch, router, gateway, or
any network service appliance that supports standardized or
proprietary programmable interface.
Measurement Units:
N/A
See Also:
None
2.1.2. SDN Application
Definition:
Any business logic that alter the network behaviour dynamically
through controller's northbound interface.
Discussion:
SDN application can be any business application, cloud orchestration
system, network services orchestration etc.,
Measurement Units:
N/A
See Also:
None
2.1.3. Flow 2.1.1. Flow
Definition: Definition:
A flow is a uni-directional sequence of packets having common The definition of Flow is same as microflows defined in RFC 4689
properties derived from the data contained in the packet. Section 3.1.5.
Discussion: Discussion:
A flow can be set of packets having same source address, destination A flow can be set of packets having same source address, destination
address, source port and destination port, or any of these address, source port and destination port, or any of these
combinations. combinations.
Measurement Units: Measurement Units:
N/A N/A
See Also: See Also:
None None
2.1.4. Northbound Interface 2.1.2. Northbound Interface
Definition: Definition:
The northbound interface is the application programming interface The definition of northbound interface is same Service Interface
provided by the SDN controller for the SDN services and applications defined in RFC 7426.
to interact with the SDN controller.
Discussion: Discussion:
The northbound interface allows SDN applications and orchestration The northbound interface allows SDN applications and orchestration
systems to program and retrieve the network information through the systems to program and retrieve the network information through the
SDN controller. SDN controller.
Measurement Units: Measurement Units:
N/A N/A
See Also: See Also:
None None
2.1.5. Southbound Interface 2.1.3. Controller Forwarding Table
Definition: Definition:
The southbound interface is the application programming interface
provided by the SDN controller to interact with the SDN nodes.
Discussion:
Southbound interface enables controller to interact with the SDN
nodes in the infrastructure for dynamically defining the traffic
forwarding behaviour.
Measurement Units:
N/A
See Also:
None
2.1.6. Controller Forwarding Table
Definition:
A controller forwarding table contains flow entries learned in one A controller forwarding table contains flow entries learned in one
of two ways: first, entries could be learned from traffic received of two ways: first, entries could be learned from traffic received
through the data plane, or, second, these entries could be through the data plane, or, second, these entries could be
statically provisioned on the controller, and distributed to devices statically provisioned on the controller, and distributed to devices
via the southbound interface. via the southbound interface.
Discussion: Discussion:
The controller forwarding table has an aging mechanism which will be The controller forwarding table has an aging mechanism which will be
applied only for dynamically learnt entries. applied only for dynamically learnt entries.
Measurement Units: Measurement Units:
N/A N/A
See Also: See Also:
None None
2.1.7. Proactive Flow Provisioning Mode 2.1.4. Proactive Flow Provisioning Mode
Definition: Definition:
Controller programming flows in SDN nodes based on the flow entries Controller programming flows in Network Devices based on the flow
provisioned through controller's northbound interface. entries provisioned through controller's northbound interface.
Discussion: Discussion:
Orchestration systems and SDN applications can define the network Orchestration systems and SDN applications can define the network
forwarding behaviour by programming the controller using proactive forwarding behaviour by programming the controller using proactive
flow provisioning. The controller can then program the SDN nodes flow provisioning. The controller can then program the Network
with the pre-provisioned entries. Devices with the pre-provisioned entries.
Measurement Units: Measurement Units:
N/A N/A
See Also: See Also:
None None
2.1.8. Reactive Flow Provisioning Mode 2.1.5. Reactive Flow Provisioning Mode
Definition: Definition:
Controller programming flows in SDN nodes based on the traffic Controller programming flows in Network Devices based on the traffic
received from SDN nodes through controller's southbound interface received from Network Devices through controller's southbound
interface
Discussion: Discussion:
The SDN controller dynamically decides the forwarding behaviour The SDN controller dynamically decides the forwarding behaviour
based on the incoming traffic from the SDN nodes. The controller based on the incoming traffic from the Network Devices. The
then programs the SDN nodes using Reactive Flow Provisioning. controller then programs the Network Devices using Reactive Flow
Provisioning.
Measurement Units: Measurement Units:
N/A N/A
See Also: See Also:
None None
2.1.9. Path 2.1.6. Path
Definition: Definition:
A path is a sequence of SDN nodes and links traversed by a flow. Refer to Section 5 in RFC 2330.
Discussion: Discussion:
As defined in RFC 2330, path is a sequence of the form < h0, l1, h1, None
..., ln, hn >, where n >=0, h0 and hn is a Host, h1...hn-1 is an SDN
Node, each li is a link between hi-1 and hi. A pair <li, hi> is
termed a 'hop'. Note that path is a unidirectional concept.
Measurement Units: Measurement Units:
N/A N/A
See Also: See Also:
None None
2.1.10. Standalone Mode 2.1.7. Standalone Mode
Definition: Definition:
Single controller handling all control plane functionalities without Single controller handling all control plane functionalities without
redundancy, or the ability to provide high availability and/or redundancy, or the ability to provide high availability and/or
automatic failover. automatic failover.
Discussion: Discussion:
In standalone mode, one controller manages one or more network In standalone mode, one controller manages one or more network
domains. domains.
Measurement Units: Measurement Units:
N/A N/A
See Also: See Also:
None None
2.1.11. Cluster/Redundancy Mode 2.1.8. Cluster/Redundancy Mode
Definition: Definition:
A group of 2 or more controllers handling all control plane A group of 2 or more controllers handling all control plane
functionalities. functionalities.
Discussion: Discussion:
In cluster mode, multiple controllers are teamed together for the In cluster mode, multiple controllers are teamed together for the
purpose of load sharing and/or high availability. The controllers in purpose of load sharing and/or high availability. The controllers in
the group may work in active/standby (master/slave) or active/active the group may work in active/standby (master/slave) or active/active
(equal) mode depending on the intended purpose. (equal) mode depending on the intended purpose.
Measurement Units: Measurement Units:
N/A N/A
See Also: See Also:
None None
skipping to change at page 8, line 13 skipping to change at page 7, line 16
purpose of load sharing and/or high availability. The controllers in purpose of load sharing and/or high availability. The controllers in
the group may work in active/standby (master/slave) or active/active the group may work in active/standby (master/slave) or active/active
(equal) mode depending on the intended purpose. (equal) mode depending on the intended purpose.
Measurement Units: Measurement Units:
N/A N/A
See Also: See Also:
None None
2.1.12. Asynchronous Message 2.1.9. Asynchronous Message
Definition: Definition:
Any message from the SDN node that is generated for network events. Any message from the Network Device that is generated for network
events.
Discussion: Discussion:
Control messages like flow setup request and response message is Control messages like flow setup request and response message is
classified as asynchronous message. The controller has to return a classified as asynchronous message. The controller has to return a
response message. Note that the SDN node will not be in blocking response message. Note that the Network Device will not be in
mode and continues to send/receive other control messages blocking mode and continues to send/receive other control messages
Measurement Units: Measurement Units:
N/A N/A
See Also: See Also:
None None
2.1.13. Test Traffic Generator 2.1.10. Test Traffic Generator
Definition: Definition:
Test Traffic Generator is an entity that generates/receives network Test Traffic Generator is an entity that generates/receives network
traffic. traffic.
Discussion: Discussion:
Test Traffic Generator can be an entity that interfaces with SDN Test Traffic Generator can be an entity that interfaces with Network
Nodes to send/receive real-time network traffic. Devices to send/receive real-time network traffic.
Measurement Units: Measurement Units:
N/A N/A
See Also: See Also:
None None
2.2. Test Configuration/Setup Terms 2.2. Test Configuration/Setup Terms
2.2.1. Number of SDN Nodes 2.2.1. Number of Network Devices
Definition: Definition:
The number of SDN nodes present in the defined test topology. The number of Network Devices present in the defined test topology.
Discussion: Discussion:
The SDN nodes defined in the test topology can be deployed using The Network Devices defined in the test topology can be deployed
real hardware or emulated in hardware platforms. using real hardware or emulated in hardware platforms.
Measurement Units: Measurement Units:
N/A N/A
See Also: See Also:
None None
2.2.2. Test Iterations 2.2.2. Test Iterations
Definition: Definition:
skipping to change at page 11, line 21 skipping to change at page 10, line 21
2.3.1.2. Asynchronous Message Processing Time 2.3.1.2. Asynchronous Message Processing Time
Definition: Definition:
To measure the time taken by the controller to process an To measure the time taken by the controller to process an
asynchronous message. asynchronous message.
Discussion: Discussion:
For SDN to support dynamic network provisioning, it is important to For SDN to support dynamic network provisioning, it is important to
measure how quickly the controller responds to an event triggered measure how quickly the controller responds to an event triggered
from the network. The event could be any notification messages from the network. The event could be any notification messages
generated by an SDN node upon arrival of a new flow, link down etc. generated by an Network Device upon arrival of a new flow, link down
This benchmark is obtained by sending asynchronous messages from etc. This benchmark is obtained by sending asynchronous messages
every connected SDN nodes one at a time for the defined test from every connected Network Devices one at a time for the defined
duration. This test assumes that the controller will respond to the test duration. This test assumes that the controller will respond to
received asynchronous message. the received asynchronous message.
Measurement Units: Measurement Units:
milliseconds milliseconds
See Also: See Also:
None None
2.3.1.3. Asynchronous Message Processing Rate 2.3.1.3. Asynchronous Message Processing Rate
Definition: Definition:
To measure the maximum number of asynchronous messages that a To measure the maximum number of asynchronous messages that a
controller can process within the test duration. controller can process within the test duration.
Discussion: Discussion:
As SDN assures flexible network and agile provisioning, it is As SDN assures flexible network and agile provisioning, it is
important to measure how many network events that the controller can important to measure how many network events that the controller can
handle at a time. This benchmark is obtained by sending asynchronous handle at a time. This benchmark is obtained by sending asynchronous
messages from every connected SDN nodes at full connection capacity messages from every connected Network Devices at full connection
for the given test duration. This test assumes that the controller capacity for the given test duration. This test assumes that the
will respond to all the received asynchronous messages. controller will respond to all the received asynchronous messages.
Measurement Units: Measurement Units:
Messages processed per second. Messages processed per second.
See Also: See Also:
None None
2.3.1.4. Reactive Path Provisioning Time 2.3.1.4. Reactive Path Provisioning Time
Definition: Definition:
The time taken by the controller to setup a path reactively between The time taken by the controller to setup a path reactively between
source and destination node, expressed in milliseconds. source and destination node, expressed in milliseconds.
Discussion: Discussion:
As SDN supports agile provisioning, it is important to measure how As SDN supports agile provisioning, it is important to measure how
fast that the controller provisions an end-to-end flow in the fast that the controller provisions an end-to-end flow in the
dataplane. The benchmark is obtained by sending traffic from a dataplane. The benchmark is obtained by sending traffic from a
source endpoint to the destination endpoint, finding the time source endpoint to the destination endpoint, finding the time
difference between the first and the last flow provisioning message difference between the first and the last flow provisioning message
exchanged between the controller and the SDN nodes for the traffic exchanged between the controller and the Network Devices for the
path. traffic path.
Measurement Units: Measurement Units:
milliseconds. milliseconds.
See Also: See Also:
None None
2.3.1.5. Proactive Path Provisioning Time 2.3.1.5. Proactive Path Provisioning Time
Definition: Definition:
The time taken by the controller to setup a path proactively between The time taken by the controller to setup a path proactively between
source and destination node, expressed in milliseconds. source and destination node, expressed in milliseconds.
Discussion: Discussion:
For SDN to support pre-provisioning of traffic path from For SDN to support pre-provisioning of traffic path from
application, it is important to measure how fast that the controller application, it is important to measure how fast that the controller
provisions an end-to-end flow in the dataplane. The benchmark is provisions an end-to-end flow in the dataplane. The benchmark is
obtained by provisioning a flow on controller's northbound interface obtained by provisioning a flow on controller's northbound interface
for the traffic to reach from a source to a destination endpoint, for the traffic to reach from a source to a destination endpoint,
finding the time difference between the first and the last flow finding the time difference between the first and the last flow
provisioning message exchanged between the controller and the SDN provisioning message exchanged between the controller and the
nodes for the traffic path. Network Devices for the traffic path.
Measurement Units: Measurement Units:
milliseconds. milliseconds.
See Also: See Also:
None None
2.3.1.6. Reactive Path Provisioning Rate 2.3.1.6. Reactive Path Provisioning Rate
Definition: Definition:
Measure the maximum number of independent paths a controller can Measure the maximum number of independent paths a controller can
concurrently establish between source and destination nodes concurrently establish between source and destination nodes
reactively within the test duration, expressed in paths per second. reactively within the test duration, expressed in paths per second.
Discussion: Discussion:
For SDN to support agile traffic forwarding, it is important to For SDN to support agile traffic forwarding, it is important to
measure how many end-to-end flows that the controller could setup in measure how many end-to-end flows that the controller could setup in
the dataplane. This benchmark is obtained by sending traffic each the dataplane. This benchmark is obtained by sending traffic each
with unique source and destination pairs from the source SDN node with unique source and destination pairs from the source Network
and determine the number of frames received at the destination SDN Device and determine the number of frames received at the
node. destination Network Device.
Measurement Units: Measurement Units:
Paths provisioned per second. Paths provisioned per second.
See Also: See Also:
None None
2.3.1.7. Proactive Path Provisioning Rate 2.3.1.7. Proactive Path Provisioning Rate
Definition: Definition:
Measure the maximum number of independent paths a controller can Measure the maximum number of independent paths a controller can
concurrently establish between source and destination nodes concurrently establish between source and destination nodes
proactively within the test duration, expressed in paths per second. proactively within the test duration, expressed in paths per second.
Discussion: Discussion:
For SDN to support pre-provisioning of traffic path for a larger For SDN to support pre-provisioning of traffic path for a larger
network from the application, it is important to measure how many network from the application, it is important to measure how many
end-to-end flows that the controller could setup in the dataplane. end-to-end flows that the controller could setup in the dataplane.
This benchmark is obtained by sending traffic each with unique This benchmark is obtained by sending traffic each with unique
source and destination pairs from the source SDN node. Program the source and destination pairs from the source Network Device. Program
flows on controller's northbound interface for traffic to reach from the flows on controller's northbound interface for traffic to reach
each of the unique source and destination pairs and determine the from each of the unique source and destination pairs and determine
number of frames received at the destination SDN node. the number of frames received at the destination Network Device.
Measurement Units: Measurement Units:
Paths provisioned per second. Paths provisioned per second.
See Also: See Also:
None None
2.3.1.8. Network Topology Change Detection Time 2.3.1.8. Network Topology Change Detection Time
Definition: Definition:
skipping to change at page 14, line 34 skipping to change at page 13, line 34
2.3.2.1. Control Sessions Capacity 2.3.2.1. Control Sessions Capacity
Definition: Definition:
To measure the maximum number of control sessions the controller To measure the maximum number of control sessions the controller
can maintain. can maintain.
Discussion: Discussion:
Measuring the controller's control sessions capacity is important to Measuring the controller's control sessions capacity is important to
determine the controller's system and bandwidth resource determine the controller's system and bandwidth resource
requirements. This benchmark is obtained by establishing control requirements. This benchmark is obtained by establishing control
session with the controller from each of the SDN node until it session with the controller from each of the Network Device until it
fails. The number of sessions that were successfully established fails. The number of sessions that were successfully established
will provide the Control Sessions Capacity. will provide the Control Sessions Capacity.
Measurement Units: Measurement Units:
N/A N/A
See Also: See Also:
None None
2.3.2.2. Network Discovery Size 2.3.2.2. Network Discovery Size
Definition: Definition:
To measure the network size (number of nodes, links and hosts) that To measure the network size (number of nodes, links and hosts) that
a controller can discover. a controller can discover.
Discussion: Discussion:
For optimal network planning, it is key to measure the maximum For optimal network planning, it is key to measure the maximum
network size that the controller can discover. This benchmark is network size that the controller can discover. This benchmark is
obtained by presenting an initial set of SDN nodes for discovery to obtained by presenting an initial set of Network Devices for
the controller. Based on the initial discovery, the number of SDN discovery to the controller. Based on the initial discovery, the
nodes is increased or decreased to determine the maximum nodes that number of Network Devices is increased or decreased to determine the
the controller can discover. maximum nodes that the controller can discover.
Measurement Units: Measurement Units:
N/A N/A
See Also: See Also:
None None
2.3.2.3. Forwarding Table Capacity 2.3.2.3. Forwarding Table Capacity
Definition: Definition:
skipping to change at page 16, line 5 skipping to change at page 15, line 5
2.3.3.1. Exception Handling 2.3.3.1. Exception Handling
Definition: Definition:
To determine the effect of handling error packets and notifications To determine the effect of handling error packets and notifications
on performance tests. on performance tests.
Discussion: Discussion:
This benchmark test is to be performed after obtaining the baseline This benchmark test is to be performed after obtaining the baseline
performance of the performance tests defined in Section 2.3.1. This performance of the performance tests defined in Section 2.3.1. This
benchmark determines the deviation from the baseline performance due benchmark determines the deviation from the baseline performance due
to the handling of error or failure messages from the connected SDN to the handling of error or failure messages from the connected
nodes. Network Devices.
Measurement Units: Measurement Units:
N/A N/A
See Also: See Also:
None None
2.3.3.2. Denial of Service Handling 2.3.3.2. Denial of Service Handling
Definition: Definition:
skipping to change at page 17, line 17 skipping to change at page 16, line 17
2.3.4.2. Network Re-Provisioning Time 2.3.4.2. Network Re-Provisioning Time
Definition: Definition:
The time taken to re-route the traffic by the Controller, when there The time taken to re-route the traffic by the Controller, when there
is a failure in existing traffic paths. is a failure in existing traffic paths.
Discussion: Discussion:
This benchmark determines the controller's re-provisioning ability This benchmark determines the controller's re-provisioning ability
upon network failures. This benchmark test assumes the following: upon network failures. This benchmark test assumes the following:
i. Network topology supports redundant path between i. Network topology supports redundant path between
source and destination endpoints. source and destination endpoints.
ii. Controller does not pre-provision the redundant path. ii. Controller does not pre-provision the redundant path.
Measurement Units: Measurement Units:
milliseconds. milliseconds.
See Also: See Also:
None None
3. Test Coverage 3. Test Setup
This section provides common reference topologies that are later
referred to in individual tests defined in the companion methodology
document.
3.1. Test setup - Controller working in Standalone Mode
+-----------------------------------------------------------+
| Application Plane Test Emulator |
| |
| +-----------------+ +-------------+ |
| | Application | | Service | |
| +-----------------+ +-------------+ |
| |
+-----------------------------+(I2)-------------------------+
|
|
| (Northbound interface)
+-------------------------------+
| +----------------+ |
| | SDN Controller | |
| +----------------+ |
| |
| Device Under Test (DUT) |
+-------------------------------+
| (Southbound interface)
|
|
+-----------------------------+(I1)-------------------------+
| |
| +-----------+ +-----------+ |
| | Network |l1 ln-1| Network | |
| | Device 1 |---- .... ----| Device n | |
| +-----------+ +-----------+ |
| |l0 |ln |
| | | |
| | | |
| +---------------+ +---------------+ |
| | Test Traffic | | Test Traffic | |
| | Generator | | Generator | |
| | (TP1) | | (TP2) | |
| +---------------+ +---------------+ |
| |
| Forwarding Plane Test Emulator |
+-----------------------------------------------------------+
Figure 1
3.2. Test setup - Controller working in Cluster Mode
+-----------------------------------------------------------+
| Application Plane Test Emulator |
| |
| +-----------------+ +-------------+ |
| | Application | | Service | |
| +-----------------+ +-------------+ |
| |
+-----------------------------+(I2)-------------------------+
|
|
| (Northbound interface)
+---------------------------------------------------------+
| |
| ------------------ ------------------ |
| | SDN Controller 1 | <--E/W--> | SDN Controller n | |
| ------------------ ------------------ |
| |
| Device Under Test (DUT) |
+---------------------------------------------------------+
| (Southbound interface)
|
|
+-----------------------------+(I1)-------------------------+
| |
| +-----------+ +-----------+ |
| | Network |l1 ln-1| Network | |
| | Device 1 |---- .... ----| Device n | |
| +-----------+ +-----------+ |
| |l0 |ln |
| | | |
| | | |
| +---------------+ +---------------+ |
| | Test Traffic | | Test Traffic | |
| | Generator | | Generator | |
| | (TP1) | | (TP2) | |
| +---------------+ +---------------+ |
| |
| Forwarding Plane Test Emulator |
+-----------------------------------------------------------+
Figure 2
4. Test Coverage
+ -----------------------------------------------------------------+ + -----------------------------------------------------------------+
| | Speed | Scalability | Reliability | | | Speed | Scalability | Reliability |
+ -----------+-------------------+---------------+-----------------+ + -----------+-------------------+---------------+-----------------+
| | 1. Network Topolo-|1. Network | | | | 1. Network Topolo-|1. Network | |
| | -gy Discovery | Discovery | | | | -gy Discovery | Discovery | |
| | | Size | | | | | Size | |
| | 2. Reactive Path | | | | | 2. Reactive Path | | |
| | Provisioning | | | | | Provisioning | | |
| | Time | | | | | Time | | |
skipping to change at page 18, line 27 skipping to change at page 20, line 5
| | | |4. Network Re- | | | | |4. Network Re- |
| | | | Provisioning | | | | | Provisioning |
| | | | Time | | | | | Time |
| | | | | | | | | |
+------------+-------------------+---------------+-----------------+ +------------+-------------------+---------------+-----------------+
| | | | | | | | | |
| Tear Down | | |1. Controller | | Tear Down | | |1. Controller |
| | | | Failover Time | | | | | Failover Time |
+------------+-------------------+---------------+-----------------+ +------------+-------------------+---------------+-----------------+
4. References 5. References
4.1. Normative References
[RFC2330] V. Paxson, G. Almes, J. Mahdavi, M. Mathis, 5.1. Normative References
"Framework for IP Performance Metrics",RFC 2330,
May 1998.
[RFC6241] R. Enns, M. Bjorklund, J. Schoenwaelder, A. Bierman, [RFC7426] E. Haleplidis, K. Pentikousis, S. Denazis, J. Hadi Salim,
"Network Configuration Protocol (NETCONF)",RFC 6241, D. Meyer, O. Koufopavlou "Software-Defined Networking
June 2011. (SDN): Layers and Architecture Terminology", RFC 7426,
January 2015.
[RFC6020] M. Bjorklund, "YANG - A Data Modeling Language for [RFC4689] S. Poretsky, J. Perser, S. Erramilli, S. Khurana
the Network Configuration Protocol (NETCONF)", RFC 6020, "Terminology for Benchmarking Network-layer Traffic
October 2010 Control Mechanisms", RFC 4689, October 2006.
[RFC5440] JP. Vasseur, JL. Le Roux, "Path Computation Element (PCE) [RFC2330] V. Paxson, G. Almes, J. Mahdavi, M. Mathis,
Communication Protocol (PCEP)", RFC 5440, March 2009. "Framework for IP Performance Metrics", RFC 2330,
May 1998.
[OpenFlow Switch Specification] ONF,"OpenFlow Switch Specification" [OpenFlow Switch Specification] ONF,"OpenFlow Switch Specification"
Version 1.4.0 (Wire Protocol 0x05), October 14, 2013. Version 1.4.0 (Wire Protocol 0x05), October 14, 2013.
[I-D.sdn-controller-benchmark-meth] Bhuvaneswaran.V, Anton Basil, [I-D.sdn-controller-benchmark-meth] Bhuvaneswaran.V, Anton Basil,
Mark.T, Vishwas Manral, Sarah Banks "Benchmarking Mark.T, Vishwas Manral, Sarah Banks "Benchmarking
Methodology for SDN Controller Performance", Methodology for SDN Controller Performance",
draft-ietf-bmwg-sdn-controller-benchmark-meth-00 draft-ietf-bmwg-sdn-controller-benchmark-meth-01
(Work in progress), October 19, 2015 (Work in progress), March 21, 2016
4.2. Informative References 5.2. Informative References
[OpenContrail] Ankur Singla, Bruno Rijsman, "OpenContrail [OpenContrail] Ankur Singla, Bruno Rijsman, "OpenContrail
Architecture Documentation", Architecture Documentation",
http://opencontrail.org/opencontrail-architecture-documentation http://opencontrail.org/opencontrail-architecture-documentation
[OpenDaylight] OpenDaylight Controller:Architectural Framework, [OpenDaylight] OpenDaylight Controller:Architectural Framework,
https://wiki.opendaylight.org/view/OpenDaylight_Controller https://wiki.opendaylight.org/view/OpenDaylight_Controller
5. IANA Considerations 6. IANA Considerations
This document does not have any IANA requests. This document does not have any IANA requests.
6. Security Considerations 7. Security Considerations
Security issues are not discussed in this memo. Security issues are not discussed in this memo.
7. Acknowledgements 8. Acknowledgements
The authors would like to acknowledge Sandeep Gangadharan (HP) for The authors would like to acknowledge Al Morton (AT&T) for the
the significant contributions to the earlier versions of this significant contributions to the earlier versions of this document.
document. The authors would like to thank the following individuals The authors would like to thank the following individuals for
for providing their valuable comments to the earlier versions of providing their valuable comments to the earlier versions of this
this document: Al Morton (AT&T), M. Georgescu (NAIST), Andrew document: Sandeep Gangadharan (HP), M. Georgescu (NAIST), Andrew
McGregor (Google), Scott Bradner (Harvard University), Jay Karthik McGregor (Google), Scott Bradner (Harvard University), Jay Karthik
(Cisco), Ramakrishnan (Dell), Khasanov Boris (Huawei). (Cisco), Ramakrishnan (Dell), Khasanov Boris (Huawei).
8. Authors' Addresses 9. Authors' Addresses
Bhuvaneswaran Vengainathan Bhuvaneswaran Vengainathan
Veryx Technologies Inc. Veryx Technologies Inc.
1 International Plaza, Suite 550 1 International Plaza, Suite 550
Philadelphia Philadelphia
PA 19113 PA 19113
Email: bhuvaneswaran.vengainathan@veryxtech.com Email: bhuvaneswaran.vengainathan@veryxtech.com
Anton Basil Anton Basil
skipping to change at page 20, line 14 skipping to change at page 21, line 38
Philadelphia Philadelphia
PA 19113 PA 19113
Email: anton.basil@veryxtech.com Email: anton.basil@veryxtech.com
Mark Tassinari Mark Tassinari
Hewlett-Packard, Hewlett-Packard,
8000 Foothills Blvd, 8000 Foothills Blvd,
Roseville, CA 95747 Roseville, CA 95747
Email: mark.tassinari@hp.com Email: mark.tassinari@hpe.com
Vishwas Manral Vishwas Manral
Ionos Corp, Nano Sec,
4100 Moorpark Ave, CA
San Jose, CA
Email: vishwas@ionosnetworks.com Email: vishwas.manral@gmail.com
Sarah Banks Sarah Banks
VSS Monitoring VSS Monitoring
930 De Guigne Drive,
Sunnyvale, CA
Email: sbanks@encrypted.net Email: sbanks@encrypted.net
 End of changes. 65 change blocks. 
211 lines changed or deleted 258 lines changed or added

This html diff was produced by rfcdiff 1.44. The latest version is available from http://tools.ietf.org/tools/rfcdiff/