draft-ietf-bmwg-sdn-controller-benchmark-term-02.txt   draft-ietf-bmwg-sdn-controller-benchmark-term-03.txt 
Internet-Draft Bhuvaneswaran Vengainathan Internet-Draft Bhuvaneswaran Vengainathan
Network Working Group Anton Basil Network Working Group Anton Basil
Intended Status: Informational Veryx Technologies Intended Status: Informational Veryx Technologies
Expires: January 8, 2017 Mark Tassinari Expires: June 8, 2017 Mark Tassinari
Hewlett-Packard Hewlett-Packard
Vishwas Manral Vishwas Manral
Nano Sec Nano Sec
Sarah Banks Sarah Banks
VSS Monitoring VSS Monitoring
July 8, 2016 January 8, 2017
Terminology for Benchmarking SDN Controller Performance Terminology for Benchmarking SDN Controller Performance
draft-ietf-bmwg-sdn-controller-benchmark-term-02 draft-ietf-bmwg-sdn-controller-benchmark-term-03
Abstract Abstract
This document defines terminology for benchmarking an SDN This document defines terminology for benchmarking an SDN
controller's control plane performance. It extends the terminology controller's control plane performance. It extends the terminology
already defined in RFC 7426 for the purpose of benchmarking SDN already defined in RFC 7426 for the purpose of benchmarking SDN
controllers. The terms provided in this document help to benchmark controllers. The terms provided in this document help to benchmark
SDN controller's performance independent of the controller's SDN controller's performance independent of the controller's
supported protocols and/or network services. A mechanism for supported protocols and/or network services. A mechanism for
benchmarking the performance of SDN controllers is defined in the benchmarking the performance of SDN controllers is defined in the
skipping to change at page 1, line 44 skipping to change at page 1, line 44
Internet-Drafts are working documents of the Internet Engineering Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF). Note that other groups may also distribute Task Force (IETF). Note that other groups may also distribute
working documents as Internet-Drafts. The list of current Internet- working documents as Internet-Drafts. The list of current Internet-
Drafts is at http://datatracker.ietf.org/drafts/current. Drafts is at http://datatracker.ietf.org/drafts/current.
Internet-Drafts are draft documents valid for a maximum of six Internet-Drafts are draft documents valid for a maximum of six
months and may be updated, replaced, or obsoleted by other documents months and may be updated, replaced, or obsoleted by other documents
at any time. It is inappropriate to use Internet-Drafts as reference at any time. It is inappropriate to use Internet-Drafts as reference
material or to cite them other than as "work in progress. material or to cite them other than as "work in progress.
This Internet-Draft will expire on January 8, 2017. This Internet-Draft will expire on June 8, 2017.
Copyright Notice Copyright Notice
Copyright (c) 2016 IETF Trust and the persons identified as the Copyright (c) 2016 IETF Trust and the persons identified as the
document authors. All rights reserved. document authors. All rights reserved.
This document is subject to BCP 78 and the IETF Trust's Legal This document is subject to BCP 78 and the IETF Trust's Legal
Provisions Relating to IETF Documents Provisions Relating to IETF Documents
(http://trustee.ietf.org/license-info) in effect on the date of (http://trustee.ietf.org/license-info) in effect on the date of
publication of this document. Please review these documents publication of this document. Please review these documents
carefully, as they describe your rights and restrictions with carefully, as they describe your rights and restrictions with
respect to this document. Code Components extracted from this respect to this document. Code Components extracted from this
document must include Simplified BSD License text as described in document must include Simplified BSD License text as described in
Section 4.e of the Trust Legal Provisions and are provided without Section 4.e of the Trust Legal Provisions and are provided without
warranty as described in the Simplified BSD License. warranty as described in the Simplified BSD License.
Table of Contents Table of Contents
1. Introduction ................................................ 3 1. Introduction...................................................4
2. Term Definitions ............................................ 4 2. Term Definitions...............................................4
2.1. SDN Terms .............................................. 4 2.1. SDN Terms.................................................4
2.1.1. Flow .............................................. 4 2.1.1. Flow.................................................4
2.1.2. Northbound Interface............................... 4 2.1.2. Northbound Interface.................................5
2.1.3. Controller Forwarding Table........................ 4 2.1.3. Controller Forwarding Table..........................5
2.1.4. Proactive Flow Provisioning Mode................... 5 2.1.4. Proactive Flow Provisioning Mode.....................5
2.1.5. Reactive Flow Provisioning Mode.................... 5 2.1.5. Reactive Flow Provisioning Mode......................6
2.1.6. Path .............................................. 6 2.1.6. Path.................................................6
2.1.7. Standalone Mode.................................... 6 2.1.7. Standalone Mode......................................6
2.1.8. Cluster/Redundancy Mode............................ 6 2.1.8. Cluster/Redundancy Mode..............................7
2.1.9. Asynchronous Message............................... 7 2.1.9. Asynchronous Message.................................7
2.1.10. Test Traffic Generator............................ 7 2.1.10. Test Traffic Generator..............................8
2.2. Test Configuration/Setup Terms.......................... 8 2.2. Test Configuration/Setup Terms............................8
2.2.1. Number of Network Devices.......................... 8 2.2.1. Number of Network Devices............................8
2.2.2. Test Iterations.................................... 8 2.2.2. Test Iterations......................................8
2.2.3. Test Duration...................................... 8 2.2.3. Test Duration........................................9
2.2.4. Number of Cluster nodes............................ 9 2.2.4. Number of Cluster nodes..............................9
2.3. Benchmarking Terms...................................... 9 2.3. Benchmarking Terms.......................................10
2.3.1. Performance........................................ 9 2.3.1. Performance.........................................10
2.3.1.1. Network Topology Discovery Time............... 9 2.3.1.1. Network Topology Discovery Time................10
2.3.1.2. Asynchronous Message Processing Time.......... 10 2.3.1.2. Asynchronous Message Processing Time...........10
2.3.1.3. Asynchronous Message Processing Rate.......... 10 2.3.1.3. Asynchronous Message Processing Rate...........11
2.3.1.4. Reactive Path Provisioning Time .............. 11 2.3.1.4. Reactive Path Provisioning Time................11
2.3.1.5. Proactive Path Provisioning Time ............. 11 2.3.1.5. Proactive Path Provisioning Time...............12
2.3.1.6. Reactive Path Provisioning Rate .............. 12 2.3.1.6. Reactive Path Provisioning Rate................12
2.3.1.7. Proactive Path Provisioning Rate ............. 12 2.3.1.7. Proactive Path Provisioning Rate...............13
2.3.1.8. Network Topology Change Detection Time........ 13 2.3.1.8. Network Topology Change Detection Time.........13
2.3.2. Scalability ....................................... 14 2.3.2. Scalability.........................................14
2.3.2.1. Control Sessions Capacity .................... 14 2.3.2.1. Control Sessions Capacity......................14
2.3.2.2. Network Discovery Size ....................... 14 2.3.2.2. Network Discovery Size.........................15
2.3.2.3. Forwarding Table Capacity .................... 15 2.3.2.3. Forwarding Table Capacity......................15
2.3.3. Security ......................................... 15 2.3.3. Security............................................16
2.3.3.1. Exception Handling ........................... 15 2.3.3.1. Exception Handling.............................16
2.3.3.2. Denial of Service Handling ................... 16 2.3.3.2. Denial of Service Handling.....................16
2.3.4. Reliability ....................................... 16 2.3.4. Reliability.........................................17
2.3.4.1. Controller Failover Time ..................... 16 2.3.4.1. Controller Failover Time.......................17
2.3.4.2. Network Re-Provisioning Time ................. 16 2.3.4.2. Network Re-Provisioning Time...................17
3. Test Setup ................................................. 17 3. Test Setup....................................................18
3.1. Test setup - Controller working in Standalone Mode...... 18 3.1. Test setup - Controller working in Standalone Mode.......18
3.2. Test setup - Controller working in Cluster Mode......... 19 3.2. Test setup - Controller working in Cluster Mode..........19
4. Test Coverage .............................................. 20 4. Test Coverage.................................................20
5. References ................................................. 21 5. References....................................................21
5.1. Normative References ................................... 21 5.1. Normative References.....................................21
5.2. Informative References ................................. 21 5.2. Informative References...................................21
6. IANA Considerations ........................................ 21 6. IANA Considerations...........................................21
7. Security Considerations ..................................... 21 7. Security Considerations.......................................22
8. Acknowledgements ........................................... 22 8. Acknowledgements..............................................22
9. Authors' Addresses ......................................... 22 9. Authors' Addresses............................................22
1. Introduction 1. Introduction
Software Defined Networking (SDN) is a networking architecture in Software Defined Networking (SDN) is a networking architecture in
which network control is decoupled from the underlying forwarding which network control is decoupled from the underlying forwarding
function and is placed in a centralized location called the SDN function and is placed in a centralized location called the SDN
controller. The SDN controller abstracts the underlying network and controller. The SDN controller abstracts the underlying network and
offers a global view of the overall network to applications and offers a global view of the overall network to applications and
business logic. Thus, an SDN controller provides the flexibility to business logic. Thus, an SDN controller provides the flexibility to
program, control, and manage network behaviour dynamically through program, control, and manage network behaviour dynamically through
skipping to change at page 4, line 10 skipping to change at page 4, line 31
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
"SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
document are to be interpreted as described in RFC 2119. document are to be interpreted as described in RFC 2119.
2. Term Definitions 2. Term Definitions
2.1. SDN Terms 2.1. SDN Terms
The terms defined in this section are extensions to the terms The terms defined in this section are extensions to the terms
defined in RFC 7426 ''Software-Defined Networking (SDN): Layers and defined in RFC 7426 "Software-Defined Networking (SDN): Layers and
Architecture Terminology''. This RFC should be referred before Architecture Terminology". This RFC should be referred before
attempting to make use of this document. attempting to make use of this document.
2.1.1. Flow 2.1.1. Flow
Definition: Definition:
The definition of Flow is same as microflows defined in RFC 4689 The definition of Flow is same as microflows defined in RFC 4689
Section 3.1.5. Section 3.1.5.
Discussion: Discussion:
A flow can be set of packets having same source address, destination A flow can be set of packets having same source address, destination
skipping to change at page 10, line 41 skipping to change at page 11, line 19
Measurement Units: Measurement Units:
milliseconds milliseconds
See Also: See Also:
None None
2.3.1.3. Asynchronous Message Processing Rate 2.3.1.3. Asynchronous Message Processing Rate
Definition: Definition:
The maximum number of asynchronous messages that the controller(s) The maximum number of asynchronous messages (session aliveness check
can process, defined as the number of asynchronous messages the message, new flow arrival notification message etc.) that the
controller(s) can process at its Southbound interface between the controller(s) can process, defined as the iteration starting with
start of the test and the expiry of given test duration.. sending asynchronous messages to the controller (s) at the maximum
possible rate and ending with an iteration that the controller(s)
processes the received asynchronous messages without dropping.
Discussion: Discussion:
As SDN assures flexible network and agile provisioning, it is As SDN assures flexible network and agile provisioning, it is
important to measure how many network events that the controller can important to measure how many network events that the controller can
handle at a time. This benchmark is obtained by sending asynchronous handle at a time. This benchmark is obtained by sending asynchronous
messages from every connected Network Devices at full connection messages from every connected Network Devices at the rate that the
capacity for the given test duration. This test assumes that the controller processes without dropping. This test assumes that the
controller will respond to all the received asynchronous messages. controller will respond to all the received asynchronous messages.
Measurement Units: Measurement Units:
Messages processed per second. Messages processed per second.
See Also: See Also:
None None
2.3.1.4. Reactive Path Provisioning Time 2.3.1.4. Reactive Path Provisioning Time
skipping to change at page 17, line 4 skipping to change at page 17, line 30
Measurement Units: Measurement Units:
milliseconds. milliseconds.
See Also: See Also:
None None
2.3.4.2. Network Re-Provisioning Time 2.3.4.2. Network Re-Provisioning Time
Definition: Definition:
The time taken to re-route the traffic by the Controller, when there The time taken to re-route the traffic by the Controller, when there
is a failure in existing traffic paths, defined as the interval is a failure in existing traffic paths, defined as the interval
starting from the first failure notification message received by the starting from the first failure notification message received by the
controller, ending with the last flow re-provisioning message sent controller, ending with the last flow re-provisioning message sent
by the controller at its Southbound interface . by the controller at its Southbound interface .
Discussion: Discussion:
This benchmark determines the controller's re-provisioning ability This benchmark determines the controller's re-provisioning ability
upon network failures. This benchmark test assumes the following: upon network failures. This benchmark test assumes the following:
i. Network topology supports redundant path between i. Network topology supports redundant path between
source and destination endpoints. source and destination endpoints.
ii. Controller does not pre-provision the redundant path. ii. Controller does not pre-provision the redundant path.
Measurement Units: Measurement Units:
milliseconds. milliseconds.
See Also: See Also:
None None
3. Test Setup 3. Test Setup
This section provides common reference topologies that are later This section provides common reference topologies that are later
skipping to change at page 21, line 28 skipping to change at page 21, line 30
[RFC2330] V. Paxson, G. Almes, J. Mahdavi, M. Mathis, [RFC2330] V. Paxson, G. Almes, J. Mahdavi, M. Mathis,
"Framework for IP Performance Metrics", RFC 2330, "Framework for IP Performance Metrics", RFC 2330,
May 1998. May 1998.
[OpenFlow Switch Specification] ONF,"OpenFlow Switch Specification" [OpenFlow Switch Specification] ONF,"OpenFlow Switch Specification"
Version 1.4.0 (Wire Protocol 0x05), October 14, 2013. Version 1.4.0 (Wire Protocol 0x05), October 14, 2013.
[I-D.sdn-controller-benchmark-meth] Bhuvaneswaran.V, Anton Basil, [I-D.sdn-controller-benchmark-meth] Bhuvaneswaran.V, Anton Basil,
Mark.T, Vishwas Manral, Sarah Banks "Benchmarking Mark.T, Vishwas Manral, Sarah Banks "Benchmarking
Methodology for SDN Controller Performance", Methodology for SDN Controller Performance",
draft-ietf-bmwg-sdn-controller-benchmark-meth-02 draft-ietf-bmwg-sdn-controller-benchmark-meth-03
(Work in progress), July 8, 2016 (Work in progress), January 8, 2017
5.2. Informative References 5.2. Informative References
[OpenContrail] Ankur Singla, Bruno Rijsman, "OpenContrail [OpenContrail] Ankur Singla, Bruno Rijsman, "OpenContrail
Architecture Documentation", Architecture Documentation",
http://opencontrail.org/opencontrail-architecture-documentation http://opencontrail.org/opencontrail-architecture-documentation
[OpenDaylight] OpenDaylight Controller:Architectural Framework, [OpenDaylight] OpenDaylight Controller:Architectural Framework,
https://wiki.opendaylight.org/view/OpenDaylight_Controller https://wiki.opendaylight.org/view/OpenDaylight_Controller
 End of changes. 13 change blocks. 
67 lines changed or deleted 67 lines changed or added

This html diff was produced by rfcdiff 1.45. The latest version is available from http://tools.ietf.org/tools/rfcdiff/