draft-ietf-bmwg-sdn-controller-benchmark-meth-03.txt   draft-ietf-bmwg-sdn-controller-benchmark-meth-04.txt 
Internet-Draft Bhuvaneswaran Vengainathan Internet-Draft Bhuvaneswaran Vengainathan
Network Working Group Anton Basil Network Working Group Anton Basil
Intended Status: Informational Veryx Technologies Intended Status: Informational Veryx Technologies
Expires: June 8, 2017 Mark Tassinari Expires: December 28, 2017 Mark Tassinari
Hewlett-Packard Hewlett-Packard
Vishwas Manral Vishwas Manral
Nano Sec Nano Sec
Sarah Banks Sarah Banks
VSS Monitoring VSS Monitoring
January 8, 2017 June 28, 2017
Benchmarking Methodology for SDN Controller Performance Benchmarking Methodology for SDN Controller Performance
draft-ietf-bmwg-sdn-controller-benchmark-meth-03 draft-ietf-bmwg-sdn-controller-benchmark-meth-04
Abstract Abstract
This document defines the methodologies for benchmarking control This document defines the methodologies for benchmarking control
plane performance of SDN controllers. Terminology related to plane performance of SDN controllers. Terminology related to
benchmarking SDN controllers is described in the companion benchmarking SDN controllers is described in the companion
terminology document. SDN controllers have been implemented with terminology document. SDN controllers have been implemented with
many varying designs in order to achieve their intended network many varying designs in order to achieve their intended network
functionality. Hence, the authors have taken the approach of functionality. Hence, the authors have taken the approach of
considering an SDN controller as a black box, defining the considering an SDN controller as a black box, defining the
skipping to change at page 1, line 45 skipping to change at page 1, line 45
Internet-Drafts are working documents of the Internet Engineering Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF). Note that other groups may also distribute Task Force (IETF). Note that other groups may also distribute
working documents as Internet-Drafts. The list of current Internet- working documents as Internet-Drafts. The list of current Internet-
Drafts is at http://datatracker.ietf.org/drafts/current. Drafts is at http://datatracker.ietf.org/drafts/current.
Internet-Drafts are draft documents valid for a maximum of six Internet-Drafts are draft documents valid for a maximum of six
months and may be updated, replaced, or obsoleted by other documents months and may be updated, replaced, or obsoleted by other documents
at any time. It is inappropriate to use Internet-Drafts as reference at any time. It is inappropriate to use Internet-Drafts as reference
material or to cite them other than as "work in progress. material or to cite them other than as "work in progress.
This Internet-Draft will expire on June 8, 2017. This Internet-Draft will expire on December 28, 2017.
Copyright Notice Copyright Notice
Copyright (c) 2016 IETF Trust and the persons identified as the Copyright (c) 2017 IETF Trust and the persons identified as the
document authors. All rights reserved. document authors. All rights reserved.
This document is subject to BCP 78 and the IETF Trust's Legal This document is subject to BCP 78 and the IETF Trust's Legal
Provisions Relating to IETF Documents Provisions Relating to IETF Documents
(http://trustee.ietf.org/license-info) in effect on the date of (http://trustee.ietf.org/license-info) in effect on the date of
publication of this document. Please review these documents publication of this document. Please review these documents
carefully, as they describe your rights and restrictions with carefully, as they describe your rights and restrictions with
respect to this document. Code Components extracted from this respect to this document. Code Components extracted from this
document must include Simplified BSD License text as described in document must include Simplified BSD License text as described in
Section 4.e of the Trust Legal Provisions and are provided without Section 4.e of the Trust Legal Provisions and are provided without
warranty as described in the Simplified BSD License. warranty as described in the Simplified BSD License.
Table of Contents Table of Contents
1. Introduction...................................................4 1. Introduction..................................................4
2. Scope..........................................................4 2. Scope.........................................................4
3. Test Setup.....................................................4 3. Test Setup....................................................4
3.1. Test setup - Controller working in Standalone Mode........5 3.1. Test setup - Controller working in Standalone Mode.......5
3.2. Test setup - Controller working in Cluster Mode...........6 3.2. Test setup - Controller working in Cluster Mode..........6
4. Test Considerations............................................7 4. Test Considerations...........................................7
4.1. Network Topology..........................................7 4.1. Network Topology.........................................7
4.2. Test Traffic..............................................7 4.2. Test Traffic.............................................7
4.3. Test Emulator Requirements................................7 4.3. Test Emulator Requirements...............................7
4.4. Connection Setup..........................................7 4.4. Connection Setup.........................................7
4.5. Measurement Point Specification and Recommendation........8 4.5. Measurement Point Specification and Recommendation.......8
4.6. Connectivity Recommendation...............................8 4.6. Connectivity Recommendation..............................8
4.7. Test Repeatability........................................8 4.7. Test Repeatability.......................................8
5. Benchmarking Tests.............................................9 5. Benchmarking Tests............................................9
5.1. Performance...............................................9 5.1. Performance..............................................9
5.1.1. Network Topology Discovery Time......................9 5.1.1. Network Topology Discovery Time.....................9
5.1.2. Asynchronous Message Processing Time................11 5.1.2. Asynchronous Message Processing Time...............11
5.1.3. Asynchronous Message Processing Rate................12 5.1.3. Asynchronous Message Processing Rate...............12
5.1.4. Reactive Path Provisioning Time.....................14 5.1.4. Reactive Path Provisioning Time....................14
5.1.5. Proactive Path Provisioning Time....................15 5.1.5. Proactive Path Provisioning Time...................15
5.1.6. Reactive Path Provisioning Rate.....................17 5.1.6. Reactive Path Provisioning Rate....................17
5.1.7. Proactive Path Provisioning Rate....................18 5.1.7. Proactive Path Provisioning Rate...................18
5.1.8. Network Topology Change Detection Time..............20 5.1.8. Network Topology Change Detection Time.............20
5.2. Scalability..............................................21 5.2. Scalability.............................................21
5.2.1. Control Session Capacity............................21 5.2.1. Control Session Capacity...........................21
5.2.2. Network Discovery Size..............................22 5.2.2. Network Discovery Size.............................22
5.2.3. Forwarding Table Capacity...........................23 5.2.3. Forwarding Table Capacity..........................23
5.3. Security.................................................24 5.3. Security................................................24
5.3.1. Exception Handling..................................24 5.3.1. Exception Handling.................................24
5.3.2. Denial of Service Handling..........................26 5.3.2. Denial of Service Handling.........................26
5.4. Reliability..............................................27 5.4. Reliability.............................................27
5.4.1. Controller Failover Time............................27 5.4.1. Controller Failover Time...........................27
5.4.2. Network Re-Provisioning Time........................28 5.4.2. Network Re-Provisioning Time.......................28
6. References....................................................30 6. References...................................................30
6.1. Normative References.....................................30 6.1. Normative References....................................30
6.2. Informative References...................................31 6.2. Informative References..................................31
7. IANA Considerations...........................................31 7. IANA Considerations..........................................31
8. Security Considerations.......................................31 8. Security Considerations......................................31
9. Acknowledgments...............................................31 9. Acknowledgments..............................................31
Appendix A. Example Test Topologies..............................33 Appendix A. Example Test Topologies.............................33
A.1. Leaf-Spine Topology - Three Tier Network Architecture....33 A.1. Leaf-Spine Topology - Three Tier Network Architecture...33
A.2. Leaf-Spine Topology - Two Tier Network Architecture......33 A.2. Leaf-Spine Topology - Two Tier Network Architecture.....33
Appendix B. Benchmarking Methodology using OpenFlow Controllers..34 Appendix B. Benchmarking Methodology using OpenFlow Controllers.34
B.1. Protocol Overview........................................34 B.1. Protocol Overview.......................................34
B.2. Messages Overview........................................34 B.2. Messages Overview.......................................34
B.3. Connection Overview......................................34 B.3. Connection Overview.....................................34
B.4. Performance Benchmarking Tests...........................35 B.4. Performance Benchmarking Tests..........................35
B.4.1. Network Topology Discovery Time.....................35 B.4.1. Network Topology Discovery Time....................35
B.4.2. Asynchronous Message Processing Time................36 B.4.2. Asynchronous Message Processing Time...............36
B.4.3. Asynchronous Message Processing Rate................37 B.4.3. Asynchronous Message Processing Rate...............37
B.4.4. Reactive Path Provisioning Time.....................38 B.4.4. Reactive Path Provisioning Time....................38
B.4.5. Proactive Path Provisioning Time....................39 B.4.5. Proactive Path Provisioning Time...................39
B.4.6. Reactive Path Provisioning Rate.....................40 B.4.6. Reactive Path Provisioning Rate....................40
B.4.7. Proactive Path Provisioning Rate....................41 B.4.7. Proactive Path Provisioning Rate...................41
B.4.8. Network Topology Change Detection Time..............42 B.4.8. Network Topology Change Detection Time.............42
B.5. Scalability..............................................43 B.5. Scalability.............................................43
B.5.1. Control Sessions Capacity...........................43 B.5.1. Control Sessions Capacity..........................43
B.5.2. Network Discovery Size..............................43 B.5.2. Network Discovery Size.............................43
B.5.3. Forwarding Table Capacity...........................44 B.5.3. Forwarding Table Capacity..........................44
B.6. Security.................................................46 B.6. Security................................................46
B.6.1. Exception Handling..................................46 B.6.1. Exception Handling.................................46
B.6.2. Denial of Service Handling..........................47 B.6.2. Denial of Service Handling.........................47
B.7. Reliability..............................................49 B.7. Reliability.............................................49
B.7.1. Controller Failover Time............................49 B.7.1. Controller Failover Time...........................49
B.7.2. Network Re-Provisioning Time........................50 B.7.2. Network Re-Provisioning Time.......................50
Authors' Addresses...............................................53 Authors' Addresses..............................................53
1. Introduction 1. Introduction
This document provides generic methodologies for benchmarking SDN This document provides generic methodologies for benchmarking SDN
controller performance. An SDN controller may support many controller performance. An SDN controller may support many
northbound and southbound protocols, implement a wide range of northbound and southbound protocols, implement a wide range of
applications, and work solely, or as a group to achieve the desired applications, and work solely, or as a group to achieve the desired
functionality. This document considers an SDN controller as a black functionality. This document considers an SDN controller as a black
box, regardless of design and implementation. The tests defined in box, regardless of design and implementation. The tests defined in
the document can be used to benchmark SDN controller for the document can be used to benchmark SDN controller for
skipping to change at page 30, line 48 skipping to change at page 30, line 48
[RFC5440] JP. Vasseur, JL. Le Roux, "Path Computation Element (PCE) [RFC5440] JP. Vasseur, JL. Le Roux, "Path Computation Element (PCE)
Communication Protocol (PCEP)", RFC 5440, March 2009. Communication Protocol (PCEP)", RFC 5440, March 2009.
[OpenFlow Switch Specification] ONF,"OpenFlow Switch Specification" [OpenFlow Switch Specification] ONF,"OpenFlow Switch Specification"
Version 1.4.0 (Wire Protocol 0x05), October 14, 2013. Version 1.4.0 (Wire Protocol 0x05), October 14, 2013.
[I-D.sdn-controller-benchmark-term] Bhuvaneswaran.V, Anton Basil, [I-D.sdn-controller-benchmark-term] Bhuvaneswaran.V, Anton Basil,
Mark.T, Vishwas Manral, Sarah Banks, "Terminology for Mark.T, Vishwas Manral, Sarah Banks, "Terminology for
Benchmarking SDN Controller Performance", Benchmarking SDN Controller Performance",
draft-ietf-bmwg-sdn-controller-benchmark-term-03 draft-ietf-bmwg-sdn-controller-benchmark-term-04
(Work in progress), January 8, 2017 (Work in progress), June 28, 2017
6.2. Informative References 6.2. Informative References
[I-D.i2rs-architecture] A. Atlas, J. Halpern, S. Hares, D. Ward, [I-D.i2rs-architecture] A. Atlas, J. Halpern, S. Hares, D. Ward,
T. Nadeau, "An Architecture for the Interface to the T. Nadeau, "An Architecture for the Interface to the
Routing System", draft-ietf-i2rs-architecture-09 Routing System", draft-ietf-i2rs-architecture-09
(Work in progress), March 6, 2015 (Work in progress), March 6, 2015
[OpenContrail] Ankur Singla, Bruno Rijsman, "OpenContrail [OpenContrail] Ankur Singla, Bruno Rijsman, "OpenContrail
Architecture Documentation", Architecture Documentation",
 End of changes. 7 change blocks. 
73 lines changed or deleted 73 lines changed or added

This html diff was produced by rfcdiff 1.45. The latest version is available from http://tools.ietf.org/tools/rfcdiff/