draft-ietf-bmwg-sdn-controller-benchmark-term-05.txt   draft-ietf-bmwg-sdn-controller-benchmark-term-06.txt 
Internet-Draft Bhuvaneswaran Vengainathan Internet-Draft Bhuvaneswaran Vengainathan
Network Working Group Anton Basil Network Working Group Anton Basil
Intended Status: Informational Veryx Technologies Intended Status: Informational Veryx Technologies
Expires: April 01, 2018 Mark Tassinari Expires: May 16, 2018 Mark Tassinari
Hewlett-Packard Hewlett-Packard
Vishwas Manral Vishwas Manral
Nano Sec Nano Sec
Sarah Banks Sarah Banks
VSS Monitoring VSS Monitoring
October 01, 2017 November 16, 2017
Terminology for Benchmarking SDN Controller Performance Terminology for Benchmarking SDN Controller Performance
draft-ietf-bmwg-sdn-controller-benchmark-term-05 draft-ietf-bmwg-sdn-controller-benchmark-term-06
Abstract Abstract
This document defines terminology for benchmarking an SDN This document defines terminology for benchmarking an SDN
controller's control plane performance. It extends the terminology controller's control plane performance. It extends the terminology
already defined in RFC 7426 for the purpose of benchmarking SDN already defined in RFC 7426 for the purpose of benchmarking SDN
controllers. The terms provided in this document help to benchmark controllers. The terms provided in this document help to benchmark
SDN controller's performance independent of the controller's SDN controller's performance independent of the controller's
supported protocols and/or network services. A mechanism for supported protocols and/or network services. A mechanism for
benchmarking the performance of SDN controllers is defined in the benchmarking the performance of SDN controllers is defined in the
skipping to change at page 1, line 44 skipping to change at page 1, line 44
Internet-Drafts are working documents of the Internet Engineering Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF). Note that other groups may also distribute Task Force (IETF). Note that other groups may also distribute
working documents as Internet-Drafts. The list of current Internet- working documents as Internet-Drafts. The list of current Internet-
Drafts is at http://datatracker.ietf.org/drafts/current. Drafts is at http://datatracker.ietf.org/drafts/current.
Internet-Drafts are draft documents valid for a maximum of six Internet-Drafts are draft documents valid for a maximum of six
months and may be updated, replaced, or obsoleted by other documents months and may be updated, replaced, or obsoleted by other documents
at any time. It is inappropriate to use Internet-Drafts as reference at any time. It is inappropriate to use Internet-Drafts as reference
material or to cite them other than as "work in progress. material or to cite them other than as "work in progress.
This Internet-Draft will expire on April 01, 2018. This Internet-Draft will expire on May 16, 2018.
Copyright Notice Copyright Notice
Copyright (c) 2017 IETF Trust and the persons identified as the Copyright (c) 2017 IETF Trust and the persons identified as the
document authors. All rights reserved. document authors. All rights reserved.
This document is subject to BCP 78 and the IETF Trust's Legal This document is subject to BCP 78 and the IETF Trust's Legal
Provisions Relating to IETF Documents Provisions Relating to IETF Documents
(http://trustee.ietf.org/license-info) in effect on the date of (http://trustee.ietf.org/license-info) in effect on the date of
publication of this document. Please review these documents publication of this document. Please review these documents
skipping to change at page 2, line 37 skipping to change at page 2, line 37
2.1.3. Controller Forwarding Table..........................5 2.1.3. Controller Forwarding Table..........................5
2.1.4. Proactive Flow Provisioning Mode.....................5 2.1.4. Proactive Flow Provisioning Mode.....................5
2.1.5. Reactive Flow Provisioning Mode......................6 2.1.5. Reactive Flow Provisioning Mode......................6
2.1.6. Path.................................................6 2.1.6. Path.................................................6
2.1.7. Standalone Mode......................................6 2.1.7. Standalone Mode......................................6
2.1.8. Cluster/Redundancy Mode..............................7 2.1.8. Cluster/Redundancy Mode..............................7
2.1.9. Asynchronous Message.................................7 2.1.9. Asynchronous Message.................................7
2.1.10. Test Traffic Generator..............................8 2.1.10. Test Traffic Generator..............................8
2.2. Test Configuration/Setup Terms............................8 2.2. Test Configuration/Setup Terms............................8
2.2.1. Number of Network Devices............................8 2.2.1. Number of Network Devices............................8
2.2.2. Trails...............................................8 2.2.2. Trials...............................................8
2.2.3. Trail Duration.......................................9 2.2.3. Trial Duration.......................................9
2.2.4. Number of Cluster nodes..............................9 2.2.4. Number of Cluster nodes..............................9
2.3. Benchmarking Terms.......................................10 2.3. Benchmarking Terms.......................................10
2.3.1. Performance.........................................10 2.3.1. Performance.........................................10
2.3.1.1. Network Topology Discovery Time................10 2.3.1.1. Network Topology Discovery Time................10
2.3.1.2. Asynchronous Message Processing Time...........10 2.3.1.2. Asynchronous Message Processing Time...........10
2.3.1.3. Asynchronous Message Processing Rate...........11 2.3.1.3. Asynchronous Message Processing Rate...........11
2.3.1.4. Reactive Path Provisioning Time................12 2.3.1.4. Reactive Path Provisioning Time................12
2.3.1.5. Proactive Path Provisioning Time...............12 2.3.1.5. Proactive Path Provisioning Time...............12
2.3.1.6. Reactive Path Provisioning Rate................13 2.3.1.6. Reactive Path Provisioning Rate................13
2.3.1.7. Proactive Path Provisioning Rate...............14 2.3.1.7. Proactive Path Provisioning Rate...............14
skipping to change at page 8, line 43 skipping to change at page 8, line 43
Discussion: Discussion:
The Network Devices defined in the test topology can be deployed The Network Devices defined in the test topology can be deployed
using real hardware or emulated in hardware platforms. using real hardware or emulated in hardware platforms.
Measurement Units: Measurement Units:
N/A N/A
See Also: See Also:
None None
2.2.2. Trails 2.2.2. Trials
Definition: Definition:
The number of times the test needs to be repeated. The number of times the test needs to be repeated.
Discussion: Discussion:
The test needs to be repeated for multiple iterations to obtain a The test needs to be repeated for multiple iterations to obtain a
reliable metric. It is recommended that this test SHOULD be reliable metric. It is recommended that this test SHOULD be
performed for at least 10 iterations to increase the confidence in performed for at least 10 iterations to increase the confidence in
measured result. measured result.
Measurement Units: Measurement Units:
N/A N/A
See Also: See Also:
None None
2.2.3. Trail Duration 2.2.3. Trial Duration
Definition: Definition:
Defines the duration of test trails for each iteration. Defines the duration of test trials for each iteration.
Discussion: Discussion:
Trail duration forms the basis for stop criteria for benchmarking Trial duration forms the basis for stop criteria for benchmarking
tests. Trail not completed within this time interval is considered tests. Trial not completed within this time interval is considered
as incomplete. as incomplete.
Measurement Units: Measurement Units:
seconds seconds
See Also: See Also:
None None
2.2.4. Number of Cluster nodes 2.2.4. Number of Cluster nodes
skipping to change at page 11, line 8 skipping to change at page 11, line 8
controller(s), ending with a response message from the controller(s) controller(s), ending with a response message from the controller(s)
at its Southbound interface. at its Southbound interface.
Discussion: Discussion:
For SDN to support dynamic network provisioning, it is important to For SDN to support dynamic network provisioning, it is important to
measure how quickly the controller responds to an event triggered measure how quickly the controller responds to an event triggered
from the network. The event could be any notification messages from the network. The event could be any notification messages
generated by an Network Device upon arrival of a new flow, link down generated by an Network Device upon arrival of a new flow, link down
etc. This benchmark is obtained by sending asynchronous messages etc. This benchmark is obtained by sending asynchronous messages
from every connected Network Devices one at a time for the defined from every connected Network Devices one at a time for the defined
trail duration. This test assumes that the controller will respond trial duration. This test assumes that the controller will respond
to the received asynchronous message. to the received asynchronous message.
Measurement Units: Measurement Units:
milliseconds milliseconds
See Also: See Also:
None None
2.3.1.3. Asynchronous Message Processing Rate 2.3.1.3. Asynchronous Message Processing Rate
skipping to change at page 13, line 33 skipping to change at page 13, line 33
None None
2.3.1.6. Reactive Path Provisioning Rate 2.3.1.6. Reactive Path Provisioning Rate
Definition: Definition:
The maximum number of independent paths a controller can The maximum number of independent paths a controller can
concurrently establish between source and destination nodes concurrently establish between source and destination nodes
reactively, defined as the number of paths provisioned by the reactively, defined as the number of paths provisioned by the
controller(s) at its Southbound interface for the flow provisioning controller(s) at its Southbound interface for the flow provisioning
requests received for path provisioning at its Southbound interface requests received for path provisioning at its Southbound interface
between the start of the trail and the expiry of given trail between the start of the trial and the expiry of given trial
duration duration
Discussion: Discussion:
For SDN to support agile traffic forwarding, it is important to For SDN to support agile traffic forwarding, it is important to
measure how many end-to-end flows that the controller could setup in measure how many end-to-end flows that the controller could setup in
the dataplane. This benchmark is obtained by sending traffic each the dataplane. This benchmark is obtained by sending traffic each
with unique source and destination pairs from the source Network with unique source and destination pairs from the source Network
Device and determine the number of frames received at the Device and determine the number of frames received at the
destination Network Device. destination Network Device.
skipping to change at page 14, line 12 skipping to change at page 14, line 12
See Also: See Also:
None None
2.3.1.7. Proactive Path Provisioning Rate 2.3.1.7. Proactive Path Provisioning Rate
Definition: Definition:
Measure the maximum number of independent paths a controller can Measure the maximum number of independent paths a controller can
concurrently establish between source and destination nodes concurrently establish between source and destination nodes
proactively, defined as the number of paths provisioned by the proactively, defined as the number of paths provisioned by the
controller(s) at its Southbound interface for the paths provisioned controller(s) at its Southbound interface for the paths provisioned
in its Northbound interface between the start of the trail and the in its Northbound interface between the start of the trial and the
expiry of given trail duration expiry of given trial duration
Discussion: Discussion:
For SDN to support pre-provisioning of traffic path for a larger For SDN to support pre-provisioning of traffic path for a larger
network from the application, it is important to measure how many network from the application, it is important to measure how many
end-to-end flows that the controller could setup in the dataplane. end-to-end flows that the controller could setup in the dataplane.
This benchmark is obtained by sending traffic each with unique This benchmark is obtained by sending traffic each with unique
source and destination pairs from the source Network Device. Program source and destination pairs from the source Network Device. Program
the flows on controller's northbound interface for traffic to reach the flows on controller's northbound interface for traffic to reach
from each of the unique source and destination pairs and determine from each of the unique source and destination pairs and determine
the number of frames received at the destination Network Device. the number of frames received at the destination Network Device.
skipping to change at page 21, line 8 skipping to change at page 21, line 8
| | | | | | | | | |
| | 4. Reactive Path | | | | | 4. Reactive Path | | |
| | Provisioning | | | | | Provisioning | | |
| | Rate | | | | | Rate | | |
| | | | | | | | | |
| | 5. Proactive Path | | | | | 5. Proactive Path | | |
| | Provisioning | | | | | Provisioning | | |
| | Rate | | | | | Rate | | |
| | | | | | | | | |
+------------+-------------------+---------------+-----------------+ +------------+-------------------+---------------+-----------------+
| | 1. Asynchronous |1. Control |1. Network | | | 1. Maximum |1. Control |1. Network |
| | Message Proces-| Sessions | Topology | | | Asynchronous | Sessions | Topology |
| | -sing Rate | Capacity | Change | | | Message Proces-| Capacity | Change |
| | | | Detection Time| | | -sing Rate | | Detection Time|
| | 2. Asynchronous |2. Forwarding | | | | |2. Forwarding | |
| | Message Proces-| Table |2. Exception | | | 2. Loss-Free | Table |2. Exception |
| | -sing Time | Capacity | Handling | | | Asynchronous | Capacity | Handling |
| Operational| | | | | | Message Proces-| | Detection Time|
| Operational| -sing Rate | | |
| | | |3. Denial of | | | | |3. Denial of |
| | | | Service | | | 3. Asynchronous | | Service |
| | | | Handling | | | Message Proces-| | Handling |
| | | | | | | -sing Time | | |
| | | |4. Network Re- | | | | |4. Network Re- |
| | | | Provisioning | | | | | Provisioning |
| | | | Time | | | | | Time |
| | | | | | | | | |
+------------+-------------------+---------------+-----------------+ +------------+-------------------+---------------+-----------------+
| | | | | | | | | |
| Tear Down | | |1. Controller | | Tear Down | | |1. Controller |
| | | | Failover Time | | | | | Failover Time |
+------------+-------------------+---------------+-----------------+ +------------+-------------------+---------------+-----------------+
skipping to change at page 22, line 8 skipping to change at page 22, line 8
"Terminology for Benchmarking Network-layer Traffic "Terminology for Benchmarking Network-layer Traffic
Control Mechanisms", RFC 4689, October 2006. Control Mechanisms", RFC 4689, October 2006.
[RFC2330] V. Paxson, G. Almes, J. Mahdavi, M. Mathis, [RFC2330] V. Paxson, G. Almes, J. Mahdavi, M. Mathis,
"Framework for IP Performance Metrics", RFC 2330, "Framework for IP Performance Metrics", RFC 2330,
May 1998. May 1998.
[I-D.sdn-controller-benchmark-meth] Bhuvaneswaran.V, Anton Basil, [I-D.sdn-controller-benchmark-meth] Bhuvaneswaran.V, Anton Basil,
Mark.T, Vishwas Manral, Sarah Banks "Benchmarking Mark.T, Vishwas Manral, Sarah Banks "Benchmarking
Methodology for SDN Controller Performance", Methodology for SDN Controller Performance",
draft-ietf-bmwg-sdn-controller-benchmark-meth-05 draft-ietf-bmwg-sdn-controller-benchmark-meth-06
(Work in progress), October 1, 2017 (Work in progress), November 16, 2017
5.2. Informative References 5.2. Informative References
[OpenFlow Switch Specification] ONF,"OpenFlow Switch Specification" [OpenFlow Switch Specification] ONF,"OpenFlow Switch Specification"
Version 1.4.0 (Wire Protocol 0x05), October 14, 2013. Version 1.4.0 (Wire Protocol 0x05), October 14, 2013.
6. IANA Considerations 6. IANA Considerations
This document does not have any IANA requests. This document does not have any IANA requests.
 End of changes. 15 change blocks. 
28 lines changed or deleted 29 lines changed or added

This html diff was produced by rfcdiff 1.46. The latest version is available from http://tools.ietf.org/tools/rfcdiff/