< draft-ietf-bmwg-dcbench-terminology-12.txt   draft-ietf-bmwg-dcbench-terminology-13.txt >
Internet Engineering Task Force L. Avramov Internet Engineering Task Force L. Avramov
INTERNET-DRAFT, Intended status: Informational Google INTERNET-DRAFT, Intended status: Informational Google
Expires: December 17,2017 J. Rapp Expires: December 17,2017 J. Rapp
June 15, 2017 VMware June 15, 2017 VMware
Data Center Benchmarking Terminology Data Center Benchmarking Terminology
draft-ietf-bmwg-dcbench-terminology-12 draft-ietf-bmwg-dcbench-terminology-13
Abstract Abstract
The purpose of this informational document is to establish definitions The purpose of this informational document is to establish definitions
and describe measurement techniques for data center benchmarking, as and describe measurement techniques for data center benchmarking, as
well as it is to introduce new terminologies applicable to data center well as it is to introduce new terminologies applicable to data center
performance evaluations. The purpose of this document is not to define performance evaluations. The purpose of this document is not to define
the test methodology, but rather establish the important concepts for the test methodology, but rather establish the important concepts for
benchmarking network switches and routers in the data center. The benchmarking network switches and routers in the data center. Many of
terminologies are not only data center specific and can be seen as these terms and methods may be applicable beyond this publication's
updates to [RFC1242], [RFC2432], [RFC2544], [RFC2889] and [RFC3918], scope as the technologies originally applied in the data center are
without the intent to cover the all use cases outside of data center. deployed elsewhere.
Status of this Memo Status of this Memo
This Internet-Draft is submitted in full conformance with the provisions This Internet-Draft is submitted in full conformance with the provisions
of BCP 78 and BCP 79. of BCP 78 and BCP 79.
Internet-Drafts are working documents of the Internet Engineering Task Internet-Drafts are working documents of the Internet Engineering Task
Force (IETF). Note that other groups may also distribute working Force (IETF). Note that other groups may also distribute working
documents as Internet-Drafts. The list of current Internet-Drafts is at documents as Internet-Drafts. The list of current Internet-Drafts is at
http://datatracker.ietf.org/drafts/current. http://datatracker.ietf.org/drafts/current.
skipping to change at page 2, line 18 skipping to change at page 2, line 17
the Trust Legal Provisions and are provided without warranty as the Trust Legal Provisions and are provided without warranty as
described in the Simplified BSD License. described in the Simplified BSD License.
Table of Contents Table of Contents
1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 3 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 3
1.1. Requirements Language . . . . . . . . . . . . . . . . . . 4 1.1. Requirements Language . . . . . . . . . . . . . . . . . . 4
1.2. Definition format . . . . . . . . . . . . . . . . . . . . . 4 1.2. Definition format . . . . . . . . . . . . . . . . . . . . . 4
2. Latency . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 2. Latency . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
2.1. Definition . . . . . . . . . . . . . . . . . . . . . . . . 4 2.1. Definition . . . . . . . . . . . . . . . . . . . . . . . . 4
2.2 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . 5 2.2 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . 6
2.3 Measurement Units . . . . . . . . . . . . . . . . . . . . . 6 2.3 Measurement Units . . . . . . . . . . . . . . . . . . . . . 6
3 Jitter . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 3 Jitter . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
3.1 Definition . . . . . . . . . . . . . . . . . . . . . . . . . 6 3.1 Definition . . . . . . . . . . . . . . . . . . . . . . . . . 6
3.2 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . 6 3.2 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . 7
3.3 Measurement Units . . . . . . . . . . . . . . . . . . . . . 7 3.3 Measurement Units . . . . . . . . . . . . . . . . . . . . . 7
4 Physical Layer Calibration . . . . . . . . . . . . . . . . . . . 7 4 Physical Layer Calibration . . . . . . . . . . . . . . . . . . . 7
4.1 Definition . . . . . . . . . . . . . . . . . . . . . . . . . 7 4.1 Definition . . . . . . . . . . . . . . . . . . . . . . . . . 7
4.2 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . 7 4.2 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . 8
4.3 Measurement Units . . . . . . . . . . . . . . . . . . . . . 7 4.3 Measurement Units . . . . . . . . . . . . . . . . . . . . . 8
5 Line rate . . . . . . . . . . . . . . . . . . . . . . . . . . . 8 5 Line rate . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
5.1 Definition . . . . . . . . . . . . . . . . . . . . . . . . . 8 5.1 Definition . . . . . . . . . . . . . . . . . . . . . . . . . 8
5.2 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . 8 5.2 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . 9
5.3 Measurement Units . . . . . . . . . . . . . . . . . . . . . 9 5.3 Measurement Units . . . . . . . . . . . . . . . . . . . . . 10
6 Buffering . . . . . . . . . . . . . . . . . . . . . . . . . . . 10 6 Buffering . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
6.1 Buffer . . . . . . . . . . . . . . . . . . . . . . . . . . . 10 6.1 Buffer . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
6.1.1 Definition . . . . . . . . . . . . . . . . . . . . . . . 10 6.1.1 Definition . . . . . . . . . . . . . . . . . . . . . . . 11
6.1.2 Discussion . . . . . . . . . . . . . . . . . . . . . . . 12 6.1.2 Discussion . . . . . . . . . . . . . . . . . . . . . . . 12
6.1.3 Measurement Units . . . . . . . . . . . . . . . . . . . 12 6.1.3 Measurement Units . . . . . . . . . . . . . . . . . . . 12
6.2 Incast . . . . . . . . . . . . . . . . . . . . . . . . . . . 12 6.2 Incast . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
6.2.1 Definition . . . . . . . . . . . . . . . . . . . . . . . 12 6.2.1 Definition . . . . . . . . . . . . . . . . . . . . . . . 13
6.2.2 Discussion . . . . . . . . . . . . . . . . . . . . . . . 13 6.2.2 Discussion . . . . . . . . . . . . . . . . . . . . . . . 14
6.2.3 Measurement Units . . . . . . . . . . . . . . . . . . . 13 6.2.3 Measurement Units . . . . . . . . . . . . . . . . . . . 14
7 Application Throughput: Data Center Goodput . . . . . . . . . . 14 7 Application Throughput: Data Center Goodput . . . . . . . . . . 14
7.1. Definition . . . . . . . . . . . . . . . . . . . . . . . . 14 7.1. Definition . . . . . . . . . . . . . . . . . . . . . . . . 14
7.2. Discussion . . . . . . . . . . . . . . . . . . . . . . . . 14 7.2. Discussion . . . . . . . . . . . . . . . . . . . . . . . . 14
7.3. Measurement Units . . . . . . . . . . . . . . . . . . . . . 14 7.3. Measurement Units . . . . . . . . . . . . . . . . . . . . . 14
8. Security Considerations . . . . . . . . . . . . . . . . . . . 15 8. Security Considerations . . . . . . . . . . . . . . . . . . . 15
9. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 15 9. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 16
10. References . . . . . . . . . . . . . . . . . . . . . . . . . 15 10. References . . . . . . . . . . . . . . . . . . . . . . . . . 16
10.1. Normative References . . . . . . . . . . . . . . . . . . 15 10.1. Normative References . . . . . . . . . . . . . . . . . . 16
10.2. Informative References . . . . . . . . . . . . . . . . . 16 10.2. Informative References . . . . . . . . . . . . . . . . . 16
10.3. Acknowledgments . . . . . . . . . . . . . . . . . . . . . 16 10.3. Acknowledgments . . . . . . . . . . . . . . . . . . . . . 17
Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 17 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 17
1. Introduction 1. Introduction
Traffic patterns in the data center are not uniform and are Traffic patterns in the data center are not uniform and are
constantly changing. They are dictated by the nature and variety of constantly changing. They are dictated by the nature and variety of
applications utilized in the data center. It can be largely east-west applications utilized in the data center. It can be largely east-west
traffic flows (server to server inside the data center) in one data traffic flows (server to server inside the data center) in one data
center and north-south (outside of the data center to server) in center and north-south (outside of the data center to server) in
another, while some may combine both. Traffic patterns can be bursty another, while some may combine both. Traffic patterns can be bursty
skipping to change at page 4, line 34 skipping to change at page 4, line 34
2.1. Definition 2.1. Definition
Latency is a the amount of time it takes a frame to transit the DUT. Latency is a the amount of time it takes a frame to transit the DUT.
Latency is measured in units of time (seconds, milliseconds, Latency is measured in units of time (seconds, milliseconds,
microseconds and so on). The purpose of measuring latency is to microseconds and so on). The purpose of measuring latency is to
understand the impact of adding a device in the communication path. understand the impact of adding a device in the communication path.
The Latency interval can be assessed between different combinations The Latency interval can be assessed between different combinations
of events, regardless of the type of switching device (bit forwarding of events, regardless of the type of switching device (bit forwarding
aka cut-through or store-and-forward type of device) aka cut-through, or store-and-forward type of device). [RFC1242]
defined Latency differently for each of these types of devices.
Traditionally the latency measurement definitions are: Traditionally the latency measurement definitions are:
FILO (First In Last Out) The time interval starting when the end of FILO (First In Last Out)
the first bit of the input frame reaches the input port and ending
when the last bit of the output frame is seen on the output port
FIFO (First In First Out) The time interval starting when the end of The time interval starting when the end of the first bit of the input
the first bit of the input frame reaches the input port and ending frame reaches the input port and ending when the last bit of the
when the start of the first bit of the output frame is seen on the output frame is seen on the output port.
output port
LILO (Last In Last Out) The time interval starting when the last bit FIFO (First In First Out):
of the input frame reaches the input port and the last bit of the
output frame is seen on the output port The time interval starting when the end of the first bit of the input
LIFO (Last In First Out) The time interval starting when the last bit frame reaches the input port and ending when the start of the first
of the input frame reaches the input port and ending when the first bit of the output frame is seen on the output port. [RFC1242] Latency
bit of the output frame is seen on the output port. for bit forwarding devices uses these events.
LILO (Last In Last Out):
The time interval starting when the last bit of the input frame
reaches the input port and the last bit of the output frame is seen
on the output port.
LIFO (Last In First Out):
The time interval starting when the last bit of the input frame
reaches the input port and ending when the first bit of the output
frame is seen on the output port. [RFC1242] Latency for bit
forwarding devices uses these events.
Another possibility to summarize the four different definitions above Another possibility to summarize the four different definitions above
is to refer to the bit position as they normally occur: Input to is to refer to the bit position as they normally occur: Input to
output. output.
FILO is FL (First bit Last bit). FIFO is FF (First bit First bit). FILO is FL (First bit Last bit). FIFO is FF (First bit First bit).
LILO is LL (Last bit Last bit). LIFO is LF (Last bit First bit). LILO is LL (Last bit Last bit). LIFO is LF (Last bit First bit).
This definition explained in this section in context of data center This definition explained in this section in context of data center
switching benchmarking is in lieu of the previous definition of switching benchmarking is in lieu of the previous definition of
skipping to change at page 5, line 28 skipping to change at page 5, line 39
For store and forward devices: The time interval starting when the For store and forward devices: The time interval starting when the
last bit of the input frame reaches the input port and ending when last bit of the input frame reaches the input port and ending when
the first bit of the output frame is seen on the output port. the first bit of the output frame is seen on the output port.
For bit forwarding devices: The time interval starting when the end For bit forwarding devices: The time interval starting when the end
of the first bit of the input frame reaches the input port and ending of the first bit of the input frame reaches the input port and ending
when the start of the first bit of the output frame is seen on the when the start of the first bit of the output frame is seen on the
output port. output port.
To accommodate both types of network devices and hybrids of the two
types that have emerged, switch Latency measurements made according
to this document MUST be measured with the FILO events. FILO will
include the latency of the switch and the latency of the frame as
well as the serialization delay. It is a picture of the 'whole'
latency going through the DUT. For applications which are latency
sensitive and can function with initial bytes of the frame, FIFO (or
RFC 1242 Latency for bit forwarding devices) MAY be used. In all
cases, the event combination used in Latency measurement MUST be
reported.
2.2 Discussion 2.2 Discussion
FILO is the most important measuring definition. All type of switches As mentioned in section 2.1, FILO is the most important measuring
MUST be measured with the FILO mechanism: FILO will include the definition.
latency of the switch and the latency of the frame as well as the
serialization delay. It is a picture of the 'whole' latency going
through the DUT. For applications, which are latency sensitive and
can function with initial bytes of the frame, FIFO MAY be an
additional type of measuring to supplement FILO.
Not all DUTs are exclusively cut-through or store-and-forward. Data Not all DUTs are exclusively cut-through or store-and-forward. Data
Center DUTs are frequently store-and-forward for smaller packet sizes Center DUTs are frequently store-and-forward for smaller packet sizes
and then adopting a cut-through behavior. The change of behavior and then adopting a cut-through behavior. The change of behavior
happens at specific larger packet sizes. The value of the packet size happens at specific larger packet sizes. The value of the packet size
for the behavior to change MAY be configurable depending on the DUT for the behavior to change MAY be configurable depending on the DUT
manufacturer. FILO covers all scenarios: Store-and-forward or cut- manufacturer. FILO covers all scenarios: Store-and-forward or cut-
through. The threshold of behavior change does not matter for through. The threshold of behavior change does not matter for
benchmarking since FILO covers both possible scenarios. benchmarking since FILO covers both possible scenarios.
skipping to change at page 8, line 25 skipping to change at page 8, line 42
The transmit timing, or maximum transmitted data rate is controlled The transmit timing, or maximum transmitted data rate is controlled
by the "transmit clock" in the DUT. The receive timing (maximum by the "transmit clock" in the DUT. The receive timing (maximum
ingress data rate) is derived from the transmit clock of the ingress data rate) is derived from the transmit clock of the
connected interface. connected interface.
The line rate or physical layer frame rate is the maximum capacity to The line rate or physical layer frame rate is the maximum capacity to
send frames of a specific size at the transmit clock frequency of the send frames of a specific size at the transmit clock frequency of the
DUT. DUT.
The term "port capacity" defines the maximum speed capability for the The term "nominal value of Line Rate" defines the maximum speed
given port; for example 1GE, 10GE, 40GE, 100GE etc. capability for the given port; for example 1GE, 10GE, 40GE, 100GE
etc.
The frequency ("clock rate") of the transmit clock in any two The frequency ("clock rate") of the transmit clock in any two
connected interfaces will never be precisely the same; therefore, a connected interfaces will never be precisely the same; therefore, a
tolerance is needed. This will be expressed by Parts Per Million tolerance is needed. This will be expressed by Parts Per Million
(PPM) value. The IEEE standards allow a specific +/- variance in the (PPM) value. The IEEE standards allow a specific +/- variance in the
transmit clock rate, and Ethernet is designed to allow for small, transmit clock rate, and Ethernet is designed to allow for small,
normal variations between the two clock rates. This results in a normal variations between the two clock rates. This results in a
tolerance of the line rate value when traffic is generated from a tolerance of the line rate value when traffic is generated from a
testing equipment to a DUT. testing equipment to a DUT.
 End of changes. 16 change blocks. 
46 lines changed or deleted 63 lines changed or added

This html diff was produced by rfcdiff 1.46. The latest version is available from http://tools.ietf.org/tools/rfcdiff/