draft-ietf-bmwg-traffic-management-02.txt   draft-ietf-bmwg-traffic-management-03.txt 
Network Working Group B. Constantine Network Working Group B. Constantine
Internet Draft JDSU Internet Draft JDSU
Intended status: Informational R. Krishnan Intended status: Informational R. Krishnan
Expires: July 2015 Brocade Communications Expires: August 2015 Brocade Communications
January 26, 2015 February 23, 2015
Traffic Management Benchmarking Traffic Management Benchmarking
draft-ietf-bmwg-traffic-management-02.txt draft-ietf-bmwg-traffic-management-03.txt
Status of this Memo Status of this Memo
This Internet-Draft is submitted in full conformance with the This Internet-Draft is submitted in full conformance with the
provisions of BCP 78 and BCP 79. provisions of BCP 78 and BCP 79.
Internet-Drafts are working documents of the Internet Engineering Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF). Note that other groups may also distribute Task Force (IETF). Note that other groups may also distribute
working documents as Internet-Drafts. The list of current Internet- working documents as Internet-Drafts. The list of current Internet-
Drafts is at http://datatracker.ietf.org/drafts/current/. Drafts is at http://datatracker.ietf.org/drafts/current/.
Internet-Drafts are draft documents valid for a maximum of six months Internet-Drafts are draft documents valid for a maximum of six months
and may be updated, replaced, or obsoleted by other documents at any and may be updated, replaced, or obsoleted by other documents at any
time. It is inappropriate to use Internet-Drafts as reference time. It is inappropriate to use Internet-Drafts as reference
material or to cite them other than as "work in progress." material or to cite them other than as "work in progress."
This Internet-Draft will expire on July 26, 2015. This Internet-Draft will expire on August 23, 2015.
Copyright Notice Copyright Notice
Copyright (c) 2015 IETF Trust and the persons identified as the Copyright (c) 2015 IETF Trust and the persons identified as the
document authors. All rights reserved. document authors. All rights reserved.
This document is subject to BCP 78 and the IETF Trust's Legal This document is subject to BCP 78 and the IETF Trust's Legal
Provisions Relating to IETF Documents Provisions Relating to IETF Documents
(http://trustee.ietf.org/license-info) in effect on the date of (http://trustee.ietf.org/license-info) in effect on the date of
publication of this document. Please review these documents publication of this document. Please review these documents
skipping to change at page 5, line 22 skipping to change at page 5, line 22
|Interface | |Ingress Actions | |Egress Actions| |Interface | |Interface | |Ingress Actions | |Egress Actions| |Interface |
|Input | |(classification,| |(scheduling, | |Output | |Input | |(classification,| |(scheduling, | |Output |
|Queues | | marking, | | shaping, | |Queues | |Queues | | marking, | | shaping, | |Queues |
| |-->| policing or |-->| active queue |-->| | | |-->| policing or |-->| active queue |-->| |
| | | shaping) | | management | | | | | | shaping) | | management | | |
| | | | | remarking) | | | | | | | | remarking) | | |
|----------| |----------------| |--------------| |----------| |----------| |----------------| |--------------| |----------|
Figure 1: Generic Traffic Management capabilities of a Network Device Figure 1: Generic Traffic Management capabilities of a Network Device
Ingress actions such as classification are defined in RFC 4689 Ingress actions such as classification are defined in [RFC4689]
[RFC4689] and include IP addresses, port numbers, DSCP, etc. In and include IP addresses, port numbers, DSCP, etc. In terms of
terms of marking, RFC 2697 [RFC2697] and RFC 2698 [RFC2698] define marking, [RFC2697] and [RFC2698] define a single rate and dual rate,
a single rate and dual rate, three color marker, respectively. three color marker, respectively.
The Metro Ethernet Forum (MEF) specifies policing and shaping in The Metro Ethernet Forum (MEF) specifies policing and shaping in
terms of Ingress and Egress Subscriber/Provider Conditioning terms of Ingress and Egress Subscriber/Provider Conditioning
Functions in MEF12.1 [MEF-12.1]; Ingress and Bandwidth Profile Functions in MEF12.1 [MEF-12.1]; Ingress and Bandwidth Profile
attributes in MEF10.2 [MEF-10.2] and MEF 26 [MEF-26]. attributes in MEF10.2 [MEF-10.2] and MEF 26 [MEF-26].
1.2 Lab Configuration and Testing Overview 1.2 Lab Configuration and Testing Overview
The following is the description of the lab set-up for the traffic The following is the description of the lab set-up for the traffic
management tests: management tests:
skipping to change at page 7, line 9 skipping to change at page 7, line 9
the NDE, maximum offered load should be tested against the following the NDE, maximum offered load should be tested against the following
frame sizes: 128, 256, 512, 768, 1024, 1500,and 9600 bytes. The delay frame sizes: 128, 256, 512, 768, 1024, 1500,and 9600 bytes. The delay
accuracy at each of these packet sizes can then be used to calibrate accuracy at each of these packet sizes can then be used to calibrate
the range of expected Bandwidth Delay Product (BDP) for the TCP the range of expected Bandwidth Delay Product (BDP) for the TCP
stateful tests. stateful tests.
2. Conventions used in this document 2. Conventions used in this document
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
"SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
document are to be interpreted as described in RFC 2119 [RFC2119]. document are to be interpreted as described in [RFC2119].
The following acronyms are used: The following acronyms are used:
AQM: Active Queue Management AQM: Active Queue Management
BB: Bottleneck Bandwidth BB: Bottleneck Bandwidth
BDP: Bandwidth Delay Product BDP: Bandwidth Delay Product
BSA: Burst Size Achieved BSA: Burst Size Achieved
skipping to change at page 9, line 32 skipping to change at page 9, line 32
network device is stateful in nature (i.e. firewall, etc.), network device is stateful in nature (i.e. firewall, etc.),
stateful test pattern traffic is important to test along with stateful test pattern traffic is important to test along with
stateless, UDP traffic in specific test scenarios (i.e. stateless, UDP traffic in specific test scenarios (i.e.
applications using TCP transport and UDP VoIP, etc.) applications using TCP transport and UDP VoIP, etc.)
As mentioned earlier in the document, repeatability of test results As mentioned earlier in the document, repeatability of test results
is critical, especially considering the nature of stateful TCP is critical, especially considering the nature of stateful TCP
traffic. To this end, the stateful tests will use TCP test patterns traffic. To this end, the stateful tests will use TCP test patterns
to emulate applications. This framework also provides guidelines to emulate applications. This framework also provides guidelines
for application modeling and open source tools to achieve the for application modeling and open source tools to achieve the
repeatable stimulus. And finally, TCP metrics from RFC 6349 repeatable stimulus. And finally, TCP metrics from [RFC6349] MUST
[RFC6349] MUST be measured for each stateful test and provide the be measured for each stateful test and provide the means to compare
means to compare each repeated test. each repeated test.
4. Traffic Benchmarking Metrics 4. Traffic Benchmarking Metrics
The metrics to be measured during the benchmarks are divided into two The metrics to be measured during the benchmarks are divided into two
(2) sections: packet layer metrics used for the stateless traffic (2) sections: packet layer metrics used for the stateless traffic
testing and TCP layer metrics used for the stateful traffic testing and TCP layer metrics used for the stateful traffic
testing. testing.
4.1. Metrics for Stateless Traffic Tests 4.1. Metrics for Stateless Traffic Tests
Stateless traffic measurements require that sequence number and Stateless traffic measurements require that sequence number and
time-stamp be inserted into the payload for lost packet analysis. time-stamp be inserted into the payload for lost packet analysis.
Delay analysis may be achieved by insertion of timestamps directly Delay analysis may be achieved by insertion of timestamps directly
into the packets or timestamps stored elsewhere (packet captures). into the packets or timestamps stored elsewhere (packet captures).
This framework does not specify the packet format to carry sequence This framework does not specify the packet format to carry sequence
number or timing information. number or timing information.
However, RFC 4737 [RFC4737] and RFC 4689 provide recommendations However,[RFC4737] and [RFC4689] provide recommendations
for sequence tracking along with definitions of in-sequence and for sequence tracking along with definitions of in-sequence and
out-of-order packets. out-of-order packets.
The following are the metrics that MUST be measured during the The following are the metrics that MUST be measured during the
stateless traffic benchmarking components of the tests: stateless traffic benchmarking components of the tests:
- Burst Size Achieved (BSA): for the traffic policing and network - Burst Size Achieved (BSA): for the traffic policing and network
queue tests, the tester will be configured to send bursts to test queue tests, the tester will be configured to send bursts to test
either the Committed Burst Size (CBS) or Excess Burst Size (EBS) of either the Committed Burst Size (CBS) or Excess Burst Size (EBS) of
a policer or the queue / buffer size configured in the DUT. The a policer or the queue / buffer size configured in the DUT. The
skipping to change at page 10, line 31 skipping to change at page 10, line 31
- Lost Packets(LP): For all traffic management tests, the tester will - Lost Packets(LP): For all traffic management tests, the tester will
transmit the test packets into the DUT ingress port and the number of transmit the test packets into the DUT ingress port and the number of
packets received at the egress port will be measured. The difference packets received at the egress port will be measured. The difference
between packets transmitted into the ingress port and received at the between packets transmitted into the ingress port and received at the
egress port is the number of lost packets as measured at the egress egress port is the number of lost packets as measured at the egress
port. These packets must have unique identifiers such that only the port. These packets must have unique identifiers such that only the
test packets are measured. For cases where multiple flows are test packets are measured. For cases where multiple flows are
transmitted from ingress to egress port (e.g. IP conversations), each transmitted from ingress to egress port (e.g. IP conversations), each
flow must have sequence numbers within the test packets stream. flow must have sequence numbers within the test packets stream.
RFC 4737 and RFC 2680 [RFC2680] describe the need to establish the [RFC6703] and [RFC2680] describe the need to establish the
time threshold to wait before a packet is declared as lost, and this time threshold to wait before a packet is declared as lost, and this
threshold MUST be reported with the results. This metric shall be threshold MUST be reported with the results. This metric shall be
reported as an integer number which cannot be negative. reported as an integer number which cannot be negative. (see:
http://tools.ietf.org/html/rfc6703#section-4.1)
- Out of Sequence (OOS): in additions to the LP metric, the test - Out of Order (OOO): in additions to the LP metric, the test
packets must be monitored for sequence and the out-of-sequence (OOS) packets must be monitored for sequence. [RFC4689] defines the
packets. RFC 4689 defines the general function of sequence tracking, general function of sequence tracking, as well as definitions
as well as definitions for in-sequence and out-of-order packets. for in-sequence and out-of-order packets. Out-of-order packets
Out-of-order packets will be counted per RFC 4737 and RFC 2680 will be counted per [RFC4737]. This metric shall be reported as
[RFC2680]. This metric shall be reported as an integer number which an integer number which cannot be negative.
cannot be negative.
- Packet Delay (PD): the Packet Delay metric is the difference - Packet Delay (PD): the Packet Delay metric is the difference
between the timestamp of the received egress port packets and the between the timestamp of the received egress port packets and the
packets transmitted into the ingress port and specified in RFC 1242 packets transmitted into the ingress port and specified in [RFC1242].
[RFC1242]. The transmitting host and receiving host time must be in The transmitting host and receiving host time must be in
time sync using NTP , GPS, etc. This metric shall be reported as an time sync using NTP , GPS, etc. This metric SHALL be reported as a
real number of seconds which cannot be negative, which usually real number of seconds, where a negative measurement usually
indicates a time synchronization problem. indicates a time synchronization problem between test devices.
- Packet Delay Variation (PDV): the Packet Delay Variation metric is - Packet Delay Variation (PDV): the Packet Delay Variation metric is
the variation between the timestamp of the received egress port the variation between the timestamp of the received egress port
packets and specified in RFC 5481 [RFC5481]. Note that per RFC 5481, packets and specified in [RFC5481]. Note that per [RFC5481],
this PDV is the variation of one-way delay across many packets in this PDV is the variation of one-way delay across many packets in
the traffic flow. Per the measurement formulat in RFC 5481, select the traffic flow. Per the measurement formula in [RFC5481], select
the high percentile of 99% and units of measure will be a real the high percentile of 99% and units of measure will be a real
number of seconds (negative is not possible for PDV and would number of seconds (negative is not possible for PDV and would
indicate a measurement error). indicate a measurement error).
- Shaper Rate (SR): the Shaper Rate is only applicable to the - Shaper Rate (SR): The SR represents the average DUT output
traffic shaping tests. The SR represents the average egress output rate (bps) over the test interval. The Shaper Rate is only
rate (bps) over the test interval. applicable to the traffic shaping tests.
- Shaper Burst Bytes (SBB): the Shaper Burst Bytes is only applicable - Shaper Burst Bytes (SBB): A traffic shaper will emit packets in
to the traffic shaping tests. A traffic shaper will emit packets in different size "trains"; these are frames "back-to-back", respect
different size "trains" (bytes back-to-back). This metric the mandatory inter-frame gap. This metric characterizes the method
characterizes the method by which the shaper emits traffic. Some by which the shaper emits traffic. Some shapers transmit larger
shapers transmit larger bursts per interval, and a burst of 1 packet bursts per interval, and a burst of 1 packet would apply to the
would apply to the extreme case of a shaper sending a CBR stream of extreme case of a shaper sending a CBR stream of single packets.
single packets. This metric shall be reported in units of bytes, This metric SHALL be reported in units of bytes, KBytes, or MBytes.
KBytes, or MBytes. Shaper Burst Bytes is only applicable to thetraffic shaping tests.
- Shaper Burst Interval(SBI): the interval is only applicable to the - Shaper Burst Interval(SBI): the SBI is the time between shaper
traffic shaping tests and again is the time between shaper emitted emitted bursts and is measured at the DUT egress port. This metric
bursts. This metric shall be reported as an real number of shall be reported as an real number of seconds. Shaper Burst
seconds which cannot be negative, which usually indicates a time Interval is only applicable to the traffic shaping tests,
synchronization problem.
4.2. Metrics for Stateful Traffic Tests 4.2. Metrics for Stateful Traffic Tests
The stateful metrics will be based on RFC 6349 TCP metrics and MUST The stateful metrics will be based on [RFC6349] TCP metrics and
include: MUST include:
- TCP Test Pattern Execution Time (TTPET): RFC 6349 defined the TCP - TCP Test Pattern Execution Time (TTPET): [RFC6349] defined the TCP
Transfer Time for bulk transfers, which is simply the measured time Transfer Time for bulk transfers, which is simply the measured time
to transfer bytes across single or concurrent TCP connections. The to transfer bytes across single or concurrent TCP connections. The
TCP test patterns used in traffic management tests will include bulk TCP test patterns used in traffic management tests will include bulk
transfer and interactive applications. The interactive patterns transfer and interactive applications. The interactive patterns
include instances such as HTTP business applications, database include instances such as HTTP business applications, database
applications, etc. The TTPET will be the measure of the time for a applications, etc. The TTPET will be the measure of the time for a
single execution of a TCP Test Pattern (TTP). Average, minimum, and single execution of a TCP Test Pattern (TTP). Average, minimum, and
maximum times will be measured or calculated and expressed as a real maximum times will be measured or calculated and expressed as a real
number of seconds. number of seconds.
skipping to change at page 11, line 11 skipping to change at page 11, line 11
Transmitted Bytes are the total number of TCP Bytes to be transmitted Transmitted Bytes are the total number of TCP Bytes to be transmitted
including the original and the retransmitted Bytes. These including the original and the retransmitted Bytes. These
retransmitted bytes should be recorded from the sender's TCP/IP stack retransmitted bytes should be recorded from the sender's TCP/IP stack
perspective, to avoid any misinterpretation that a reordered packet perspective, to avoid any misinterpretation that a reordered packet
is a retransmitted packet (as may be the case with packet decode is a retransmitted packet (as may be the case with packet decode
interpretation). interpretation).
- Buffer Delay: represents the increase in RTT during a TCP test - Buffer Delay: represents the increase in RTT during a TCP test
versus the baseline DUT RTT (non congested, inherent latency). RTT versus the baseline DUT RTT (non congested, inherent latency). RTT
and the technique to measure RTT (average versus baseline) are and the technique to measure RTT (average versus baseline) are
defined in RFC 6349. Referencing RFC 6349, the average RTT is defined in [RFC6349]. Referencing [RFC6349], the average RTT is
derived from the total of all measured RTTs during the actual test derived from the total of all measured RTTs during the actual test
sampled at every second divided by the test duration in seconds. sampled at every second divided by the test duration in seconds.
Total RTTs during transfer Total RTTs during transfer
Average RTT during transfer = ----------------------------- Average RTT during transfer = -----------------------------
Transfer duration in seconds Transfer duration in seconds
Average RTT during Transfer - Baseline RTT Average RTT during Transfer - Baseline RTT
Buffer Delay % = ------------------------------------------ X 100 Buffer Delay % = ------------------------------------------ X 100
Baseline RTT Baseline RTT
Note that even though this was not explicitly stated in RFC 6349, Note that even though this was not explicitly stated in [RFC6349],
retransmitted packets should not be used in RTT measurements. retransmitted packets should not be used in RTT measurements.
Also, the test results should record the average RTT in millisecond Also, the test results should record the average RTT in millisecond
across the entire test duration and number of samples. across the entire test duration and number of samples.
5. Tester Capabilities 5. Tester Capabilities
The testing capabilities of the traffic management test environment The testing capabilities of the traffic management test environment
are divided into two (2) sections: stateless traffic testing and are divided into two (2) sections: stateless traffic testing and
stateful traffic testing stateful traffic testing
5.1. Stateless Test Traffic Generation 5.1. Stateless Test Traffic Generation
The test device MUST be capable of generating traffic at up to the The test device MUST be capable of generating traffic at up to the
link speed of the DUT. The test device must be calibrated to verify link speed of the DUT. The test device must be calibrated to verify
that it will not drop any packets. The test device's inherent PD and that it will not drop any packets. The test device's inherent PD and
PDV must also be calibrated and subtracted from the PD and PDV PDV must also be calibrated and subtracted from the PD and PDV
metrics. The test device must support the encapsulation to be metrics. The test device must support the encapsulation to be
tested such as IEEE 802.1Q VLAN, IEEE 802.1ad Q-in-Q, Multiprotocol tested such as IEEE 802.1Q VLAN, IEEE 802.1ad Q-in-Q, Multiprotocol
Label Switching (MPLS), etc. Also, the test device must allow Label Switching (MPLS), etc. Also, the test device must allow
control of the classification techniques defined in RFC 4689 control of the classification techniques defined in [RFC4689]
(i.e. IP address, DSCP, TOS, etc classification). (i.e. IP address, DSCP, TOS, etc classification).
The open source tool "iperf" can be used to generate stateless UDP The open source tool "iperf" can be used to generate stateless UDP
traffic and is discussed in Appendix A. Since iperf is a software traffic and is discussed in Appendix A. Since iperf is a software
based tool, there will be performance limitations at higher link based tool, there will be performance limitations at higher link
speeds (e.g. GigE, 10 GigE, etc.). Careful calibration of any test speeds (e.g. GigE, 10 GigE, etc.). Careful calibration of any test
environment using iperf is important. At higher link speeds, it is environment using iperf is important. At higher link speeds, it is
recommended to use hardware based packet test equipment. recommended to use hardware based packet test equipment.
5.1.1 Burst Hunt with Stateless Traffic 5.1.1 Burst Hunt with Stateless Traffic
skipping to change at page 12, line 26 skipping to change at page 12, line 26
maximum burst supported by the DUT is discovered. The recommended maximum burst supported by the DUT is discovered. The recommended
granularity of the incremental burst size increase is 1 KB. granularity of the incremental burst size increase is 1 KB.
Optionally for a policer function and if the burst size passes, the Optionally for a policer function and if the burst size passes, the
burst should be increased by increments of 1 KB to verify that the burst should be increased by increments of 1 KB to verify that the
policer is truly configured properly (or enabled at all). policer is truly configured properly (or enabled at all).
5.2. Stateful Test Pattern Generation 5.2. Stateful Test Pattern Generation
The TCP test host will have many of the same attributes as the TCP The TCP test host will have many of the same attributes as the TCP
test host defined in RFC 6349. The TCP test device may be a standard test host defined in [RFC6349]. The TCP test device may be a
computer or a dedicated communications test instrument. In both standard computer or a dedicated communications test instrument. In
cases, it must be capable of emulating both a client and a server. both cases, it must be capable of emulating both a client and a
server.
For any test using stateful TCP test traffic, the Network Delay For any test using stateful TCP test traffic, the Network Delay
Emulator (NDE function from the lab set-up diagram) must be used in Emulator (NDE function from the lab set-up diagram) must be used in
order to provide a meaningful BDP. As referenced in section 2, the order to provide a meaningful BDP. As referenced in section 2, the
target traffic rate and configured RTT MUST be verified independently target traffic rate and configured RTT MUST be verified independently
using just the NDE for all stateful tests (to ensure the NDE can using just the NDE for all stateful tests (to ensure the NDE can
delay without loss). delay without loss).
The TCP test host MUST be capable to generate and receive stateful The TCP test host MUST be capable to generate and receive stateful
TCP test traffic at the full link speed of the DUT. As a general TCP test traffic at the full link speed of the DUT. As a general
skipping to change at page 12, line 55 skipping to change at page 12, line 56
BDP for bulk transfer TCP test application traffic. BDP for bulk transfer TCP test application traffic.
Measuring RTT and retransmissions per connection will generally Measuring RTT and retransmissions per connection will generally
require a dedicated communications test instrument. In the absence of require a dedicated communications test instrument. In the absence of
dedicated hardware based test tools, these measurements may need to dedicated hardware based test tools, these measurements may need to
be conducted with packet capture tools, i.e. conduct TCP Throughput be conducted with packet capture tools, i.e. conduct TCP Throughput
tests and analyze RTT and retransmissions in packet captures. tests and analyze RTT and retransmissions in packet captures.
The TCP implementation used by the test host MUST be specified in The TCP implementation used by the test host MUST be specified in
the test results (e.g. TCP New Reno, TCP options supported, etc.). the test results (e.g. TCP New Reno, TCP options supported, etc.).
Additionally, RFC 3148 recommends that specific congestion control Additionally, the test results SHALL provide specific congestion
algorithm details that should also be included in the test results. control algorithm details, as per [RFC3148].
While RFC 6349 defined the means to conduct throughput tests of TCP While [RFC6349] defined the means to conduct throughput tests of TCP
bulk transfers, the traffic management framework will extend TCP test bulk transfers, the traffic management framework will extend TCP test
execution into interactive TCP application traffic. Examples include execution into interactive TCP application traffic. Examples include
email, HTTP, business applications, etc. This interactive traffic is email, HTTP, business applications, etc. This interactive traffic is
bi-directional and can be chatty, meaning many turns in traffic bi-directional and can be chatty, meaning many turns in traffic
communication during the course of a transaction (versus the communication during the course of a transaction (versus the
relatively uni-directional flow of bulk transfer applications). relatively uni-directional flow of bulk transfer applications).
The test device must not only support bulk TCP transfer application The test device must not only support bulk TCP transfer application
traffic but MUST also support chatty traffic. A valid stress test traffic but MUST also support chatty traffic. A valid stress test
SHOULD include both traffic types. This is due to the non-uniform, SHOULD include both traffic types. This is due to the non-uniform,
skipping to change at page 15, line 16 skipping to change at page 15, line 16
Runs and if the variation is characterized as a significant portion Runs and if the variation is characterized as a significant portion
of the measured values, the next step may be to revise the methods to of the measured values, the next step may be to revise the methods to
achieve better consistency. achieve better consistency.
6.1. Policing Tests 6.1. Policing Tests
A policer is defined as the entity performing the policy function. A policer is defined as the entity performing the policy function.
The intent of the policing tests is to verify the policer performance The intent of the policing tests is to verify the policer performance
(i.e. CIR-CBS and EIR-EBS parameters). The tests will verify that the (i.e. CIR-CBS and EIR-EBS parameters). The tests will verify that the
network device can handle the CIR with CBS and the EIR with EBS and network device can handle the CIR with CBS and the EIR with EBS and
will use back-back packet testing concepts from RFC 2544 (but adapted will use back-back packet testing concepts from [RFC2544] (but
to burst size algorithms and terminology). Also MEF-14,19,37 provide adapted to burst size algorithms and terminology). Also MEF-14,19,
some basis for specific components of this test. The burst hunt 37 provide some basis for specific components of this test. The
algorithm defined in section 5.1.1 can also be used to automate the burst hunt algorithm defined in section 5.1.1 can also be used to
measurement of the CBS value. automate the measurement of the CBS value.
The tests are divided into two (2) sections; individual policer The tests are divided into two (2) sections; individual policer
tests and then full capacity policing tests. It is important to tests and then full capacity policing tests. It is important to
benchmark the basic functionality of the individual policer then benchmark the basic functionality of the individual policer then
proceed into the fully rated capacity of the device. This capacity proceed into the fully rated capacity of the device. This capacity
may include the number of policing policies per device and the may include the number of policing policies per device and the
number of policers simultaneously active across all ports. number of policers simultaneously active across all ports.
6.1.1 Policer Individual Tests 6.1.1 Policer Individual Tests
Objective: Objective:
Test a policer as defined by RFC 4115 or MEF 10.2, depending upon the Test a policer as defined by [RFC4115] or MEF 10.2, depending upon
equipment's specification. In addition to verifying that the policer the equipment's specification. In addition to verifying that the
allows the specified CBS and EBS bursts to pass, the policer test policer allows the specified CBS and EBS bursts to pass, the policer
MUST verify that the policer will remark or drop excess, and pass test MUST verify that the policer will remark or drop excess, and
traffic at the specified CBS/EBS values. pass traffic at the specified CBS/EBS values.
Test Summary: Test Summary:
Policing tests should use stateless traffic. Stateful TCP test Policing tests should use stateless traffic. Stateful TCP test
traffic will generally be adversely affected by a policer in the traffic will generally be adversely affected by a policer in the
absence of traffic shaping. So while TCP traffic could be used, absence of traffic shaping. So while TCP traffic could be used,
it is more accurate to benchmark a policer with stateless traffic. it is more accurate to benchmark a policer with stateless traffic.
As an example for RFC 4115, consider a CBS and EBS of 64KB and CIR As an example for [RFC4115], consider a CBS and EBS of 64KB and CIR
and EIR of 100 Mbps on a 1GigE physical link (in color-blind mode). and EIR of 100 Mbps on a 1GigE physical link (in color-blind mode).
A stateless traffic burst of 64KB would be sent into the policer at A stateless traffic burst of 64KB would be sent into the policer at
the GigE rate. This equates to approximately a 0.512 millisecond the GigE rate. This equates to approximately a 0.512 millisecond
burst time (64 KB at 1 GigE). The traffic generator must space these burst time (64 KB at 1 GigE). The traffic generator must space these
bursts to ensure that the aggregate throughput does not exceed the bursts to ensure that the aggregate throughput does not exceed the
CIR. The Ti between the bursts would equal CBS * 8 / CIR = 5.12 CIR. The Ti between the bursts would equal CBS * 8 / CIR = 5.12
millisecond in this example. millisecond in this example.
Test Metrics: Test Metrics:
The metrics defined in section 4.1 (BSA, LP, OOS, PD, and PDV) SHALL The metrics defined in section 4.1 (BSA, LP, OOS, PD, and PDV) SHALL
skipping to change at page 21, line 32 skipping to change at page 21, line 32
stateful traffic bursts up to the queue depth. stateful traffic bursts up to the queue depth.
Test Background and Summary: Test Background and Summary:
To provide a more realistic benchmark and to test queues in layer 4 To provide a more realistic benchmark and to test queues in layer 4
devices such as firewalls, stateful traffic testing is recommended devices such as firewalls, stateful traffic testing is recommended
for the queue tests. Stateful traffic tests will also utilize the for the queue tests. Stateful traffic tests will also utilize the
Network Delay Emulator (NDE) from the network set-up configuration in Network Delay Emulator (NDE) from the network set-up configuration in
section 2. section 2.
The BDP of the TCP test traffic must be calibrated to the QL of the The BDP of the TCP test traffic must be calibrated to the QL of the
device queue. Referencing RFC 6349, the BDP is equal to: device queue. Referencing [RFC6349], the BDP is equal to:
BB * RTT / 8 (in bytes) BB * RTT / 8 (in bytes)
The NDE must be configured to an RTT value which is large enough to The NDE must be configured to an RTT value which is large enough to
allow the BDP to be greater than QL. An example test scenario is allow the BDP to be greater than QL. An example test scenario is
defined below: defined below:
- Ingress link = GigE - Ingress link = GigE
- Egress link = 100 Mbps (BB) - Egress link = 100 Mbps (BB)
- QL = 32KB - QL = 32KB
skipping to change at page 22, line 54 skipping to change at page 22, line 54
multiple TCP connections, then all of the TCP connections are multiple TCP connections, then all of the TCP connections are
aggregated in the application throughput measurement / metrics for aggregated in the application throughput measurement / metrics for
that application. that application.
Then there is the case of mulitlple instances of an application Then there is the case of mulitlple instances of an application
session (i.e. multiple FTPs emulating multiple clients). In this session (i.e. multiple FTPs emulating multiple clients). In this
situation, the test should measure / record each FTP application situation, the test should measure / record each FTP application
session independently, tabulating the minimum, maximum, and average session independently, tabulating the minimum, maximum, and average
for all FTP sessions. for all FTP sessions.
Finally, application throughput measurements are based off of Layer 4 Finally, application throughput measurements are based on Layer 4
TCP throughput and do not include bytes retransmitted. The TCP TCP throughput and do not include bytes retransmitted. The TCP
Efficiency metric MUST be measured during the test and provides a Efficiency metric MUST be measured during the test and provides a
measure of "goodput" during each test. measure of "goodput" during each test.
Reporting Format: Reporting Format:
The Queue/Scheduler Stateful Traffic individual report MUST contain The Queue/Scheduler Stateful Traffic individual report MUST contain
all results for each traffic scheduler and QL/BB test run and a all results for each traffic scheduler and QL/BB test run and a
recommended format is as follows: recommended format is as follows:
******************************************************** ********************************************************
skipping to change at page 28, line 46 skipping to change at page 28, line 46
shaping function, the cumulative TCP window should exceed the BDP shaping function, the cumulative TCP window should exceed the BDP
which will stress the shaper. BDP factors of 1.1 to 1.5 are which will stress the shaper. BDP factors of 1.1 to 1.5 are
recommended, but the values are the discretion of the tester and recommended, but the values are the discretion of the tester and
should be documented. should be documented.
The cumulative TCP Window Sizes* (RWND at the receiving end & CWND The cumulative TCP Window Sizes* (RWND at the receiving end & CWND
at the transmitting end) equates to: at the transmitting end) equates to:
TCP window size* for each connection x number of connections TCP window size* for each connection x number of connections
* as described in section 3 of RFC6349, the SSB MUST be large * as described in section 3 of [RFC6349], the SSB MUST be large
enough to fill the BDP enough to fill the BDP
Example, if the BDP is equal to 256 Kbytes and a connection size of Example, if the BDP is equal to 256 Kbytes and a connection size of
64Kbytes is used for each connection, then it would require four (4) 64Kbytes is used for each connection, then it would require four (4)
connections to fill the BDP and 5-6 connections (over subscribe the connections to fill the BDP and 5-6 connections (over subscribe the
BDP) to stress test the traffic shaping function. BDP) to stress test the traffic shaping function.
Two types of TCP tests MUST be performed: Bulk Transfer test and Two types of TCP tests MUST be performed: Bulk Transfer test and
Micro Burst Test Pattern as documented in Appendix B. The Bulk Micro Burst Test Pattern as documented in Appendix B. The Bulk
Transfer Test only bursts during the TCP Slow Start (or Congestion Transfer Test only bursts during the TCP Slow Start (or Congestion
skipping to change at page 33, line 41 skipping to change at page 33, line 41
Appendix B: Stateful TCP Test Patterns Appendix B: Stateful TCP Test Patterns
This framework recommends at a minimum the following TCP test This framework recommends at a minimum the following TCP test
patterns since they are representative of real world application patterns since they are representative of real world application
traffic (section 5.2.1 describes some methods to derive other traffic (section 5.2.1 describes some methods to derive other
application-based TCP test patterns). application-based TCP test patterns).
- Bulk Transfer: generate concurrent TCP connections whose aggregate - Bulk Transfer: generate concurrent TCP connections whose aggregate
number of in-flight data bytes would fill the BDP. Guidelines number of in-flight data bytes would fill the BDP. Guidelines
from RFC 6349 are used to create this TCP traffic pattern. from [RFC6349] are used to create this TCP traffic pattern.
- Micro Burst: generate precise burst patterns within a single or - Micro Burst: generate precise burst patterns within a single or
multiple TCP connections(s). The idea is for TCP to establish multiple TCP connections(s). The idea is for TCP to establish
equilibrium and then burst application bytes at defined sizes. The equilibrium and then burst application bytes at defined sizes. The
test tool must allow the burst size and burst time interval to be test tool must allow the burst size and burst time interval to be
configurable. configurable.
- Web Site Patterns: The HTTP traffic model from - Web Site Patterns: The HTTP traffic model from
"3GPP2 C.R1002-0 v1.0" is referenced (Table 4.1.3.2.1) to develop "3GPP2 C.R1002-0 v1.0" is referenced (Table 4.1.3.2.1) to develop
these TCP test patterns. In summary, the HTTP traffic model consists these TCP test patterns. In summary, the HTTP traffic model consists
skipping to change at page 37, line 41 skipping to change at page 37, line 41
8. IANA Considerations 8. IANA Considerations
This document does not REQUIRE an IANA registration for ports This document does not REQUIRE an IANA registration for ports
dedicated to the TCP testing described in this document. dedicated to the TCP testing described in this document.
9. Acknowledgments 9. Acknowledgments
We would like to thank Al Morton for his continuous review and We would like to thank Al Morton for his continuous review and
invaluable input to the document. We would also like to thank invaluable input to the document. We would also like to thank
Scott Bradnor for providing guidance early in the drafts Scott Bradner for providing guidance early in the drafts
conception in the area of benchmarking scope of traffic management conception in the area of benchmarking scope of traffic management
functions. Additionally, we would like to thank Tim Copley for this functions. Additionally, we would like to thank Tim Copley for this
original input and David Taht, Gory Erg, Toke Hoiland-Jorgensen for original input and David Taht, Gory Erg, Toke Hoiland-Jorgensen for
their review and input for the AQM group. And for the formal reviews their review and input for the AQM group. And for the formal reviews
of this document, we would like to thank Gilles Forget, of this document, we would like to thank Gilles Forget,
Vijay Gurbani, Reinhard Schrage, and Bhuvaneswaran Vengainathan Vijay Gurbani, Reinhard Schrage, and Bhuvaneswaran Vengainathan
10. References 10. References
10.1. Normative References 10.1. Normative References
[RFC2119] Bradner, S., "Key words for use in RFCs to Indicate [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate
Requirement Levels", BCP 14, RFC 2119, March 1997. Requirement Levels", BCP 14, RFC2119, March 1997.
[RFC1242] S. Bradnor, "Benchmarking Terminology for Network [RFC1242] S. Bradner, "Benchmarking Terminology for Network
Interconnection Devices," RFC 1242 July 1991 Interconnection Devices," RFC1242 July 1991
[RFC5481] A. Morton etal., "Packet Delay Variation Applicability [RFC2544] S. Bradner, "Benchmarking Methodology for Network
Statement," RFC 5481 March 2009 Interconnect Devices," RFC2544 March 1999
[RFC3148] M. Mathis et al., A Framework for Defining Empirical
Bulk Transfer Capacity Metrics," RFC3148 July 2001
[RFC5481] A. Morton et al., "Packet Delay Variation Applicability
Statement," RFC5481 March 2009
[RFC6703] A. Morton et al., "Reporting IP Network Performance
Metrics: Different Points of View." RFC 6703 August 2012
[RFC2680] G. Almes et al., "A One-way Packet Loss Metric for IPPM," [RFC2680] G. Almes et al., "A One-way Packet Loss Metric for IPPM,"
RFC 2680 September 1999 RFC2680 September 1999
[RFC2697] J. Heinanen et al., "A Single Rate Three Color Marker," [RFC2697] J. Heinanen et al., "A Single Rate Three Color Marker,"
RFC 2697, September 1999 RFC2697, September 1999
[RFC2698] J. Heinanen et al., "A Two Rate Three Color Marker, " [RFC2698] J. Heinanen et al., "A Two Rate Three Color Marker, "
RFC 2698, September 1999 RFC2698, September 1999
[RFC4689] S. Poretsky et al., "Terminology for Benchmarking [RFC4689] S. Poretsky et al., "Terminology for Benchmarking
Network-layer Traffic Control Mechanisms," RFC 4689, Network-layer Traffic Control Mechanisms," RFC4689,
October 2006 October 2006
[RFC4737] A. Morton et al., "Packet Reordering Metrics," RFC 4737, [RFC4737] A. Morton et al., "Packet Reordering Metrics," RFC4737,
November 2006 February 2006
[RFC4115] O. Aboul-Magd et al., "A Differentiated Service Two-Rate,
Three-Color Marker with Efficient Handling of in-Profile Traffic."
RFC4115 July 2005
[RFC6349] Barry Constantine et al., "Framework for TCP Throughput [RFC6349] Barry Constantine et al., "Framework for TCP Throughput
Testing," RFC 6349, August 2011 Testing," RFC6349, August 2011
[AQM-RECO] Fred Baker et al., "IETF Recommendations Regarding [AQM-RECO] Fred Baker et al., "IETF Recommendations Regarding
Active Queue Management," August 2014, Active Queue Management," August 2014,
https://datatracker.ietf.org/doc/draft-ietf-aqm- https://datatracker.ietf.org/doc/draft-ietf-aqm-
recommendation/ recommendation/
[MEF-10.2] "MEF 10.2: Ethernet Services Attributes Phase 2," October [MEF-10.2] "MEF 10.2: Ethernet Services Attributes Phase 2," October
2009, http://metroethernetforum.org/PDF_Documents/ 2009, http://metroethernetforum.org/PDF_Documents/
technical-specifications/MEF10.2.pdf technical-specifications/MEF10.2.pdf
skipping to change at page 39, line 7 skipping to change at page 39, line 11
Part 2: Ethernet Services Layer - Base Elements," April Part 2: Ethernet Services Layer - Base Elements," April
2010, https://www.metroethernetforum.org/Assets/Technical 2010, https://www.metroethernetforum.org/Assets/Technical
_Specifications/PDF/MEF12.1.pdf _Specifications/PDF/MEF12.1.pdf
[MEF-26] "MEF 26: External Network Network Interface (ENNI) - [MEF-26] "MEF 26: External Network Network Interface (ENNI) -
Phase 1,"January 2010, http://www.metroethernetforum.org Phase 1,"January 2010, http://www.metroethernetforum.org
/PDF_Documents/technical-specifications/MEF26.pdf /PDF_Documents/technical-specifications/MEF26.pdf
10.2. Informative References 10.2. Informative References
Authors' Addresses Authors' Addresses
Barry Constantine Barry Constantine
JDSU, Test and Measurement Division JDSU, Test and Measurement Division
Germantown, MD 20876-7100, USA Germantown, MD 20876-7100, USA
Phone: +1 240 404 2227 Phone: +1 240 404 2227
Email: barry.constantine@jdsu.com Email: barry.constantine@jdsu.com
Ram Krishnan Ram Krishnan
Brocade Communications Brocade Communications
San Jose, 95134, USA San Jose, 95134, USA
Phone: +001-408-406-7890 Phone: +001-408-406-7890
Email: ramk@brocade.com Email: ramk@brocade.com
 End of changes. 50 change blocks. 
95 lines changed or deleted 100 lines changed or added

This html diff was produced by rfcdiff 1.42. The latest version is available from http://tools.ietf.org/tools/rfcdiff/