draft-ietf-bmwg-hash-stuffing-02.txt   draft-ietf-bmwg-hash-stuffing-03.txt 
Network Working Group D. Newman Network Working Group D. Newman
Internet-Draft Network Test Internet-Draft Network Test
Expires: August 19, 2005 T. Player Expires: December 23, 2005 T. Player
Spirent Communications Spirent Communications
February 15, 2005 June 21, 2005
Hash and Stuffing: Overlooked Factors in Network Device Benchmarking Hash and Stuffing: Overlooked Factors in Network Device Benchmarking
draft-ietf-bmwg-hash-stuffing-02.txt draft-ietf-bmwg-hash-stuffing-03.txt
Status of this Memo Status of this Memo
This document is an Internet-Draft and is subject to all provisions By submitting this Internet-Draft, each author represents that any
of Section 3 of RFC 3667. By submitting this Internet-Draft, each applicable patent or other IPR claims of which he or she is aware
author represents that any applicable patent or other IPR claims of have been or will be disclosed, and any of which he or she becomes
which he or she is aware have been or will be disclosed, and any of aware will be disclosed, in accordance with Section 6 of BCP 79.
which he or she become aware will be disclosed, in accordance with
RFC 3668.
Internet-Drafts are working documents of the Internet Engineering Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF), its areas, and its working groups. Note that Task Force (IETF), its areas, and its working groups. Note that
other groups may also distribute working documents as other groups may also distribute working documents as Internet-
Internet-Drafts. Drafts.
Internet-Drafts are draft documents valid for a maximum of six months Internet-Drafts are draft documents valid for a maximum of six months
and may be updated, replaced, or obsoleted by other documents at any and may be updated, replaced, or obsoleted by other documents at any
time. It is inappropriate to use Internet-Drafts as reference time. It is inappropriate to use Internet-Drafts as reference
material or to cite them other than as "work in progress." material or to cite them other than as "work in progress."
The list of current Internet-Drafts can be accessed at The list of current Internet-Drafts can be accessed at
http://www.ietf.org/ietf/1id-abstracts.txt. http://www.ietf.org/ietf/1id-abstracts.txt.
The list of Internet-Draft Shadow Directories can be accessed at The list of Internet-Draft Shadow Directories can be accessed at
http://www.ietf.org/shadow.html. http://www.ietf.org/shadow.html.
This Internet-Draft will expire on August 19, 2005. This Internet-Draft will expire on December 23, 2005.
Copyright Notice Copyright Notice
Copyright (C) The Internet Society (2005). Copyright (C) The Internet Society (2005).
Abstract Abstract
Test engineers take pains to declare all factors that affect a given Test engineers take pains to declare all factors that affect a given
measurement, including offered load, packet length, test duration, measurement, including offered load, packet length, test duration,
and traffic orientation. However, current benchmarking practice and traffic orientation. However, current benchmarking practice
skipping to change at page 3, line 34 skipping to change at page 3, line 34
to ensure sequences in test traffic will not be confused with flag or to ensure sequences in test traffic will not be confused with flag or
control characters. control characters.
Stuffing adds significant and variable overhead. Currently there is Stuffing adds significant and variable overhead. Currently there is
no standard method for determining the probability that stuffing will no standard method for determining the probability that stuffing will
occur for a given pattern, and thus no way to determine what impact occur for a given pattern, and thus no way to determine what impact
stuffing will have on test results. stuffing will have on test results.
This document covers two areas. First, we discuss strategies for This document covers two areas. First, we discuss strategies for
dealing with randomness and nonrandomness in test traffic. Second, dealing with randomness and nonrandomness in test traffic. Second,
we present formulas to determine the probability of bit- and we present formulas to determine the probability of bit- and byte-
byte-stuffing on PPP and POS circuits. In both areas, we provide stuffing on PPP and POS circuits. In both areas, we provide
recommendations for obtaining more repeatability in test results. recommendations for obtaining more repeatability in test results.
2. Requirements 2. Requirements
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
"SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
document are to be interpreted as described in [RFC2119]. document are to be interpreted as described in [RFC2119].
3. General considerations 3. General considerations
3.1 Repeatability 3.1 Repeatability
Repeatability is a desirable trait in benchmarking, but it can be an Repeatability is a desirable trait in benchmarking, but it can be an
elusive goal. It is a common but mistaken belief that test results elusive goal. It is a common but mistaken belief that test results
can always be reproduced provided the device under test and test can always be reproduced provided the device under test and test
instrument are configured are configured identically for each test instrument are configured identically for each test iteration. In
iteration. In fact, even identical configurations may introduce some fact, even identical configurations may introduce some variations in
variations in test traffic, such as changes in timestamps, TCP test traffic, such as changes in timestamps, TCP sequence numbers, or
sequence numbers, or other naturally occurring phenomena. other naturally occurring phenomena.
While this variability does not necessarily invalidate test results, While this variability does not necessarily invalidate test results,
it is important to recognize such variation exists. Exact it is important to recognize such variation exists. Exact bit-for-
bit-for-bit reproduction of test traffic in all cases is a hard bit reproduction of test traffic in all cases is a hard problem. A
problem. A simpler approach is to acknowledge that some variation simpler approach is to acknowledge that some variation exists,
exists, characterize that variation, and describe it when analyzing characterize that variation, and describe it when analyzing test
test results. results.
3.2 Randomness 3.2 Randomness
This document recommends the use of pseudorandom patterns in test This document recommends the use of pseudorandom patterns in test
traffic under controlled lab conditions. The rand() functions traffic under controlled lab conditions. The rand() functions
available in many programming languages produce output that is available in many programming languages produce output that is
pseudorandom rather than truly random. Pseudorandom patterns are pseudorandom rather than truly random. Pseudorandom patterns are
sufficient for the recommendations given in this document, provided sufficient for the recommendations given in this document, provided
they produce output that is uniformly distributed across the pattern they produce output that is uniformly distributed across the pattern
space. space.
skipping to change at page 9, line 6 skipping to change at page 9, line 6
multiple hosts, even on a single interface. This is desirable in multiple hosts, even on a single interface. This is desirable in
assessing DUT/SUT scalability. assessing DUT/SUT scalability.
However, test instruments may emulate multiple MAC addresses by However, test instruments may emulate multiple MAC addresses by
incrementing and/or decrementing addresses from a fixed starting incrementing and/or decrementing addresses from a fixed starting
point. This leads to situations as described above in "Address point. This leads to situations as described above in "Address
Pattern Variations" where hashing algorithms produce nonoptimal Pattern Variations" where hashing algorithms produce nonoptimal
outcomes. outcomes.
The outcome can be nonoptimal even if the set of addresses begins The outcome can be nonoptimal even if the set of addresses begins
with a pseudorandom number. For example, the following with a pseudorandom number. For example, the following source/
source/destination pairs will not be equally distributed by the 3-bit destination pairs will not be equally distributed by the 3-bit
hashing algorithm discussed above: hashing algorithm discussed above:
Source Destination Source Destination
00:00:01:FC:B3:45 00:00:19:38:8C:80 00:00:01:FC:B3:45 00:00:19:38:8C:80
00:00:01:FC:B3:46 00:00:19:38:8C:81 00:00:01:FC:B3:46 00:00:19:38:8C:81
00:00:01:FC:B3:47 00:00:19:38:8C:82 00:00:01:FC:B3:47 00:00:19:38:8C:82
00:00:01:FC:B3:48 00:00:19:38:8C:83 00:00:01:FC:B3:48 00:00:19:38:8C:83
00:00:01:FC:B3:49 00:00:19:38:8C:84 00:00:01:FC:B3:49 00:00:19:38:8C:84
00:00:01:FC:B3:50 00:00:19:38:8C:85 00:00:01:FC:B3:4A 00:00:19:38:8C:85
00:00:01:FC:B3:51 00:00:19:38:8C:86 00:00:01:FC:B3:4B 00:00:19:38:8C:86
00:00:01:FC:B3:52 00:00:19:38:8C:87 00:00:01:FC:B3:4C 00:00:19:38:8C:87
...
Again working with our 3-bit XOR hashing algorithm, we get the Again working with our 3-bit XOR hashing algorithm, we get the
following outcomes: following outcomes:
101 ^ 000 = 101 101 ^ 000 = 101
110 ^ 001 = 111 110 ^ 001 = 111
111 ^ 010 = 101 111 ^ 010 = 101
000 ^ 011 = 011 000 ^ 011 = 011
001 ^ 100 = 101 001 ^ 100 = 101
010 ^ 101 = 111 010 ^ 101 = 111
011 ^ 110 = 101 011 ^ 110 = 101
100 ^ 111 = 011 100 ^ 111 = 011
Note that only three of eight possible outcomes are achieved when Note that only three of eight possible outcomes are achieved when
incrementing addresses. This is actually the best case. incrementing addresses. This is actually the best case.
Incrementing from other combinations of pseudorandom address pairs Incrementing from other combinations of pseudorandom address pairs
produces only one or two out of eight possible outcomes. produces only one or two out of eight possible outcomes.
Every MAC address should be pseudorandom, not just the starting one. Every MAC address SHOULD be pseudorandom, not just the starting one.
When generating traffic with multiple addresses, it is RECOMMENDED When generating traffic with multiple addresses, it is RECOMMENDED
that all addresses use pseudorandom values. There are multiple ways that all addresses use pseudorandom values. There are multiple ways
to use sets of pseudorandom numbers. One strategy would be for the to use sets of pseudorandom numbers. One strategy would be for the
test instrument to iterate over an array of pseudorandom values test instrument to iterate over an array of pseudorandom values
rather than incrementing/decrementing from a starting address. The rather than incrementing/decrementing from a starting address. The
actual method is an implementation detail; in the end, any method actual method is an implementation detail; in the end, any method
that uses multiple addresses and avoids hash table collisions will be that uses multiple addresses and avoids hash table collisions will be
sufficient. sufficient.
skipping to change at page 10, line 47 skipping to change at page 10, line 47
range of IP addresses. range of IP addresses.
4.5 Transport-layer Addressing 4.5 Transport-layer Addressing
Some devices with transport- or application-layer awareness use TCP Some devices with transport- or application-layer awareness use TCP
or UDP port numbers in making forwarding decisions. Examples of such or UDP port numbers in making forwarding decisions. Examples of such
devices include load balancers and application-layer firewalls. devices include load balancers and application-layer firewalls.
Test instruments have the capability of generating packets with Test instruments have the capability of generating packets with
random TCP and UDP source and destination port numbers. Known random TCP and UDP source and destination port numbers. Known
destination port numbers are often required for testing destination port numbers are often required for testing application-
application-layer devices. However, unless known port numbers are layer devices. However, unless known port numbers are specifically
specifically required for a test, it is RECOMMENDED to use randomly required for a test, it is RECOMMENDED to use randomly distributed
distributed values for both source and destination port numbers. values for both source and destination port numbers.
In addition, it may be desirable to pick pseudorandom values from a In addition, it may be desirable to pick pseudorandom values from a
selected pool of numbers. Many services identify themselves through selected pool of numbers. Many services identify themselves through
use of reserved destination port numbers between 1 and 1023 use of reserved destination port numbers between 1 and 1023
inclusive. Unless specific port numbers are required, it is inclusive. Unless specific port numbers are required, it is
RECOMMENDED to pick randomly distributed destination port numbers RECOMMENDED to pick randomly distributed destination port numbers
between these lower and upper boundaries. between these lower and upper boundaries.
Similarly, clients typically choose source port numbers in the space Similarly, clients typically choose source port numbers in the space
between 1024 and 65535 inclusive. Unless specific port numbers are between 1024 and 65535 inclusive. Unless specific port numbers are
skipping to change at page 16, line 9 skipping to change at page 16, line 13
percentage overhead is simply the probability of stuffing expressed percentage overhead is simply the probability of stuffing expressed
as a percentage. as a percentage.
Given that the overhead added by bit-stuffing is at most 1 in 62, or Given that the overhead added by bit-stuffing is at most 1 in 62, or
approximately 1.6 percent, it is RECOMMENDED that testers reduce the approximately 1.6 percent, it is RECOMMENDED that testers reduce the
maximum offered load by 1.6 percent to avoid introducing congestion maximum offered load by 1.6 percent to avoid introducing congestion
when testing devices using bit-synchronous interfaces (such as T1/E1, when testing devices using bit-synchronous interfaces (such as T1/E1,
DS-3, and the like). DS-3, and the like).
The percentage given above is an approximation. For greatest The percentage given above is an approximation. For greatest
precision, the actual offered load should be calculated using the precision, the actual offered load SHOULD be calculated using the
percentage overhead formula above and then expressed in frames per percentage overhead formula above and then expressed in frames per
second, rounded down to the nearest integer. second, rounded down to the nearest integer.
Note that the DUT/SUT may be able to forward offered loads higher Note that the DUT/SUT may be able to forward offered loads higher
than the calculated theoretical maximum without packet loss. Such than the calculated theoretical maximum rate without packet loss.
results are the result of queuing on the part of the DUT/SUT. While Such results are the result of queuing on the part of the DUT/SUT.
a device's throughput may be above this level, delay-related While a device's throughput may be above this level, delay-related
measurements such as latency or jitter may be affected. Accordingly, measurements may be affected. Accordingly, it is RECOMMENDED to
it is RECOMMENDED to reduce offered levels by the amount of reduce offered levels by the amount of bit-stuffing overhead when
bit-stuffing overhead when testing devices using bit-synchronous testing devices using bit-synchronous links. This recommendation
links. This recommendation applies for all measurements, including applies for all measurements, including throughput.
throughput.
5.3 POS Byte Stuffing 5.3 POS Byte Stuffing
[RFC1662] requires that "Each Flag Sequence, Control Escape octet, [RFC1662] requires that "Each Flag Sequence, Control Escape octet,
and any octet which is flagged in the sending and any octet which is flagged in the sending Async-Control-
Async-Control-Character-Map (ACCM), is replaced by a two octet Character-Map (ACCM), is replaced by a two octet sequence consisting
sequence consisting of the Control Escape octet followed by the of the Control Escape octet followed by the original octet exclusive-
original octet exclusive-or'd with hexadecimal 0x20." The practical or'd with hexadecimal 0x20." The practical effect of this is to
effect of this is to insert a stuff byte for instances of up to 34 insert a stuff byte for instances of up to 34 characters: 0x7E, 0x7D,
characters: 0x7E, 0x7D, or any of 32 ACCM values. or any of 32 ACCM values.
A common implementation of PPP in HDLC-like framing is in PPP over A common implementation of PPP in HDLC-like framing is in PPP over
Sonet/SDH (POS), as defined in [RFC2615]. Sonet/SDH (POS), as defined in [RFC2615].
As with the bit-stuffing case, the requirement in characterizing POS As with the bit-stuffing case, the requirement in characterizing POS
test traffic is to determine the probability that byte-stuffing will test traffic is to determine the probability that byte-stuffing will
occur for a given sequence. This is much simpler to do than with occur for a given sequence. This is much simpler to do than with
bit-synchronous links, since there is no possibility of overlap bit-synchronous links, since there is no possibility of overlap
across byte boundaries. across byte boundaries.
skipping to change at page 17, line 39 skipping to change at page 17, line 41
When testing a DUT/SUT that implements PPP in HDLC-like framing and When testing a DUT/SUT that implements PPP in HDLC-like framing and
L2TP (or any other technology that uses nonzero ACCM values), it is L2TP (or any other technology that uses nonzero ACCM values), it is
RECOMMENDED that testers reduce the maximum offered load by 13.3 RECOMMENDED that testers reduce the maximum offered load by 13.3
percent to avoid introducing congestion. percent to avoid introducing congestion.
When testing a DUT/SUT that implements PPP in HDLC-like framing and When testing a DUT/SUT that implements PPP in HDLC-like framing and
an ACCM value of 0x00, it is RECOMMENDED that testers reduce the an ACCM value of 0x00, it is RECOMMENDED that testers reduce the
maximum offered load by 0.8 percent to avoid introducing congestion. maximum offered load by 0.8 percent to avoid introducing congestion.
Note that the percentages given above are approximations. For Note that the percentages given above are approximations. For
greatest precision, the actual offered load should be calculated greatest precision, the actual offered load SHOULD be calculated
using the percentage overhead formula above and then expressed in using the percentage overhead formula above and then expressed in
frames per second (rounded down to the nearest integer). frames per second (rounded down to the nearest integer).
Note also that the DUT/SUT may be able to forward offered loads Note also that the DUT/SUT may be able to forward offered loads
higher than the calculated theoretical maximum without packet loss. higher than the calculated theoretical maximum rate without packet
Such results are the result of queuing on the part of the DUT/SUT. loss. Such results are the result of queuing on the part of the DUT/
While a device's throughput may be above this level, delay-related SUT. While a device's throughput may be above this level, delay-
measurements such as latency or jitter may be affected. Accordingly, related measurements may be affected. Accordingly, it is RECOMMENDED
it is RECOMMENDED to reduce offered levels by the amount of to reduce offered levels by the amount of byte-stuffing overhead when
byte-stuffing overhead when testing devices using byte-synchronous testing devices using byte-synchronous links. This recommendation
links. This recommendation applies for all measurements, including applies for all measurements, including throughput.
throughput.
6. Security Considerations 6. Security Considerations
This document recommends the use of pseudorandom patterns in test This document recommends the use of pseudorandom patterns in test
traffic. Testers should be aware that the rand() functions of many traffic. The rand() functions of many programming languages produce
programming languages produce output that is pseudorandom rather than output that is pseudorandom rather than truly random. As far as the
truly random. As far as the authors are aware, pseudorandom patterns authors are aware, pseudorandom patterns are sufficient for
are sufficient for generating test traffic in lab conditions. generating test traffic in lab conditions.
[RFC2615], section 6, discusses a denial-of-service attack involving [RFC2615], section 6, discusses a denial-of-service attack involving
the intentional transmission of characters that require stuffing. the intentional transmission of characters that require stuffing.
This attack could consume up to 100 percent of available bandwidth. This attack could consume up to 100 percent of available bandwidth.
However, the test networks described in BMWG documents generally However, the test networks described in BMWG documents generally
SHOULD NOT be reachable by anyone other than the tester(s). SHOULD NOT be reachable by anyone other than the tester(s).
7. IANA Considerations 7. IANA Considerations
This document has no actions for IANA. This document has no actions for IANA.
skipping to change at page 20, line 24 skipping to change at page 20, line 24
[RFC2119] Bradner, S., "Key words for use in RFCs to Indicate [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate
Requirement Levels", BCP 14, RFC 2119, March 1997. Requirement Levels", BCP 14, RFC 2119, March 1997.
[RFC2544] Bradner, S. and J. McQuaid, "Benchmarking Methodology for [RFC2544] Bradner, S. and J. McQuaid, "Benchmarking Methodology for
Network Interconnect Devices", RFC 2544, March 1999. Network Interconnect Devices", RFC 2544, March 1999.
[RFC2615] Malis, A. and W. Simpson, "PPP over SONET/SDH", RFC 2615, [RFC2615] Malis, A. and W. Simpson, "PPP over SONET/SDH", RFC 2615,
June 1999. June 1999.
[RFC2661] Townsley, W., Valencia, A., Rubens, A., Pall, G., Zorn, G. [RFC2661] Townsley, W., Valencia, A., Rubens, A., Pall, G., Zorn,
and B. Palter, "Layer Two Tunneling Protocol "L2TP"", G., and B. Palter, "Layer Two Tunneling Protocol "L2TP"",
RFC 2661, August 1999. RFC 2661, August 1999.
[RFC2889] Mandeville, R. and J. Perser, "Benchmarking Methodology [RFC2889] Mandeville, R. and J. Perser, "Benchmarking Methodology
for LAN Switching Devices", RFC 2889, August 2000. for LAN Switching Devices", RFC 2889, August 2000.
8.2 Informative References 8.2 Informative References
[Ca63] Campbell, D. and J. Stanley, "Experimental and [Ca63] Campbell, D. and J. Stanley, "Experimental and Quasi-
Quasi-Experimental Designs for Research", 1963. Experimental Designs for Research", 1963.
[Go97] Goralski, W., "SONET: A Guide to Synchronous Optical [Go97] Goralski, W., "SONET: A Guide to Synchronous Optical
Networks", 1997. Networks", 1997.
[Kn97] Knuth, D., "The Art of Computer Programming, Volume 2, Third [Kn97] Knuth, D., "The Art of Computer Programming, Volume 2, Third
Edition", 1997. Edition", 1997.
Authors' Addresses Authors' Addresses
David Newman David Newman
Network Test Network Test
Email: dnewman@networktest.com Email: dnewman@networktest.com
Timmons C. Player Timmons C. Player
Spirent Communications Spirent Communications
Email: timmons.player@spirentcom.com Email: timmons.player@spirentcom.com
Appendix A. Acknowledgements Appendix A. Acknowledgements
The authors gratefully acknowledge reviews and contributions by Glenn The authors gratefully acknowledge reviews and contributions by Neil
Chagnot, Rafael Francis, Paul Hoffman, David Joyner, Joe Perches, and Carter, Glenn Chagnot, Rafael Francis, Paul Hoffman, David Joyner,
Scott Poretsky. Joe Perches, and Scott Poretsky.
Intellectual Property Statement Intellectual Property Statement
The IETF takes no position regarding the validity or scope of any The IETF takes no position regarding the validity or scope of any
Intellectual Property Rights or other rights that might be claimed to Intellectual Property Rights or other rights that might be claimed to
pertain to the implementation or use of the technology described in pertain to the implementation or use of the technology described in
this document or the extent to which any license under such rights this document or the extent to which any license under such rights
might or might not be available; nor does it represent that it has might or might not be available; nor does it represent that it has
made any independent effort to identify any such rights. Information made any independent effort to identify any such rights. Information
on the procedures with respect to rights in RFC documents can be on the procedures with respect to rights in RFC documents can be
 End of changes. 

This html diff was produced by rfcdiff 1.24, available from http://www.levkowetz.com/ietf/tools/rfcdiff/