draft-ietf-bmwg-methodology-01.txt   draft-ietf-bmwg-methodology-02.txt 
INTERNET-DRAFT
Expires in six months Network Working Group Scott Bradner
Internet Draft Harvard University
Expires in six months Jim McQuaid
Wandel & Goltermann
August 1995
Benchmarking Methodology for Network Interconnect Devices Benchmarking Methodology for Network Interconnect Devices
<draft-ietf-bmwg-methodology-01.txt> <draft-ietf-bmwg-methodology-02.txt>
Status of this Document Status of this Document
This document is an Internet-Draft. Internet-Drafts are working This document is an Internet-Draft. Internet-Drafts are working
documents of the Internet Engineering Task Force (IETF), its areas, documents of the Internet Engineering Task Force (IETF), its areas,
and its working groups. Note that other groups may also distribute and its working groups. Note that other groups may also distribute
working documents as Internet-Drafts. working documents as Internet-Drafts.
Internet-Drafts are draft documents valid for a maximum of six Internet-Drafts are draft documents valid for a maximum of six months
months and may be updated, replaced, or obsoleted by other documents and may be updated, replaced, or obsoleted by other documents at any
at any time. It is inappropriate to use Internet- Drafts as time. It is inappropriate to use Internet-Drafts as reference
reference material or to cite them other than as ``work in material or to cite them other than as ``work in progress.''
progress.''
To learn the current status of any Internet-Draft, please check the To learn the current status of any Internet-Draft, please check the
``1id-abstracts.txt'' listing contained in the Internet-Drafts ``1id-abstracts.txt'' listing contained in the Internet-Drafts Shadow
Shadow Directories on ds.internic.net (US East Coast), nic.nordu.net Directories on ds.internic.net (US East Coast), nic.nordu.net
(Europe), ftp.isi.edu (US West Coast), or munnari.oz.au (Pacific (Europe), ftp.isi.edu (US West Coast), or munnari.oz.au (Pacific
Rim). Rim).
Distribution of this document is unlimited. Please send comments to Distribution of this document is unlimited. Please send comments to
bmwg@harvard.edu or to the editor. bmwg@harvard.edu or to the editors.
Abstract Abstract
This document discusses and defines a number of tests that may be This document discusses and defines a number of tests that may be
used to describe the performance characteristics of a network used to describe the performance characteristics of a network
interconnecting device. In addition to defining the tests this interconnecting device. In addition to defining the tests this
document also describes specific formats for reporting the results document also describes specific formats for reporting the results of
of the tests. Appendix A lists the tests and conditions that we the tests. Appendix A lists the tests and conditions that we believe
believe should be included for specific cases and gives additional should be included for specific cases and gives additional
information about testing practices. Appendix B is a reference information about testing practices. Appendix B is a reference
listing of maximum frame rates to be used with specific frame sizes listing of maximum frame rates to be used with specific frame sizes
on various media and Appendix C gives some examples of frame formats on various media and Appendix C gives some examples of frame formats
to be used in testing. to be used in testing.
1. Introduction 1. Introduction
Vendors often engage in "specsmanship" in an attempt to give their Vendors often engage in "specsmanship" in an attempt to give their
products a better position in the marketplace. This often involves products a better position in the marketplace. This often involves
"smoke & mirrors" to confuse the potential users of the products. "smoke & mirrors" to confuse the potential users of the products.
This document and follow up memos attempt to define a specific set
of tests that vendors can use to measure and report the performance This document defines a specific set of tests that vendors can use to
characteristics of network devices. The results of these tests will measure and report the performance characteristics of network
provide the user comparable data from different vendors with which devices. The results of these tests will provide the user comparable
to evaluate these devices. data from different vendors with which to evaluate these devices.
A previous document, "Benchmarking Terminology for Network A previous document, "Benchmarking Terminology for Network
Interconnect Devices" (RFC 1242), defined many of the terms that are Interconnect Devices" (RFC 1242), defined many of the terms that are
used in this document. The terminology document should be consulted used in this document. The terminology document should be consulted
before attempting to make use of this document. before attempting to make use of this document.
2. Real world 2. Real world
In producing this document the authors attempted to keep in mind the In producing this document the authors attempted to keep in mind the
requirement that apparatus to perform the described tests must requirement that apparatus to perform the described tests must
actually be built. We do not know of "off the shelf" equipment actually be built. We do not know of "off the shelf" equipment
available to implement all of the tests but it is our opinion that available to implement all of the tests but it is our opinion that
such equipment can be constructed. such equipment can be constructed.
3. Tests to be run 3. Tests to be run
There are a number of tests described in this document. Not all of There are a number of tests described in this document. Not all of
the tests apply to all types of devices. It is expected that a the tests apply to all types of devices under test (DUTs). Vendors
vendor will perform all of the tests that apply to a specific type should perform all of the tests that can be supported by a specific
of product. The authors understand that it will take a considerable type of product. The authors understand that it will take a
period of time to perform all of the recommended tests under all of considerable period of time to perform all of the recommended tests
the recommended conditions. We believe that the results are worth under all of the recommended conditions. We believe that the results
the effort. Appendix A lists the tests and conditions that we are worth the effort. Appendix A lists some of the tests and
believe should be included for specific cases. conditions that we believe should be included for specific cases.
4. Evaluating the results 4. Evaluating the results
Performing all of the recommended tests will result in a great deal Performing all of the recommended tests will result in a great deal
of data. Much of this data will not apply to the evaluation of the of data. Much of this data will not apply to the evaluation of the
devices under each circumstance. For example, the rate at which a devices under each circumstance. For example, the rate at which a
router forwards IPX frames will be of little use in selecting a router forwards IPX frames will be of little use in selecting a
router for an environment that does not (and will not) support that router for an environment that does not (and will not) support that
protocol. Evaluating even that data which is relevant to a protocol. Evaluating even that data which is relevant to a
particular network installation will require experience which may particular network installation will require experience which may not
not be readily available. be readily available. Furthermore, selection of the tests to be run
and evaluation of the test data must be done with an understanding of
generally accepted testing practices regarding repeatability,
variance and statistical significance of small numbers of trials.
5. Requirements 5. Requirements
In this document, the words that are used to define the significance In this document, the words that are used to define the significance
of each particular requirement are capitalized. These words are: of each particular requirement are capitalized. These words are:
* "MUST" * "MUST" This word, or the words "REQUIRED" and "SHALL" mean that
This word or the adjective "REQUIRED" means that the item is the item is an absolute requirement of the specification.
an absolute requirement of the specification.
* "SHOULD" * "SHOULD" This word or the adjective "RECOMMENDED" means that there
This word or the adjective "RECOMMENDED" means that there may may exist valid reasons in particular circumstances to ignore this
exist valid reasons in particular circumstances to ignore this item, but the full implications should be understood and the case
item, but the full implications should be understood and the carefully weighed before choosing a different course.
case carefully weighed before choosing a different course.
* "MAY" * "MAY" This word or the adjective "OPTIONAL" means that this item
This word or the adjective "OPTIONAL" means that this item is is truly optional. One vendor may choose to include the item because
truly optional. One vendor may choose to include the item a particular marketplace requires it or because it enhances the
because a particular marketplace requires it or because it product, for example; another vendor may omit the same item.
enhances the product, for example; another vendor may omit the
same item.
An implementation is not compliant if it fails to satisfy one or An implementation is not compliant if it fails to satisfy one or more
more of the MUST requirements for the protocols it implements. An of the MUST requirements for the protocols it implements. An
implementation that satisfies all the MUST and all the SHOULD implementation that satisfies all the MUST and all the SHOULD
requirements for its protocols is said to be "unconditionally requirements for its protocols is said to be "unconditionally
compliant"; one that satisfies all the MUST requirements but not all compliant"; one that satisfies all the MUST requirements but not all
the SHOULD requirements for its protocols is said to be the SHOULD requirements for its protocols is said to be
"conditionally compliant". "conditionally compliant".
6. Device set up 6. Test set up
Before starting to perform the tests, the device to be tested MUST The ideal way to implement this series of tests is to use a tester
be configured following the instructions provided to the user. with both transmitting and receiving ports. Connections are made
Specifically, it is expected that all of the supported protocols from the sending ports of the tester to the receiving ports of the
will be configured and enabled during this set up (See Appendix A). DUT and from the sending ports of the DUT back to the tester. (see
It is expected that all of the tests will be run without changing figure 1) Since the tester both sends the test traffic and receives
the configuration or setup of the device in any way other than that it back, after the traffic has been forwarded but the DUT, the tester
required to do the specific test. For example, it is not acceptable can easily determine if all of the transmitted packets were received
to change the size of frame handling buffers between tests of frame and verify that the correct packets were received. The same
handling rates or to disable all but one transport protocol when functionality can be obtained with separate transmitting and
testing the throughput of that protocol. It is necessary to modify receiving devices (see figure 2) but unless they are remotely
the configuration when starting a test to determine the effect of controlled by some computer in a way that simulates the single
filters on throughput, but the only change MUST be to enable the tester, the labor required to accurately perform some of the tests
specific filter. The device set up SHOULD include the normally (particularly the throughput test) can be prohibitive.
recommended routing update intervals and keep alive frequency. The
specific version of the software and the exact device configuration,
including what device functions are disabled, used during the tests
SHOULD be included as part of the report of the results.
7. Frame formats +------------+
| |
+------------| tester |<-------------+
| | | |
| +------------+ |
| |
| +------------+ |
| | | |
+----------->| DUT |--------------+
| |
+------------+
figure 1
+--------+ +------------+ +----------+
| | | | | |
| sender |-------->| DUT |--------->| receiver |
| | | | | |
+--------+ +------------+ +----------+
figure 2
6.1 Test set up for multiple media types
Two different setups could be used to test a DUT which is used in
real-world networks to connect networks of differing media type,
local Ethernet to a backbone FDDI ring for example. The tester could
support both media types in which case the set up shown in figure 1
would be used.
Two identical DUTs are used in the other test set up. (see figure 3)
In many cases this set up may more accurately simulate the real
world. For example, connecting two LANs together with a WAN link or
high speed backbone. This set up would not be as good at simulating
a system where clients on a Ethernet LAN were interacting with a
server on an FDDI backbone.
+-----------+
| |
+---------------------| tester |<---------------------+
| | | |
| +-----------+ |
| |
| +----------+ +----------+ |
| | | | | |
+------->| DUT 1 |-------------->| DUT 2 |---------+
| | | |
+----------+ +----------+
figure 3
7. DUT set up
Before starting to perform the tests, the DUT to be tested MUST be
configured following the instructions provided to the user.
Specifically, it is expected that all of the supported protocols will
be configured and enabled during this set up (See Appendix A). It is
expected that all of the tests will be run without changing the
configuration or setup of the DUT in any way other than that required
to do the specific test. For example, it is not acceptable to change
the size of frame handling buffers between tests of frame handling
rates or to disable all but one transport protocol when testing the
throughput of that protocol. It is necessary to modify the
configuration when starting a test to determine the effect of filters
on throughput, but the only change MUST be to enable the specific
filter. The DUT set up SHOULD include the normally recommended
routing update intervals and keep alive frequency. The specific
version of the software and the exact DUT configuration, including
what functions are disabled, used during the tests MUST be included
as part of the report of the results.
8. Frame formats
The formats of the test frames to use for TCP/IP over Ethernet are The formats of the test frames to use for TCP/IP over Ethernet are
shown in Appendix C: Test Frame Formats. It is expected that these shown in Appendix C: Test Frame Formats. These exact frame formats
exact frame formats will be used in the tests described in this SHOULD be used in the tests described in this document for this
document for this protocol/media combination and that these frames protocol/media combination and that these frames will be used as a
will be used as a template for testing other protocol/media template for testing other protocol/media combinations. The specific
combinations. The specific formats that are used to define the test formats that are used to define the test frames for a particular test
frames for a particular test series MUST be included in the report series MUST be included in the report of the results.
of the results.
8. Frame sizes 9. Frame sizes
All of the described tests SHOULD be performed at a number of frame All of the described tests SHOULD be performed at a number of frame
sizes. Specifically, the sizes SHOULD include the maximum and sizes. Specifically, the sizes SHOULD include the maximum and minimum
minimum legitimate sizes for the protocol under test on the media legitimate sizes for the protocol under test on the media under test
under test and enough sizes in between to be able to get a full and enough sizes in between to be able to get a full characterization
characterization of the device performance. of the DUT performance. Except where noted, at least five frame
sizes SHOULD be tested for each test condition.
Except where noted, it is expected that at least five frame sizes
will be tested for each test condition.
Theoretically the minimum size UDP Echo request frame would consist Theoretically the minimum size UDP Echo request frame would consist
of an IP header (minimum length 20 octets), a UDP header (8 octets) of an IP header (minimum length 20 octets), a UDP header (8 octets)
and whatever MAC level header is required by the media in use. The and whatever MAC level header is required by the media in use. The
theoretical maximum frame size is determined by the size of the theoretical maximum frame size is determined by the size of the
length field in the IP header. In almost all cases the actual length field in the IP header. In almost all cases the actual
maximum and minimum sizes are determined by the limitations of the maximum and minimum sizes are determined by the limitations of the
media. media.
In theory it would be ideal to distribute the frame sizes in a way In theory it would be ideal to distribute the frame sizes in a way
that would evenly distribute the theoretical frame rates. These that would evenly distribute the theoretical frame rates. These
recommendations incorporate this theory but specify frame sizes recommendations incorporate this theory but specify frame sizes which
which are easy to understand and remember. In addition, many of the are easy to understand and remember. In addition, many of the same
same frame sizes are specified on each of the media types to allow frame sizes are specified on each of the media types to allow for
for easy performance comparisons. easy performance comparisons.
The inclusion of an unrealistically small frame size on some of the Note: The inclusion of an unrealistically small frame size on some of
media types (i.e. with little or no space for data) is to help the media types (i.e. with little or no space for data) is to help
characterize the per-frame processing overhead of the network characterize the per-frame processing overhead of the DUT.
connection device.
9.1 Frame sizes to be used on Ethernet
8.1 Frame sizes to be used on Ethernet
64, 128, 256, 512, 1024, 1280, 1518 64, 128, 256, 512, 1024, 1280, 1518
These sizes include the maximum and minimum frame sizes permitted by These sizes include the maximum and minimum frame sizes permitted
the Ethernet standard and a selection of sizes between these by the Ethernet standard and a selection of sizes between these
extremes with a finer granularity for the smaller frame sizes and extremes with a finer granularity for the smaller frame sizes and
higher frame rates. higher frame rates.
8.2 Frame sizes to be used on 4Mb and 16Mb token ring 9.2 Frame sizes to be used on 4Mb and 16Mb token ring
54, 64, 128, 256, 1024, 1518, 2048, 4472 54, 64, 128, 256, 1024, 1518, 2048, 4472
The frame size recommendations for token ring assume that there is The frame size recommendations for token ring assume that there is
no RIF field in the frames of routed protocols. A RIF field would no RIF field in the frames of routed protocols. A RIF field would
be present in any direct source route bridge performance test. The be present in any direct source route bridge performance test.
minimum size frame for UDP on token ring is 54 octets. The maximum The minimum size frame for UDP on token ring is 54 octets. The
size of 4472 octets is recommended for 16Mb token ring instead of maximum size of 4472 octets is recommended for 16Mb token ring
the theoretical size of 17.9Kb because of the size limitations instead of the theoretical size of 17.9Kb because of the size
imposed by many token ring interfaces. The reminder of the sizes limitations imposed by many token ring interfaces. The reminder
are selected to permit direct comparisons with other types of media. of the sizes are selected to permit direct comparisons with other
An IP (i.e. not UDP) frame may be used in addition if a higher data types of media. An IP (i.e. not UDP) frame may be used in
rate is desired, in which case the minimum frame size is 46 octets. addition if a higher data rate is desired, in which case the
minimum frame size is 46 octets.
9.3 Frame sizes to be used on FDDI
8.3 Frame sizes to be used on FDDI
54, 64, 128, 256, 1024, 1518, 2048, 4472 54, 64, 128, 256, 1024, 1518, 2048, 4472
The minimum size frame for UDP on FDDI is 53 octets, the minimum The minimum size frame for UDP on FDDI is 53 octets, the minimum
size of 54 is recommended to allow direct comparison to token ring size of 54 is recommended to allow direct comparison to token ring
performance. The maximum size of 4472 is recommended instead of the performance. The maximum size of 4472 is recommended instead of
theoretical maximum size of 4500 octets to permit the same type of the theoretical maximum size of 4500 octets to permit the same
comparison. An IP (i.e. not UDP) frame may be used in addition if a type of comparison. An IP (i.e. not UDP) frame may be used in
higher data rate is desired, in which case the minimum frame size is addition if a higher data rate is desired, in which case the
45 octets. minimum frame size is 45 octets.
8.4 Frame sizes in the presence of disparate MTUs
When the interconnect device supports connecting links with 9.4 Frame sizes in the presence of disparate MTUs
disparate MTUs, the frame sizes for the link with the *larger* MTU When the interconnect DUT supports connecting links with disparate
SHOULD be used, up to the limit of the protocol being tested. If the MTUs, the frame sizes for the link with the *larger* MTU SHOULD be
interconnect device does not support the fragmenting of frames in used, up to the limit of the protocol being tested. If the
the presence of MTU mismatch, the forwarding rate for that frame interconnect DUT does not support the fragmenting of frames in the
size shall be reported as zero (0). presence of MTU mismatch, the forwarding rate for that frame size
shall be reported as zero.
For example, the test of IP forwarding with a bridge or router that For example, the test of IP forwarding with a bridge or router
joins FDDI and Ethernet should use the frame sizes of FDDI when that joins FDDI and Ethernet should use the frame sizes of FDDI
going from the FDDI to the Ethernet link. If the bridge does not when going from the FDDI to the Ethernet link. If the bridge does
support IP fragmentation, the forwarding rate for those frames too not support IP fragmentation, the forwarding rate for those frames
large for Ethernet should be reported as zero. too large for Ethernet should be reported as zero.
9. Verifying received frames 10. Verifying received frames
The test equipment SHOULD discard any frames received during a test The test equipment SHOULD discard any frames received during a test
run that are not actual forwarded test frames. For example, keep- run that are not actual forwarded test frames. For example, keep-
alive and routing update frames SHOULD NOT be included in the count alive and routing update frames SHOULD NOT be included in the count
of received frames. In any case, the test equipment SHOULD verify of received frames. In any case, the test equipment SHOULD verify
the length of the received frames and check that they match the the length of the received frames and check that they match the
expected length. expected length.
Preferably, the test equipment SHOULD include sequence numbers in Preferably, the test equipment SHOULD include sequence numbers in the
the transmitted frames and check for these numbers on the received transmitted frames and check for these numbers on the received
frames. If this is done, the reported results SHOULD include in frames. If this is done, the reported results SHOULD include in
addition to the number of frames dropped, the number of frames that addition to the number of frames dropped, the number of frames that
were received out of order, the number of duplicate frames received were received out of order, the number of duplicate frames received
and the number of gaps in the received frame numbering sequence. and the number of gaps in the received frame numbering sequence.
This functionality is required for some of the described tests. This functionality is required for some of the described tests.
10. Modifiers 11. Modifiers
It might be useful to know the device performance under a number of It might be useful to know the DUT performance under a number of
conditions; some of these conditions are noted below. It is conditions; some of these conditions are noted below. The reported
expected that the reported results will include as many of these results SHOULD include as many of these conditions as the test
conditions as the test equipment is able to generate. The suite of equipment is able to generate. The suite of tests SHOULD be first
tests SHOULD be first run without any modifying conditions and then run without any modifying conditions and then repeated under each of
repeated under each of the conditions separately. To preserve the the conditions separately. To preserve the ability to compare the
ability to compare the results of these tests any frames that are results of these tests any frames that are required to generate the
required to generate the modifying conditions (management queries modifying conditions (management queries for example) will be
for example) will be included in the same data stream as the normal included in the same data stream as the normal test frames in place
test frames in place of one of the test frames and not be supplied of one of the test frames and not be supplied to the DUT on a
to the device on a separate network port. separate network port.
10.1 Broadcast frames 11.1 Broadcast frames
In most router designs special processing is required when frames In most router designs special processing is required when frames
addressed to the hardware broadcast address are received. In addressed to the hardware broadcast address are received. In
bridges (or in bridge mode on routers) these broadcast frames must bridges (or in bridge mode on routers) these broadcast frames must
be flooded to a number of ports. The stream of test frames SHOULD be flooded to a number of ports. The stream of test frames SHOULD
be augmented with 1% frames addressed to the hardware broadcast be augmented with 1% frames addressed to the hardware broadcast
address. The specific frames that should be used are included in address. The frames sent to the broadcast address should be of a
the test frame format document. The broadcast frames SHOULD be type that the router will not need to process. The aim of this
evenly distributed throughout the data stream, for example, every test is to determine if there is any effect on the forwarding rate
100th frame. of the other data in the stream. The specific frames that should
be used are included in the test frame format document. The
broadcast frames SHOULD be evenly distributed throughout the data
stream, for example, every 100th frame.
The same test SHOULD be performed on bridge-like DUTs but in this
case the broadcast packets will be processed and flooded to all
outputs.
It is understood that a level of broadcast frames of 1% is much It is understood that a level of broadcast frames of 1% is much
higher than many networks experience but, as in drug toxicity higher than many networks experience but, as in drug toxicity
evaluations, the higher level is required to be able to gage the evaluations, the higher level is required to be able to gage the
effect which would otherwise often fall within the normal effect which would otherwise often fall within the normal
variability of the system performance. Due to design factors some variability of the system performance. Due to design factors some
test equipment will not be able to generate a level of alternate test equipment will not be able to generate a level of alternate
frames this low. In these cases it is expected that the percentage frames this low. In these cases the percentage SHOULD be as small
would be as small as the equipment can provide and that the actual as the equipment can provide and that the actual level be
level be described in the report of the test results. described in the report of the test results.
10.2 Management frames 11.2 Management frames
Most data networks now make use of management protocols such as Most data networks now make use of management protocols such as
SNMP. In many environments there can be a number of management SNMP. In many environments there can be a number of management
stations sending queries to the same device at the same time. stations sending queries to the same DUT at the same time.
The stream of test frames SHOULD be augmented with one management The stream of test frames SHOULD be augmented with one management
query as the first frame sent each second during the duration of the query as the first frame sent each second during the duration of
trial. The result of the query must fit into one response frame. the trial. The result of the query must fit into one response
The response frame SHOULD be verified by the test equipment. One frame. The response frame SHOULD be verified by the test
example of the specific query frame that should be used is shown in equipment. One example of the specific query frame that should be
Appendix C. used is shown in Appendix C.
10.3 Routing update frames 11.3 Routing update frames
The processing of dynamic routing protocol updates could have a The processing of dynamic routing protocol updates could have a
significant impact on the ability of a router to forward data significant impact on the ability of a router to forward data
frames. The stream of test frames SHOULD be augmented with one frames. The stream of test frames SHOULD be augmented with one
routing update frame transmitted as the first frame transmitted routing update frame transmitted as the first frame transmitted
during the trial. Routing update frames SHOULD be sent at the rate during the trial. Routing update frames SHOULD be sent at the
specified in Appendix C for the specific routing protocol being used rate specified in Appendix C for the specific routing protocol
in the test. Two routing update frames are defined in Appendix C for being used in the test. Two routing update frames are defined in
the TCP/IP over Ethernet example. The routing frames are designed Appendix C for the TCP/IP over Ethernet example. The routing
to change the routing to a number of networks that are not involved frames are designed to change the routing to a number of networks
in the forwarding of the test data. The first frame sets the that are not involved in the forwarding of the test data. The
routing table state to "A", the second one changes the state to "B". first frame sets the routing table state to "A", the second one
The frames MUST be alternated during the trial. changes the state to "B". The frames MUST be alternated during
the trial.
The test SHOULD verify that the routing update was processed by the The test SHOULD verify that the routing update was processed by
device under test. the DUT.
10.4 Filters 11.4 Filters
Filters are added to routers and bridges to selectively inhibit the Filters are added to routers and bridges to selectively inhibit
forwarding of frames that would normally be forwarded. This is the forwarding of frames that would normally be forwarded. This
usually done to implement security controls on the data that is is usually done to implement security controls on the data that is
accepted between one area and another. Different products have accepted between one area and another. Different products have
different capabilities to implement filters. different capabilities to implement filters.
The device SHOULD be first configured to add one filter condition The DUT SHOULD be first configured to add one filter condition and
and the tests performed. This filter SHOULD permit the forwarding the tests performed. This filter SHOULD permit the forwarding of
of the test data stream. In routers this filter SHOULD be of the the test data stream. In routers this filter SHOULD be of the
form: form:
forward input_protocol_address to output_protocol_address forward input_protocol_address to output_protocol_address
In bridges the filter SHOULD be of the form: In bridges the filter SHOULD be of the form:
forward destination_hardware_address forward destination_hardware_address
The device SHOULD be then reconfigured to implement a total of 25 The DUT SHOULD be then reconfigured to implement a total of 25
filters. The first 24 of these filters SHOULD be of the form: filters. The first 24 of these filters SHOULD be of the form:
block input_protocol_address to output_protocol_address block input_protocol_address to output_protocol_address
The 24 input and output protocol addresses SHOULD not be any that The 24 input and output protocol addresses SHOULD not be any that
are represented in the test data stream. The last filter SHOULD are represented in the test data stream. The last filter SHOULD
permit the forwarding of the test data stream. By "first" and permit the forwarding of the test data stream. By "first" and
"last" we mean to ensure that in the second case, 25 conditions must "last" we mean to ensure that in the second case, 25 conditions
be checked before the data frames will match the conditions that must be checked before the data frames will match the conditions
permit the forwarding of the frame. that permit the forwarding of the frame. Of course, if the DUT
reorders the filters or does not use a linear scan of the filter
rules the effect of the sequence in which the filters are input is
properly lost.
The exact filters configuration command lines used SHOULD be The exact filters configuration command lines used SHOULD be
included with the report of the results. included with the report of the results.
10.4.1 Filter Addresses 11.4.1 Filter Addresses
Two sets of filter addresses are required, one for the single filter Two sets of filter addresses are required, one for the single
case and one for the 25 filter case. filter case and one for the 25 filter case.
The single filter case should permit traffic from IP address The single filter case should permit traffic from IP address
198.18.1.2 to IP address 198.19.65.2 and deny all other traffic. 198.18.1.2 to IP address 198.19.65.2 and deny all other
traffic.
The 25 filter case should follow the following sequence. The 25 filter case should follow the following sequence.
allow aa.ba.1.1 to aa.ba.100.1 deny aa.ba.1.1 to aa.ba.100.1
allow aa.ba.2.2 to aa.ba.101.2 deny aa.ba.2.2 to aa.ba.101.2
allow aa.ba.3.3 to aa.ba.103.3 deny aa.ba.3.3 to aa.ba.103.3
... ...
allow aa.ba.12.12 to aa.ba.112.12 deny aa.ba.12.12 to aa.ba.112.12
allow aa.bc.1.2 to aa.bc.65.1 allow aa.bc.1.2 to aa.bc.65.1
allow aa.ba.13.13 to aa.ba.113.13 deny aa.ba.13.13 to aa.ba.113.13
allow aa.ba.14.14 to aa.ba.114.14 deny aa.ba.14.14 to aa.ba.114.14
... ...
allow aa.ba.24.24 to aa.ba.124.24 deny aa.ba.24.24 to aa.ba.124.24
deny all else deny all else
All previous filter conditions should be cleared from the router All previous filter conditions should be cleared from the
before this sequence is entered. The sequence is selected to test router before this sequence is entered. The sequence is
to see if the router sorts the filter conditions or accepts them in selected to test to see if the router sorts the filter
the order that they were entered. Both of these procedures will conditions or accepts them in the order that they were entered.
result in a greater reduction in performance than will some form of Both of these procedures will result in a greater impact on
hash coding. performance than will some form of hash coding.
11. Protocol addresses 12. Protocol addresses
It is easier to implement these tests using a single logical stream It is easier to implement these tests using a single logical stream
of data, with one source protocol address and one destination of data, with one source protocol address and one destination
protocol address, and for some conditions like the filters described protocol address, and for some conditions like the filters described
above, a practical requirement. Networks in the real world are not above, a practical requirement. Networks in the real world are not
limited to single streams of data. The test suite SHOULD be first limited to single streams of data. The test suite SHOULD be first run
run with a single protocol (or hardware for bridge tests) source and with a single protocol (or hardware for bridge tests) source and
destination address pair. The tests SHOULD then be repeated with destination address pair. The tests SHOULD then be repeated with
using a random destination address. While testing routers the using a random destination address. While testing routers the
addresses SHOULD be random over a range of 256 networks and random addresses SHOULD be random and uniformly distributed over a range of
over the full MAC range for bridges. The specific address ranges to 256 networks and random and uniformly distributed over the full MAC
use for IP are shown in Appendix C. range for bridges. The specific address ranges to use for IP are
shown in Appendix C.
12. Route Set Up 13. Route Set Up
It is not expected that all of the routing information necessary to It is not reasonable that all of the routing information necessary to
forward the test stream, especially in the multiple address case, forward the test stream, especially in the multiple address case,
will be manually set up. At the start of each trial a routing will be manually set up. At the start of each trial a routing update
update MUST be sent to the device. This routing update MUST include MUST be sent to the DUT. This routing update MUST include all of the
all of the network addresses that will be required for the trial. network addresses that will be required for the trial. All of the
All of the addresses SHOULD resolve to the same "next-hop" and it is addresses SHOULD resolve to the same "next-hop". Normally this will
expected that this will be the address of the receiving side of the be the address of the receiving side of the test equipment. This
test equipment. This routing update will have to be repeated at the routing update will have to be repeated at the interval required by
interval required by the routing protocol being used. An example of the routing protocol being used. An example of the format and
the format and repetition interval of the update frames is given in repetition interval of the update frames is given in Appendix C.
Appendix C.
13. Bidirectional traffic 14. Bidirectional traffic
Normal network activity is not all in a single direction. To test Normal network activity is not all in a single direction. To test
the bidirectional performance of a device, the test series SHOULD be the bidirectional performance of a DUT, the test series SHOULD be run
run with the same data rate being offered from each direction. The with the same data rate being offered from each direction. The sum of
sum of the data rates should not exceed the theoretical limit for the data rates should not exceed the theoretical limit for the media.
the media.
14. Single stream path 15. Single stream path
The full suite of tests SHOULD be run along with whatever modifier The full suite of tests SHOULD be run along with whatever modifier
conditions that are relevant using a single input and output network conditions that are relevant using a single input and output network
port on the device. If the internal design of the device has port on the DUT. If the internal design of the DUT has multiple
multiple distinct pathways, for example, multiple interface cards distinct pathways, for example, multiple interface cards each with
each with multiple network ports, then all possible types of multiple network ports, then all possible types of pathways SHOULD be
pathways SHOULD be tested separately. tested separately.
15. Multi-port 16. Multi-port
Many current router and bridge products provide many network ports Many current router and bridge products provide many network ports in
in the same device. In performing these tests first half of the the same module. In performing these tests first half of the ports
ports are designated as "input ports" and half are designated as are designated as "input ports" and half are designated as "output
"output ports". These ports SHOULD be evenly distributed across the ports". These ports SHOULD be evenly distributed across the DUT
device architecture. For example if a device has two interface cards architecture. For example if a DUT has two interface cards each of
each of which has four ports, two ports on each interface card are which has four ports, two ports on each interface card are designated
designated as input and two are designated as output. as input and two are designated as output. The specified tests are
run using the same data rate being offered to each of the input
ports. The addresses in the input data streams SHOULD be set so that
a frame will be directed to each of the output ports in sequence so
that all "output" ports will get an even distribution of packets from
this input. The same configuration MAY be used to perform a
bidirectional multi-stream test. In this case all of the ports are
considered both input and output ports and each data stream MUST
consist of frames addressed to all of the other ports.
The specified tests are run using the same data rate being offered Consider the following 6 port DUT:
to each of the input ports. The addresses in the input data streams
SHOULD be set so that a frame will be directed to each of the output
ports in sequence. The stream offered to input one SHOULD consist
of a series of frames (one destined to each of the output ports), as
SHOULD the frame stream offered to input two. The same configuration
MAY be used to perform a bidirectional multi-stream test. In this
case all of the ports are considered both input and output ports and
each data stream MUST consist of frames addressed to all of the
other ports.
16. Multiple protocols --------------
---------| in A out X|--------
---------| in B out Y|--------
---------| in C out Z|--------
--------------
The addressing of the data streams for each of the inputs SHOULD be:
stream sent to input A:
packet to out X, packet to out Y, packet to out Z
stream sent to input B:
packet to out X, packet to out Y, packet to out Z
stream sent to input C
packet to out X, packet to out Y, packet to out Z
Note that these streams each follow the same sequence so that 3
packets will arrive at output X at the same time, then 3 packets at
Y, then 3 packets at Z. This procedure ensures that, as in the real
world, the DUT will have to deal with multiple packets addressed to
the same output at the same time.
17. Multiple protocols
This document does not address the issue of testing the effects of a This document does not address the issue of testing the effects of a
mixed protocol environment other than to suggest that if such tests mixed protocol environment other than to suggest that if such tests
are wanted then frames SHOULD be distributed between all of the test are wanted then frames SHOULD be distributed between all of the test
protocols. The distribution MAY approximate the conditions on the protocols. The distribution MAY approximate the conditions on the
network in which the device would be used. network in which the DUT would be used.
17. Multiple frame sizes 18. Multiple frame sizes
This document does not address the issue of testing the effects of a This document does not address the issue of testing the effects of a
mixed frame size environment other than to suggest that if such mixed frame size environment other than to suggest that if such tests
tests are wanted then frames SHOULD be distributed between all of are wanted then frames SHOULD be distributed between all of the
the listed sizes for the protocol under test. The distribution MAY listed sizes for the protocol under test. The distribution MAY
approximate the conditions on the network in which the device would approximate the conditions on the network in which the DUT would be
be used. used. The authors do not have any idea how the results of such a test
would be interpreted other than to directly compare multiple DUTs in
some very specific simulated network.
18. Testing performance beyond a single device. 19. Testing performance beyond a single DUT.
In the performance testing of a single device, the paradigm can be In the performance testing of a single DUT, the paradigm can be
described as applying some input to a device under test and described as applying some input to a DUT and monitoring the output.
monitoring the output. The results of which can be used to form a The results of which can be used to form a basis of characterization
basis of characterization of that device under those test of that device under those test conditions.
conditions.
This model is useful when the test input and output are homogenous This model is useful when the test input and output are homogenous
(e.g., 64-byte IP, 802.3 frames into the device under test; 64 IP, (e.g., 64-byte IP, 802.3 frames into the DUT; 64 byte IP, 802.3
802.3 frames out), or the method of test can distinguish between frames out), or the method of test can distinguish between dissimilar
dissimilar input/output. (E.g., 1518 byte, IP, 802.3 frames in; 576 input/output. (E.g., 1518 byte IP, 802.3 frames in; 576 byte,
byte, fragmented IP, X.25 frames out.) fragmented IP, X.25 frames out.)
By extending the single device test model, reasonable benchmarks By extending the single DUT test model, reasonable benchmarks
regarding multiple devices or heterogeneous environments may be regarding multiple DUTs or heterogeneous environments may be
collected. In this extension, the single device under test is collected. In this extension, the single DUT is replaced by a system
replaced by a system of interconnected network devices. This test of interconnected network DUTs. This test methodology would support
methodology would support the benchmarking of a variety of the benchmarking of a variety of device/media/service/protocol
device/media/service/protocol combinations. For example, a combinations. For example, a configuration for a LAN-to-WAN-to-LAN
configuration for a LAN-to-WAN-to-LAN test might be: test might be:
(1) 802.3-> device 1 -> X.25 @ 64kbps -> device 2 -> 802.3 (1) 802.3-> DUT 1 -> X.25 @ 64kbps -> DUT 2 -> 802.3
Or a mixed LAN configuration might be: Or a mixed LAN configuration might be:
(2) 802.3 -> device 1 -> FDDI -> device 2 -> FDDI -> device 3 -> (2) 802.3 -> DUT 1 -> FDDI -> DUT 2 -> FDDI -> DUT 3 -> 802.3
802.3
In both examples 1 and 2, end-to-end benchmarks of each system could In both examples 1 and 2, end-to-end benchmarks of each system could
be empirically ascertained. Other behavior may be characterized be empirically ascertained. Other behavior may be characterized
through the use of intermediate devices. In example 2, the through the use of intermediate devices. In example 2, the
configuration may be used to give an indication of the FDDI to FDDI configuration may be used to give an indication of the FDDI to FDDI
capability exhibited by device 2. capability exhibited by DUT 2.
Because multiple devices are treated as a single system, there are Because multiple DUTs are treated as a single system, there are
limitations to this methodology. For instance, this methodology may limitations to this methodology. For instance, this methodology may
yield an aggregate benchmark for a tested system. That benchmark yield an aggregate benchmark for a tested system. That benchmark
alone, however, may not necessarily reflect asymmetries in behavior alone, however, may not necessarily reflect asymmetries in behavior
between the devices, latencies introduce by other apparatus (e.g., between the DUTs, latencies introduce by other apparatus (e.g.,
CSUs/DSUs, switches), etc. CSUs/DSUs, switches), etc.
Further, care must be used when comparing benchmarks of different Further, care must be used when comparing benchmarks of different
systems by ensuring that the devices' features/configuration of the systems by ensuring that the DUTs' features/configuration of the
tested systems have the appropriate common denominators to allow tested systems have the appropriate common denominators to allow
comparison. comparison.
20. Maximum frame rate
The maximum frame rates that should be used when testing LAN
connections SHOULD be the listed theoretical maximum rate for the
frame size on the media.
The maximum frame rate that should be used when testing WAN The maximum frame rate that should be used when testing WAN
connections SHOULD be greater than the listed theoretical maximum connections SHOULD be greater than the listed theoretical maximum
rate for the frame size on that speed connection. The higher rate rate for the frame size on that speed connection. The higher rate
for WAN tests is to compensate for the fact that some vendors employ for WAN tests is to compensate for the fact that some vendors employ
various forms of header compression. See Appendix A. various forms of header compression.
19. Maximum frame rate A list of maximum frame rates for LAN connections is included in
The maximum frame rate that should be used when testing LAN Appendix B.
connections SHOULD be the listed theoretical maximum rate for the
frame size on the media. A list of maximum frame rates for LAN
connections is included in Appendix B.
20. Bursty traffic 21. Bursty traffic
It is convenient to measure the device performance under steady It is convenient to measure the DUT performance under steady state
state load but this is an unrealistic way to gage the functioning of load but this is an unrealistic way to gauge the functioning of a DUT
a device since actual network traffic normally consists of bursts of since actual network traffic normally consists of bursts of frames.
frames. Some of the tests described below SHOULD be performed with Some of the tests described below SHOULD be performed with both
both steady state traffic and with traffic consisting of repeated steady state traffic and with traffic consisting of repeated bursts
bursts of frames. The frames within a burst are transmitted with of frames. The frames within a burst are transmitted with the
the minimum legitimate inter-frame gap. minimum legitimate inter-frame gap.
The objective of the test is to determine the minimum interval The objective of the test is to determine the minimum interval
between bursts which the device under test can process with no frame between bursts which the DUT can process with no frame loss. During
loss. During each test the number of frames in each burst is held each test the number of frames in each burst is held constant and the
constant and the inter-burst interval varied. Tests SHOULD be run inter-burst interval varied. Tests SHOULD be run with burst sizes of
with burst sizes of 16, 64, 256 and 1024 frames. 16, 64, 256 and 1024 frames.
21. Frames per token 22. Frames per token
Although it is possible to configure some token ring and FDDI Although it is possible to configure some token ring and FDDI
interfaces to transmit more than one frame each time that the token interfaces to transmit more than one frame each time that the token
is received, most of the network devices currently available is received, most of the network devices currently available transmit
transmit only one frame per token. These tests SHOULD first be only one frame per token. These tests SHOULD first be performed
performed while transmitting only one frame per token. while transmitting only one frame per token.
Some current high-performance workstation servers do transmit more Some current high-performance workstation servers do transmit more
than one frame per token on FDDI to maximize throughput. Since this than one frame per token on FDDI to maximize throughput. Since this
may be a common feature in future workstations and servers, may be a common feature in future workstations and servers,
interconnect devices with FDDI interfaces SHOULD be tested with 1, interconnect devices with FDDI interfaces SHOULD be tested with 1, 4,
4, 8, and 16 frames per token. The reported frame rate SHOULD be 8, and 16 frames per token. The reported frame rate SHOULD be the
the average rate of frame transmission over the total trial period. average rate of frame transmission over the total trial period.
22. Trial description 23. Trial description
A particular test consists of multiple trials. Each trial returns A particular test consists of multiple trials. Each trial returns
one piece of information, for example the loss rate at a particular one piece of information, for example the loss rate at a particular
input frame rate. Each trial consists of a number of phases: input frame rate. Each trial consists of a number of phases:
a) If the test device is a router, send the routing update to a) If the DUT is a router, send the routing update to the "input"
the "input" port and pause two seconds to be sure that the port and pause two seconds to be sure that the routing has settled.
routing has settled.
b) Send the "learning frames" to the "output" port and wait 2 b) Send the "learning frames" to the "output" port and wait 2
seconds to be sure that the learning has settled. Bridge seconds to be sure that the learning has settled. Bridge learning
learning frames are frames with source addresses that are the frames are frames with source addresses that are the same as the
same as the destination addresses used by the test frames. destination addresses used by the test frames. Learning frames for
Learning frames for other protocols are used to prime the other protocols are used to prime the address resolution tables in
address resolution tables in the device. The formats of the the DUT. The formats of the learning frame that should be used are
learning frame that should be used are shown in the Test Frame shown in the Test Frame Formats document.
Formats document.
c) Run the test trial. c) Run the test trial.
d) Wait for two sec for any residual frames to be received. d) Wait for two seconds for any residual frames to be received.
e) Wait for at least five seconds for the device to e) Wait for at least five seconds for the DUT to restabilize.
restabilize.
23. Trial duration 24. Trial duration
The aim of these tests is to determine the rate continuously The aim of these tests is to determine the rate continuously
supportable by the device. The actual duration of the test trials supportable by the DUT. The actual duration of the test trials must
must be a compromise between this aim and the duration of the be a compromise between this aim and the duration of the benchmarking
benchmarking test suite. The duration of the test portion of each test suite. The duration of the test portion of each trial SHOULD be
trial SHOULD be at least 60 seconds. The tests that involve some at least 60 seconds. The tests that involve some form of "binary
form of "binary search", for example the throughput test, to search", for example the throughput test, to determine the exact
determine the exact result MAY use a shorter trial duration to result MAY use a shorter trial duration to minimize the length of the
minimize the length of the search procedure, but it is expected that search procedure, but the final determination SHOULD be made with
the final determination will be made with full length trials. full length trials.
24 Address resolution 25. Address resolution
The test device SHOULD be able to respond to address resolution The DUT SHOULD be able to respond to address resolution requests sent
requests sent by the device under test wherever the protocol by the DUT wherever the protocol requires such a process.
requires such a process.
25 Benchmarking tests: 26. Benchmarking tests:
Note: The notation "type of data stream" refers to the above Note: The notation "type of data stream" refers to the above
modifications to a frame stream with a constant inter-frame gap, for modifications to a frame stream with a constant inter-frame gap, for
example, the addition of traffic filters to the configuration of the example, the addition of traffic filters to the configuration of the
device under test. DUT.
26.1 Throughput
25.1 Throughput
Objective: Objective:
To determine the device throughput as defined in RFC 1242. To determine the DUT throughput as defined in RFC 1242.
Procedure: Procedure:
Send a specific number of frames at a specific rate through the Send a specific number of frames at a specific rate through the
device and then count the frames that are transmitted by the device. DUT and then count the frames that are transmitted by the DUT. If
If the count of offered frames is equal to the count of received the count of offered frames is equal to the count of received
frames, the rate of the offered stream is raised and the test rerun. frames, the rate of the offered stream is raised and the test
If fewer frames are received than were transmitted, the rate of the rerun. If fewer frames are received than were transmitted, the
offered stream is reduced and the test is rerun. rate of the offered stream is reduced and the test is rerun.
The throughput is the fastest rate at which the count of test frames The throughput is the fastest rate at which the count of test
transmitted by the DUT is equal to the number of test frames sent to frames transmitted by the DUT is equal to the number of test
it by the test equipment. frames sent to it by the test equipment.
Reporting format: Reporting format:
The results of the throughput test SHOULD be reported in the form of The results of the throughput test SHOULD be reported in the form
a graph. If it is, the x coordinate SHOULD be the frame size, the y of a graph. If it is, the x coordinate SHOULD be the frame size,
coordinate SHOULD be the frame rate. There SHOULD be at least two the y coordinate SHOULD be the frame rate. There SHOULD be at
lines on the graph. There SHOULD be one line showing the least two lines on the graph. There SHOULD be one line showing
theoretical frame rate for the media at the various frame sizes. the theoretical frame rate for the media at the various frame
The second line SHOULD be the plot of the test results. Additional sizes. The second line SHOULD be the plot of the test results.
lines MAY be used on the graph to report the results for each type Additional lines MAY be used on the graph to report the results
of data stream tested. Text accompanying the graph SHOULD indicate for each type of data stream tested. Text accompanying the graph
the protocol, data stream format, and type of media used in the SHOULD indicate the protocol, data stream format, and type of
tests. media used in the tests.
We assume that if a single value is desired for advertising purposes We assume that if a single value is desired for advertising
the vendor will select the rate for the minimum frame size for the purposes the vendor will select the rate for the minimum frame
media. If this is done then the figure MUST be expressed in frames size for the media. If this is done then the figure MUST be
per second. The rate MAY also be expressed in bits (or bytes) per expressed in frames per second. The rate MAY also be expressed in
second if the vendor so desires. The statement of performance MUST bits (or bytes) per second if the vendor so desires. The
include a/ the measured maximum frame rate, b/ the size of the frame statement of performance MUST include a/ the measured maximum
used, c/ the theoretical limit of the media for that frame size, and frame rate, b/ the size of the frame used, c/ the theoretical
d/ the type of protocol used in the test. Even if a single value is limit of the media for that frame size, and d/ the type of
used as part of the advertising copy, the full table of results protocol used in the test. Even if a single value is used as part
SHOULD be included in the product data sheet. of the advertising copy, the full table of results SHOULD be
included in the product data sheet.
26.2 Latency
25.2 Latency
Objective: Objective:
To determine the latency as defined in RFC 1242. To determine the latency as defined in RFC 1242.
Procedure: Procedure:
First determine the throughput for device at each of the listed First determine the throughput for DUT at each of the listed frame
frame sizes. Send a stream of frames at a particular frame size through
sizes. the DUT at the determined throughput rate to a specific
destination. The stream SHOULD be at least 120 seconds in
Send a stream of frames at a particular frame size through the duration. An identifying tag SHOULD be included in one frame
device at the determined throughput rate to a specific destination. after 60 seconds with the type of tag being implementation
The stream SHOULD be at least 120 seconds in duration. An dependent. The time at which this frame is fully transmitted is
identifying tag SHOULD be included in one frame after 60 seconds recorded, i.e. the last bit has been transmitted (timestamp A).
with the type of tag being implementation dependent. The time at The receiver logic in the test equipment MUST be able to recognize
which this frame is fully transmitted is recorded, i.e. the last bit the tag information in the frame stream and record the time at
has been transmitted (timestamp A). The receiver logic in the test which the entire tagged frame was received (timestamp B).
equipment MUST be able to recognize the tag information in the frame
stream and record the time at which the entire tagged frame was
received (timestamp B).
The latency is timestamp B minus timestamp A minus the transit time The latency is timestamp B minus timestamp A minus the transit
for a frame of the tested size on the tested media. This time for a frame of the tested size on the tested media. This
calculation may result in a negative value for those devices that calculation may result in a negative value for those DUTs that
begin to transmit the output frame before the entire input frame has begin to transmit the output frame before the entire input frame
been received. has been received.
The test MUST be repeated at least 20 times with the reported value The test MUST be repeated at least 20 times with the reported
being the average of the recorded values. value being the average of the recorded values.
This test SHOULD be performed with the test frame addressed to the This test SHOULD be performed with the test frame addressed to the
same destination as the rest of the data stream and also with each same destination as the rest of the data stream and also with each
of the test frames addressed to a new destination network. of the test frames addressed to a new destination network.
Reporting format: Reporting format:
The latency results SHOULD be reported in the format of a table with The latency results SHOULD be reported in the format of a table
a row for each of the tested frame sizes. There SHOULD be columns with a row for each of the tested frame sizes. There SHOULD be
for the frame size, the rate at which the latency test was run for columns for the frame size, the rate at which the latency test was
that frame size, for the media types tested, and for the resultant run for that frame size, for the media types tested, and for the
latency values for each type of data stream tested. resultant latency values for each type of data stream tested.
26.3 Frame loss rate
25.3 Frame loss rate
Objective: Objective:
To determine the frame loss rate, as defined in RFC 1242, of a To determine the frame loss rate, as defined in RFC 1242, of a DUT
device throughout the entire range of input data rates and frame throughout the entire range of input data rates and frame sizes.
sizes.
Procedure: Procedure:
Send a specific number of frames at a specific rate through the Send a specific number of frames at a specific rate through the
device to be tested and count the frames that are transmitted by the DUT to be tested and count the frames that are transmitted by the
device. The frame loss rate at each point is calculated using the DUT. The frame loss rate at each point is calculated using the
following following equation:
equation:
( ( input_count - output_count ) * 100 ) / input_count ( ( input_count - output_count ) * 100 ) / input_count
The first trial SHOULD be run for the frame rate that corresponds to The first trial SHOULD be run for the frame rate that corresponds
100% of the maximum rate for the frame size on the input media. to 100% of the maximum rate for the frame size on the input media.
Repeat the procedure for the rate that corresponds to 90% of the Repeat the procedure for the rate that corresponds to 90% of the
maximum rate used and then for 80% of this rate. This sequence maximum rate used and then for 80% of this rate. This sequence
SHOULD be continued (at reducing 10% intervals) until there are two SHOULD be continued (at reducing 10% intervals) until there are
successive trials in which no frames are lost. The maximum two successive trials in which no frames are lost. The maximum
granularity of the trials MUST be 10% of the maximum rate, a finer granularity of the trials MUST be 10% of the maximum rate, a finer
granularity is encouraged. granularity is encouraged.
Reporting format: Reporting format:
The results of the frame loss rate test SHOULD be plotted as a The results of the frame loss rate test SHOULD be plotted as a
graph. If this is done then the X axis MUST be the input frame rate graph. If this is done then the X axis MUST be the input frame
as a percent of the theoretical rate for the media at the specific rate as a percent of the theoretical rate for the media at the
frame size. The Y axis MUST be the percent loss at the particular specific frame size. The Y axis MUST be the percent loss at the
input rate. The left end of the X axis and the bottom of the Y axis particular input rate. The left end of the X axis and the bottom
MUST be 0 percent; the right end of the X axis and the top of the Y of the Y axis MUST be 0 percent; the right end of the X axis and
axis MUST be 100 percent. Multiple lines on the graph MAY used to the top of the Y axis MUST be 100 percent. Multiple lines on the
report the frame loss rate for different frame sizes, protocols, and graph MAY used to report the frame loss rate for different frame
types of data streams. sizes, protocols, and types of data streams.
Note: See section 18 for the maximum frame rates that SHOULD be Note: See section 18 for the maximum frame rates that SHOULD be
used. used.
25.4 Back-to-back frames 26.4 Back-to-back frames
Objective: Objective:
To characterize the ability of a device to process back-to-back To characterize the ability of a DUT to process back-to-back
frames as defined in RFC 1242. frames as defined in RFC 1242.
Procedure: Procedure:
Send a burst of frames with minimum inter-frame gaps to the device Send a burst of frames with minimum inter-frame gaps to the DUT
and count the number of frames forwarded by the device. If the and count the number of frames forwarded by the DUT. If the count
count of transmitted frames is equal to the number of frames of transmitted frames is equal to the number of frames forwarded
forwarded the length of the burst is increased and the test is the length of the burst is increased and the test is rerun. If
rerun. If the number of forwarded frames is less than the number the number of forwarded frames is less than the number
transmitted, the length of the burst is reduced and the test is transmitted, the length of the burst is reduced and the test is
rerun. rerun.
The back-to-back value is the number of frames in the longest burst The back-to-back value is the number of frames in the longest
that the device will handle without the loss of any frames. burst that the DUT will handle without the loss of any frames.
The trial length MUST be at least 2 seconds and SHOULD be
The trial length MUST be at least 2 seconds and SHOULD be repeated repeated at least 50 times with the average of the recorded values
at least 50 times with the average of the recorded values being being reported.
reported.
Reporting format: Reporting format:
The back-to-back results SHOULD be reported in the format of a table The back-to-back results SHOULD be reported in the format of a
with a row for each of the tested frame sizes. There SHOULD be table with a row for each of the tested frame sizes. There SHOULD
columns for the frame size and for the resultant average frame count be columns for the frame size and for the resultant average frame
for each type of data stream tested. The standard deviation for count for each type of data stream tested. The standard deviation
each measurement MAY also be reported. for each measurement MAY also be reported.
26.5 System recovery
25.5 System recovery
Objective: Objective:
To characterize the speed at which a device recovers from an To characterize the speed at which a DUT recovers from an overload
overload condition. condition.
Procedure: Procedure:
First determine the throughput for a device at each of the listed First determine the throughput for a DUT at each of the listed
frame sizes. frame sizes.
Send a stream of frames at a rate 110% of the recorded throughput Send a stream of frames at a rate 110% of the recorded throughput
rate or the maximum rate for the media, whichever is lower, for at rate or the maximum rate for the media, whichever is lower, for at
least 60 seconds. At Timestamp A reduce the frame rate to 50% of least 60 seconds. At Timestamp A reduce the frame rate to 50% of
the above rate and record the time of the last frame lost (Timestamp the above rate and record the time of the last frame lost
B). The system recovery time is determined by subtracting Timestamp (Timestamp B). The system recovery time is determined by
A from Timestamp B. The test SHOULD be repeated a number of times subtracting Timestamp A from Timestamp B. The test SHOULD be
and the average of the recorded values being reported. repeated a number of times and the average of the recorded values
being reported.
Reporting format: Reporting format:
The system recovery results SHOULD be reported in the format of a The system recovery results SHOULD be reported in the format of a
table with a row for each of the tested frame sizes. There SHOULD table with a row for each of the tested frame sizes. There SHOULD
be columns for the frame size, the frame rate used as the throughput be columns for the frame size, the frame rate used as the
rate for each type of data stream tested, and for the measured throughput rate for each type of data stream tested, and for the
recovery time for each type of data stream tested. measured recovery time for each type of data stream tested.
26.6 Reset
25.6 Reset
Objective: Objective:
To characterize the speed at which a device recovers from a device To characterize the speed at which a DUT recovers from a device or
or software reset. software reset.
Procedure: Procedure:
First determine the throughput for the device for the minimum frame First determine the throughput for the DUT for the minimum frame
size on the media used in the testing. size on the media used in the testing.
Send a continuous stream of frames at the determined throughput rate Send a continuous stream of frames at the determined throughput
for the minimum sized frames. Cause a reset in the device. Monitor rate for the minimum sized frames. Cause a reset in the DUT.
the output until frames begin to be forwarded and record the time Monitor the output until frames begin to be forwarded and record
that the last frame (Timestamp A) of the initial stream and the the time that the last frame (Timestamp A) of the initial stream
first frame of the new stream (Timestamp B) are received. and the first frame of the new stream (Timestamp B) are received.
A power interruption reset test is performed as above except that A power interruption reset test is performed as above except that
the power to the device should be interrupted for 10 seconds in the power to the DUT should be interrupted for 10 seconds in place
place of causing a reset. of causing a reset.
This test SHOULD only be run using frames addressed to networks This test SHOULD only be run using frames addressed to networks
directly connected to the device under test so that there is no directly connected to the DUT so that there is no requirement to
requirement to delay until a routing update is received. delay until a routing update is received.
The reset value is obtained by subtracting Timestamp A from The reset value is obtained by subtracting Timestamp A from
Timestamp B. Timestamp B.
Hardware and software resets, as well as a power interruption SHOULD Hardware and software resets, as well as a power interruption
be tested. SHOULD be tested.
Reporting format: Reporting format:
The reset value SHOULD be reported in a simple set of statements, The reset value SHOULD be reported in a simple set of statements,
one for each reset type. one for each reset type.
26. Security Considerations 27. Security Considerations
Security issues are not addressed in this document. Security issues are not addressed in this document.
27. Editor's Address 28. Editor's Addresses
Scott Bradner Scott Bradner
Holyoke Center Phone +1 617 495-3864 Harvard University Phone +1 617 495-3864
Harvard University Fax +1 617 495-0914 1350 Mass. Ave, room 813 Fax +1 617 496-8500
Cambridge, MA 02138 Email: sob@harvard.edu Cambridge, MA 02138 Email: sob@harvard.edu
Jim McQuaid Jim McQuaid
Wandel & Goltermann Technologies, Inc Phone +1 919 941-4730 Wandel & Goltermann Technologies, Inc Phone +1 919 941-4730
P. O. Box 13585 Fax: +1 919 941-5751 P. O. Box 13585 Fax: +1 919 941-5751
Research Triangle Park, NC 27709 Email: mcquaid@wg.com Research Triangle Park, NC 27709 Email: mcquaid@wg.com
Appendix A: Testing Considerations Appendix A: Testing Considerations
A.1 Scope Of This Appendix A.1 Scope Of This Appendix
skipping to change at page 1, line 792 skipping to change at page 19, line 24
Cambridge, MA 02138 Email: sob@harvard.edu Cambridge, MA 02138 Email: sob@harvard.edu
Jim McQuaid Jim McQuaid
Wandel & Goltermann Technologies, Inc Phone +1 919 941-4730 Wandel & Goltermann Technologies, Inc Phone +1 919 941-4730
P. O. Box 13585 Fax: +1 919 941-5751 P. O. Box 13585 Fax: +1 919 941-5751
Research Triangle Park, NC 27709 Email: mcquaid@wg.com Research Triangle Park, NC 27709 Email: mcquaid@wg.com
Appendix A: Testing Considerations Appendix A: Testing Considerations
A.1 Scope Of This Appendix A.1 Scope Of This Appendix
This appendix discusses certain issues in the benchmarking This appendix discusses certain issues in the benchmarking
methodology where experience or judgement may play a role in the methodology where experience or judgment may play a role in the tests
tests selected to be run or in the approach to constructing the test selected to be run or in the approach to constructing the test with a
with a particular device. As such, this appendix MUST not be read particular DUT. As such, this appendix MUST not be read as an
as an amendment to the methodology described in the body of this amendment to the methodology described in the body of this document
document but as a guide to testing practice. but as a guide to testing practice.
1. Typical testing practice has been to enable all protocols to 1. Typical testing practice has been to enable all protocols to be
be tested and conduct all testing with no further tested and conduct all testing with no further configuration of
configuration of protocols, even though a given set of trials protocols, even though a given set of trials may exercise only one
may exercise only one protocol at a time. This minimizes the protocol at a time. This minimizes the opportunities to "tune" a
opportunities to "tune" a device under test for a single DUT for a single protocol.
protocol.
2. The least common denominator of the available filter functions 2. The least common denominator of the available filter functions
should be used to ensure that there is a basis for comparison should be used to ensure that there is a basis for comparison
between vendors. Because of product differences, those between vendors. Because of product differences, those conducting
conducting and evaluating tests must make a judgement about and evaluating tests must make a judgment about this issue.
this issue.
3. Architectural considerations may need to be considered. For 3. Architectural considerations may need to be considered. For
example, first perform the tests with the stream going between example, first perform the tests with the stream going between
ports on the same interface card and the repeat the tests with ports on the same interface card and the repeat the tests with the
the stream going into a port on one interface card and out of stream going into a port on one interface card and out of a port
a port on a second interface card. There will almost always be on a second interface card. There will almost always be a best
a best case and worst case configuration for a given device case and worst case configuration for a given DUT architecture.
under test architecture.
4. Testing done using traffic streams consisting of mixed 4. Testing done using traffic streams consisting of mixed protocols
protocols has not shown much difference between testing with has not shown much difference between testing with individual
individual protocols. That is, if protocol A testing and protocols. That is, if protocol A testing and protocol B testing
protocol B testing give two different performance results, give two different performance results, mixed protocol testing
mixed protocol testing appears to give a result which is the appears to give a result which is the average of the two.
average of the two.
5. Wide Area Network (WAN) performance may be tested by setting 5. Wide Area Network (WAN) performance may be tested by setting up
up two identical devices connected by the appropriate short- two identical devices connected by the appropriate short- haul
haul versions of the WAN modems. Performance is then measured versions of the WAN modems. Performance is then measured between
between a LAN interface on one device to a LAN interface on a LAN interface on one DUT to a LAN interface on the other DUT.
the other device.
The maximum frame rate to be used for LAN-WAN-LAN The maximum frame rate to be used for LAN-WAN-LAN configurations is a
configurations is a judgement that can be based on known judgment that can be based on known characteristics of the overall
characteristics of the overall system including compression system including compression effects, fragmentation, and gross link
effects, fragmentation, and gross link speeds. Practice speeds. Practice suggests that the rate should be at least 110% of
suggests that the rate should be at least 110% of the slowest the slowest link speed. Substantive issues of testing compression
link speed. Substantive issues of testing compression itself itself are beyond the scope of this document.
are beyond the scope of this document.
Appendix B: Maximum frame rates reference Appendix B: Maximum frame rates reference
(Provided by Roger Beeman) (Provided by Roger Beeman, Cisco Systems)
Ethernet Size Ethernet 16Mb Token Ring FDDI Size Ethernet 16Mb Token Ring FDDI
(bytes) (pps) (pps) (pps) (bytes) (pps) (pps) (pps)
64 14880 24691 152439 64 14880 24691 152439
128 8445 13793 85616 128 8445 13793 85616
256 4528 7326 45620 256 4528 7326 45620
512 2349 3780 23585 512 2349 3780 23585
768 1586 2547 15903 768 1586 2547 15903
1024 1197 1921 11996 1024 1197 1921 11996
1280 961 1542 9630 1280 961 1542 9630
1518 812 1302 8138 1518 812 1302 8138
skipping to change at page 1, line 880 skipping to change at page 21, line 10
DSAP 8 bits DSAP 8 bits
SSAP 8 bits SSAP 8 bits
Control 8 bits Control 8 bits
Vendor 24 bits Vendor 24 bits
Type 16 bits Type 16 bits
Data 8 x ( N - 18) bits Data 8 x ( N - 18) bits
FCS 32 bits FCS 32 bits
ED 8 bits ED 8 bits
FS 8 bits FS 8 bits
No accounting for token or idles between packets (theoretical Tokens or idles between packets are not included
minimums hard to pin down)
FDDI size FDDI size
Preamble 64 bits Preamble 64 bits
SD 8 bits SD 8 bits
FC 8 bits FC 8 bits
DA 48 bits DA 48 bits
SA 48 bits SA 48 bits
SNAP SNAP
DSAP 8 bits DSAP 8 bits
SSAP 8 bits SSAP 8 bits
Control 8 bits Control 8 bits
Vendor 24 bits Vendor 24 bits
Type 16 bits Type 16 bits
Data 8 x ( N - 18) bits Data 8 x ( N - 18) bits
FCS 32 bits FCS 32 bits
ED 4 bits ED 4 bits
FS 12 bits FS 12 bits
No accounting for token or idles between packets (theoretical
minimums hard to pin down)
Appendix C: Test Frame Formats Appendix C: Test Frame Formats
This appendix defines the frame formats that may be used with these This appendix defines the frame formats that may be used with these
tests. It also includes protocol specific parameters for TCP/IP tests. It also includes protocol specific parameters for TCP/IP over
over Ethernet to be used with the tests as an example. Ethernet to be used with the tests as an example.
C.1. Introduction C.1. Introduction
The general logic used in the selection of the parameters and the The general logic used in the selection of the parameters and the
design of the frame formats is explained for each case within the design of the frame formats is explained for each case within the
TCP/IP section. The same logic has been used in the other sections. TCP/IP section. The same logic has been used in the other sections.
Comments are used in these sections only if there is a protocol Comments are used in these sections only if there is a protocol
specific feature to be explained. Parameters and frame formats for specific feature to be explained. Parameters and frame formats for
additional protocols can be defined by the reader by using the same additional protocols can be defined by the reader by using the same
logic. logic.
C.2. TCP/IP Information C.2. TCP/IP Information
The following section deals with the TCP/IP protocol suite. The following section deals with the TCP/IP protocol suite.
C.2.1 Frame Type. C.2.1 Frame Type.
An application level datagram echo request is used for the test data An application level datagram echo request is used for the test
frame in the protocols that support such a function. A datagram data frame in the protocols that support such a function. A
protocol is used to minimize the chance that a router might expect a datagram protocol is used to minimize the chance that a router
specific session initialization sequence, as might be the case for a might expect a specific session initialization sequence, as might
reliable stream protocol. A specific defined protocol is used be the case for a reliable stream protocol. A specific defined
because some routers verify the protocol field and refuse to forward protocol is used because some routers verify the protocol field
unknown protocols. and refuse to forward unknown protocols.
For TCP/IP a UDP Echo Request is used. For TCP/IP a UDP Echo Request is used.
C.2.2 Protocol Addresses C.2.2 Protocol Addresses
Two sets of addresses must be defined: first the addresses assigned Two sets of addresses must be defined: first the addresses
to the router ports, and second the address that are to be used in assigned to the router ports, and second the address that are to
the frames themselves and in the routing updates. be used in the frames themselves and in the routing updates.
The following specific network addresses are have been assigned to The network addresses 192.18.0.0 through 192.19.255.255 are have
the BMWG by the NIC for this purpose. This assignment was made to been assigned to the BMWG by the IANA for this purpose. This
minimize the chance of conflict in case a testing device were to be assignment was made to minimize the chance of conflict in case a
accidentally connected to part of the Internet. testing device were to be accidentally connected to part of the
Internet. The specific use of the addresses is detailed below.
C.2.2.1 Router port protocol addresses C.2.2.1 Router port protocol addresses
Half of the ports on a multi-port router are referred to as "input" Half of the ports on a multi-port router are referred to as
ports and the other half as "output" ports even though some of the "input" ports and the other half as "output" ports even though
tests use all ports both as input and output. A contiguous series some of the tests use all ports both as input and output. A
of IP Class C network addresses from 198.18.1.0 to 198.18.64.0 have contiguous series of IP Class C network addresses from
been assigned for use on the "input" ports. A second series from 198.18.1.0 to 198.18.64.0 have been assigned for use on the
198.19.1.0 to 198.19.64.0 have been assigned for use on the "output" "input" ports. A second series from 198.19.1.0 to 198.19.64.0
ports. In all cases the router port is node 1 on the appropriate have been assigned for use on the "output" ports. In all cases
network. For example, a two port device would have an IP address of the router port is node 1 on the appropriate network. For
198.18.1.1 on one port and 198.19.1.1 on the other port. example, a two port DUT would have an IP address of 198.18.1.1
on one port and 198.19.1.1 on the other port.
Some of the tests described in the methodology memo make use of an Some of the tests described in the methodology memo make use of
SNMP management connection to the device under test. The management an SNMP management connection to the DUT. The management
access address for the device is assumed to be the first of the access address for the DUT is assumed to be the first of the
"input" ports (198.18.1.1). "input" ports (198.18.1.1).
C.2.2.2 Frame addresses C.2.2.2 Frame addresses
Some of the described tests assume adjacent network routing (the Some of the described tests assume adjacent network routing
reboot time test for example). The IP address used in the test (the reboot time test for example). The IP address used in the
frame is that of node 2 on the appropriate Class C network. test frame is that of node 2 on the appropriate Class C
(198.19.1.2 for example) network. (198.19.1.2 for example)
If the test involves non-adjacent network routing the phantom If the test involves non-adjacent network routing the phantom
routers are located at node 10 of each of the appropriate Class C routers are located at node 10 of each of the appropriate Class
networks. A series of Class C network addresses from 198.18.65.0 to C networks. A series of Class C network addresses from
198.18.254.0 has been assigned for use as the networks accessible 198.18.65.0 to 198.18.254.0 has been assigned for use as the
through the phantom routers on the "input" side of device under networks accessible through the phantom routers on the "input"
test. The series of Class C networks from 198.19.65.0 to side of DUT. The series of Class C networks from 198.19.65.0
198.19.254.0 have been assigned to be used as the networks visible to 198.19.254.0 have been assigned to be used as the networks
through the phantom routers on the "output" side of the device under visible through the phantom routers on the "output" side of the
test. DUT.
C.2.3 Routing Update Frequency C.2.3 Routing Update Frequency
The update interval for each routing protocol is may have to be The update interval for each routing protocol is may have to be
determined by the specifications of the individual protocol. For IP determined by the specifications of the individual protocol. For
RIP, Cisco IGRP and for OSPF a routing update frame or frames should IP RIP, Cisco IGRP and for OSPF a routing update frame or frames
precede each stream of test frames by 5 seconds. This frequency is should precede each stream of test frames by 5 seconds. This
sufficient for trial durations of up to 60 seconds. Routing updates frequency is sufficient for trial durations of up to 60 seconds.
must be mixed with the stream of test frames if longer trial periods Routing updates must be mixed with the stream of test frames if
are selected. The frequency of updates should be taken from the longer trial periods are selected. The frequency of updates
following table. should be taken from the following table.
IP-RIP 30 sec IP-RIP 30 sec
IGRP 90 sec IGRP 90 sec
OSPF 90 sec OSPF 90 sec
C.2.4 Frame Formats - detailed discussion C.2.4 Frame Formats - detailed discussion
C.2.4.1 Learning Frame C.2.4.1 Learning Frame
In most protocols a procedure is used to determine the mapping In most protocols a procedure is used to determine the mapping
between the protocol node address and the MAC address. The Address between the protocol node address and the MAC address. The
Resolution Protocol (ARP) is used to perform this function in Address Resolution Protocol (ARP) is used to perform this
TCP/IP. No such procedure is required in XNS or IPX because the MAC function in TCP/IP. No such procedure is required in XNS or
address is used as the protocol node address. IPX because the MAC address is used as the protocol node
address.
In the ideal case the tester would be able to respond to ARP In the ideal case the tester would be able to respond to ARP
requests from the device under test. In cases where this is not requests from the DUT. In cases where this is not possible an
possible an ARP request should be sent to the router's "output" ARP request should be sent to the router's "output" port. This
port. This request should be seen as coming from the immediate request should be seen as coming from the immediate destination
destination of the test frame stream. (i.e. the phantom router or of the test frame stream. (i.e. the phantom router (figure 2)
the end node if adjacent network routing is being used.) It is or the end node if adjacent network routing is being used.) It
assumed that the router will cache the MAC address of the requesting is assumed that the router will cache the MAC address of the
device. The ARP request should be sent 5 seconds before the test requesting device. The ARP request should be sent 5 seconds
frame stream starts in each trial. Trial lengths of longer than 50 before the test frame stream starts in each trial. Trial
seconds may require that the router be configured for an extended lengths of longer than 50 seconds may require that the router
ARP timeout. be configured for an extended ARP timeout.
+--------+ +------------+
| | | phantom |------ P LAN
A
IN A------| DUT |------------| |------ P LAN
B
| | OUT A | router |------ P LAN
C
+--------+ +------------+
figure 2
In the case where full routing is being used
C.2.4.2 Routing Update Frame C.2.4.2 Routing Update Frame
If the test does not involve adjacent net routing the tester must If the test does not involve adjacent net routing the tester
supply proper routing information using a routing update. A single must supply proper routing information using a routing update.
routing update is used before each trial on each "destination" port A single routing update is used before each trial on each
(see section C.24). This update includes the network addresses that "destination" port (see section C.24). This update includes
are reachable through a phantom router on the network attached to the network addresses that are reachable through a phantom
the port. For a full mesh test, one destination network address is router on the network attached to the port. For a full mesh
present in the routing update for each of the "input" ports. The test, one destination network address is present in the routing
test stream on each "input" port consists of a repeating sequence of update for each of the "input" ports. The test stream on each
frames, one to each of the "output" ports. "input" port consists of a repeating sequence of frames, one to
each of the "output" ports.
C.2.4.3 Management Query Frame C.2.4.3 Management Query Frame
The management overhead test uses SNMP to query a set of variables The management overhead test uses SNMP to query a set of
that should be present in all devices that support SNMP. The variables that should be present in all DUTs that support SNMP.
variables are read by an NMS at the appropriate intervals. The list The variables are read by an NMS at the appropriate intervals.
of variables to retrieve follow: The list of variables to retrieve follow:
sysUpTime sysUpTime
ifInOctets ifInOctets
ifOutOctets ifOutOctets
ifInUcastPkts ifInUcastPkts
ifOutUcastPkts ifOutUcastPkts
C.2.4.4 Test Frames C.2.4.4 Test Frames
The test frame is an UDP Echo Request with enough data to fill out The test frame is an UDP Echo Request with enough data to fill
the required frame size. The data should not be all bits off or all out the required frame size. The data should not be all bits
bits on since these patters can cause a "bit stuffing" process to be off or all bits on since these patters can cause a "bit
used to maintain clock synchronization on WAN links. This process stuffing" process to be used to maintain clock synchronization
will result in a longer frame than was intended. on WAN links. This process will result in a longer frame than
was intended.
C.2.4.5 Frame Formats - TCP/IP on Ethernet C.2.4.5 Frame Formats - TCP/IP on Ethernet
Each of the frames below are described for the 1st pair of device Each of the frames below are described for the 1st pair of DUT
ports, i.e. "input" port #1 and "output" port #1. Addresses must be ports, i.e. "input" port #1 and "output" port #1. Addresses
changed if the frame is to be used for other ports. must be changed if the frame is to be used for other ports.
C.2.6.1 Learning Frame C.2.6.1 Learning Frame
ARP Request on Ethernet ARP Request on Ethernet
-- DATAGRAM HEADER -- DATAGRAM HEADER
offset data (hex) description offset data (hex) description
00 FF FF FF FF FF FF dest MAC address 00 FF FF FF FF FF FF dest MAC address send to
send to broadcast address broadcast address
06 xx xx xx xx xx xx set to source MAC address 06 xx xx xx xx xx xx set to source MAC address
12 08 06 ARP type 12 08 06 ARP type
14 00 01 hardware type 14 00 01 hardware type Ethernet = 1
Ethernet = 1 16 08 00 protocol type IP = 800
16 08 00 protocol type 18 06 hardware address length 48 bits
IP = 800 on Ethernet
18 06 hardware address length 19 04 protocol address length 4 octets
48 bits on Ethernet for IP
19 04 protocol address length 20 00 01 opcode request = 1
4 octets for IP
20 00 01 opcode
request = 1
22 xx xx xx xx xx xx source MAC address 22 xx xx xx xx xx xx source MAC address
28 xx xx xx xx source IP address 28 xx xx xx xx source IP address
32 FF FF FF FF FF FF requesting DUT's MAC address 32 FF FF FF FF FF FF requesting DUT's MAC address
38 xx xx xx xx DUT's IP address 38 xx xx xx xx DUT's IP address
C.2.6.2 Routing Update Frame C.2.6.2 Routing Update Frame
-- DATAGRAM HEADER -- DATAGRAM HEADER
offset date description offset data (hex) description
00 FF FF FF FF FF FF dest MAC address is broadcast 00 FF FF FF FF FF FF dest MAC address is broadcast
06 xx xx xx xx xx xx source hardware address 06 xx xx xx xx xx xx source hardware address
12 08 00 type 12 08 00 type
-- IP HEADER -- IP HEADER
14 45 IP version - 4, 14 45 IP version - 4, header length (4
header length byte units) - 5
(4 byte units) - 5
15 00 service field 15 00 service field
16 00 EE total length 16 00 EE total length
18 00 00 ID 18 00 00 ID
20 40 00 flags (3 bits) 20 40 00 flags (3 bits) 4 (do not
4 (do not fragment),fragment fragment),
offset-0 fragment offset-0
22 0A TTL 22 0A TTL
23 11 protocol - 17 (UDP) 23 11 protocol - 17 (UDP)
24 C4 8D header checksum 24 C4 8D header checksum
26 xx xx xx xx source IP address 26 xx xx xx xx source IP address
30 xx xx xx destination IP address 30 xx xx xx destination IP address
33 FF host part = FF for broadcast 33 FF host part = FF for broadcast
-- UDP HEADER -- UDP HEADER
34 02 08 source port 34 02 08 source port 208 = RIP
208 = RIP 36 02 08 destination port 208 = RIP
36 02 08 destination port
208 = RIP
38 00 DA UDP message length 38 00 DA UDP message length
40 00 00 UDP checksum 40 00 00 UDP checksum
-- RIP packet -- RIP packet
42 02 command = response 42 02 command = response
43 01 version = 1 43 01 version = 1
44 00 00 0 44 00 00 0
-- net 1 -- net 1
46 00 02 family = IP 46 00 02 family = IP
skipping to change at page 1, line 1166 skipping to change at page 27, line 24
153 00 net not node 153 00 net not node
154 00 00 00 00 0 154 00 00 00 00 0
158 00 00 00 00 0 158 00 00 00 00 0
162 00 00 00 07 metric 7 162 00 00 00 07 metric 7
C.2.4.6 Management Query Frame C.2.4.6 Management Query Frame
To be defined. To be defined.
C.2.6.4 Test Frames C.2.6.4 Test Frames
UDP echo request on Ethernet UDP echo request on Ethernet
-- DATAGRAM HEADER -- DATAGRAM HEADER
offset data description offset data (hex) description
00 xx xx xx xx xx xx set to dest MAC address 00 xx xx xx xx xx xx set to dest MAC address
06 xx xx xx xx xx xx set to source MAC address 06 xx xx xx xx xx xx set to source MAC address
12 08 00 type 12 08 00 type
-- IP HEADER -- IP HEADER
14 45 IP version - 4 14 45 IP version - 4 header length 5 4
header length 5 4 byte units byte units
15 00 TOS 15 00 TOS
16 00 2E total length* 16 00 2E total length*
18 00 00 ID 18 00 00 ID
20 00 00 flags (3 bits) - 0 20 00 00 flags (3 bits) - 0 fragment
fragment offset-0 offset-0
22 0A TTL 22 0A TTL
23 11 protocol - 17 (UDP) 23 11 protocol - 17 (UDP)
24 C4 8D header checksum* 24 C4 8D header checksum*
26 xx xx xx xx set to source IP address** 26 xx xx xx xx set to source IP address**
30 xx xx xx xx set to destination IP address** 30 xx xx xx xx set to destination IP address**
-- UDP HEADER -- UDP HEADER
34 C0 20 source port 34 C0 20 source port
36 00 07 destination port 36 00 07 destination port 07 = Echo
07 = Echo
38 00 1A UDP message length* 38 00 1A UDP message length*
40 00 00 UDP checksum 40 00 00 UDP checksum
-- UDP DATA -- UDP DATA
42 00 01 02 03 04 05 06 07 some data*** 42 00 01 02 03 04 05 06 07 some data***
50 08 09 0A 0B 0C 0D 0E 0F 50 08 09 0A 0B 0C 0D 0E 0F
* - change for different length frames * - change for different length frames
** - change for different logical streams ** - change for different logical streams
*** - fill remainder of frame with incrementing octets, repeated if *** - fill remainder of frame with incrementing octets,
required by frame length repeated if required by frame length
Values to be used in Total Length and UDP message length fields: Values to be used in Total Length and UDP message length fields:
frame size total length UDP message length frame size total length UDP message length
64 00 2E 00 1A 64 00 2E 00 1A
128 00 6E 00 5A 128 00 6E 00 5A
256 00 EE 00 9A 256 00 EE 00 9A
512 01 EE 01 9A 512 01 EE 01 9A
768 02 EE 02 9A 768 02 EE 02 9A
1024 03 EE 03 9A 1024 03 EE 03 9A
1280 04 EE 04 9A 1280 04 EE 04 9A
1518 05 DC 05 C8 1518 05 DC 05 C8
Benchmarking Methodology Working Group Scott Bradner
Internet Draft - February 1995 Harvard University
Jim McQuaid
Wandel & Goltermann Technologies, Inc
 End of changes. 

This html diff was produced by rfcdiff 1.23, available from http://www.levkowetz.com/ietf/tools/rfcdiff/