draft-ietf-bmwg-mcastm-07.txt   draft-ietf-bmwg-mcastm-08.txt 
Network Working Group Debra Stopp Network Working Group Debra Stopp
Hardev Soor Hardev Soor
INTERNET-DRAFT IXIA INTERNET-DRAFT IXIA
Expires in: August 2001 Expires in: November 2002
Methodology for IP Multicast Benchmarking Methodology for IP Multicast Benchmarking
<draft-ietf-bmwg-mcastm-07.txt> <draft-ietf-bmwg-mcastm-08.txt>
Status of this Memo Status of this Memo
This document is an Internet-Draft and is in full conformance This document is an Internet-Draft and is in full conformance with
with all provisions of Section 10 of RFC2026. all provisions of Section 10 of RFC2026.
Internet-Drafts are working documents of the Internet Internet-Drafts are working documents of the Internet Engineering
Engineering Task Force (IETF), its areas, and its working Task Force (IETF), its areas, and its working groups. Note that
groups. Note that other groups may also distribute working other groups may also distribute working documents as Internet-
documents as Internet- Drafts. Drafts.
Internet-Drafts are draft documents valid for a maximum of six Internet-Drafts are draft documents valid for a maximum of six
months and may be updated, replaced, or obsoleted by other months and may be updated, replaced, or obsoleted by other
documents at any time. It is inappropriate to use Internet- documents at any time. It is inappropriate to use Internet-Drafts
Drafts as reference material or to cite them other than as "work as reference material or to cite them other than as "work in
in progress." progress."
The list of current Internet-Drafts can be accessed The list of current Internet-Drafts can be accessed at
at http://www.ietf.org/ietf/1id-abstracts.txt http://www.ietf.org/ietf/1id-abstracts.txt
The list of Internet-Draft Shadow Directories can be accessed The list of Internet-Draft Shadow Directories can be accessed at
at http://www.ietf.org/shadow.html. http://www.ietf.org/shadow.html.
Copyright Notice Copyright Notice
Copyright (C) The Internet Society (2001). All Rights Reserved. Copyright (C) The Internet Society (2002). All Rights Reserved.
Abstract Abstract
The purpose of this draft is to describe methodology specific to The purpose of this draft is to describe methodology specific to
the benchmarking of multicast IP forwarding devices. It builds the benchmarking of multicast IP forwarding devices. It builds upon
upon the tenets set forth in RFC 2544, RFC 2432 and other IETF the tenets set forth in RFC 2544, RFC 2432 and other IETF
Benchmarking Methodology Working Group (BMWG) efforts. This Benchmarking Methodology Working Group (BMWG) efforts. This
document seeks to extend these efforts to the multicast paradigm. document seeks to extend these efforts to the multicast paradigm.
The BMWG produces two major classes of documents: The BMWG produces two major classes of documents: Benchmarking
Benchmarking Terminology documents and Benchmarking Methodology Terminology documents and Benchmarking Methodology documents. The
documents. The Terminology documents present the benchmarks and Terminology documents present the benchmarks and other related
other related terms. The Methodology documents define the terms. The Methodology documents define the procedures required to
procedures required to collect the benchmarks cited in the collect the benchmarks cited in the corresponding Terminology
corresponding Terminology documents. documents.
Table of Contents Table of Contents
1. INTRODUCTION...................................................3 1. INTRODUCTION...................................................3
2. KEY WORDS TO REFLECT REQUIREMENTS..............................3 2. KEY WORDS TO REFLECT REQUIREMENTS..............................3
3. TEST SET UP....................................................3 3. TEST SET UP....................................................3
3.1. Test Considerations..........................................4 3.1. Test Considerations..........................................5
3.1.1. IGMP Support..............................................4 3.1.1. IGMP Support..............................................5
3.1.2. Group Addresses...........................................5 3.1.2. Group Addresses...........................................5
3.1.3. Frame Sizes...............................................5 3.1.3. Frame Sizes...............................................5
3.1.4. TTL.......................................................5 3.1.4. TTL.......................................................6
3.2. Layer 2 Support..............................................5 3.1.5. Trial Duration............................................6
4. FORWARDING AND THROUGHPUT......................................5 3.2. Layer 2 Support..............................................6
4. FORWARDING AND THROUGHPUT......................................6
4.1. Mixed Class Throughput.......................................6 4.1. Mixed Class Throughput.......................................6
4.2. Scaled Group Forwarding Matrix...............................7 4.2. Scaled Group Forwarding Matrix...............................7
4.3. Aggregated Multicast Throughput..............................7 4.3. Aggregated Multicast Throughput..............................8
4.4. Encapsulation/Decapsulation (Tunneling) Throughput...........8 4.4. Encapsulation/Decapsulation (Tunneling) Throughput...........9
4.4.1. Encapsulation Throughput..................................9 4.4.1. Encapsulation Throughput..................................9
4.4.2. Decapsulation Throughput..................................9 4.4.2. Decapsulation Throughput..................................9
4.4.3. Re-encapsulation Throughput..............................10 4.4.3. Re-encapsulation Throughput..............................10
5. FORWARDING LATENCY............................................10 5. FORWARDING LATENCY............................................10
5.1. Multicast Latency...........................................11 5.1. Multicast Latency...........................................11
5.2. Min/Max Multicast Latency...................................13 5.2. Min/Max Multicast Latency...................................13
6. OVERHEAD......................................................14 6. OVERHEAD......................................................14
6.1. Group Join Delay............................................14 6.1. Group Join Delay............................................14
6.2. Group Leave Delay...........................................15 6.2. Group Leave Delay...........................................15
7. CAPACITY......................................................16 7. CAPACITY......................................................16
skipping to change at page 2, line 46 skipping to change at page 3, line 4
8.2. Forwarding Burdened Group Join Delay........................17 8.2. Forwarding Burdened Group Join Delay........................17
9. SECURITY CONSIDERATIONS.......................................17 9. SECURITY CONSIDERATIONS.......................................17
10. ACKNOWLEDGEMENTS.............................................17 10. ACKNOWLEDGEMENTS.............................................17
11. REFERENCES...................................................18 11. REFERENCES...................................................18
12. AUTHOR'S ADDRESSES...........................................19 12. AUTHOR'S ADDRESSES...........................................19
13. FULL COPYRIGHT STATEMENT.....................................19 13. FULL COPYRIGHT STATEMENT.....................................19
APPENDIX A: DETERMINING AN EVEN DISTRIBUTION.....................20
1. Introduction 1. Introduction
This document defines a specific set of tests that vendors can use This document defines a specific set of tests that vendors can use
to measure and report the performance characteristics and to measure and report the performance characteristics and
forwarding capabilities of network devices that support IP forwarding capabilities of network devices that support IP
multicast protocols. The results of these tests will provide the multicast protocols. The results of these tests will provide the
user comparable data from different vendors with which to evaluate user comparable data from different vendors with which to evaluate
these devices. these devices.
A previous document, " Terminology for IP Multicast Benchmarking" A previous document, " Terminology for IP Multicast Benchmarking"
skipping to change at page 3, line 27 skipping to change at page 3, line 27
This methodology will focus on one source to many destinations, This methodology will focus on one source to many destinations,
although many of the tests described may be extended to use although many of the tests described may be extended to use
multiple source to multiple destination IP multicast communication. multiple source to multiple destination IP multicast communication.
2. Key Words to Reflect Requirements 2. Key Words to Reflect Requirements
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL
NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL"
in this document are to be interpreted as described in RFC 2119. in this document are to be interpreted as described in RFC 2119.
RFC 2119 defines the use of these key words to help make the intent
of standards track documents as clear as possible. While this
document uses these keywords, this document is not a standards
track document.
3. Test set up 3. Test set up
The set of methodologies presented in this draft are for single
ingress, multiple egress scenarios as exemplified by Figures 1 and
2. Methodologies for multiple ingress, multiple egress scenarios
are beyond the scope of this document.
Figure 1 shows a typical setup for an IP multicast test, with one Figure 1 shows a typical setup for an IP multicast test, with one
source to multiple destinations, although this MAY be extended to source to multiple destinations.
multiple source to multiple destinations.
+----------------+ +----------------+
+------------+ | Egress | +------------+ | Egress |
+--------+ | (-)-------->| destination(E1)| +--------+ | (-)-------->| destination(E1)|
| | | | | | | | | | | |
| source |------->(|)Ingress | +----------------+ | source |------->(|)Ingress | +----------------+
| | | | +----------------+ | | | | +----------------+
+--------+ | D U T (-)-------->| Egress | +--------+ | D U T (-)-------->| Egress |
| | | destination(E2)| | | | destination(E2)|
| | | | | | | |
skipping to change at page 4, line 32 skipping to change at page 5, line 5
--------- ---------
Generally , the destination ports first join the desired number of Generally , the destination ports first join the desired number of
multicast groups by sending IGMP Join Group messages to the multicast groups by sending IGMP Join Group messages to the
DUT/SUT. To verify that all destination ports successfully joined DUT/SUT. To verify that all destination ports successfully joined
the appropriate groups, the source port MUST transmit IP multicast the appropriate groups, the source port MUST transmit IP multicast
frames destined for these groups. The destination ports MAY send frames destined for these groups. The destination ports MAY send
IGMP Leave Group messages after the transmission of IP Multicast IGMP Leave Group messages after the transmission of IP Multicast
frames to clear the IGMP table of the DUT/SUT. frames to clear the IGMP table of the DUT/SUT.
In addition, all transmitted frames MUST contain a recognizable In addition, test equipment MUST validate the correct and proper
pattern that can be filtered on in order to ensure the receipt of forwarding actions of the devices they test in order to ensure the
only the frames that are involved in the test. receipt of only the frames that are involved in the test.
3.1. Test Considerations 3.1. Test Considerations
The procedures outlined below are written without regard for The procedures outlined below are written without regard for
specific physical layer or link layer protocols. The methodology specific physical layer or link layer protocols. The methodology
further assumes a uniform medium topology. Issues regarding mixed further assumes a uniform medium topology. Issues regarding mixed
transmission media, such as speed mismatch, headers differences, transmission media, such as speed mismatch, headers differences,
etc., are not specifically addressed. Moreover, no provisions are etc., are not specifically addressed. Flow control, QoS and other
made for traffic-affecting factors, such as congestion control or traffic-affecting mechanisms MUST be disabled. Modifications to
service differentiation mechanisms. Modifications to the specified the specified collection procedures might need to be made to
collection procedures might need to be made to accommodate the accommodate the transmission media actually tested. These
transmission media actually tested. These accommodations MUST be accommodations MUST be presented with the test results.
presented with the test results.
3.1.1. IGMP Support 3.1.1. IGMP Support
Each of the destination ports should support and be able to test Each of the destination ports should support and be able to test
all IGMP versions 1, 2 and 3. The minimum requirement, however, is all IGMP versions 1, 2 and 3. The minimum requirement, however, is
IGMP version 2. IGMP version 2.
Each destination port should be able to respond to IGMP queries Each destination port should be able to respond to IGMP queries
during the test. during the test.
skipping to change at page 5, line 23 skipping to change at page 5, line 45
The Class D Group address SHOULD be changed between tests. Many The Class D Group address SHOULD be changed between tests. Many
DUTs have memory or cache that is not cleared properly and can bias DUTs have memory or cache that is not cleared properly and can bias
the results. the results.
The following group addresses are recommended by use in a test: The following group addresses are recommended by use in a test:
224.0.1.27-224.0.1.255 224.0.1.27-224.0.1.255
224.0.5.128-224.0.5.255 224.0.5.128-224.0.5.255
224.0.6.128-224.0.6.255 224.0.6.128-224.0.6.255
If the number of group addresses accommodated by these ranges do If the number of group addresses accommodated by these ranges does
not satisfy the requirements of the test, then these ranges may be not satisfy the requirements of the test, then these ranges may be
overlapped. The total number of configured group addresses must be overlapped. The total number of configured group addresses must be
less than or equal to the IGMP table size of the DUT/SUT. less than or equal to the IGMP table size of the DUT/SUT.
3.1.3. Frame Sizes 3.1.3. Frame Sizes
Each test SHOULD be run with different Multicast Frame Sizes. The Each test SHOULD be run with different Multicast Frame Sizes. The
recommended frame sizes are 64, 128, 256, 512, 1024, 1280, and 1518 recommended frame sizes are 64, 128, 256, 512, 1024, 1280, and 1518
byte frames. byte frames.
3.1.4. TTL 3.1.4. TTL
The source frames should have a TTL value large enough to The source frames should have a TTL value large enough to
accommodate the DUT/SUT. accommodate the DUT/SUT.
3.1.5. Trial Duration
The duration of the test portion of each trial SHOULD be at least
30 seconds. This parameter MUST be included as part of the results
reporting for each methodology.
3.2. Layer 2 Support 3.2. Layer 2 Support
Each of the destination ports should support GARP/GMRP protocols to Each of the destination ports should support GARP/GMRP protocols to
join groups on Layer 2 DUTs/SUTs. join groups on Layer 2 DUTs/SUTs.
4. Forwarding and Throughput 4. Forwarding and Throughput
This section contains the description of the tests that are related This section contains the description of the tests that are related
to the characterization of the packet forwarding of a DUT/SUT in a to the characterization of the packet forwarding of a DUT/SUT in a
multicast environment. Some metrics extend the concept of throughput multicast environment. Some metrics extend the concept of throughput
skipping to change at page 6, line 18 skipping to change at page 6, line 42
To determine the maximum throughput rate at which none of the To determine the maximum throughput rate at which none of the
offered frames, comprised from a unicast Class and a multicast offered frames, comprised from a unicast Class and a multicast
Class, to be forwarded are dropped by the device across a fixed Class, to be forwarded are dropped by the device across a fixed
number of ports as defined in RFC 2432. number of ports as defined in RFC 2432.
Procedure Procedure
Multicast and unicast traffic are mixed together in the same Multicast and unicast traffic are mixed together in the same
aggregated traffic stream in order to simulate the non-homogenous aggregated traffic stream in order to simulate the non-homogenous
networking environment. While the multicast traffic is transmitted networking environment. The DUT/SUT MUST learn the appropriate
from one source to multiple destinations, the unicast traffic MAY unicast IP addresses, either by sending ARP frames from each
be evenly distributed across the DUT/SUT architecture. In addition, unicast address, sending a RIP packet or by assigning static
the DUT/SUT MUST learn the appropriate unicast IP addresses, either entries into the DUT/SUT address table.
by sending ARP frames from each unicast address, sending a RIP
packet or by assigning static entries into the DUT/SUT address
table.
The mixture of multicast and unicast traffic MUST be set up in one The mixture of multicast and unicast traffic MUST be set up in one
of two ways: of two ways:
a) As a percentage of the total traffic flow employing maximum a) Input frame rate for each class of traffic [Br91] or as a
bandwidth utilization. Thus, each type of traffic is percentage of media_maximum-octets [Ma98]. Frame rate should
transmitted at the maximum available bandwidth. This also be specified independently for each traffic class.
implies that the intended load, regardless of the type of
traffic, remains constant.
b) As a percentage of the total traffic flow employing a b) As an aggregate rate (given either in frames per second or
proportionate bandwidth utilization. Thus, each type of as a percentage), with the ratio of multicast to unicast
traffic is transmitted at a fraction of the available traffic declared.
bandwidth proportional to the specified ratio. This also
implies that the intended load for each traffic type varies in
proportion to its specified ratio.
The transmission of the frames MUST be set up so that they form a While the multicast traffic is transmitted from one source to
deterministic distribution while still maintaining the specified multiple destinations, the unicast traffic MAY be evenly
forwarding rates. See Appendix A for a discussion on non-homogenous distributed across the DUT/SUT architecture. Unicast traffic
vs. homogenous packet distribution. distribution can either be non-meshed or meshed [Ma98] as specified
in RFC2544 or RFC2289.
Similar to the Frame loss rate test in RFC 2544, the first trial Throughput measurement is defined in RFC1242 [Br91]. A search
SHOULD be run for the frame rate that corresponds to 100% of the algorithm MUST be utilized to determine the maximum offered frame
maximum rate for the frame size on the input media. Repeat the rate with a zero frame loss rate.
procedure for the rate that corresponds to 90% of the maximum rate
used and then for 80% of this rate. This sequence SHOULD be
continued (at reducing 10% intervals) until there are two
successive trials in which no frames are lost. The maximum
granularity of the trials MUST be 10% of the maximum rate, a finer
granularity is encouraged.
Result Result
Parameters to be measured SHOULD include the frame loss and percent Parameters to be measured MUST include the aggregate offered load,
loss for each class of traffic per destination port. The ratio of number of multicast frames offered, number of unicast frames
unicast traffic to multicast traffic MUST be reported. offered, number multicast frames received, number of unicast frames
received and transmit duration of offered frames.
The nature of the traffic stream contributing to the result MUST be
reported. All required reporting parameters of mixed class
throughput MUST be reflected in the results report, such as the
transmitted packet size(s) and offered load of the packet stream.
4.2. Scaled Group Forwarding Matrix 4.2. Scaled Group Forwarding Matrix
Objective Objective
A table that demonstrates Forwarding Rate as a function of tested A table that demonstrates Forwarding Rate as a function of tested
multicast groups for a fixed number of tested DUT/SUT ports. multicast groups for a fixed number of tested DUT/SUT ports.
Procedure Procedure
Multicast traffic is sent at a fixed percent of maximum offered Multicast traffic is sent at a fixed percent of maximum offered
load with a fixed number of receive ports of the tester at a fixed load with a fixed number of receive ports of the tester at a fixed
frame length. frame length.
The receive ports SHOULD continue joining incrementally by 10 On each iteration, the receive ports SHOULD incrementally join 10
multicast groups until a user defined maximum is reached. multicast groups until a user defined maximum number of groups is
reached.
The receive ports will continue joining in the incremental fashion
until a user defined maximum is reached.
Results Results
Parameters to be measured SHOULD include the frame loss and percent Parameters to be measured MUST include the offered load and
loss per destination port for each multicast group address. forwarding rate as a function of the total number of multicast
groups, for each test iteration.
The nature of the traffic stream contributing to the result MUST be The nature of the traffic stream contributing to the result MUST be
reported. All required reporting parameters MUST be reflected in reported, specifically number of source and destination ports
the results report, such as the transmitted packet size(s) and within the multicast group. In addition, all other reporting
offered load of the packet stream. parameters of the scaled group forwarding matrix methodology MUST
be reflected in the results report, such as the transmitted packet
size(s) and offered load of the packet stream for each source port.
Result reports MUST include the following parameters for each
iteration: the number of frames offered, number of frames received
per each group, number of multicast groups and forwarding rate, in
frames per second, and transmit duration of offered frames.
Constructing a table that contains the forwarding rate vs. number
of groups is desirable.
4.3. Aggregated Multicast Throughput 4.3. Aggregated Multicast Throughput
Objective Objective
The maximum rate at which none of the offered frames to be The maximum rate at which none of the offered frames to be
forwarded through N destination interfaces of the same multicast forwarded through N destination interfaces of the same multicast
group are dropped. group is dropped.
Procedure Procedure
Multicast traffic is sent at a fixed percent of maximum offered Multicast traffic is sent at a fixed percent of maximum offered
load with a fixed number of groups at a fixed frame length for a load with a fixed number of groups at a fixed frame length for a
fixed duration of time. fixed duration of time.
The initial number of receive ports of the tester will join the The initial number of receive ports of the tester will join the
group(s) and the sender will transmit to the same groups after a group(s) and the sender will transmit to the same groups after a
certain delay (a few seconds). certain delay (a few seconds).
Then the an incremental number of receive ports will join the same If any frame loss is detected, one receive port MUST leave the
groups and then the Multicast traffic is sent as stated. group(s) and the sender will transmit again. Continue in this
iterative fashion until either there are no ports left joined to
The receive ports will continue to be added and multicast traffic the multicast group(s) OR 0% frame loss is achieved.
sent until a user defined maximum number of ports is reached.
Results Results
Parameters to be measured SHOULD include the frame loss and percent Parameters to be measured MUST include the maximum offered load at
loss per destination port for each multicast group address. which no frame loss occurred (as defined by RFC 2544)
The nature of the traffic stream contributing to the result MUST be The nature of the traffic stream contributing to the result MUST be
reported. All required reporting parameters of aggregated reported. All required reporting parameters of aggregated
throughput MUST be reflected in the results report, such as the throughput MUST be reflected in the results report, such as the
transmitted packet size(s) and offered load of the packet stream. initial number of receive ports, the final number of receive ports,
total number of multicast group addresses, the transmitted packet
size(s), offered load of the packet stream and transmit duration of
offered frames.
Constructing a table from the measurements might be useful in
illustrating the effect of modifying the number of active egress
ports on the tested system.
4.4. Encapsulation/Decapsulation (Tunneling) Throughput 4.4. Encapsulation/Decapsulation (Tunneling) Throughput
This sub-section provides the description of tests that help in This sub-section provides the description of tests that help in
obtaining throughput measurements when a DUT/SUT or a set of DUTs obtaining throughput measurements when a DUT/SUT or a set of DUTs
are acting as tunnel endpoints. The following Figure 3 presents the are acting as tunnel endpoints
a tunneled network.
Client A DUT/SUT A Network DUT/SUT B Client B
---------- ----------
| | ------ | |
-----(a) (b)| |(c) ( ) (d)| |(e) (f)-----
||||| -----> | |---->( )----->| |-----> |||||
----- | | ------ | | -----
| | | |
---------- ----------
Figure 3
--------
A tunnel is created between DUT/SUT A (the encapsulator) and
DUT/SUT B (the decapsulator). Client A is acting as a source and
Client B is the destination. Client B joins a multicast group (for
example, 224.0.1.1) by sending an IGMP Join message to DUT/SUT B to
join that group. Client A now wants to transmit some traffic to
Client B. It will send the multicast traffic to DUT/SUT A which
encapsulates the multicast frames, sends it to DUT/SUT B which will
decapsulate the same frames and forward them to Client B.
4.4.1. Encapsulation Throughput 4.4.1. Encapsulation Throughput
Objective Objective
The maximum rate at which frames offered a DUT/SUT are encapsulated The maximum rate at which frames offered a DUT/SUT are encapsulated
and correctly forwarded by the DUT/SUT without loss. and correctly forwarded by the DUT/SUT without loss.
Procedure Procedure
Traffic is sent through a DUT/SUT that has been configured to Traffic is sent through a DUT/SUT that has been configured to
encapsulate the frames. Traffic is received on a test port prior to encapsulate the frames. Traffic is received on a test port prior to
decapsulation and throughput is calculated based on RFC2544. decapsulation and throughput is calculated based on RFC2544.
Results Results
Parameters to be measured SHOULD include the measured throughput Parameters to be measured SHOULD include the measured throughput
per tunnel. per tunnel,
The nature of the traffic stream contributing to the result MUST be The nature of the traffic stream contributing to the result MUST be
reported. All required reporting parameters of encapsulation reported. All required reporting parameters of encapsulation
throughput MUST be reflected in the results report, such as the throughput MUST be reflected in the results report, such as the
transmitted packet size(s) and offered load of the packet stream. transmitted packet size(s), offered load of the packet stream and
transmit duration of offered frames.
4.4.2. Decapsulation Throughput 4.4.2. Decapsulation Throughput
Objective Objective
The maximum rate at which frames offered a DUT/SUT are decapsulated The maximum rate at which frames offered a DUT/SUT are decapsulated
and correctly forwarded by the DUT/SUT without loss. and correctly forwarded by the DUT/SUT without loss.
Procedure Procedure
skipping to change at page 9, line 54 skipping to change at page 10, line 5
RFC2544. RFC2544.
Results Results
Parameters to be measured SHOULD include the measured throughput Parameters to be measured SHOULD include the measured throughput
per tunnel. per tunnel.
The nature of the traffic stream contributing to the result MUST be The nature of the traffic stream contributing to the result MUST be
reported. All required reporting parameters of decapsulation reported. All required reporting parameters of decapsulation
throughput MUST be reflected in the results report, such as the throughput MUST be reflected in the results report, such as the
transmitted packet size(s) and offered load of the packet stream. transmitted packet size(s), offered load of the packet stream and
transmit duration of offered frames.
4.4.3. Re-encapsulation Throughput 4.4.3. Re-encapsulation Throughput
Objective Objective
The maximum rate at which frames of one encapsulated format offered The maximum rate at which frames of one encapsulated format offered
a DUT/SUT are converted to another encapsulated format and a DUT/SUT are converted to another encapsulated format and
correctly forwarded by the DUT/SUT without loss. correctly forwarded by the DUT/SUT without loss.
Procedure Procedure
skipping to change at page 10, line 29 skipping to change at page 10, line 32
RFC2544. RFC2544.
Results Results
Parameters to be measured SHOULD include the measured throughput Parameters to be measured SHOULD include the measured throughput
per tunnel. per tunnel.
The nature of the traffic stream contributing to the result MUST be The nature of the traffic stream contributing to the result MUST be
reported. All required reporting parameters of re-encapsulation reported. All required reporting parameters of re-encapsulation
throughput MUST be reflected in the results report, such as the throughput MUST be reflected in the results report, such as the
transmitted packet size(s) and offered load of the packet stream. transmitted packet size(s), offered load of the packet stream and
transmit duration of offered frames.
5. Forwarding Latency 5. Forwarding Latency
This section presents methodologies relating to the This section presents methodologies relating to the
characterization of the forwarding latency of a DUT/SUT in a characterization of the forwarding latency of a DUT/SUT in a
multicast environment. It extends the concept of latency multicast environment. It extends the concept of latency
characterization presented in RFC 2544. characterization presented in RFC 2544.
In order to lessen the effect of packet buffering in the DUT/SUT, In order to lessen the effect of packet buffering in the DUT/SUT,
the latency tests MUST be run such that the offered load is less the latency tests MUST be run such that the offered load is less
than the multicast throughput of the DUT/SUT as determined in the than the multicast throughput of the DUT/SUT as determined in the
previous section. The tests should also take into account the previous section. The tests should also take into account the
DUT's/SUT's need to cache the traffic in its IP cache, fastpath DUT's/SUT's need to cache the traffic in its IP cache, fastpath
cache or shortcut tables since the initial part of the traffic will cache or shortcut tables since the initial part of the traffic will
be utilized to build these tables. be utilized to build these tables.
Lastly, RFC 1242 and RFC 2544 draws distinction between two classes Lastly, RFC 1242 and RFC 2544 draw distinction between two classes
of devices: "store and forward" and "bit-forwarding." Each class of devices: "store and forward" and "bit-forwarding." Each class
impacts how latency is collected and subsequently presented. See impacts how latency is collected and subsequently presented. See
the related RFCs for more information. In practice, much of the the related RFCs for more information. In practice, much of the
test equipment will collect the latency measurement for one class test equipment will collect the latency measurement for one class
or the other, and, if needed, mathematically derive the reported or the other, and, if needed, mathematically derive the reported
value by the addition or subtraction of values accounting for value by the addition or subtraction of values accounting for
medium propagation delay of the packet, bit times to the timestamp medium propagation delay of the packet, bit times to the timestamp
trigger within the packet, etc. Test equipment vendors SHOULD trigger within the packet, etc. Test equipment vendors SHOULD
provide documentation regarding the composition and calculation provide documentation regarding the composition and calculation
latency values being reported. The user of this data SHOULD latency values being reported. The user of this data SHOULD
skipping to change at page 11, line 48 skipping to change at page 11, line 53
If the multicast latencies are to be taken across multiple devices If the multicast latencies are to be taken across multiple devices
forming a System Under Test (SUT), an example test topology might forming a System Under Test (SUT), an example test topology might
take the form of Figure 2 in section 3. take the form of Figure 2 in section 3.
The trial duration SHOULD be 120 seconds. Departures to the The trial duration SHOULD be 120 seconds. Departures to the
suggested traffic class guidelines MUST be disclosed with the suggested traffic class guidelines MUST be disclosed with the
respective trial results. The nature of the latency measurement, respective trial results. The nature of the latency measurement,
"store and forward" or "bit forwarding," MUST be associated with "store and forward" or "bit forwarding," MUST be associated with
the related test trial(s) and disclosed in the results report. the related test trial(s) and disclosed in the results report.
End-to-end reachability of the test traffic path SHOULD be verified End-to-end reach ability of the test traffic path SHOULD be
prior to the engagement of a test trial. This implies that verified prior to the engagement of a test trial. This implies
subsequent measurements are intended to characterize the latency that subsequent measurements are intended to characterize the
across the tested device's or devices' normal traffic forwarding latency across the tested device's or devices' normal traffic
path (e.g., faster hardware-based engines) of the device(s) as forwarding path (e.g., faster hardware-based engines) of the
opposed a non-standard traffic processing path (e.g. slower, device(s) as opposed a non-standard traffic processing path (e.g.
software-based exception handlers). If the test trial is to be slower, software-based exception handlers). If the test trial is
executed with the intent of characterizing a non-optimal, to be executed with the intent of characterizing a non-optimal,
forwarding condition, then a description of the exception forwarding condition, then a description of the exception
processing conditions being characterized MUST be included with the processing conditions being characterized MUST be included with the
trial's results. trial's results.
A test traffic stream is presented to the DUT. At the mid-point of A test traffic stream is presented to the DUT. At the mid-point of
the trial's duration, the test apparatus MUST inject a uniquely the trial's duration, the test apparatus MUST inject a uniquely
identifiable ("tagged") packet into the test traffic packets being identifiable ("tagged") packet into the test traffic packets being
presented. This tagged packet will be the basis for the latency presented. This tagged packet will be the basis for the latency
measurements. By "uniquely identifiable," it is meant that the test measurements. By "uniquely identifiable," it is meant that the test
apparatus MUST be able to discern the "tagged" packet from the apparatus MUST be able to discern the "tagged" packet from the
skipping to change at page 13, line 32 skipping to change at page 13, line 32
Specifically, when reporting the results of a VALID test trial, the Specifically, when reporting the results of a VALID test trial, the
set of ALL latencies related to the tested ingress interface and set of ALL latencies related to the tested ingress interface and
each tested egress DUT/SUT interface of MUST be presented. The each tested egress DUT/SUT interface of MUST be presented. The
time units of the presented latency MUST be uniform and with time units of the presented latency MUST be uniform and with
sufficient precision for the medium or media being tested. Results sufficient precision for the medium or media being tested. Results
MAY be offered in tabular format and SHOULD preserve the MAY be offered in tabular format and SHOULD preserve the
relationship of latency to ingress/egress interface to assist in relationship of latency to ingress/egress interface to assist in
trending across multiple trials. trending across multiple trials.
The Offered Load of the test traffic presented the DUT/SUT, size of The Offered Load of the test traffic presented the DUT/SUT, size of
the "tagged" packet, trial duration, and nature (i.e., store-and- the "tagged" packet, transmit duration of offered frames and nature
forward or bit-forwarding) of the trial's measurement MUST be (i.e., store-and-forward or bit-forwarding) of the trial's
associated with any reported test trial's result. measurement MUST be associated with any reported test trial's
result.
5.2. Min/Max Multicast Latency 5.2. Min/Max Multicast Latency
Objective Objective
The difference between the maximum latency measurement and the The difference between the maximum latency measurement and the
minimum latency measurement from a collected set of latencies minimum latency measurement from a collected set of latencies
produced by the Multicast Latency benchmark. produced by the Multicast Latency benchmark.
Procedure Procedure
Collect a set of multicast latency measurements, as prescribed in Collect a set of multicast latency measurements, as prescribed in
section 5.1. This will produce a set of multicase latencies, M, section 5.1. This will produce a set of multicast latencies, M,
where M is composed of individual forwarding altencies between DUT where M is composed of individual forwarding latencies between DUT
packet ingress and DUT packet egress port pairs. E.g.: packet ingress and DUT packet egress port pairs. E.g.:
M = {L(I,E1),L(I,E2), , L(I,En)} M = {L(I,E1),L(I,E2), , L(I,En)}
where L is the latency between a tested ingress port, I, of the where L is the latency between a tested ingress port, I, of the
DUT, and Ex a specific, tested multicast egress port of the DUT. DUT, and Ex a specific, tested multicast egress port of the DUT.
E1 through En are unique egress ports on the DUT. E1 through En are unique egress ports on the DUT.
From the collected multicast latency measurements in set M, From the collected multicast latency measurements in set M,
identify MAX(M), where MAX is a function that yields the largest identify MAX(M), where MAX is a function that yields the largest
latency value from set M. latency value from set M.
Identify MIN(M), when MIN is a funtion that yields the smallest Identify MIN(M), when MIN is a function that yields the smallest
latency value from set M. latency value from set M.
The Max/Min value is determined from the following formula: The Max/Min value is determined from the following formula:
Result = MAX(M) MIN(M) Result = MAX(M) MIN(M)
Results Results
The result MUST be represented as a single numerical value in time The result MUST be represented as a single numerical value in time
units consistent with the corresponding latency measurements. In units consistent with the corresponding latency measurements. In
addition the number of tested egress ports on the DUT MUST be addition, the number of tested egress ports on the DUT MUST be
reported. reported.
The nature of the traffic stream contributing to the result MUST be The nature of the traffic stream contributing to the result MUST be
reported. All required reporting parameters of multicast latency reported. All required reporting parameters of multicast latency
MUST be reflected in the min/max results report, such as the MUST be reflected in the min/max results report, such as the
transmitted packet size(s) and offered load of the packet stream in transmitted packet size(s), offered load of the packet stream in
which the tagged packet was presented to the DUT. which the tagged packet was presented to the DUT and transmit
duration of offered frames.
6. Overhead 6. Overhead
This section presents methodology relating to the characterization This section presents methodology relating to the characterization
of the overhead delays associated with explicit operations found in of the overhead delays associated with explicit operations found in
multicast environments. multicast environments.
6.1. Group Join Delay 6.1. Group Join Delay
Objective Objective
skipping to change at page 16, line 53 skipping to change at page 16, line 53
In addition, the nature of the traffic stream contributing to the In addition, the nature of the traffic stream contributing to the
result MUST be reported. All required reporting parameters MUST be result MUST be reported. All required reporting parameters MUST be
reflected in the results report, such as the transmitted packet reflected in the results report, such as the transmitted packet
size(s) and offered load of the packet stream. size(s) and offered load of the packet stream.
8. Interaction 8. Interaction
Network forwarding devices are generally required to provide more Network forwarding devices are generally required to provide more
functionality than just the forwarding of traffic. Moreover, functionality than just the forwarding of traffic. Moreover,
network forwarding devices may be asked to provide those functions network-forwarding devices may be asked to provide those functions
in a variety of environments. This section offers terms to assist in a variety of environments. This section offers terms to assist
in the characterization of DUT/SUT behavior in consideration of in the characterization of DUT/SUT behavior in consideration of
potentially interacting factors. potentially interacting factors.
8.1. Forwarding Burdened Multicast Latency 8.1. Forwarding Burdened Multicast Latency
The Multicast Latency metrics can be influenced by forcing the The Multicast Latency metrics can be influenced by forcing the
DUT/SUT to perform extra processing of packets while multicast DUT/SUT to perform extra processing of packets while multicast
traffic is being forwarded for latency measurements. In this test, traffic is being forwarded for latency measurements. In this test,
a set of ports on the tester will be designated to be source and a set of ports on the tester will be designated to be source and
skipping to change at page 17, line 31 skipping to change at page 17, line 31
ports 5, 6, 7, and 8 form the non-burdened response setup (setup B) ports 5, 6, 7, and 8 form the non-burdened response setup (setup B)
which will afflict the burdened response setup, then setup B which will afflict the burdened response setup, then setup B
traffic will join multicast group addresses not joined by the ports traffic will join multicast group addresses not joined by the ports
in this setup. By sending such multicast traffic, the DUT/SUT will in this setup. By sending such multicast traffic, the DUT/SUT will
perform a lookup on the packets that will affect the processing of perform a lookup on the packets that will affect the processing of
setup A traffic. setup A traffic.
8.2. Forwarding Burdened Group Join Delay 8.2. Forwarding Burdened Group Join Delay
The port configuration in this test is similar to the one described The port configuration in this test is similar to the one described
in section 8.1, but in this test, the multicast traffic is not sent in section 8.1, but in this test, the ports in setup B do not send
by the ports in setup B. In this test, the setup A traffic must be the multicast traffic. Rather, setup A traffic must be influenced
influenced in such a way that will affect the DUT's/SUT's ability in such a way that will affect the DUT's/SUT's ability to process
to process Group Join messages. Therefore, in this test, the ports Group Join messages. Therefore, in this test, the ports in setup B
in setup B will send a set of IGMP Group Join messages while the will send a set of IGMP Group Join messages while the ports in
ports in setup A are also joining its own set of group addresses. setup A are also joining its own set of group addresses. Since the
Since the two sets of group addresses are independent of each two sets of group addresses are independent of each other, the
other, the group join delay for setup A may be different than in group join delay for setup A may be different than in the case when
the case when there were no other group addresses being joined. there were no other group addresses being joined.
9. Security Considerations 9. Security Considerations
As this document is solely for the purpose of providing metric As this document is solely for the purpose of providing metric
methodology and describes neither a protocol nor a protocol's methodology and describes neither a protocol nor a protocol's
implementation, there are no security considerations associated implementation, there are no security considerations associated
with this document. with this document.
10. Acknowledgements 10. Acknowledgements
The authors would like to acknowledge the following individuals for The authors would like to acknowledge the following individuals for
their help and participation of the compilation and editing of this their help and participation of the compilation and editing of this
document Ralph Daniels, Netcom Systems, who made significant document Ralph Daniels, Netcom Systems, who made significant
contributions to earlier versions of this draft and Kevin Dubray, contributions to earlier versions of this draft, Daniel Bui, IXIA,
Juniper Networks. and Kevin Dubray, Juniper Networks.
11. References 11. References
[Br91] Bradner, S., "Benchmarking Terminology for Network [Br91] Bradner, S., "Benchmarking Terminology for Network
Interconnection Devices", RFC 1242, July 1991. Interconnection Devices", RFC 1242, July 1991.
[Br96] Bradner, S., and J. McQuaid, "Benchmarking Methodology for [Br96] Bradner, S., and J. McQuaid, "Benchmarking Methodology for
Network Interconnect Devices", RFC 2544, March 1999. Network Interconnect Devices", RFC 2544, March 1999.
[Br97] Bradner, S. "Use of Keywords in RFCs to Reflect Requirement [Br97] Bradner, S. "Use of Keywords in RFCs to Reflect Requirement
skipping to change at page 20, line 4 skipping to change at line 899
it or assist in its implementation may be prepared, copied, it or assist in its implementation may be prepared, copied,
published and distributed, in whole or in part, without restriction published and distributed, in whole or in part, without restriction
of any kind, provided that the above copyright notice and this of any kind, provided that the above copyright notice and this
paragraph are included on all such copies and derivative works. paragraph are included on all such copies and derivative works.
However, this document itself may not be modified in any way, such However, this document itself may not be modified in any way, such
as by removing the copyright notice or references to the Internet as by removing the copyright notice or references to the Internet
Society or other Internet organizations, except as needed for the Society or other Internet organizations, except as needed for the
purpose of developing Internet standards in which case the purpose of developing Internet standards in which case the
procedures for copyrights defined in the Internet Standards process procedures for copyrights defined in the Internet Standards process
must be followed, or as required to translate it into. must be followed, or as required to translate it into.
Appendix A: Determining an even distribution
It is important to understand and fully define the distribution of
frames among all multicast and unicast destinations. If the
distribution is not well defined or understood, the throughput and
forwarding metrics are not meaningful.
In a homogeneous environment, a large single burst of multicast
frames may be followed by a large burst of unicast frames. This is
a very different distribution than that of a non-homogeneous
environment, where the multicast and unicast frames are
intermingled throughout the entire transmission.
The recommended distribution is that of the non-homogeneous
environment because it more closely represents a real-world
scenario. The distribution is modeled by calculating the number of
multicast frames per destination port as a burst, then calculating
the number of unicast frames to transmit as a percentage of the
total frames transmitted. The overall effect of the distribution is
small bursts of multicast frames intermingled with small bursts of
unicast frames.
 End of changes. 

This html diff was produced by rfcdiff 1.23, available from http://www.levkowetz.com/ietf/tools/rfcdiff/