draft-ietf-bmwg-mcastm-13.txt   draft-ietf-bmwg-mcastm-14.txt 
Network Working Group Debra Stopp Network Working Group Debra Stopp
INTERNET-DRAFT Ixia INTERNET-DRAFT Ixia
Expires in: August 2003 Brooks Hickman Expires in: February 2004 Brooks Hickman
Spirent Communications Spirent Communications
July 2003 January 2004
Methodology for IP Multicast Benchmarking Methodology for IP Multicast Benchmarking
<draft-ietf-bmwg-mcastm-13.txt> <draft-ietf-bmwg-mcastm-14.txt>
Status of this Memo Status of this Memo
This document is an Internet-Draft and is in full conformance with This document is an Internet-Draft and is in full conformance with
all provisions of Section 10 of RFC2026. all provisions of Section 10 of RFC2026.
Internet-Drafts are working documents of the Internet Engineering Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF), its areas, and its working groups. Note that Task Force (IETF), its areas, and its working groups. Note that
other groups may also distribute working documents as Internet- other groups may also distribute working documents as Internet-
Drafts. Drafts.
skipping to change at page 1, line 36 skipping to change at page 1, line 36
progress." progress."
The list of current Internet-Drafts can be accessed at The list of current Internet-Drafts can be accessed at
http://www.ietf.org/ietf/1id-abstracts.txt http://www.ietf.org/ietf/1id-abstracts.txt
The list of Internet-Draft Shadow Directories can be accessed at The list of Internet-Draft Shadow Directories can be accessed at
http://www.ietf.org/shadow.html. http://www.ietf.org/shadow.html.
Copyright Notice Copyright Notice
Copyright (C) The Internet Society (2003). All Rights Reserved. Copyright (C) The Internet Society (2004). All Rights Reserved.
Abstract Abstract
The purpose of this document is to describe methodology specific to The purpose of this document is to describe methodology specific to
the benchmarking of multicast IP forwarding devices. It builds upon the benchmarking of multicast IP forwarding devices. It builds upon
the tenets set forth in RFC 2544, RFC 2432 and other IETF the tenets set forth in RFC 2544, RFC 2432 and other IETF
Benchmarking Methodology Working Group (BMWG) efforts. This Benchmarking Methodology Working Group (BMWG) efforts. This
document seeks to extend these efforts to the multicast paradigm. document seeks to extend these efforts to the multicast paradigm.
The BMWG produces two major classes of documents: Benchmarking The BMWG produces two major classes of documents: Benchmarking
skipping to change at page 2, line 13 skipping to change at page 2, line 13
documents. documents.
Table of Contents Table of Contents
1. INTRODUCTION...................................................3 1. INTRODUCTION...................................................3
2. KEY WORDS TO REFLECT REQUIREMENTS..............................3 2. KEY WORDS TO REFLECT REQUIREMENTS..............................3
3. TEST SET UP....................................................3 3. TEST SET UP....................................................3
3.1. Test Considerations..........................................5 3.1. Test Considerations..........................................5
3.1.1. IGMP Support..............................................5 3.1.1. IGMP Support..............................................6
3.1.2. Group Addresses...........................................5 3.1.2. Group Addresses...........................................6
3.1.3. Frame Sizes...............................................6 3.1.3. Frame Sizes...............................................6
3.1.4. TTL.......................................................6 3.1.4. TTL.......................................................6
3.1.5. Trial Duration............................................6 3.1.5. Trial Duration............................................6
4. FORWARDING AND THROUGHPUT......................................6 4. FORWARDING AND THROUGHPUT......................................7
4.1. Mixed Class Throughput.......................................7 4.1. Mixed Class Throughput.......................................7
4.2. Scaled Group Forwarding Matrix...............................8 4.2. Scaled Group Forwarding Matrix...............................8
4.3. Aggregated Multicast Throughput..............................9 4.3. Aggregated Multicast Throughput..............................9
4.4. Encapsulation/Decapsulation (Tunneling) Throughput..........10 4.4. Encapsulation/Decapsulation (Tunneling) Throughput..........10
4.4.1. Encapsulation Throughput.................................10 4.4.1. Encapsulation Throughput.................................10
4.4.2. Decapsulation Throughput.................................12 4.4.2. Decapsulation Throughput.................................12
4.4.3. Re-encapsulation Throughput..............................14 4.4.3. Re-encapsulation Throughput..............................14
5. FORWARDING LATENCY............................................16 5. FORWARDING LATENCY............................................16
5.1. Multicast Latency...........................................16 5.1. Multicast Latency...........................................17
5.2. Min/Max Multicast Latency...................................18 5.2. Min/Max Multicast Latency...................................19
6. OVERHEAD......................................................20 6. OVERHEAD......................................................20
6.1. Group Join Delay............................................20 6.1. Group Join Delay............................................20
6.2. Group Leave Delay...........................................22 6.2. Group Leave Delay...........................................23
7. CAPACITY......................................................24 7. CAPACITY......................................................25
7.1. Multicast Group Capacity....................................24 7.1. Multicast Group Capacity....................................25
8. INTERACTION...................................................25 8. INTERACTION...................................................26
8.1. Forwarding Burdened Multicast Latency.......................25 8.1. Forwarding Burdened Multicast Latency.......................26
8.2. Forwarding Burdened Group Join Delay........................26 8.2. Forwarding Burdened Group Join Delay........................27
9. SECURITY CONSIDERATIONS.......................................28 9. SECURITY CONSIDERATIONS.......................................28
10. ACKNOWLEDGEMENTS.............................................28 10. ACKNOWLEDGEMENTS.............................................29
11. CONTRIBUTIONS................................................28 11. CONTRIBUTIONS................................................29
12. REFERENCES...................................................29 12. REFERENCES...................................................30
13. AUTHOR'S ADDRESSES...........................................30 13. AUTHOR'S ADDRESSES...........................................31
14. FULL COPYRIGHT STATEMENT.....................................30 14. FULL COPYRIGHT STATEMENT.....................................31
1. Introduction 1. Introduction
This document defines tests for measuring and reporting the This document defines tests for measuring and reporting the
throughput, forwarding, latency and IGMP group membership throughput, forwarding, latency and IGMP group membership
characteristics of devices that support IP multicast protocols. characteristics of devices that support IP multicast protocols.
The results of these tests will provide the user with meaningful The results of these tests will provide the user with meaningful
data on multicast performance. data on multicast performance.
A previous document, " Terminology for IP Multicast Benchmarking" A previous document, " Terminology for IP Multicast Benchmarking"
(RFC 2432), defined many of the terms that are used in this (RFC 2432), defined many of the terms that are used in this
skipping to change at page 5, line 34 skipping to change at page 5, line 34
mechanisms, (e.g., process RPF events, build device caches, etc.) mechanisms, (e.g., process RPF events, build device caches, etc.)
to optimally forward subsequent traffic. Therefore, prior to to optimally forward subsequent traffic. Therefore, prior to
running any tests that require forwarding of multicast or unicast running any tests that require forwarding of multicast or unicast
packets, the test apparatus MUST generate test traffic utilizing packets, the test apparatus MUST generate test traffic utilizing
the same addressing characteristics to the DUT/SUT that will the same addressing characteristics to the DUT/SUT that will
subsequently be used to measure the DUT/SUT response. The test subsequently be used to measure the DUT/SUT response. The test
monitor should ensure the correct forwarding of traffic by the monitor should ensure the correct forwarding of traffic by the
DUT/SUT. The priming action need only be repeated to keep the DUT/SUT. The priming action need only be repeated to keep the
associated information current. associated information current.
It is the intent of this memo to provide the methodology for basic
characterizations regarding the forwarding of multicast packets by
a device or simple system of devices. These characterizations may
be useful in illustrating the impact of device architectural
features (e.g., message passing versus shared memory; handling
multicast traffic as an exception by the general purpose processor
versus the by a primary data path, etc.) in the forwarding of
multicast traffic.
It has been noted that the formation of the multicast distribution
tree may be a significant component of multicast performance.
While this component may be present in some of the measurements or
scenarios presented in this memo, this memo does not seek to
explicitly benchmark the formation of the multicast distribution
tree. The benchmarking of the multicast distribution tree
formation is left as future, more targeted work specific to a given
tree formation vehicle.
3.1.1. IGMP Support 3.1.1. IGMP Support
All of the ingress and egress interfaces MUST support a version of All of the ingress and egress interfaces MUST support a version of
IGMP. The IGMP version on the ingress interface MUST be the same IGMP. The IGMP version on the ingress interface MUST be the same
version of IGMP that is being tested on the egress interfaces. version of IGMP that is being tested on the egress interfaces.
Each of the ingress and egress interfaces SHOULD be able to respond Each of the ingress and egress interfaces SHOULD be able to respond
to IGMP queries during the test. to IGMP queries during the test.
Each of the ingress and egress interfaces SHOULD also send LEAVE Each of the ingress and egress interfaces SHOULD also send LEAVE
(running IGMP version 2 or later) after each test. (running IGMP version 2 or later) after each test.
3.1.2. Group Addresses 3.1.2. Group Addresses
It is intended that the collection of benchmarks prescribed in this
document be executed in an isolated lab environment. That is to
say, the test traffic offered the tested devices MUST NOT traverse
a live internet, intranet, or other production network.
There is no restriction to the use of multicast addresses to There is no restriction to the use of multicast addresses to
compose the test traffic other than those assignments imposed by compose the test traffic other than those assignments imposed by
IANA. The IANA assignments MUST be regarded for operational IANA. The IANA assignments for multicast addresses[IANA1] MUST be
consistency. For multicast address assignments see: regarded for operational consistency. Address selection does not
need to be restricted to Administratively Scoped IP Multicast
http://www.iana.org/assignments/multicast-addresses addresses[Me89].
Address selection does not need to be restricted to
Administratively Scoped IP Multicast addresses.
3.1.3. Frame Sizes 3.1.3. Frame Sizes
Each test SHOULD be run with different multicast frame sizes. For Each test SHOULD be run with different multicast frame sizes. For
Ethernet, the recommended sizes are 64, 128, 256, 512, 1024, 1280, Ethernet, the recommended sizes are 64, 128, 256, 512, 1024, 1280,
and 1518 byte frames. and 1518 byte frames.
Other link layer technologies MAY be used. The minimum and maximum Other link layer technologies MAY be used. The minimum and maximum
frame lengths of the link layer technology in use SHOULD be tested. frame lengths of the link layer technology in use SHOULD be tested.
skipping to change at page 7, line 16 skipping to change at page 7, line 23
Objective: Objective:
To determine the throughput of a DUT/SUT when both unicast class To determine the throughput of a DUT/SUT when both unicast class
frames and multicast class frames are offered simultaneously to a frames and multicast class frames are offered simultaneously to a
fixed number of interfaces as defined in RFC 2432. fixed number of interfaces as defined in RFC 2432.
Procedure: Procedure:
Multicast and unicast traffic are mixed together in the same Multicast and unicast traffic are mixed together in the same
aggregated traffic stream in order to simulate the non-homogenous aggregated traffic stream in order to simulate a heterogeneous
networking environment. networking environment.
The following events MUST occur before offering test traffic: The following events MUST occur before offering test traffic:
o All destination test ports configured to receive multicast o All destination test ports configured to receive multicast
traffic MUST join all configured multicast groups; traffic MUST join all configured multicast groups;
o The DUT/SUT MUST learn the appropriate unicast and o The DUT/SUT MUST learn the appropriate unicast and
multicast addresses; and multicast addresses; and
o Group membership and unicast address learning MUST be o Group membership and unicast address learning MUST be
verified through some externally observable method. verified through some externally observable method.
The intended load [Ma98] SHOULD be configured as alternating The intended load [Ma98] SHOULD be configured as alternating
multicast class frames and unicast class frames to a single ingress multicast class frames and unicast class frames to a single ingress
interface. The unicast class frames MUST be configured to transmit interface. The unicast class frames MUST be configured to transmit
in an unweighted round-robin fashion to all of the destination in an unweighted round-robin fashion to all of the destination
ports. ports.
For example, with six multicast groups and 3 destination ports with
one unicast addresses per port, the source test port will offer
frames in the following order:
m1 u1 m2 u2 m3 u3 m4 u1 m5 u2 m6 u3 m1 ...
Where:
m<Number> = Multicast Frame<Group>
u<Number> = Unicast Frame<Target Port>
Mixed class throughput measurement is defined in RFC2432 [Du98]. A Mixed class throughput measurement is defined in RFC2432 [Du98]. A
search algorithm MUST be utilized to determine the Mixed Class search algorithm MUST be utilized to determine the Mixed Class
Throughput. The ratio of unicast to multicast frames MUST remain Throughput. The ratio of unicast to multicast frames MUST remain
the same when varying the intended load. the same when varying the intended load.
Reporting Format: Reporting Format:
The following configuration parameters MUST be reflected in the The following configuration parameters MUST be reflected in the
test report: test report:
skipping to change at page 10, line 37 skipping to change at page 10, line 43
per the recommendations in section 3.1.3. Each row or iteration per the recommendations in section 3.1.3. Each row or iteration
SHOULD specify offered load, total number of offered frames and the SHOULD specify offered load, total number of offered frames and the
measured Aggregated Multicast Throughput. measured Aggregated Multicast Throughput.
4.4. Encapsulation/Decapsulation (Tunneling) Throughput 4.4. Encapsulation/Decapsulation (Tunneling) Throughput
This sub-section provides the description of tests related to the This sub-section provides the description of tests related to the
determination of throughput measurements when a DUT/SUT or a set of determination of throughput measurements when a DUT/SUT or a set of
DUTs are acting as tunnel endpoints. DUTs are acting as tunnel endpoints.
For this specific testing scenario, encapsulation or tunneling
refers to a packet that contains an unsupported protocol feature in
a format that is supported by the DUT/SUT.
4.4.1. Encapsulation Throughput 4.4.1. Encapsulation Throughput
Objective: Objective:
To determine the maximum rate at which frames offered to one To determine the maximum rate at which frames offered to one
ingress interface of a DUT/SUT are encapsulated and correctly ingress interface of a DUT/SUT are encapsulated and correctly
forwarded on one or more egress interfaces of the DUT/SUT without forwarded on one or more egress interfaces of the DUT/SUT without
loss. loss.
Procedure: Procedure:
skipping to change at page 12, line 18 skipping to change at page 12, line 18
test report: test report:
o Number of tested egress interfaces on the DUT/SUT o Number of tested egress interfaces on the DUT/SUT
o Test duration o Test duration
o IGMP version o IGMP version
o Total number of multicast groups o Total number of multicast groups
o MTU size of DUT/SUT interfaces o MTU size of DUT/SUT interfaces
o Originating un-encapsulated frame size o Originating un-encapsulated frame size
o Number of tunnels per egress interface o Number of tunnels per egress interface
o Number of multicast groups per tunnel o Number of multicast groups per tunnel
o Encapsulation algorithm or format used to tunnel the
packets
The following results MUST be reflected in the test report: The following results MUST be reflected in the test report:
o Measured Encapsulated Throughput as defined in RFC2432 o Measured Encapsulated Throughput as defined in RFC2432
[Du98] [Du98]
o Encapsulated frame size o Encapsulated frame size
The Encapsulated Throughput results SHOULD be reported in the form The Encapsulated Throughput results SHOULD be reported in the form
of a table and specific to this test there SHOULD be rows for each of a table and specific to this test there SHOULD be rows for each
originating un-encapsulated frame size. Each row or iteration originating un-encapsulated frame size. Each row or iteration
skipping to change at page 14, line 9 skipping to change at page 14, line 14
Reporting Format: Reporting Format:
The following configuration parameters MUST be reflected in the The following configuration parameters MUST be reflected in the
test report: test report:
o Number of tested egress interfaces on the DUT/SUT o Number of tested egress interfaces on the DUT/SUT
o Test duration o Test duration
o IGMP version o IGMP version
o Total number of multicast groups o Total number of multicast groups
o Originating encapsulation format o Originating encapsulation algorithm or format used to
tunnel the packets
o Originating encapsulated frame size o Originating encapsulated frame size
o Number of tunnels o Number of tunnels
o Number of multicast groups per tunnel o Number of multicast groups per tunnel
The following results MUST be reflected in the test report: The following results MUST be reflected in the test report:
o Measured Decapsulated Throughput as defined in RFC2432 o Measured Decapsulated Throughput as defined in RFC2432
[Du98] [Du98]
o Decapsulated frame size o Decapsulated frame size
skipping to change at page 15, line 46 skipping to change at page 16, line 13
Reporting Format: Reporting Format:
The following configuration parameters MUST be reflected in the The following configuration parameters MUST be reflected in the
test report: test report:
o Number of tested egress interfaces on the DUT/SUT o Number of tested egress interfaces on the DUT/SUT
o Test duration o Test duration
o IGMP version o IGMP version
o Total number of multicast groups o Total number of multicast groups
o MTU size of DUT/SUT interfaces o MTU size of DUT/SUT interfaces
o Originating encapsulation format o Originating encapsulation algorithm or format used to
tunnel the packets
o Re-encapsulation algorithm or format used to tunnel the
packets
o Originating encapsulated frame size o Originating encapsulated frame size
o Number of tunnels per interface o Number of tunnels per interface
o Number of multicast groups per tunnel o Number of multicast groups per tunnel
The following results MUST be reflected in the test report: The following results MUST be reflected in the test report:
o Measured Re-encapsulated Throughput as defined in RFC2432 o Measured Re-encapsulated Throughput as defined in RFC2432
[Du98] [Du98]
o Re-encapsulated frame size o Re-encapsulated frame size
The Re-encapsulated Throughput results SHOULD be reported in the The Re-encapsulated Throughput results SHOULD be reported in the
format of a table and specific to this test there SHOULD be rows format of a table and specific to this test there SHOULD be rows
for each originating encapsulated frame size. Each row or for each originating encapsulated frame size. Each row or
iteration SHOULD specify the offered load, decapsulated frame size, iteration SHOULD specify the offered load, decapsulated frame size,
skipping to change at page 27, line 4 skipping to change at page 27, line 40
recommended frame sizes in section 3.1.3, and SHOULD preserve the recommended frame sizes in section 3.1.3, and SHOULD preserve the
relationship of latency to ingress/egress interface(s) to assist in relationship of latency to ingress/egress interface(s) to assist in
trending across multiple trials. trending across multiple trials.
8.2. Forwarding Burdened Group Join Delay 8.2. Forwarding Burdened Group Join Delay
Objective: Objective:
To determine the time duration it takes a DUT/SUT to start To determine the time duration it takes a DUT/SUT to start
forwarding multicast frames from the time a successful IGMP Group forwarding multicast frames from the time a successful IGMP Group
Membership Report has been issued to the DUT/SUT while while Membership Report has been issued to the DUT/SUT while forwarding
forwarding meshed unicast traffic. meshed unicast traffic.
Procedure: Procedure:
The Forwarding Burdened Group Join Delay test MUST follow the The Forwarding Burdened Group Join Delay test MUST follow the
described setup for the Group Join Delay test in Section 6.1. In described setup for the Group Join Delay test in Section 6.1. In
addition, another set of test ports MUST be used to burden the addition, another set of test ports MUST be used to burden the
DUT/SUT (burdening ports). The burdening ports will be used to DUT/SUT (burdening ports). The burdening ports will be used to
transmit unicast class traffic to the DUT/SUT in a fully meshed transmit unicast class traffic to the DUT/SUT in a fully meshed
traffic pattern as described in RFC 2285 [Ma98]. The DUT/SUT MUST traffic pattern as described in RFC 2285 [Ma98]. The DUT/SUT MUST
learn the appropriate unicast addresses and verified through some learn the appropriate unicast addresses and verified through some
skipping to change at page 28, line 10 skipping to change at page 28, line 43
the recommendations in section 3.1.3. Each row or iteration MAY the recommendations in section 3.1.3. Each row or iteration MAY
specify the group join delay time per egress interface, number of specify the group join delay time per egress interface, number of
frames transmitted and number of frames received for that frames transmitted and number of frames received for that
iteration. iteration.
9. Security Considerations 9. Security Considerations
As this document is solely for the purpose of providing metric As this document is solely for the purpose of providing metric
methodology and describes neither a protocol nor a protocol's methodology and describes neither a protocol nor a protocol's
implementation, there are no security considerations associated implementation, there are no security considerations associated
with this document. with this document specifically. Results from these methodologies
may identify a performance capability or limit of a device or
system in a particular test context. However, such results might
not be representative of the tested entity in an operational
network.
10. Acknowledgements 10. Acknowledgements
The Benchmarking Methodology Working Group of the IETF and The Benchmarking Methodology Working Group of the IETF and
particularly Kevin Dubray, Juniper Networks, are to be thanked for particularly Kevin Dubray, Juniper Networks, are to be thanked for
the many suggestions they collectively made to help complete this the many suggestions they collectively made to help complete this
document. document.
11. Contributions 11. Contributions
skipping to change at page 29, line 21 skipping to change at page 30, line 21
[Br96] Bradner, S., and J. McQuaid, "Benchmarking Methodology for [Br96] Bradner, S., and J. McQuaid, "Benchmarking Methodology for
Network Interconnect Devices", RFC 2544, March 1999. Network Interconnect Devices", RFC 2544, March 1999.
[Br97] Bradner, S. "Use of Keywords in RFCs to Reflect Requirement [Br97] Bradner, S. "Use of Keywords in RFCs to Reflect Requirement
Levels, RFC 2119, March 1997 Levels, RFC 2119, March 1997
[Du98] Dubray, K., "Terminology for IP Multicast Benchmarking", RFC [Du98] Dubray, K., "Terminology for IP Multicast Benchmarking", RFC
2432, October 1998. 2432, October 1998.
[IANA1] IANA multicast address assignments,
http://www.iana.org/assignments/multicast-addresses
[Ma98] Mandeville, R., "Benchmarking Terminology for LAN Switching [Ma98] Mandeville, R., "Benchmarking Terminology for LAN Switching
Devices", RFC 2285, February 1998. Devices", RFC 2285, February 1998.
Informative References Informative References
[Ca02] Cain, B., et al., "Internet Group Management Protocol, Version [Ca02] Cain, B., et al., "Internet Group Management Protocol, Version
3", RFC 3376, October 2002. 3", RFC 3376, October 2002.
[De89] Deering, S., "Host Extensions for IP Multicasting", STD 5, RFC [De89] Deering, S., "Host Extensions for IP Multicasting", STD 5, RFC
1112, August 1989. 1112, August 1989.
skipping to change at page 30, line 27 skipping to change at page 31, line 27
Spirent Communications Spirent Communications
26750 Agoura Rd. 26750 Agoura Rd.
Calabasas, CA 91302 Calabasas, CA 91302
USA USA
Phone: + 1 818 676 2412 Phone: + 1 818 676 2412
EMail: brooks.hickman@spirentcom.com EMail: brooks.hickman@spirentcom.com
14. Full Copyright Statement 14. Full Copyright Statement
"Copyright (C) The Internet Society (2003). All Rights Reserved. "Copyright (C) The Internet Society (2004). All Rights Reserved.
This document and translations of it may be copied and furnished to This document and translations of it may be copied and furnished to
others, and derivative works that comment on or otherwise explain others, and derivative works that comment on or otherwise explain
it or assist in its implementation may be prepared, copied, it or assist in its implementation may be prepared, copied,
published and distributed, in whole or in part, without restriction published and distributed, in whole or in part, without restriction
of any kind, provided that the above copyright notice and this of any kind, provided that the above copyright notice and this
paragraph are included on all such copies and derivative works. paragraph are included on all such copies and derivative works.
However, this document itself may not be modified in any way, such However, this document itself may not be modified in any way, such
as by removing the copyright notice or references to the Internet as by removing the copyright notice or references to the Internet
Society or other Internet organizations, except as needed for the Society or other Internet organizations, except as needed for the
purpose of developing Internet standards in which case the purpose of developing Internet standards in which case the
 End of changes. 

This html diff was produced by rfcdiff 1.23, available from http://www.levkowetz.com/ietf/tools/rfcdiff/