draft-ietf-bmwg-mcastm-11.txt   draft-ietf-bmwg-mcastm-12.txt 
Network Working Group Debra Stopp Network Working Group Debra Stopp
INTERNET-DRAFT Ixia INTERNET-DRAFT Ixia
Expires in: August 2003 Brooks Hickman Expires in: August 2003 Brooks Hickman
Spirent Communications Spirent Communications
February 2003 April 2003
Methodology for IP Multicast Benchmarking Methodology for IP Multicast Benchmarking
<draft-ietf-bmwg-mcastm-11.txt> <draft-ietf-bmwg-mcastm-12.txt>
Status of this Memo Status of this Memo
This document is an Internet-Draft and is in full conformance with This document is an Internet-Draft and is in full conformance with
all provisions of Section 10 of RFC2026. all provisions of Section 10 of RFC2026.
Internet-Drafts are working documents of the Internet Engineering Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF), its areas, and its working groups. Note that Task Force (IETF), its areas, and its working groups. Note that
other groups may also distribute working documents as Internet- other groups may also distribute working documents as Internet-
Drafts. Drafts.
skipping to change at page 2, line 25 skipping to change at page 2, line 25
3.1.3. Frame Sizes...............................................6 3.1.3. Frame Sizes...............................................6
3.1.4. TTL.......................................................6 3.1.4. TTL.......................................................6
3.1.5. Trial Duration............................................6 3.1.5. Trial Duration............................................6
4. FORWARDING AND THROUGHPUT......................................6 4. FORWARDING AND THROUGHPUT......................................6
4.1. Mixed Class Throughput.......................................7 4.1. Mixed Class Throughput.......................................7
4.2. Scaled Group Forwarding Matrix...............................8 4.2. Scaled Group Forwarding Matrix...............................8
4.3. Aggregated Multicast Throughput..............................9 4.3. Aggregated Multicast Throughput..............................9
4.4. Encapsulation/Decapsulation (Tunneling) Throughput..........10 4.4. Encapsulation/Decapsulation (Tunneling) Throughput..........10
4.4.1. Encapsulation Throughput.................................10 4.4.1. Encapsulation Throughput.................................10
4.4.2. Decapsulation Throughput.................................12 4.4.2. Decapsulation Throughput.................................12
4.4.3. Re-encapsulation Throughput..............................13 4.4.3. Re-encapsulation Throughput..............................14
5. FORWARDING LATENCY............................................15 5. FORWARDING LATENCY............................................16
5.1. Multicast Latency...........................................16 5.1. Multicast Latency...........................................16
5.2. Min/Max Multicast Latency...................................18 5.2. Min/Max Multicast Latency...................................18
6. OVERHEAD......................................................20 6. OVERHEAD......................................................19
6.1. Group Join Delay............................................20 6.1. Group Join Delay............................................20
6.2. Group Leave Delay...........................................21 6.2. Group Leave Delay...........................................22
7. CAPACITY......................................................23 7. CAPACITY......................................................24
7.1. Multicast Group Capacity....................................23 7.1. Multicast Group Capacity....................................24
8. INTERACTION...................................................24 8. INTERACTION...................................................25
8.1. Forwarding Burdened Multicast Latency.......................24 8.1. Forwarding Burdened Multicast Latency.......................25
8.2. Forwarding Burdened Group Join Delay........................25 8.2. Forwarding Burdened Group Join Delay........................26
9. SECURITY CONSIDERATIONS.......................................26 9. SECURITY CONSIDERATIONS.......................................28
10. ACKNOWLEDGEMENTS.............................................27 10. ACKNOWLEDGEMENTS.............................................28
11. CONTRIBUTIONS................................................27 11. CONTRIBUTIONS................................................28
12. REFERENCES...................................................28 12. REFERENCES...................................................29
13. AUTHOR'S ADDRESSES...........................................29 13. AUTHOR'S ADDRESSES...........................................30
14. FULL COPYRIGHT STATEMENT.....................................29 14. FULL COPYRIGHT STATEMENT.....................................30
1. Introduction 1. Introduction
This document defines tests for measuring and reporting the This document defines tests for measuring and reporting the
forwarding, latency and IGMP group membership characteristics of throughput, forwarding, latency and IGMP group membership
devices that support IP multicast routing protocols. The results characteristics of devices that support IP multicast protocols.
of these tests will provide the user with meaningful data on The results of these tests will provide the user with meaningful
multicast performance. data on multicast performance.
A previous document, " Terminology for IP Multicast Benchmarking" A previous document, " Terminology for IP Multicast Benchmarking"
(RFC 2432), defined many of the terms that are used in this (RFC 2432), defined many of the terms that are used in this
document. The terminology document should be consulted before document. The terminology document should be consulted before
attempting to make use of this document. attempting to make use of this document.
This methodology will focus on one source to many destinations, This methodology will focus on one source to many destinations,
although many of the tests described may be extended to use although many of the tests described may be extended to use
multiple source to multiple destination topologies. multiple source to multiple destination topologies.
skipping to change at page 3, line 34 skipping to change at page 3, line 34
NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL"
in this document are to be interpreted as described in RFC 2119. in this document are to be interpreted as described in RFC 2119.
RFC 2119 defines the use of these key words to help make the intent RFC 2119 defines the use of these key words to help make the intent
of standards track documents as clear as possible. While this of standards track documents as clear as possible. While this
document uses these keywords, this document is not a standards document uses these keywords, this document is not a standards
track document. track document.
3. Test set up 3. Test set up
The set of methodologies presented in this document are for single The set of methodologies presented in this document are for single
ingress, multiple egress scenarios as exemplified by Figures 1 and ingress, multiple egress multicast scenarios as exemplified by
2. Methodologies for multiple ingress and multiple egress Figures 1 and 2. Methodologies for multiple ingress and multiple
scenarios are beyond the scope of this document. egress multicast scenarios are beyond the scope of this document.
Figure 1 shows a typical setup for an IP multicast test, with one Figure 1 shows a typical setup for an IP multicast test, with one
source to multiple destinations. source to multiple destinations.
+------------+ +--------------+ +------------+ +--------------+
| | | destination | | | | destination |
+--------+ | Egress(-)------->| test | +--------+ | Egress(-)------->| test |
| source | | | | port(E1) | | source | | | | port(E1) |
| test |------>(|)Ingress | +--------------+ | test |------>(|)Ingress | +--------------+
| port | | | +--------------+ | port | | | +--------------+
skipping to change at page 4, line 40 skipping to change at page 4, line 40
interface(s) of devices in the SUT. Figure 2 offers an example SUT interface(s) of devices in the SUT. Figure 2 offers an example SUT
test topology. If a SUT is tested, the test topology and all test topology. If a SUT is tested, the test topology and all
relevant configuration details MUST be disclosed with the relevant configuration details MUST be disclosed with the
corresponding test results. corresponding test results.
*-----------------------------------------* *-----------------------------------------*
| | | |
+--------+ | +----------------+ | +--------+ +--------+ | +----------------+ | +--------+
| | | +------------+ |DUT B Egress E0(-)-|->| | | | | +------------+ |DUT B Egress E0(-)-|->| |
| | | |DUT A |--->| | | | | | | | |DUT A |--->| | | | |
| Test | | | | | Egress E1(-)-|->| Test | | source | | | | | Egress E1(-)-|->| dest. |
| App. |--|->(-)Ingress, I | +----------------+ | | App. | | test |--|->(-)Ingress, I | +----------------+ | | test |
| Traffic| | | | +----------------+ | | Traffic| | port | | | | +----------------+ | | port |
| Src. | | | |--->|DUT C Egress E2(-)-|->| Dest. | | | | | |--->|DUT C Egress E2(-)-|->| |
| | | +------------+ | | | | | | | | +------------+ | | | | |
| | | | Egress En(-)--|->| | | | | | Egress En(-)-|->| |
+--------+ | +----------------+ | +--------+ +--------+ | +----------------+ | +--------+
| | | |
*------------------SUT--------------------* *------------------SUT--------------------*
Figure 2 Figure 2
--------- ---------
Generally, the destination test ports first join the desired number Generally, the destination test ports first join the desired number
of multicast groups by sending IGMP Group Report messages to the of multicast groups by sending IGMP Group Report messages to the
DUT/SUT. To verify that all destination test ports successfully DUT/SUT. To verify that all destination test ports successfully
joined the appropriate groups, the source port MUST transmit IP joined the appropriate groups, the source test port MUST transmit
multicast frames destined for these groups. The destination test IP multicast frames destined for these groups. After test
ports MAY send IGMP Leave Group messages after the transmission of completion, the destination test ports MAY send IGMP Leave Group
IP Multicast frames to clear the IGMP table of the DUT/SUT. messages to clear the IGMP table of the DUT/SUT.
In addition, test equipment MUST validate the correct and proper In addition, test equipment MUST validate the correct and proper
forwarding actions of the devices they test in order to ensure the forwarding actions of the devices they test in order to ensure the
receipt of the frames that are involved in the test. receipt of the frames that are involved in the test.
3.1. Test Considerations 3.1. Test Considerations
The methodology assumes a uniform medium topology. Issues regarding The methodology assumes a uniform medium topology. Issues regarding
mixed transmission media, such as speed mismatch, headers mixed transmission media, such as speed mismatch, headers
differences, etc., are not specifically addressed. Flow control, differences, etc., are not specifically addressed. Flow control,
QoS and other non-essential traffic or traffic-affecting mechanisms QoS and other non-essential traffic or traffic-affecting mechanisms
affecting the variable under test MUST be disabled. Modifications affecting the variable under test MUST be disabled. Modifications
to the collection procedures might need to be made to accommodate to the collection procedures might need to be made to accommodate
the transmission media actually tested. These accommodations MUST the transmission media actually tested. These accommodations MUST
be presented with the test results. be presented with the test results.
An actual flow of test traffic may be required to prime related An actual flow of test traffic MAY be required to prime related
mechanisms, (e.g., process RPF events, build device caches, etc.) mechanisms, (e.g., process RPF events, build device caches, etc.)
to optimally forward subsequent traffic. Therefore, before an to optimally forward subsequent traffic. Therefore, prior to
initial, measured forwarding test trial, the test apparatus MUST running any tests that require forwarding of multicast or unicast
generate test traffic utilizing the same addressing characteristics packets, the test apparatus MUST generate test traffic utilizing
to the DUT/SUT that will subsequently be used to measure the the same addressing characteristics to the DUT/SUT that will
DUT/SUT response. The test monitor should ensure the correct subsequently be used to measure the DUT/SUT response. The test
forwarding of traffic by the DUT/SUT. The priming action need only monitor should ensure the correct forwarding of traffic by the
be repeated to keep the associated information current. DUT/SUT. The priming action need only be repeated to keep the
associated information current.
3.1.1. IGMP Support 3.1.1. IGMP Support
All of the ingress and egress interfaces MAY support any version of All of the ingress and egress interfaces MUST support a version of
IGMP. The IGMP version on the ingress interface MUST be the same IGMP. The IGMP version on the ingress interface MUST be the same
version of IGMP that is being tested on the egress interfaces. version of IGMP that is being tested on the egress interfaces.
Each of the ingress and egress interfaces SHOULD be able to respond Each of the ingress and egress interfaces SHOULD be able to respond
to IGMP queries during the test. to IGMP queries during the test.
Each of the ingress and egress interfaces SHOULD also send LEAVE Each of the ingress and egress interfaces SHOULD also send LEAVE
(running IGMP version 2 or later) after each test. (running IGMP version 2 or later) after each test.
3.1.2. Group Addresses 3.1.2. Group Addresses
It is intended that the collection of benchmarks prescribed in this It is intended that the collection of benchmarks prescribed in this
document be executed in an isolated lab environment. That is to document be executed in an isolated lab environment. That is to
say, the test traffic offered the tested devices MUST NOT traverse say, the test traffic offered the tested devices MUST NOT traverse
a live internet, intranet, or other production network. a live internet, intranet, or other production network.
Assuming the above, there is no restriction to the use of multicast There is no restriction to the use of multicast addresses to
addresses to compose the test traffic other than those assignments compose the test traffic other than those assignments imposed by
imposed by IANA. The IANA assignments MUST be regarded for IANA. The IANA assignments MUST be regarded for operational
operational consistency. For multicast address assignments see: consistency. For multicast address assignments see:
http://www.iana.org/assignments/multicast-addresses http://www.iana.org/assignments/multicast-addresses
Address selection does not need to be restricted to Address selection does not need to be restricted to
Administratively Scoped IP Multicast addresses. Administratively Scoped IP Multicast addresses.
3.1.3. Frame Sizes 3.1.3. Frame Sizes
Each test SHOULD be run with different multicast frame sizes. For Each test SHOULD be run with different multicast frame sizes. For
Ethernet, the recommended sizes are 64, 128, 256, 512, 1024, 1280, Ethernet, the recommended sizes are 64, 128, 256, 512, 1024, 1280,
skipping to change at page 6, line 32 skipping to change at page 6, line 32
frame lengths of the link layer technology in use SHOULD be tested. frame lengths of the link layer technology in use SHOULD be tested.
When testing with different frame sizes, the DUT/SUT configuration When testing with different frame sizes, the DUT/SUT configuration
MUST remain the same. MUST remain the same.
3.1.4. TTL 3.1.4. TTL
The data plane test traffic should have a TTL value large enough to The data plane test traffic should have a TTL value large enough to
traverse the DUT/SUT. traverse the DUT/SUT.
The TTL in IGMP control plane messages is in compliance with the The TTL in IGMP control plane messages MUST be in compliance with
version of IGMP in use. the version of IGMP in use.
3.1.5. Trial Duration 3.1.5. Trial Duration
The duration of the test portion of each trial SHOULD be at least The duration of the test portion of each trial SHOULD be at least
30 seconds. This parameter MUST be included as part of the results 30 seconds. This parameter MUST be included as part of the results
reporting for each methodology. reporting for each methodology.
4. Forwarding and Throughput 4. Forwarding and Throughput
This section contains the description of the tests that are related This section contains the description of the tests that are related
to the characterization of the frame forwarding of a DUT/SUT in a to the characterization of the frame forwarding of a DUT/SUT in a
multicast environment. Some metrics extend the concept of throughput multicast environment. Some metrics extend the concept of throughput
presented in RFC 1242. Forwarding Rate is cited in RFC 2285. presented in RFC 1242. Forwarding Rate is cited in RFC 2285 [Ma98].
4.1. Mixed Class Throughput 4.1. Mixed Class Throughput
Objective: Objective:
To determine the throughput of a DUT/SUT when both unicast class To determine the throughput of a DUT/SUT when both unicast class
frames and multicast class frames are offered simultaneously to a frames and multicast class frames are offered simultaneously to a
fixed number of interfaces as defined in RFC 2432. fixed number of interfaces as defined in RFC 2432.
Procedure: Procedure:
Multicast and unicast traffic are mixed together in the same Multicast and unicast traffic are mixed together in the same
aggregated traffic stream in order to simulate the non-homogenous aggregated traffic stream in order to simulate the non-homogenous
networking environment. networking environment.
The following events MUST occur before offering test traffic: The following events MUST occur before offering test traffic:
o All DUT/SUT egress interfaces configured to receive o All destination test ports configured to receive multicast
multicast traffic MUST join all configured multicast traffic MUST join all configured multicast groups;
groups;
o The DUT/SUT MUST learn the appropriate unicast addresses; o The DUT/SUT MUST learn the appropriate unicast addresses;
and and
o Group membership and unicast address learning MUST be o Group membership and unicast address learning MUST be
verified through some externally observable method. verified through some externally observable method.
The intended load [Ma98] SHOULD be configured as alternating The intended load [Ma98] SHOULD be configured as alternating
multicast frames and unicast frames to a single ingress interface multicast class frames and unicast class frames to a single ingress
in a 50-50 ratio. The unicast frames MUST be configured to interface. The unicast class frames MUST be configured to transmit
transmit in a round-robin fashion to all of the egress interfaces. in a round-robin fashion to all of the destination ports.
The multicast frames MUST be configured to transmit to all of the
egress interfaces.
Mixed class throughput measurement is defined in RFC2432 [Du98]. A Mixed class throughput measurement is defined in RFC2432 [Du98]. A
search algorithm MUST be utilized to determine the throughput for search algorithm MUST be utilized to determine the Mixed Class
both unicast class and multicast class traffic in a mixed class Throughput.
environment.
Reporting Format: Reporting Format:
The following configuration parameters MUST be reflected in the The following configuration parameters MUST be reflected in the
results specific to this methodology: test report:
o Frame size(s) o Frame size(s)
o Number of tested egress interfaces on the DUT/SUT o Number of tested egress interfaces on the DUT/SUT
o Test duration o Test duration
o IGMP version o IGMP version
o Total number of multicast groups o Total number of multicast groups
o Traffic distribution for unicast and multicast traffic o Traffic distribution for unicast and multicast traffic
classes classes
o The ratio of multicast and unicast traffic must be declared o The ratio of multicast to unicast class traffic
The following results MUST be reflected in the results specific to The following results MUST be reflected in the test report:
this methodology:
o Mixed Class Throughput as defined in RFC2432 [Du98], o Mixed Class Throughput as defined in RFC2432 [Du98],
including: Throughput per unicast and multicast traffic including: Throughput per unicast and multicast traffic
classes. classes.
The Mixed Class Throughput results for each test SHOULD be reported The Mixed Class Throughput results for each test SHOULD be reported
in the form of a table with a row for each of the tested frame in the form of a table with a row for each of the tested frame
sizes per the recommendations in section 3.1.3. Each row SHOULD sizes per the recommendations in section 3.1.3. Each row SHOULD
specify the intended load, number of multicast frames offered, specify the intended load, number of multicast frames offered,
number of unicast frames offered and measured throughput per class. number of unicast frames offered and measured throughput per class.
skipping to change at page 8, line 28 skipping to change at page 8, line 27
Objective: Objective:
To determine Forwarding Rate as a function of tested multicast To determine Forwarding Rate as a function of tested multicast
groups for a fixed number of tested DUT/SUT ports. groups for a fixed number of tested DUT/SUT ports.
Procedure: Procedure:
This is an iterative procedure. The destination test port(s) MUST This is an iterative procedure. The destination test port(s) MUST
join an initial number of multicast groups on the first iteration. join an initial number of multicast groups on the first iteration.
All DUT/SUT destination test port(s) configured to receive All destination test ports configured to receive multicast traffic
multicast traffic MUST join all configured multicast groups. The MUST join all configured multicast groups. The recommended number
recommended number of groups to join on the first iteration is 10 of groups to join on the first iteration is 10 groups. Multicast
groups. Multicast traffic is subsequently transmitted to all traffic is subsequently transmitted to all groups joined on this
groups joined on this iteration. iteration and the forwarding rate is measured.
The number of multicast groups joined by each destination test port The number of multicast groups joined by each destination test port
is then incremented, or scaled, by an additional number of is then incremented, or scaled, by an additional number of
multicast groups. The recommended granularity of additional groups multicast groups. The recommended granularity of additional groups
to join per iteration is 10, although the tester MAY choose a finer to join per iteration is 10, although the tester MAY choose a finer
granularity. Multicast traffic is subsequently transmitted to all granularity. Multicast traffic is subsequently transmitted to all
groups joined during this iteration. groups joined during this iteration and the forwarding rate is
measured.
The total number of multicast groups joined MUST not exceed the The total number of multicast groups joined MUST not exceed the
capacity of the DUT/SUT. Both Group Join Delay and Group Capacity multicast group capacity of the DUT/SUT. The Group Capacity
results MUST be known prior to running this test. (Section 7.1) results MUST be known prior to running this test.
Reporting Format: Reporting Format:
The following configuration parameters MUST be reflected in the The following configuration parameters MUST be reflected in the
results specific to this methodology: test report:
o Frame size(s) o Frame size(s)
o Number of tested egress interfaces on the DUT/SUT o Number of tested egress interfaces on the DUT/SUT
o Test duration o Test duration
o IGMP version o IGMP version
The following results MUST be reflected in the results specific to The following results MUST be reflected in the test report:
this methodology:
o The total number of multicast groups joined for that o The total number of multicast groups joined for that
iteration iteration
o Total number of frames transmitted
o Total number of frames received
o Offered load
o Forwarding rate determined for that iteration o Forwarding rate determined for that iteration
The Scaled Group Forwarding results for each test SHOULD be The Scaled Group Forwarding results for each test SHOULD be
reported in the form of a table with a row representing each reported in the form of a table with a row representing each
iteration of the test. Each row or iteration SHOULD specify the iteration of the test. Each row or iteration SHOULD specify the
total number of groups joined for that iteration, total number of total number of groups joined for that iteration, offered load,
frames transmitted, total number of frames received and the total number of frames transmitted, total number of frames received
aggregate forwarding rate determined for that iteration. and the aggregate forwarding rate determined for that iteration.
4.3. Aggregated Multicast Throughput 4.3. Aggregated Multicast Throughput
Objective: Objective:
To determine the maximum rate at which none of the offered frames To determine the maximum rate at which none of the offered frames
to be forwarded through N destination interfaces of the same to be forwarded through N destination interfaces of the same
multicast groups are dropped. multicast groups are dropped.
Procedure: Procedure:
Offer multicast traffic at an initial fixed offered load to a fixed Offer multicast traffic at an initial maximum offered load to a
set of interfaces with a fixed number of groups at a fixed frame fixed set of interfaces with a fixed number of groups at a fixed
length for a fixed duration of time. All destination test ports frame length for a fixed duration of time. All destination test
MUST join all specified multicast groups. ports MUST join all specified multicast groups.
If any frame loss is detected, the offered load is decreased and If any frame loss is detected, the offered load is decreased and
the sender will transmit again. An iterative search algorithm MUST the sender will transmit again. An iterative search algorithm MUST
be utilized to determine the maximum offered frame rate with a zero be utilized to determine the maximum offered frame rate with a zero
frame loss. frame loss.
Each iteration will involve varying the offered load of the Each iteration will involve varying the offered load of the
multicast traffic, while keeping the set of interfaces, number of multicast traffic, while keeping the set of interfaces, number of
multicast groups, frame length and test duration fixed, until the multicast groups, frame length and test duration fixed, until the
maximum rate at which none of the offered frames are dropped is maximum rate at which none of the offered frames are dropped is
determined. determined.
Parameters to be measured MUST include the offered load at which no Parameters to be measured MUST include the maximum offered load at
frame loss occurred. which no frame loss occurred. Other offered loads MAY be measured
for diagnostic purposes.
Reporting Format: Reporting Format:
The following configuration parameters MUST be reflected in the The following configuration parameters MUST be reflected in the
results specific to this methodology: test report:
o Frame size(s) o Frame size(s)
o Number of tested egress interfaces on the DUT/SUT o Number of tested egress interfaces on the DUT/SUT
o Test duration o Test duration
o IGMP version o IGMP version
o Total number of multicast groups o Total number of multicast groups
The following results MUST be reflected in the results specific to The following results MUST be reflected in the test report:
this methodology:
o Aggregated Multicast Throughput as defined in RFC2432 o Aggregated Multicast Throughput as defined in RFC2432
[Du98] [Du98]
The Aggregated Multicast Throughput results SHOULD be reported in The Aggregated Multicast Throughput results SHOULD be reported in
the format of a table with a row for each of the tested frame sizes the format of a table with a row for each of the tested frame sizes
per the recommendations in section 3.1.3. Each row or iteration per the recommendations in section 3.1.3. Each row or iteration
SHOULD specify offered load, total number of offered frames and the SHOULD specify offered load, total number of offered frames and the
measured Aggregated Multicast Throughput. measured Aggregated Multicast Throughput.
4.4. Encapsulation/Decapsulation (Tunneling) Throughput 4.4. Encapsulation/Decapsulation (Tunneling) Throughput
This sub-section provides the description of tests that help in This sub-section provides the description of tests related to the
obtaining throughput measurements when a DUT/SUT or a set of DUTs determination of throughput measurements when a DUT/SUT or a set of
are acting as tunnel endpoints. DUTs are acting as tunnel endpoints.
4.4.1. Encapsulation Throughput 4.4.1. Encapsulation Throughput
Objective: Objective:
To determine the maximum rate at which frames offered to one To determine the maximum rate at which frames offered to one
ingress interface of a DUT/SUT are encapsulated and correctly ingress interface of a DUT/SUT are encapsulated and correctly
forwarded on one or more egress interfaces of the DUT/SUT without forwarded on one or more egress interfaces of the DUT/SUT without
loss. loss.
skipping to change at page 11, line 44 skipping to change at page 11, line 44
b) Multiple tunnels, each encapsulating one or more multicast b) Multiple tunnels, each encapsulating one or more multicast
address groups. address groups.
The number of multicast groups per tunnel MUST be the same when the The number of multicast groups per tunnel MUST be the same when the
DUT/SUT is configured in a multiple tunnel configuration. In DUT/SUT is configured in a multiple tunnel configuration. In
addition, it is RECOMMENDED to test with the same number of tunnels addition, it is RECOMMENDED to test with the same number of tunnels
on each egress interface. All destination test ports MUST join all on each egress interface. All destination test ports MUST join all
multicast group addresses offered by the source test port. Each multicast group addresses offered by the source test port. Each
egress interface MUST be configured with the same MTU. egress interface MUST be configured with the same MTU.
Note: when offering large frames sizes, the encapsulation process
may require the DUT/SUT to fragment the IP datagrams prior to being
forwarded on the egress interface. It is RECOMMENDED to limit the
offered frame size such that no fragmentation is required by the
DUT/SUT.
A search algorithm MUST be utilized to determine the encapsulation A search algorithm MUST be utilized to determine the encapsulation
throughput as defined in [Du98]. throughput as defined in [Du98].
Reporting Format: Reporting Format:
The following configuration parameters MUST be reflected in the The following configuration parameters MUST be reflected in the
results specific to this methodology: test report:
o Number of tested egress interfaces on the DUT/SUT o Number of tested egress interfaces on the DUT/SUT
o Test duration o Test duration
o IGMP version o IGMP version
o Total number of multicast groups o Total number of multicast groups
o MTU size of DUT/SUT interfaces o MTU size of DUT/SUT interfaces
The following results MUST be reflected in the results specific to o Originating un-encapsulated frame size
this methodology: o Number of tunnels per egress interface
o Number of multicast groups per tunnel
The following results MUST be reflected in the test report:
o Measured Encapsulated Throughput as defined in RFC2432 o Measured Encapsulated Throughput as defined in RFC2432
[Du98] [Du98]
o Encapsulated frame size o Encapsulated frame size
o Originating un-encapsulated frame size
o Number of tunnels
o Number of multicast groups per tunnel
The Encapsulated Throughput results SHOULD be reported in the form The Encapsulated Throughput results SHOULD be reported in the form
of a table and specific to this test there SHOULD be rows for each of a table and specific to this test there SHOULD be rows for each
originating un-encapsulated frame size. Each row or iteration originating un-encapsulated frame size. Each row or iteration
SHOULD specify the offered load, encapsulation method, encapsulated SHOULD specify the offered load, encapsulation method, encapsulated
frame size, total number of offered frames, and the encapsulation frame size, total number of offered frames, and the encapsulation
throughput. throughput.
4.4.2. Decapsulation Throughput 4.4.2. Decapsulation Throughput
skipping to change at page 13, line 5 skipping to change at page 13, line 28
Figure 4 Figure 4
--------- ---------
Figure 4 shows the setup for testing the decapsulation throughput Figure 4 shows the setup for testing the decapsulation throughput
of the DUT/SUT. One or more tunnels are created between the source of the DUT/SUT. One or more tunnels are created between the source
test port and the DUT/SUT. Encapsulated multicast traffic will test port and the DUT/SUT. Encapsulated multicast traffic will
then be offered by the source test port, decapsulated by the then be offered by the source test port, decapsulated by the
DUT/SUT and forwarded to the destination test port(s). DUT/SUT and forwarded to the destination test port(s).
The DUT/SUT SHOULD be configured such that the traffic across each The DUT/SUT SHOULD be configured such that the traffic across the
egress interface will consist of either: ingress interface will consist of either:
a) A single tunnel encapsulating one or more multicast address a) A single tunnel encapsulating one or more multicast address
groups OR groups OR
b) Multiple tunnels, each encapsulating one or more multicast b) Multiple tunnels, each encapsulating one or more multicast
address groups. address groups.
The number of multicast groups per tunnel MUST be the same when the The number of multicast groups per tunnel MUST be the same when the
DUT/SUT is configured in a multiple tunnel configuration. All DUT/SUT is configured in a multiple tunnel configuration. All
destination test ports MUST join all multicast group addresses destination test ports MUST join all multicast group addresses
offered by the source test port. Each egress interface MUST offered by the source test port. Each egress interface MUST
be configured with the same MTU. be configured with the same MTU.
A search algorithm MUST be utilized to determine the decapsulation A search algorithm MUST be utilized to determine the decapsulation
throughput as defined in [Du98]. throughput as defined in [Du98].
When making performance comparisons between the encapsulation and
decapsulation process of the DUT/SUT, the offered frame sizes
SHOULD reflect the encapsulated frame sizes reported in the
encapsulation test (See section 4.4.1) in place of those noted in
section 3.1.3.
Reporting Format: Reporting Format:
The following configuration parameters MUST be reflected in the The following configuration parameters MUST be reflected in the
results specific to this methodology: test report:
o Number of tested egress interfaces on the DUT/SUT o Number of tested egress interfaces on the DUT/SUT
o Test duration o Test duration
o IGMP version o IGMP version
o Total number of multicast groups o Total number of multicast groups
o MTU size of DUT/SUT interfaces o Originating encapsulation format
o Originating encapsulated frame size
o Number of tunnels
o Number of multicast groups per tunnel
The following results MUST be reflected in the results specific to The following results MUST be reflected in the test report:
this methodology:
o Measured Decapsulated Throughput as defined in RFC2432 o Measured Decapsulated Throughput as defined in RFC2432
[Du98] [Du98]
o Originating encapsulation format
o Decapsulated frame size o Decapsulated frame size
o Originating encapsulated frame size
o Number of tunnels
o Number of multicast groups per tunnel
The Decapsulated Throughput results SHOULD be reported in the The Decapsulated Throughput results SHOULD be reported in the
format of a table and specific to this test there SHOULD be rows format of a table and specific to this test there SHOULD be rows
for each originating encapsulated frame size. Each row or for each originating encapsulated frame size. Each row or
iteration SHOULD specify the offered load, decapsulated frame size, iteration SHOULD specify the offered load, decapsulated frame size,
total number of offered frames and the decapsulation throughput. total number of offered frames and the decapsulation throughput.
4.4.3. Re-encapsulation Throughput 4.4.3. Re-encapsulation Throughput
Objective: Objective:
To determine the maximum rate at which frames of one encapsulated To determine the maximum rate at which frames of one encapsulated
format offered to one ingress interface of a DUT/SUT are converted format offered to one ingress interface of a DUT/SUT are converted
to another encapsulated format and correctly forwarded by the to another encapsulated format and correctly forwarded by the
DUT/SUT to one or more egress interfaces without loss. DUT/SUT on one or more egress interfaces without loss.
Procedure: Procedure:
Source DUT/SUT Destination Source DUT/SUT Destination
Test Port Test Port(s) Test Port Test Port(s)
+---------+ +---------+ +---------+ +---------+ +---------+ +---------+
| | | | | | | | | | | |
| | | Egress|-(Tunnel)->| | | | | Egress|-(Tunnel)->| |
| | | | | | | | | | | |
| |-(Tunnel)->|Ingress | | | | |-(Tunnel)->|Ingress | | |
skipping to change at page 14, line 23 skipping to change at page 15, line 4
| | | Egress|-(Tunnel)->| | | | | Egress|-(Tunnel)->| |
| | | | | | | | | | | |
| |-(Tunnel)->|Ingress | | | | |-(Tunnel)->|Ingress | | |
| | | | | | | | | | | |
| | | Egress|-(Tunnel)->| | | | | Egress|-(Tunnel)->| |
| | | | | | | | | | | |
+---------+ +---------+ +---------+ +---------+ +---------+ +---------+
Figure 5 Figure 5
--------- ---------
Figure 5 shows the setup for testing the Re-encapsulation Figure 5 shows the setup for testing the Re-encapsulation
throughput of the DUT/SUT. The source test port will offer throughput of the DUT/SUT. The source test port will offer
encapsulated traffic of one type to the DUT/SUT, which has been encapsulated traffic of one type to the DUT/SUT, which has been
configured to re-encapsulate the offered frames using a different configured to re-encapsulate the offered frames using a different
encapsulation format. The DUT/SUT will then forward the re- encapsulation format. The DUT/SUT will then forward the re-
encapsulated frames to the destination test port(s). encapsulated frames to the destination test port(s).
The DUT/SUT SHOULD be configured such that the traffic across each The DUT/SUT SHOULD be configured such that the traffic across the
egress interface will consist of either: ingress and each egress interface will consist of either:
a) A single tunnel encapsulating one or more multicast address a) A single tunnel encapsulating one or more multicast address
groups OR groups OR
b) Multiple tunnels, each encapsulating one or more multicast b) Multiple tunnels, each encapsulating one or more multicast
address groups. address groups.
The number of multicast groups per tunnel MUST be the same when the The number of multicast groups per tunnel MUST be the same when the
DUT/SUT is configured in a multiple tunnel configuration. DUT/SUT is configured in a multiple tunnel configuration. In
addition, the DUT/SUT SHOULD be configured such that the number of
In addition, the DUT/SUT SHOULD be configured such that the number tunnels on the ingress and each egress interface are the same. All
of tunnels on the ingress and each egress interface are the same. destination test ports MUST join all multicast group addresses
All destination test ports MUST join all multicast group addresses
offered by the source test port. Each egress interface MUST be offered by the source test port. Each egress interface MUST be
configured with the same MTU. configured with the same MTU.
Note that when offering large frames sizes, the encapsulation
process may require the DUT/SUT to fragment the IP datagrams prior
to being forwarded on the egress interface. It is RECOMMENDED to
limit the offered frame sizes, such that no fragmentation is
required by the DUT/SUT.
A search algorithm MUST be utilized to determine the re- A search algorithm MUST be utilized to determine the re-
encapsulation throughput as defined in [Du98]. encapsulation throughput as defined in [Du98].
Reporting Format: Reporting Format:
The following configuration parameters MUST be reflected in the The following configuration parameters MUST be reflected in the
results specific to this methodology: test report:
o Number of tested egress interfaces on the DUT/SUT o Number of tested egress interfaces on the DUT/SUT
o Test duration o Test duration
o IGMP version o IGMP version
o Total number of multicast groups o Total number of multicast groups
o MTU size of DUT/SUT interfaces o MTU size of DUT/SUT interfaces
The following results MUST be reflected in the results specific to
this methodology:
o Measured Re-encapsulated Throughput as defined in RFC2432
[Du98]
o Originating encapsulation format o Originating encapsulation format
o Decapsulated frame size
o Originating encapsulated frame size o Originating encapsulated frame size
o Number of tunnels o Number of tunnels per interface
o Number of multicast groups per tunnel o Number of multicast groups per tunnel
The following results MUST be reflected in the test report:
The Decapsulated Throughput results SHOULD be reported in the o Measured Re-encapsulated Throughput as defined in RFC2432
[Du98]
o Re-encapsulated frame size
The Re-encapsulated Throughput results SHOULD be reported in the
format of a table and specific to this test there SHOULD be rows format of a table and specific to this test there SHOULD be rows
for each originating encapsulated frame size. Each row or for each originating encapsulated frame size. Each row or
iteration SHOULD specify the offered load, decapsulated frame size, iteration SHOULD specify the offered load, decapsulated frame size,
total number of offered frames and the decapsulation throughput total number of offered frames and the Re-encapsulated Throughput.
5. Forwarding Latency 5. Forwarding Latency
This section presents methodologies relating to the This section presents methodologies relating to the
characterization of the forwarding latency of a DUT/SUT in a characterization of the forwarding latency of a DUT/SUT in a
multicast environment. It extends the concept of latency multicast environment. It extends the concept of latency
characterization presented in RFC 2544. characterization presented in RFC 2544.
To lessen the effect of frame buffering in the DUT/SUT, the latency The offered load accompanying the latency-measured packet can
tests MUST be run at the measured multicast throughput level of the affect the DUT/SUT packet buffering, which may subsequently impact
DUT; multicast latency at other offered loads is optional. measured packet latency. This SHOULD be a consideration when
selecting the intended load for the described methodologies below.
Lastly, RFC 1242 and RFC 2544 draw a distinction between device RFC 1242 and RFC 2544 draw a distinction between device types:
types: "store and forward" and "bit-forwarding." Each type impacts "store and forward" and "bit-forwarding." Each type impacts how
how latency is collected and subsequently presented. See the latency is collected and subsequently presented. See the related
related RFCs for more information. In practice, much of the test RFCs for more information.
equipment will collect the latency measurement for one type or the
other, and, if needed, mathematically derive the reported value by
the addition or subtraction of values accounting for medium
propagation delay of the frame, bit times to the timestamp trigger
within the frame, etc.
5.1. Multicast Latency 5.1. Multicast Latency
Objective: Objective:
To produce a set of multicast latency measurements from a single, To produce a set of multicast latency measurements from a single,
multicast ingress interface of a DUT/SUT through multiple, egress multicast ingress interface of a DUT/SUT through multiple, egress
multicast interfaces of that same DUT/SUT as provided for by the multicast interfaces of that same DUT/SUT as provided for by the
metric "Multicast Latency" in RFC 2432. metric "Multicast Latency" in RFC 2432 [Du98].
The Procedures highlighted below attempt to draw from the The procedures below draw from the collection methodology for
collection methodology for latency in RFC 2544 to the degree latency in RFC 2544 [Br96]. The methodology addresses two
possible. The methodology addresses two topological scenarios: one topological scenarios: one for a single device (DUT)
for a single device (DUT) characterization; a second scenario is characterization; a second scenario is presented or multiple device
presented or multiple device (SUT) characterization. (SUT) characterization.
Procedure: Procedure:
If the test trial is to characterize latency across a single Device If the test trial is to characterize latency across a single Device
Under Test (DUT), an example test topology might take the form of Under Test (DUT), an example test topology might take the form of
Figure 1 in section 3. That is, a single DUT with one ingress Figure 1 in section 3. That is, a single DUT with one ingress
interface receiving the multicast test traffic from frame- interface receiving the multicast test traffic from frame-
transmitting component of the test apparatus and n egress transmitting component of the test apparatus and n egress
interfaces on the same DUT forwarding the multicast test traffic interfaces on the same DUT forwarding the multicast test traffic
back to the frame-receiving component of the test apparatus. Note back to the frame-receiving component of the test apparatus. Note
that n reflects the number of TESTED egress interfaces on the DUT that n reflects the number of TESTED egress interfaces on the DUT
actually expected to forward the test traffic (as opposed to actually expected to forward the test traffic (as opposed to
configured but untested, non-forwarding interfaces, for example). configured but untested, non-forwarding interfaces, for example).
If the multicast latencies are to be taken across multiple devices If the multicast latencies are to be taken across multiple devices
forming a System Under Test (SUT), an example test topology might forming a System Under Test (SUT), an example test topology might
take the form of Figure 2 in section 3. take the form of Figure 2 in section 3.
The trial duration SHOULD be 120 seconds to be consistent with RFC The trial duration SHOULD be 120 seconds to be consistent with RFC
2544. The nature of the latency measurement, "store and forward" 2544 [Br96]. The nature of the latency measurement, "store and
or "bit forwarding," MUST be associated with the related test forward" or "bit forwarding," MUST be associated with the related
trial(s) and disclosed in the results report. test trial(s) and disclosed in the results report.
End-to-end reachability of the test traffic path MUST be verified
prior to the engagement of a test trial. This implies that
subsequent measurements are intended to characterize the latency
across the tested device's or devices' normal traffic forwarding
path (e.g., faster hardware-based engines) of the device(s) as
opposed a non-standard traffic processing path (e.g. slower,
software-based exception handlers). If the test trial is to be
executed with the intent of characterizing a non-optimal,
forwarding condition, then a description of the exception
processing conditions being characterized MUST be included with the
trial's results.
A test traffic stream is presented to the DUT. It is RECOMMENDED to A test traffic stream is presented to the DUT. It is RECOMMENDED to
offer traffic at the measured aggregated multicast throughput rate offer traffic at the measured aggregated multicast throughput rate
(Section 4.3). At the mid-point of the trial's duration, the test (Section 4.3). At the mid-point of the trial's duration, the test
apparatus MUST inject a uniquely identifiable ("tagged") frame into apparatus MUST inject a uniquely identifiable ("tagged") frame into
the test traffic frames being presented. This tagged frame will be the test traffic frames being presented. This tagged frame will be
the basis for the latency measurements. By "uniquely identifiable," the basis for the latency measurements. By "uniquely identifiable,"
it is meant that the test apparatus MUST be able to discern the it is meant that the test apparatus MUST be able to discern the
"tagged" frame from the other frames comprising the test traffic "tagged" frame from the other frames comprising the test traffic
set. A frame generation timestamp, Timestamp A, reflecting the set. A frame generation timestamp, Timestamp A, reflecting the
completion of the transmission of the tagged frame by the test completion of the transmission of the tagged frame by the test
apparatus, MUST be determined. apparatus, MUST be determined.
The test apparatus then monitors frames from the DUT's tested The test apparatus will monitor frames from the DUT's tested egress
egress interface(s) for the expected tagged frame(s) until the interface(s) for the expected tagged frame(s) and MUST record the
cessation of traffic generation at the end of the configured trial time of the successful detection of a tagged frame from a tested
duration. egress interface with a timestamp, Timestamp B. A set of Timestamp
B values MUST be collected for all tested egress interfaces of the
The test apparatus MUST record the time of the successful detection DUT/SUT. See RFC 1242 [Br91] for additional discussion regarding
of a tagged frame from a tested egress interface with a timestamp, store and forward devices and bit forwarding devices.
Timestamp B. A set of Timestamp B values MUST be collected for all
tested egress interfaces of the DUT/SUT. See RFC 1242 [Br91] for
additional discussion regarding store and forward devices and bit
forwarding devices.
A trial MUST be considered INVALID should any of the following A trial MUST be considered INVALID should any of the following
conditions occur in the collection of the trial data: conditions occur in the collection of the trial data:
o Forwarded test frames directed to improper destinations.
o Unexpected differences between Intended Load and Offered o Unexpected differences between Intended Load and Offered
Load or unexpected differences between Offered Load and the Load or unexpected differences between Offered Load and the
resulting Forwarding Rate(s) on the DUT/SUT egress ports. resulting Forwarding Rate(s) on the DUT/SUT egress ports.
o Forwarded test frames improperly formed or frame header o Forwarded test frames improperly formed or frame header
fields improperly manipulated. fields improperly manipulated.
o Failure to forward required tagged frame(s) on all expected o Failure to forward required tagged frame(s) on all expected
egress interfaces. egress interfaces.
o Reception of a tagged frame by the test apparatus outside o Reception of tagged frames by the test apparatus more than
the configured test duration interval or 5 seconds, 5 seconds after the cessation of test traffic by the source
whichever is greater. test port.
Data from invalid trials SHOULD be considered inconclusive. Data
from invalid trials MUST not form the basis of comparison.
The set of latency measurements, M, composed from each latency The set of latency measurements, M, composed from each latency
measurement taken from every ingress/tested egress interface measurement taken from every ingress/tested egress interface
pairing MUST be determined from a valid test trial: pairing MUST be determined from a valid test trial:
M = { (Timestamp B(E0) - Timestamp A), M = { (Timestamp B(E0) - Timestamp A),
(Timestamp B(E1) - Timestamp A), ... (Timestamp B(E1) - Timestamp A), ...
(Timestamp B(En) - Timestamp A) } (Timestamp B(En) - Timestamp A) }
where (E0 ... En) represents the range of all tested egress where (E0 ... En) represents the range of all tested egress
interfaces and Timestamp B represents a tagged frame detection interfaces and Timestamp B represents a tagged frame detection
event for a given DUT/SUT tested egress interface. event for a given DUT/SUT tested egress interface.
A more continuous profile MAY be built from a series of individual A more continuous profile MAY be built from a series of individual
measurements. measurements.
Reporting Format: Reporting Format:
The following configuration parameters MUST be reflected in the The following configuration parameters MUST be reflected in the
skipping to change at page 18, line 14 skipping to change at page 18, line 23
where (E0 ... En) represents the range of all tested egress where (E0 ... En) represents the range of all tested egress
interfaces and Timestamp B represents a tagged frame detection interfaces and Timestamp B represents a tagged frame detection
event for a given DUT/SUT tested egress interface. event for a given DUT/SUT tested egress interface.
A more continuous profile MAY be built from a series of individual A more continuous profile MAY be built from a series of individual
measurements. measurements.
Reporting Format: Reporting Format:
The following configuration parameters MUST be reflected in the The following configuration parameters MUST be reflected in the
results specific to this methodology: test report:
o Frame size(s) o Frame size(s)
o Number of tested egress interfaces on the DUT/SUT o Number of tested egress interfaces on the DUT/SUT
o Test duration o Test duration
o IGMP version o IGMP version
o Offered load o Offered load
o Total number of multicast groups o Total number of multicast groups
The following results MUST be reflected in the results specific to The following results MUST be reflected in the test report:
this methodology:
o The time units of the presented latency MUST be uniform and o The set of all latencies with respective time units related
with sufficient precision for the medium or media being to the tested ingress and each tested egress DUT/SUT
tested. interface.
o Specifically, when reporting the results of a valid test
trial, the set of all latencies related to the tested
ingress and each tested egress DUT/SUT interface of MUST be
presented.
The latency results for each test SHOULD be reported in the form of The time units of the presented latency MUST be uniform and with
a table, with a row for each of the tested frame sizes per the sufficient precision for the medium or media being tested.
recommended frame sizes in section 3.1.3, and SHOULD preserve the
relationship of latency to ingress/egress interface(s) to assist in The results MAY be offered in a tabular format and should preserve
trending across multiple trials. the relationship of latency to ingress/egress interface for each
multicast group to assist in trending across multiple trials.
5.2. Min/Max Multicast Latency 5.2. Min/Max Multicast Latency
Objective: Objective:
To determine the difference between the maximum latency measurement To determine the difference between the maximum latency measurement
and the minimum latency measurement from a collected set of and the minimum latency measurement from a collected set of
latencies produced by the Multicast Latency benchmark. latencies produced by the Multicast Latency benchmark.
Procedure: Procedure:
skipping to change at page 19, line 25 skipping to change at page 19, line 28
identify MAX(M), where MAX is a function that yields the largest identify MAX(M), where MAX is a function that yields the largest
latency value from set M. latency value from set M.
Identify MIN(M), when MIN is a function that yields the smallest Identify MIN(M), when MIN is a function that yields the smallest
latency value from set M. latency value from set M.
The Max/Min value is determined from the following formula: The Max/Min value is determined from the following formula:
Result = MAX(M) - MIN(M) Result = MAX(M) - MIN(M)
A more continuous profile MAY be built from a series of individual
measurements.
Reporting Format: Reporting Format:
The following configuration parameters MUST be reflected in the The following configuration parameters MUST be reflected in the
results specific to this methodology: test report:
o Frame size(s) o Frame size(s)
o Number of tested egress interfaces on the DUT/SUT o Number of tested egress interfaces on the DUT/SUT
o Test duration o Test duration
o IGMP version o IGMP version
o Offered load o Offered load
o Total number of multicast groups o Total number of multicast groups
The following results MUST be reflected in the results specific to The following results MUST be reflected in the test report:
this methodology:
o The result of the min/max value represented as a single o The Max/Min value
numerical value in time units consistent with the o The set of all latencies with respective time units related
corresponding latency measurements. to the tested ingress and each tested egress DUT/SUT
o Specifically, when reporting the results of a valid test interface.
trial, the set of all latencies related to the tested
ingress interface MUST be reported.
The time units of the presented latency MUST be uniform and with The time units of the presented latency MUST be uniform and with
sufficient precision for the medium or media being tested. The sufficient precision for the medium or media being tested.
latency results for each test SHOULD be reported in the form of a
table, with a row for each of the tested frame sizes per the
recommendations in section 3.1.3, and SHOULD preserve the
relationship of latency to ingress/egress interface(s) to assist in
trending across multiple trials.
6. Overhead The results MAY be offered in a tabular format and should preserve
the relationship of latency to ingress/egress interface for each
multicast group.
6. Overhead
This section presents methodology relating to the characterization This section presents methodology relating to the characterization
of the overhead delays associated with explicit operations found in of the overhead delays associated with explicit operations found in
multicast environments. multicast environments.
6.1. Group Join Delay 6.1. Group Join Delay
Objective: Objective:
To determine the time duration it takes a DUT/SUT to start To determine the time duration it takes a DUT/SUT to start
forwarding multicast frames from the time a successful IGMP group forwarding multicast frames from the time a successful IGMP group
membership report has been issued to the DUT/SUT. membership report has been issued to the DUT/SUT.
Procedure: Procedure:
The Multicast Group Join Delay measurement may be influenced by the
state of the Multicast Forwarding Database <MFDB> of the DUT/SUT.
The states of the MFDB may be described as follows:
. State 0, where the MFDB does not contain the specified
multicast group address. In this state, the delay measurement
includes the time the DUT/SUT requires to add the address to
the MFDB and begin forwarding. Delay measured from State 0
provides information about how the DUT/SUT is able to add new
addresses into MFDB.
. State 1, where the MFDB does contain the specified multicast
group address. In this state, the delay measurement includes
the time the DUT/SUT requires to update the MFDB with the
newly joined node<s> and begin forwarding to the new node<s>
plus packet replication time. Delay measured from State 1
provides information about how well the DUT/SUT is able to
update the MFDB for new nodes while transmitting packets to
other nodes for the same IP multicast address. Examples
include adding a new user to an event that is being promoted
via multicast packets.
The methodology for the Multicast Group Join Delay measurement
provides two alternate methods, based on the state of the MFDB, to
measure the delay metric. The methods MAY be used independently or
in conjunction to provide meaningful insight into the DUT/SUT
ability to manage the MFDB.
Users MAY elect to use either method to determine the Multicast
Group Join Delay; however the collection method MUST be specified
as part of the reporting format.
In order to minimize the variation in delay calculations as well as
minimize burden on the DUT/SUT, the test SHOULD be performed with
one multicast group. In addition, all destination test ports MUST
join the specified multicast group offered to the ingress interface
of the DUT/SUT.
Method A:
Method A assumes that the Multicast Forwarding Database <MFDB> of
the DUT/SUT does not contain or has not learned the specified
multicast group address. In this scenario, the metric represents
the time the DUT/SUT takes to add the multicast address to the MFDB
and begin forwarding the multicast packet. Only one ingress and
one egress MUST be used to determine this metric.
Prior to sending any IGMP Group Membership Reports used to Prior to sending any IGMP Group Membership Reports used to
calculate the group join delay, it MUST be verified through calculate the Multicast Group Join Delay, it MUST be verified
externally observable means that the destination test ports are not through externally observable means that the destination test port
currently a member of any of the specified multicast groups. If is not currently a member of the specified multicast group. In
any of the egress interfaces forward multicast frames, the test is addition, it MUST be verified through externally observable means
not valid. that the MFDB of the DUT/SUT does not contain the specified
multicast address.
Once verification is complete, multicast traffic for all relevant Method B:
multicast group addresses MUST be offered to the ingress interface
prior to receipt or processing of any IGMP Group Membership Report Method B assumes that the MFDB of the DUT/SUT does contain the
specified multicast group address. In this scenario, the metric
represents the time the DUT/SUT takes to update the MFDB with the
additional nodes and their corresponding interfaces and to begin
forwarding the multicast packet. One or more egress ports MAY be
used to determine this metric.
Prior to sending any IGMP Group Membership Reports used to
calculate the Group Join Delay, it MUST be verified through
externally observable means that the MFDB contains the specified
multicast group address. A single un-instrumented test port MUST
be used to join the specified multicast group address prior to
sending any test traffic. This port will be used only for insuring
that the MFDB has been populated with the specified multicast group
address and can successfully forward traffic to the un-instrumented
port.
Join Delay Calculation
Once verification is complete, multicast traffic for the specified
multicast group address MUST be offered to the ingress interface
prior to the DUT/SUT receiving any IGMP Group Membership Report
messages. It is RECOMMENDED to offer traffic at the measured messages. It is RECOMMENDED to offer traffic at the measured
aggregated multicast throughput rate (Section 4.3). aggregated multicast throughput rate (Section 4.3).
After the multicast traffic has been started, each destination test After the multicast traffic has been started, the destination test
port (See Figure 1) SHOULD send one IGMP Group Membership Report port (See Figure 1) MUST send one IGMP Group Membership Report for
with one or more (IGMP V3) multicast group(s) specified. All the specified multicast group.
destination test ports MUST join all multicast groups offered on
the ingress interface of the DUT/SUT. The test MUST be performed
with one multicast group and SHOULD be performed with multiple
groups.
The join delay is the difference in time from when the IGMP Group The join delay is the difference in time from when the IGMP Group
Membership message is sent (timestamp A) and the first frame of the Membership message is sent (timestamp A) and the first frame of the
multicast group is forwarded to a receiving egress interface multicast group is forwarded to a receiving egress interface
(timestamp B). (timestamp B).
Group Join delay time = timestamp B - timestamp A Group Join delay time = timestamp B - timestamp A
Timestamp A MUST be the time the last bit of the IGMP group Timestamp A MUST be the time the last bit of the IGMP group
membership report is sent from the destination test port; timestamp membership report is sent from the destination test port; timestamp
B MUST be the time the first bit of the first valid multicast frame B MUST be the time the first bit of the first valid multicast frame
is forwarded on the egress interface of the DUT/SUT. is forwarded on the egress interface of the DUT/SUT.
Reporting Format: Reporting Format:
The following configuration parameters MUST be reflected in the The following configuration parameters MUST be reflected in the
results specific to this methodology: test report:
o Frame size(s) o Frame size(s)
o Number of tested egress interfaces on the DUT/SUT o Number of tested egress interfaces on the DUT/SUT
o Test duration
o IGMP version o IGMP version
o Total number of multicast groups o Total number of multicast groups
o Offered load to ingress interface
o Method used to measure the join delay metric
The following results MUST be reflected in the results specific to The following results MUST be reflected in the test report:
this methodology:
o The group join delay time per multicast group address o The group join delay time in microseconds per egress
o The group join delay time per egress interface(s) interface(s)
The Group Join Delay results for each test SHOULD be reported in The Group Join Delay results for each test MAY be reported in the
the form of a table, with a row for each of the tested frame sizes form of a table, with a row for each of the tested frame sizes per
per the recommendations in section 3.1.3. Each row or iteration the recommendations in section 3.1.3. Each row or iteration MAY
SHOULD specify the group join delay time for each multicast group specify the group join delay time per egress interface for that
per destination interface, number of frames transmitted and number iteration.
of frames received for that iteration.
6.2. Group Leave Delay 6.2. Group Leave Delay
Objective: Objective:
To determine the time duration it takes a DUT/SUT to cease To determine the time duration it takes a DUT/SUT to cease
forwarding multicast frames after a corresponding IGMP Leave Group forwarding multicast frames after a corresponding IGMP Leave Group
message has been successfully offered to the DUT/SUT. message has been successfully offered to the DUT/SUT.
Procedure: Procedure:
Prior to sending any IGMP Group Leave Group messages used to In order to minimize the variation in delay calculations as well as
calculate the group leave delay, it MUST be verified through minimize burden on the DUT/SUT, the test SHOULD be performed with
externally observable means that the destination test ports are one multicast group. In addition, all destination test ports MUST
currently a member of all the specified multicast groups. If any join the specified multicast group offered to the ingress interface
of the destination test ports do not receive multicast frames, the of the DUT/SUT.
test is not valid.
Once verification is complete, multicast traffic for all relevant Prior to sending any IGMP Leave Group messages used to calculate
multicast group addresses MUST be offered to the ingress interface the group leave delay, it MUST be verified through externally
observable means that the destination test ports are currently
members of the specified multicast group. If any of the egress
interfaces do not forward validation multicast frames then the test
is invalid.
Once verification is complete, multicast traffic for the specified
multicast group address MUST be offered to the ingress interface
prior to receipt or processing of any IGMP Leave Group messages. prior to receipt or processing of any IGMP Leave Group messages.
It is RECOMMENDED to offer traffic at the measured aggregated It is RECOMMENDED to offer traffic at the measured aggregated
multicast throughput rate (Section 4.3). multicast throughput rate (Section 4.3).
After the multicast traffic has been started, each destination test After the multicast traffic has been started, each destination test
port (See Figure 1) MUST send one IGMP Leave Group message for each port (See Figure 1) MUST send one IGMP Leave Group message for the
multicast group specified. All destination test ports MUST leave specified multicast group.
all relevant multicast groups offered on the ingress interface of
the DUT/SUT. The test MUST be performed with one multicast group
and SHOULD be performed with multiple groups.
The leave delay is the difference in time from when the IGMP Leave The leave delay is the difference in time from when the IGMP Leave
Group message is sent (timestamp A) and the last frame of the Group message is sent (timestamp A) and the last frame of the
multicast group is forwarded to a receiving egress interface multicast group is forwarded to a receiving egress interface
(timestamp B). (timestamp B).
Group Leave delay time = timestamp B - timestamp A Group Leave delay time = timestamp B - timestamp A
Timestamp A MUST be the time the last bit of the IGMP Leave Group Timestamp A MUST be the time the last bit of the IGMP Leave Group
message is sent from the destination test port; timestamp B MUST be message is sent from the destination test port; timestamp B MUST be
the time the last bit of the last valid multicast frame is the time the last bit of the last valid multicast frame is
forwarded on the egress interface of the DUT/SUT. forwarded on the egress interface of the DUT/SUT.
Reporting Format: Reporting Format:
The following configuration parameters MUST be reflected in the The following configuration parameters MUST be reflected in the
results specific to this methodology: test report:
o Frame size(s) o Frame size(s)
o Number of tested egress interfaces on the DUT/SUT o Number of tested egress interfaces on the DUT/SUT
o Test duration
o IGMP version o IGMP version
o Total number of multicast groups o Total number of multicast groups
o Offered load to ingress interface
The following results MUST be reflected in the results specific to The following results MUST be reflected in the test report:
this methodology:
o The group leave delay time per multicast group address
o The group leave delay time per egress interface(s)
The Group Leave Delay results for each test SHOULD be reported in o The group leave delay time in microseconds per egress
the form of a table, with a row for each of the tested frame sizes interface(s)
per the recommendations in section 3.1.3. Each row or iteration The Group Leave Delay results for each test MAY be reported in the
SHOULD specify the group leave delay time for each multicast group form of a table, with a row for each of the tested frame sizes per
per destination interface, number of frames transmitted and number the recommendations in section 3.1.3. Each row or iteration MAY
of frames received for that iteration. specify the group leave delay time per egress interface for that
iteration.
7. Capacity 7. Capacity
This section offers terms relating to the identification of This section offers a procedure relating to the identification of
multicast group limits of a DUT/SUT. multicast group limits of a DUT/SUT.
7.1. Multicast Group Capacity 7.1. Multicast Group Capacity
Objective: Objective:
To determine the maximum number of multicast groups a DUT/SUT can To determine the maximum number of multicast groups a DUT/SUT can
support while maintaining the ability to forward multicast frames support while maintaining the ability to forward multicast frames
to all multicast groups registered to that DUT/SUT. to all multicast groups registered to that DUT/SUT.
Procedure: Procedure:
One or more egress interfaces of DUT/SUT will join an initial One or more destination test ports of DUT/SUT will join an initial
number of multicast groups. number of multicast groups.
After a delay as determined by section 6.1, the ingress interface After a minimum delay as measured by section 6.1, the source test
MUST transmit to each group at a specified offered load. ports MUST transmit to each group at a specified offered load.
If at least one frame for each multicast group is forwarded If at least one frame for each multicast group is forwarded
properly by the DUT/SUT to each participating egress interface, the properly by the DUT/SUT on each participating egress interface, the
iteration is said to pass at the current capacity. iteration is said to pass at the current capacity.
If the iteration passes, the test will add a user defined For each successful iteration, each destination test port will join
incremental value of groups to each egress interface. At the new an additional user-defined number of multicast groups and the test
group level and resultant capacity as stated above, run the repeats. The test stops iterating when one or more of the egress
iteration again. interfaces fails to forward traffic on one or more of the
configured multicast groups.
Once the test fails, the last/previous iteration capacity that Once the iteration fails, the last successful iteration is the
passed is the stated Maximum Group Capacity result. stated Maximum Group Capacity result.
Reporting Format: Reporting Format:
The following configuration parameters MUST be reflected in the The following configuration parameters MUST be reflected in the
results specific to this methodology: test report:
o Frame size(s) o Frame size(s)
o Number of tested egress interfaces on the DUT/SUT o Number of tested egress interfaces on the DUT/SUT
o Test duration
o IGMP version o IGMP version
o Offered load o Offered load
The following results MUST be reflected in the results specific to The following results MUST be reflected in the test report:
this methodology:
o The total number of multicast group addresses that were o The total number of multicast group addresses that were
successfully forwarded through the DUT/SUT successfully forwarded through the DUT/SUT
The Multicast Group Capacity results for each test SHOULD be The Multicast Group Capacity results for each test SHOULD be
reported in the form of a table, with a row for each of the tested reported in the form of a table, with a row for each of the tested
frame sizes per the recommendations in section 3.1.3. Each row or frame sizes per the recommendations in section 3.1.3. Each row or
iteration SHOULD specify the number of multicast groups joined per iteration SHOULD specify the number of multicast groups joined per
destination interface, number of frames transmitted and number of destination interface, number of frames transmitted and number of
frames received for that iteration. frames received for that iteration.
8. Interaction 8. Interaction
Network forwarding devices are generally required to provide more Network forwarding devices are generally required to provide more
skipping to change at page 24, line 27 skipping to change at page 25, line 43
assist in the characterization of DUT/SUT behavior in consideration assist in the characterization of DUT/SUT behavior in consideration
of potentially interacting factors. of potentially interacting factors.
8.1. Forwarding Burdened Multicast Latency 8.1. Forwarding Burdened Multicast Latency
Objective: Objective:
To produce a set of multicast latency measurements from a single To produce a set of multicast latency measurements from a single
multicast ingress interface of a DUT/SUT through multiple egress multicast ingress interface of a DUT/SUT through multiple egress
multicast interfaces of that same DUT/SUT as provided for by the multicast interfaces of that same DUT/SUT as provided for by the
metric "Multicast Latency" in RFC 2432 while under the influence of metric "Multicast Latency" in RFC 2432 [Du96] while forwarding
a traffic forwarding requirement. meshed unicast traffic.
Procedure: Procedure:
The Multicast Latency metrics can be influenced by forcing the The Multicast Latency metrics can be influenced by forcing the
DUT/SUT to perform extra processing of packets while multicast DUT/SUT to perform extra processing of packets while multicast
class traffic is being forwarded for latency measurements. class traffic is being forwarded for latency measurements.
The Burdened Forwarding Latency test MUST follow the described The Burdened Forwarding Multicast Latency test MUST follow the
setup in Section 5.1. described setup for the Multicast Latency test in Section 5.1. In
addition, another set of test ports MUST be used to burden the
Perform a baseline measurement of latency as described in Section DUT/SUT (burdening ports). The burdening ports will be used to
5.1. After the baseline measurement is obtained, the test is transmit unicast class traffic to the DUT/SUT in a fully meshed
repeated with the ingress interface offering an additional set of traffic distribution as described in RFC 2285 [Ma98]. The DUT/SUT
user specified multicast group addresses which have not been joined MUST learn the appropriate unicast addresses and verified through
by the destination test port(s). The offered load MUST be the same some externally observable method.
as was used in the baseline measurement.
By sending such multicast class traffic, the DUT/SUT may perform a Perform a baseline measurement of Multicast Latency as described in
lookup on the frames that may affect the processing of traffic Section 5.1. After the baseline measurement is obtained, start
destined for the egress interface(s). transmitting the unicast class traffic at a user-specified offered
load on the set of burdening ports and rerun the Multicast Latency
test. The offered load to the ingress port MUST be the same as was
used in the baseline measurement.
Reporting Format: Reporting Format:
Similar to Section 5.1, the following configuration parameters MUST Similar to Section 5.1, the following configuration parameters MUST
be reflected in the results specific to this methodology: be reflected in the test report:
o Frame size(s) o Frame size(s)
o Number of tested egress interfaces on the DUT/SUT o Number of tested egress interfaces on the DUT/SUT
o Test duration o Test duration
o IGMP version o IGMP version
o Total number of multicast groups in the baseline setup o Offered load to ingress interface
o Total number of additional multicast groups used to burden o Total number of multicast groups
the setup o Offered load to burdening ports
o Total number of burdening ports
The following results MUST be reflected in the results specific to The following results MUST be reflected in the test report:
this methodology:
o The time units of the presented latency MUST be uniform and o The set of all latencies related to the tested ingress and
with sufficient precision for the medium or media being each tested egress DUT/SUT interface for both the baseline
tested and burdened response.
o Specifically, when reporting the results of a valid test
trial, the set of all latencies related to the tested The time units of the presented latency MUST be uniform and with
ingress and each tested egress DUT/SUT interface of MUST be sufficient precision for the medium or media being tested.
presented
o Reported results from baseline measurement; section 5.1
The latency results for each test SHOULD be reported in the form of The latency results for each test SHOULD be reported in the form of
a table, with a row for each of the tested frame sizes per the a table, with a row for each of the tested frame sizes per the
recommended frame sizes in section 3.1.3, and SHOULD preserve the recommended frame sizes in section 3.1.3, and SHOULD preserve the
relationship of latency to ingress/egress interface(s) to assist in relationship of latency to ingress/egress interface(s) to assist in
trending across multiple trials. trending across multiple trials.
8.2. Forwarding Burdened Group Join Delay 8.2. Forwarding Burdened Group Join Delay
Objective: Objective:
To determine the time duration it takes a DUT/SUT to start To determine the time duration it takes a DUT/SUT to start
forwarding multicast frames from the time a successful IGMP Group forwarding multicast frames from the time a successful IGMP Group
Membership Report has been issued to the DUT/SUT while under the Membership Report has been issued to the DUT/SUT while while
influence of a traffic forwarding requirement. forwarding meshed unicast traffic.
Procedure: Procedure:
The Group Join Delay metrics can be influenced by forcing the
DUT/SUT to perform extra process of packets while attempting to
update and maintain the IP multicast address forwarding table.
The Forwarding Burdened Group Join Delay test MUST follow the The Forwarding Burdened Group Join Delay test MUST follow the
described setup in Section 6.1. described setup for the Group Join Delay test in Section 6.1. In
addition, another set of test ports MUST be used to burden the
Perform a baseline measurement of group join delay as described in DUT/SUT (burdening ports). The burdening ports will be used to
Section 6.1. After the baseline measurement is obtained, the test transmit unicast class traffic to the DUT/SUT in a fully meshed
is repeated with the ingress interface offering an additional set traffic pattern as described in RFC 2285 [Ma98]. The DUT/SUT MUST
of user specified multicast group address which have not been learn the appropriate unicast addresses and verified through some
joined by the destination test port(s). The offered load MUST be externally observable method.
the same as was used in the baseline measurement.
By sending such multicast class traffic, the DUT/SUT may perform a Perform a baseline measurement of Group Join Delay as described in
lookup on the frames that may affect the processing of the IGMP Section 6.1. After the baseline measurement is obtained, start
Group Report messages. transmitting the unicast class traffic at a user-specified offered
load on the set of burdening ports and rerun the Group Join Delay
test. The offered load to the ingress port MUST be the same as was
used in the baseline measurement.
Reporting Format: Reporting Format:
Similar to Section 6.1, the following configuration parameters MUST Similar to Section 6.1, the following configuration parameters MUST
be reflected in the results specific to this methodology: be reflected in the test report:
o Frame size(s) o Frame size(s)
o Number of tested egress interfaces on the DUT/SUT o Number of tested egress interfaces on the DUT/SUT
o Test duration
o IGMP version o IGMP version
o Total number of multicast groups in the baseline setup o Offered load to ingress interface
o Total number of additional multicast groups used to burden o Total number of multicast groups
the setup o Offered load to burdening ports
o Total number of burdening ports
The following results MUST be reflected in the results specific to The following results MUST be reflected in the test report:
this methodology:
o The group join delay time per multicast group address o The group join delay time in microseconds per egress
o The group join delay time per egress interface(s) interface(s) for both the baseline and burdened response.
o Reported results from baseline measurement; section 6.1
The Group Join Delay results for each test SHOULD be reported in The Group Join Delay results for each test MAY be reported in the
the form of a table, with a row for each of the tested frame sizes form of a table, with a row for each of the tested frame sizes per
per the recommendations in section 3.1.3. Each row or iteration the recommendations in section 3.1.3. Each row or iteration MAY
SHOULD specify the group join delay time for each multicast group specify the group join delay time per egress interface, number of
per destination interface, number of frames transmitted and number frames transmitted and number of frames received for that
of frames received for that iteration. iteration.
9. Security Considerations 9. Security Considerations
As this document is solely for the purpose of providing metric As this document is solely for the purpose of providing metric
methodology and describes neither a protocol nor a protocol's methodology and describes neither a protocol nor a protocol's
implementation, there are no security considerations associated implementation, there are no security considerations associated
with this document. with this document.
10. Acknowledgements 10. Acknowledgements
 End of changes. 

This html diff was produced by rfcdiff 1.23, available from http://www.levkowetz.com/ietf/tools/rfcdiff/