draft-ietf-bmwg-mcastm-02.txt   draft-ietf-bmwg-mcastm-03.txt 
Network Working Group Hardev Soor Network Working Group Hardev Soor
INTERNET-DRAFT Debra Stopp INTERNET-DRAFT Debra Stopp
Expires in: March 2000 Ixia Communications Expires in: August 2000 Ixia Communications
Ralph Daniels Ralph Daniels
Netcom Systems Netcom Systems
October 1999 March 2000
Methodology for IP Multicast Benchmarking Methodology for IP Multicast Benchmarking
<draft-ietf-bmwg-mcastm-02.txt> <draft-ietf-bmwg-mcastm-03.txt>
Status of this Memo Status of this Memo
This document is an Internet-Draft and is in full conformance with This document is an Internet-Draft and is in full conformance with
all provisions of Section 10 of RFC2026. all provisions of Section 10 of RFC2026.
Internet-Drafts are working documents of the Internet Engineering Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF), its areas, and its working groups. Note that Task Force (IETF), its areas, and its working groups. Note that
other groups may also distribute working documents as Internet- other groups may also distribute working documents as Internet-
Drafts. Drafts.
skipping to change at page 2, line 5 skipping to change at page 2, line 5
tenets set forth in RFC 2544, RFC 2432 and other IETF Benchmarking tenets set forth in RFC 2544, RFC 2432 and other IETF Benchmarking
Methodology Working Group (BMWG) efforts. This document seeks to Methodology Working Group (BMWG) efforts. This document seeks to
extend these efforts to the multicast paradigm. extend these efforts to the multicast paradigm.
The BMWG produces two major classes of documents: Benchmarking The BMWG produces two major classes of documents: Benchmarking
Terminology documents and Benchmarking Methodology documents. The Terminology documents and Benchmarking Methodology documents. The
Terminology documents present the benchmarks and other related terms. Terminology documents present the benchmarks and other related terms.
The Methodology documents define the procedures required to collect The Methodology documents define the procedures required to collect
the benchmarks cited in the corresponding Terminology documents. the benchmarks cited in the corresponding Terminology documents.
1. Introduction 1 Introduction
This document defines a specific set of tests that vendors can use to This document defines a specific set of tests that vendors can use to
measure and report the performance characteristics and forwarding measure and report the performance characteristics and forwarding
capabilities of network devices that support IP multicast protocols. capabilities of network devices that support IP multicast protocols.
The results of these tests will provide the user comparable data from The results of these tests will provide the user comparable data from
different vendors with which to evaluate these devices. different vendors with which to evaluate these devices.
A previous document, " Terminology for IP Multicast Benchmarking" A previous document, " Terminology for IP Multicast Benchmarking"
(RFC 2432), defined many of the terms that are used in this document. (RFC 2432), defined many of the terms that are used in this document.
The terminology document should be consulted before attempting to The terminology document should be consulted before attempting to
make use of this document. make use of this document.
This methodology will focus on one source to many destinations, This methodology will focus on one source to many destinations,
although many of the tests described may be extended to use multiple although many of the tests described may be extended to use multiple
source to multiple destination IP multicast communication. source to multiple destination IP multicast communication.
2. Key Words to Reflect Requirements 2 Key Words to Reflect Requirements
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
"SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
document are to be interpreted as described in RFC 2119. document are to be interpreted as described in RFC 2119.
3. Test set up 3 Test set up
Figure 1 shows a typical setup for an IP multicast test, with one Figure 1 shows a typical setup for an IP multicast test, with one
source to multiple destinations, although this MAY be extended to source to multiple destinations, although this MAY be extended to
multiple source to multiple destinations. multiple source to multiple destinations.
+----------------+ +----------------+
+------------+ | | +------------+ | |
+--------+ | |--------->| destination(1) | +--------+ | |--------->| destination(1) |
| | | | | | | | | | | |
| source |-------->| | +----------------+ | source |-------->| | +----------------+
skipping to change at page 2, line 52 skipping to change at page 2, line 52
| | | | | | | |
| | +----------------+ | | +----------------+
| | . . . | | . . .
| | +----------------+ | | +----------------+
| | | | | | | |
| |--------->| destination(n) | | |--------->| destination(n) |
| | | | | | | |
| | +----------------+ | | +----------------+
| | | |
+------------+ +------------+
Figure 1 Figure 1
Generally , the destination ports first join the desired number of Generally , the destination ports first join the desired number of
multicast groups by sending IGMP Join Group messages to the DUT/SUT. multicast groups by sending IGMP Join Group messages to the DUT/SUT. To
To verify that all destination ports successfully joined the verify that all destination ports successfully joined the appropriate
appropriate groups, the source port MUST transmit IP multicast groups, the source port MUST transmit IP multicast frames destined for
frames destined for these groups. The destination ports MAY send these groups. The destination ports MAY send IGMP Leave Group messages
IGMP Leave Group messages after the transmission of IP Multicast after the transmission of IP Multicast frames to clear the IGMP table of
frames to clear the IGMP table of the DUT/SUT. the DUT/SUT.
In addition, all transmitted frames MUST contain a recognizable In addition, all transmitted frames MUST contain a recognizable pattern
pattern that can be filtered on in order to ensure the receipt of only that can be filtered on in order to ensure the receipt of only the
the frames that are involved in the test. frames that are involved in the test.
3.1 Test Considerations 3.1 Test Considerations
3.1.1 IGMP Support 3.2 IGMP Support
Each of the receiving ports of the tester should support and be Each of the receiving ports of the tester should support and be able
able to test all IGMP versions 1, 2 and 3. The minimum requirement, to test all IGMP versions 1, 2 and 3. The minimum requirement,
however, is IGMP version 2. however, is IGMP version 2.
Each receiving port should be able to respond to IGMP queries during the Each receiving port should be able to respond to IGMP queries during
test. the test.
Each receiving port should also send LEAVE (running IGMP version 2) Each receiving port should also send LEAVE (running IGMP version 2)
after each test. after each test.
3.1.2 Group Addresses 3.3 Group Addresses
The Class D Group address should be changed between tests. Many The Class D Group address SHOULD be changed between tests. Many DUTs
DUTs have memory or cache that is not cleared properly and can have memory or cache that is not cleared properly and can bias the
bias the results. results.
The following group addresses are recommended by use in a test: The following group addresses are recommended by use in a test:
224.0.1.27-224.0.1.255 224.0.1.27-224.0.1.255
224.0.5.128-224.0.5.255 224.0.5.128-224.0.5.255
224.0.6.128-224.0.6.255 224.0.6.128-224.0.6.255
If the number of group addresses accomodated by these ranges do not If the number of group addresses accommodated by these ranges do not
satisfy the requrirements of the test, then these ranges may be satisfy the requirements of the test, then these ranges may be
overlapped. overlapped. The total number of configured group addresses must be
less than or equal to the IGMP table size of the DUT/SUT.
3.1.3 Frame Sizes 3.4 Frame Sizes
Each test should be run with different Multicast Frame Sizes. The Each test should be run with different Multicast Frame Sizes. The
recommended frame sizes are 64, 128, 256, 512, 1024, 1280, and 1518 recommended frame sizes are 64, 128, 256, 512, 1024, 1280, and 1518
byte frames. byte frames.
3.1.4 TTL 3.5 TTL
The source frames should have a TTL value large enough to accommodate The source frames should have a TTL value large enough to accommodate
the DUT/SUT. the DUT/SUT.
3.1.5 Layer 2 Support 3.6 Layer 2 Support
Each of the receiving ports of the tester should support GARP/GMRP Each of the receiving ports of the tester should support GARP/GMRP
protocols to join groups on Layer 2 DUTs/SUTs. protocols to join groups on Layer 2 DUTs/SUTs.
4. Forwarding and Throughput 4 Forwarding and Throughput
This section contains the description of the tests that are related to This section contains the description of the tests that are related to
the characterization of the packet forwarding of a DUT/SUT in a the characterization of the packet forwarding of a DUT/SUT in a
multicast environment. Some metrics extend the concept of throughput multicast environment. Some metrics extend the concept of throughput
presented in RFC 1242. The notion of Forwarding Rate is cited in RFC presented in RFC 1242. The notion of Forwarding Rate is cited in RFC
2285. 2285.
4.1 Mixed Class Throughput 4.1 Mixed Class Throughput
Definition Definition
skipping to change at page 4, line 23 skipping to change at page 4, line 23
This section contains the description of the tests that are related to This section contains the description of the tests that are related to
the characterization of the packet forwarding of a DUT/SUT in a the characterization of the packet forwarding of a DUT/SUT in a
multicast environment. Some metrics extend the concept of throughput multicast environment. Some metrics extend the concept of throughput
presented in RFC 1242. The notion of Forwarding Rate is cited in RFC presented in RFC 1242. The notion of Forwarding Rate is cited in RFC
2285. 2285.
4.1 Mixed Class Throughput 4.1 Mixed Class Throughput
Definition Definition
The maximum rate at which none of the offered frames, comprised from The maximum rate at which none of the offered frames, comprised from
a unicast Class and a multicast Class, to be forwarded are dropped a unicast Class and a multicast Class, to be forwarded are dropped by
by the device across a fixed number of ports. the device across a fixed number of ports.
Procedure Procedure
Multicast and unicast traffic are mixed together in the same Multicast and unicast traffic are mixed together in the same
aggregated traffic stream in order to simulate the non-homogenous aggregated traffic stream in order to simulate the non-homogenous
networking environment. While the multicast traffic is transmitted networking environment. While the multicast traffic is transmitted
from one source to multiple destinations, the unicast traffic MAY be from one source to multiple destinations, the unicast traffic MAY be
evenly distributed across the DUT/SUT architecture. In addition, the evenly distributed across the DUT/SUT architecture. In addition, the
DUT/SUT SHOULD learn the appropriate unicast IP addresses, either by DUT/SUT SHOULD learn the appropriate unicast IP addresses, either by
sending ARP frames from each unicast address, sending a RIP packet sending ARP frames from each unicast address, sending a RIP packet or
or by assigning static entries into the DUT/SUT address table. by assigning static entries into the DUT/SUT address table.
The rates at which traffic is transmitted for both traffic classes The mixture of multicast and unicast traffic MUST be set up in one of
MUST be set up in one of two ways: two ways:
a) A percentage of the bandwidth is allocated for each traffic class a) As a percent of the total bandwidth resulting in a ratio. For
and frames for each class are transmitted at the rate equal to example, 20 percent multicast traffic to 80 percent unicast
the allocated bandwidth. For example, 64 byte frames can be traffic.
transmitted at a theoretical maximum rate of 148810 frames/second.
If 80 percent of the bandwidth is allocated for unicast traffic
and 20 percent for multicast traffic, then unicast traffic will
be sent at a maximum rate of 119048 frames/second and the
multicast traffic at a rate of 29762 frames/second.
b) Transmission rate is fixed for both traffic classes and a percentage of b) In evenly distributed bursts of multicast and unicast
number of frames for each traffic class is specified. For example, if a traffic, resulting in a 50-50 ratio of multicast to unicast
fixed rate of 100% of theoretical maximum is desired, then 64 byte traffic.
frames will be sent at 148810 frames/second for both unicast and
multicast traffic. If 80 percent of the frames are to be unicast and
20 percent multicast, then for a duration of 30 seconds, 3571440
frames of unicast and 892860 frames of multicast will be sent. This
fixed rate scenario actually over-subscribes the bandwidth,
potentially causing congestion in the DUT/SUT.
The transmission of the frames MUST be set up so that they form a The transmission of the frames MUST be set up so that they form a
deterministic distribution while still maintaining the specified bandwidth deterministic distribution while still maintaining the specified
and transmission rates. See Appendix A for a discussion on determining an bandwidth and transmission rates. See Appendix A for a discussion on
even distribution. determining an even distribution.
Similar to the Frame loss rate test in RFC 2544, the first trial SHOULD be Similar to the Frame loss rate test in RFC 2544, the first trial
run for the frame rate that corresponds to 100% of the maximum rate for SHOULD be run for the frame rate that corresponds to 100% of the
the frame size on the input media. Repeat the procedure for the rate that maximum rate for the frame size on the input media. Repeat the
corresponds to 90% of the maximum rate used and then for 80% of this rate. procedure for the rate that corresponds to 90% of the maximum rate
This sequence SHOULD be continued (at reducing 10% intervals) until there used and then for 80% of this rate. This sequence SHOULD be continued
are two successive trials in which no frames are lost. The maximum (at reducing 10% intervals) until there are two successive trials in
granularity of the trials MUST be 10% of the maximum rate, a finer which no frames are lost. The maximum granularity of the trials MUST
granularity is encouraged. be 10% of the maximum rate, a finer granularity is encouraged.
Result Result
Transmit and Receive rates in frames per second for each source and Parameters to be measured SHOULD include the frame loss and percent
destination port for both unicast and multicast traffic for each trial loss for each class of traffic per destination port. The ratio of
percent transmit rate. The ratio of the Unicast traffic versus Multicast unicast traffic to multicast traffic MUST be reported.
traffic SHOULD be reported. The result report SHOULD contain the number of
frames transmitted and received per port per class type (unicast and In addition, the transmit and receive rates in frames per second for
multicast traffic), reported in number of frames and percent loss per each source and destination port for both unicast and multicast
port. traffic, together with the number of frames transmitted and received
per port per class type traffic SHOULD be reported.
4.2 Scaled Group Forwarding Matrix 4.2 Scaled Group Forwarding Matrix
Definition: Definition
A table that demonstrates Forwarding Rate as a function of tested A table that demonstrates Forwarding Rate as a function of tested
multicast groups for a fixed number of tested DUT/SUT ports. multicast groups for a fixed number of tested DUT/SUT ports.
Procedure: Procedure
Multicast traffic is sent at a fixed percent of line rate with a fixed Multicast traffic is sent at a fixed percent of line rate with a
number of receive ports of the tester at a fixed frame length. fixed number of receive ports of the tester at a fixed frame length.
The receive ports will join an initial number of groups and the sender The receive ports SHOULD continue joining incrementally by 10
will transmit to the same groups after a certain delay (a few seconds). multicast groups until a user defined maximum is reached.
Then the receive ports will join an incremental value of groups and the The receive ports will continue joining in the incremental fashion
transmit port will send to all groups joined (initial plus incremental). until a user defined maximum is reached.
The receive ports will continue joining in the incremental fashion until a Results
user defined maximum is reached.
Results: Parameters to be measured SHOULD include the frame loss and percent
loss per destination port for each multicast group address.
For each group load the result WILL display frame rate, frames In addition, the transmit and receive rates in frames per second for
transmitted, total frames received, total frames loss, and percent each source and destination port for all multicast groups, together
loss. The frame loss per receive port of the tester per group SHOULD also with the number of frames transmitted and received per port per
be available. multicast groups SHOULD be reported.
4.3 Aggregated Multicast Throughput 4.3 Aggregated Multicast Throughput
Definition: Definition
The maximum rate at which none of the offered frames to be forwarded
through N destination interfaces of the same multicast group are
dropped.
The maximum rate at which none of the offered frames to be Procedure
forwarded through N destination interfaces of the same multicast
group are dropped.
Procedure: Multicast traffic is sent at a fixed percent of line rate with a
fixed number of groups at a fixed frame length for a fixed duration
of time.
Multicast traffic is sent at a fixed percent of line rate with a fixed The initial number of receive ports of the tester will join the
number of groups at a fixed frame length for a fixed duration of time. group(s) and the sender will transmit to the same groups after a
certain delay (a few seconds).
The initial number of receive ports of the tester will join the group(s) Then the an incremental number of receive ports will join the same
and the sender will transmit to the same groups after a certain delay groups and then the Multicast traffic is sent as stated.
(a few seconds).
Then the an incremental or decremental number of receive ports will The receive ports will continue to be added and multicast traffic
join the same groups and then the Multicast traffic is sent as stated. sent until a user defined maximum number of ports is reached.
The receive ports will continue to be added or deleted and the Multicast Results
traffic sent until a user defined maximum number of ports is reached.
Results: Parameters to be measured SHOULD include the frame loss and percent
loss per destination port for each multicast group address.
For each number of receive ports the result WILL display frame rate, In addition, the transmit and receive rates in frames per second for
frames transmitted, total frames received, total frames loss, and percent each source and destination port for all multicast groups, together
loss. The frame loss per receive port per group SHOULD also be available. with the number of frames transmitted and received per port per
multicast groups SHOULD be reported.
4.4 Encapsulation (Tunneling) Throughput 4.4 Encapsulation (Tunneling) Throughput
This sub-section provides the description of tests that help in obtaining This sub-section provides the description of tests that help in
throughput measurements when a DUT/SUT or a set of DUTs are acting as tunnel obtaining throughput measurements when a DUT/SUT or a set of DUTs are
endpoints. The following Figure 2 presents the scenario for the tests. acting as tunnel endpoints. The following Figure 2 presents the
scenario for the tests.
Client A DUT/SUT A Network DUT/SUT B Client B Client A DUT/SUT A Network DUT/SUT B Client B
---------- ---------- ---------- ----------
| | ------ | | | | ------ | |
-------(a) (b)| |(c) ( ) (d)| |(e) (f)------- -----(a) (b)| |(c) ( ) (d)| |(e) (f)-----
||||||| -----> | |---->( )----->| |-----> ||||||| ||||| -----> | |---->( )----->| |-----> |||||
------- | | ------ | | ------- ----- | | ------ | | -----
| | | | | | | |
---------- ---------- ---------- ----------
Figure 2 Figure 2
--------
A tunnel is created between DUT/SUT A (the encapsulator) and DUT/SUT B (the --------
decapsulator). Client A is acting as a source and Client B is the A tunnel is created between DUT/SUT A (the encapsulator) and DUT/SUT
destination. Client B joins a multicast group (for example, 224.0.1.1) and it B (the decapsulator). Client A is acting as a source and Client B is
sends an IGMP Join message to DUT/SUT B to join that group. Client A now the destination. Client B joins a multicast group (for example,
wants to transmit some traffic to Client B. It will send the multicast 224.0.1.1) and it sends an IGMP Join message to DUT/SUT B to join
traffic to DUT/SUT A which encapsulates the multicast frames, sends it to that group. Client A now wants to transmit some traffic to Client B.
DUT/SUT B which will decapsulate the same frames and forward them to Client It will send the multicast traffic to DUT/SUT A which encapsulates
B. the multicast frames, sends it to DUT/SUT B which will decapsulate
the same frames and forward them to Client B.
4.4.1 Encapsulation Throughput 4.4.1 Encapsulation Throughput
Definition Definition
The maximum rate at which frames offered a DUT/SUT are encapsulated and The maximum rate at which frames offered a DUT/SUT are
correctly forwarded by the DUT/SUT without loss. encapsulated and correctly forwarded by the DUT/SUT without loss.
Procedure Procedure
To test the forwarding rate of the DUT/SUT when it has to go through the To test the forwarding rate of the DUT/SUT when it has to go
process of encapsulation, a test port B is injected at the other end of through the process of encapsulation, a test port B is injected at
DUT/SUT A (Figure B) that will receive the encapsulated frames and measure the other end of DUT/SUT A (Figure B) that will receive the
the throughput. Also, a test port A is used to generate multicast frames that encapsulated frames and measure the throughput. Also, a test port
will be passed through the tunnel. A is used to generate multicast frames that will be passed through
the tunnel.
The following is the test setup: The following is the test setup:
Test port A DUT/SUT A Test port B Test port A DUT/SUT A Test port B
---------- (c') (d')--------- ---------- (c') (d')---------
| |-------------->| | | |-------------->| |
-------(a) (b)| | | | -------(a) (b)| | | |
||||||| -----> | | ------ --------- ||||||| -----> | | ------ ---------
------- | |(c) ( N/W ) ------- | |(c) ( N/W )
| |---->( ) | |---->( )
---------- ------ ---------- ------
Figure 3 Figure 3
-------- --------
In Figure 2, a tunnel is created with the local IP address of DUT/SUT A as
the beginning of the tunnel (point c) and the IP address of DUT/SUT B as the In Figure 2, a tunnel is created with the local IP address of
end of the tunnel (point d). DUT/SUT B is assumed to have the tunneling DUT/SUT A as the beginning of the tunnel (point c) and the IP
protocol enabled so that the frames can be decapsulated. When the test port B address of DUT/SUT B as the end of the tunnel (point d). DUT/SUT B
is inserted in between the DUT/SUT A and DUT/SUT B (Figure 3), the endpoint is assumed to have the tunneling protocol enabled so that the
of tunnel has to be re-configured to be directed to the test port B's IP frames can be decapsulated. When the test port B is inserted in
address. For example, in Figure 3, point c' would be assigned as the between the DUT/SUT A and DUT/SUT B (Figure 3), the endpoint of
beginning of the tunnel and point d' as the end of the tunnel. The test port tunnel has to be re-configured to be directed to the test port B's
B is acting as the end of the tunnel, and it does not have to support any IP address. For example, in Figure 3, point c' would be assigned
tunneling protocol since the frames do not have to be decapsulated. Instead, as the beginning of the tunnel and point d' as the end of the
the received encapsulated frames are used to calculate the throughput and tunnel. The test port B is acting as the end of the tunnel, and it
other necessary measurements. does not have to support any tunneling protocol since the frames
do not have to be decapsulated. Instead, the received encapsulated
frames are used to calculate the throughput and other necessary
measurements.
Result Result
Throughput in frames per second for each destination port. The results Parameters to be measured SHOULD include the frame loss and
should also contain the number of frames transmitted and received per port. percent loss per destination port for each multicast group
address.
In addition, the transmit and receive rates in frames per second
for each source and destination port for all multicast groups,
together with the number of frames transmitted and received per
port per multicast groups SHOULD be reported.
4.4.2 Decapsulation Throughput 4.4.2 Decapsulation Throughput
Definition Definition
The maximum rate at which frames offered a DUT/SUT are decapsulated and
correctly forwarded by the DUT/SUT without loss. The maximum rate at which frames offered a DUT/SUT are
decapsulated and correctly forwarded by the DUT/SUT without loss.
Procedure Procedure
The decapsulation process returns the tunneled unicast frames back to The decapsulation process returns the tunneled unicast frames back
their multicast format. This test measures the throughput of the DUT/SUT to their multicast format. This test measures the throughput of
when it has to perform the process of decapsulation, therefore, a test the DUT/SUT when it has to perform the process of decapsulation,
port C is used at the end of the tunnel to receive the decapsulated therefore, a test port C is used at the end of the tunnel to
frames (Figure 4). receive the decapsulated frames (Figure 4).
Test port A DUT/SUT A Test port B DUT/SUT B Test port C Test port A DUT/SUT A Test port B DUT/SUT B Test port C
---------- ---------- ---------- ----------
| | | | | | | |
-------(a) (b)| |(c) ------ (d)| |(e) (f)------- -----(a) (b)| |(c) ---- (d)| |(e) (f)-----
||||||| -----> | |----> |||||| ----->| |-----> ||||||| ||||| -----> | |----> |||| ----->| |-----> |||||
------- | | ------ | | ------- ----- | | ---- | | -----
| | | | | | | |
---------- ---------- ---------- ----------
Figure 4 Figure 4
-------- --------
In Figure 4, the encapsulation process takes place in DUT/SUT A. This may
effect the throughput of the DUT/SUT B. Therefore, two test ports should In Figure 4, the encapsulation process takes place in DUT/SUT A.
be used to separate the encapsulation and decapsulation processes. This may effect the throughput of the DUT/SUT B. Therefore, two
Client A is replaced with the test port A which will generate a test ports should be used to separate the encapsulation and
multicast frame that will be encapsulated by DUT/SUT A. Another test decapsulation processes. Client A is replaced with the test port A
port B is inserted between DUT/SUT A and DUT/SUT B that will receive the which will generate a multicast frame that will be encapsulated by
encapsulated frames and forward it to DUT/SUT B. Test port C will DUT/SUT A. Another test port B is inserted between DUT/SUT A and
receive the decapsulated frames and measure the throughput. DUT/SUT B that will receive the encapsulated frames and forward it
to DUT/SUT B. Test port C will receive the decapsulated frames and
measure the throughput.
Result Result
Parameters to be measured SHOULD include the frame loss and
percent loss per destination port for each multicast group
address.
Throughput in frames per second for each destination port. The In addition, the transmit and receive rates in frames per second
results should also contain the number of frames transmitted and for each source and destination port for all multicast groups,
received per port. together with the number of frames transmitted and received per
port per multicast groups SHOULD be reported.
4.4.3 Re-encapsulation Throughput 4.4.3 Re-encapsulation Throughput
Definition Definition
The maximum rate at which frames of one encapsulated format offered The maximum rate at which frames of one encapsulated format
a DUT/SUT are converted to another encapsulated format and correctly offered a DUT/SUT are converted to another encapsulated format and
forwarded by the DUT/SUT without loss. correctly forwarded by the DUT/SUT without loss.
Procedure Procedure
Re-encapsulation takes place in DUT/SUT B after test port C has received Re-encapsulation takes place in DUT/SUT B after test port C has
the decapsulated frames. These decapsulated frames will be re-inserted received the decapsulated frames. These decapsulated frames will
with a new encapsulation frame and sent to test port B which will measure be re-inserted with a new encapsulation frame and sent to test
the throughput. See Figure 5. port B which will measure the throughput. See Figure 5.
Test port A DUT/SUT A Test port B DUT/SUT B Test port C Test port A DUT/SUT A Test port B DUT/SUT B Test port C
---------- ---------- ---------- ----------
| | | | | | | |
-------(a) (b)| |(c) ------ (d)| |(e) (f)------- -----(a) (b)| |(c) ---- (d)| |(e) (f)-----
||||||| -----> | |----> |||||| <---->| |<----> ||||||| ||||| -----> | |----> |||| <---->| |<----> |||||
------- | | ------ | | ------- ----- | | ---- | | -----
| | | | | | | |
---------- ---------- ---------- ----------
Figure 5 Figure 5
-------- --------
Result Result
Throughput in frames per second for each destination port. The results Parameters to be measured SHOULD include the frame loss and
should also contain the number of frames transmitted and received per percent loss per destination port for each multicast group
port. address.
5. Forwarding Latency In addition, the transmit and receive rates in frames per second
for each source and destination port for all multicast groups,
together with the number of frames transmitted and received per
port per multicast groups SHOULD be reported.
5 Forwarding Latency
This section presents methodologies relating to the characterization of This section presents methodologies relating to the characterization of
the forwarding latency of a DUT/SUT in a multicast environment. It the forwarding latency of a DUT/SUT in a multicast environment. It
extends the concept of latency characterization presented in RFC 2544. extends the concept of latency characterization presented in RFC 2544.
5.1 Multicast Latency 5.1 Multicast Latency
Definition Definition
The set of individual latencies from a single input port on the DUT/SUT or The set of individual latencies from a single input port on the
SUT to all tested ports belonging to the destination multicast group. DUT/SUT or SUT to all tested ports belonging to the destination
multicast group.
Procedure Procedure
According to RFC 2544, a tagged frame is sent half way through the According to RFC 2544, a tagged frame is sent half way through the
transmission that contains a timestamp used for calculation of latency. transmission that contains a timestamp used for calculation of
In the multicast situation, a tagged frame is sent to all destinations latency. In the multicast situation, a tagged frame is sent to all
for each multicast group and latency calculated on a per multicast destinations for each multicast group and latency calculated on a per
group basis. Note that this test MUST be run using the transmission multicast group basis. Note that this test MUST be run using the
rate that is less than the multicast throughput of the DUT/SUT. Also, transmission rate that is less than the multicast throughput of the
the test should take into account the DUT's/SUT's need to cache the DUT/SUT. Also, the test should take into account the DUT's/SUT's need
traffic in its IP cache, fastpath cache or shortcut tables since the to cache the traffic in its IP cache, fastpath cache or shortcut
initial part of the traffic will be utilized to build these tables. tables since the initial part of the traffic will be utilized to
build these tables.
Result Result
The latency value for each multicast group address per port. An aggregate
latency MAY also be reported. The parameter to be measured is the latency value for each multicast
group address per destination port. An aggregate latency MAY also be
reported.
5.2 Min/Max/Average Multicast Latency 5.2 Min/Max/Average Multicast Latency
Definition: Definition
The difference between the maximum latency measurement and the The difference between the maximum latency measurement and the
minimum latency measurement from the set of latencies produced by minimum latency measurement from the set of latencies produced by the
the Multicast Latency benchmark. Multicast Latency benchmark.
Procedure: Procedure
For the entire duration of the Latency test the smallest latency, the First determine the throughput for DUT/SUT at each of the listed
largest latency, the sum of latencies, and the number should be tracked frame sizes determined by the forwarding and throughput tests of
per receive port of the tester. section 4. Send a stream of frames to a fixed number of multicast
groups through the DUT at the determined throughput rate. An
identifying tag SHOULD be included in all frames to ensure proper
identification of the transmitted frame on the receive side, the type
of tag being implementation dependent.
The test can also increment bucket counters that represent a range latency Latencies for each transmitted frame are calculated based on the
range. This can be used to create a histogram. From the histogram, description of latencies in RFC 2544. The average latency is the
minimum, maximum, and average the test results can show the jitter. total of all accumulated latency values divided by the total number
of those values. The minimum latency is the smallest latency; the
maximum latency is the largest latency of all accumulated latency
values.
Results: Results
For each port the results WILL display the number of frames, minimum The parameters to be measured are the minium, maximum and average
latency, maximum latency, and the average latency. The results SHOULD latency values for each multicast group address per destination port.
also display the histogram of latencies.
6. Overhead 6 Overhead
This section presents methodology relating to the characterization of This section presents methodology relating to the characterization of
the overhead delays associated with explicit operations found in the overhead delays associated with explicit operations found in
multicast environments. multicast environments.
6.1 Group Join Delay 6.1 Group Join Delay
Definition: Definition
The time duration it takes a DUT/SUT to start forwarding multicast The time duration it takes a DUT/SUT to start forwarding multicast
packets from the time a successful IGMP group membership report packets from the time a successful IGMP group membership report has
has been issued to the DUT/SUT. been issued to the DUT/SUT.
Procedure: Procedure
Traffic is sent on the source port at the same time as the IGMP JOIN Traffic is sent on the source port at the same time as the IGMP JOIN
Group message is transmitted from the destination ports. The join Group message is transmitted from the destination ports. The join
delay is the difference in time from when the IGMP Join is sent and delay is the difference in time from when the IGMP Join is sent
the first frame is received. (timestamp A) and the first frame is forwarded to a receiving member
port (timestamp B).
One of the keys is to transmit at the fastest rate the DUT/SUT can handle Group Join delay = timestamp B - timestamp A
multicast frames. This is to get the best resultion in the Join Delay.
However, you do not want to transmit the frames to fast that frames
are dropped by the DUT/SUT. Traffic should be sent at the throughput rate
determined by the forwarding tests of section 4.
Results: One of the keys is to transmit at the fastest rate the DUT/SUT can
handle multicast frames. This is to get the best resolution and the
least margin of error in the Join Delay.
The JOIN delay for each port. An error or granularity of the However, you do not want to transmit the frames so fast that frames
timestamp should be reported. This granularity may be within 20 are dropped by the DUT/SUT. Traffic should be sent at the throughput
nanoseconds of the result. The granularity depends on the hardware rate determined by the forwarding tests of section 4.
limitation of the tester.
Results
The parameter to be measured is the join delay time for each
multicast group address per destination port. In addition, the number
of frames transmitted and received and percent loss may be reported.
6.2 Group Leave Delay 6.2 Group Leave Delay
Definition Definition
The time duration it takes a DUT/SUT to cease forwarding multicast packets The time duration it takes a DUT/SUT to cease forwarding multicast
after a corresponding IGMP "Leave Group" message has been successfully packets after a corresponding IGMP "Leave Group" message has been
offered to the DUT/SUT. successfully offered to the DUT/SUT.
Procedure Procedure
Traffic is sent on the source port at the same time as the IGMP Leave Traffic is sent on the source port at the same time as the IGMP Leave
Group messages are transmitted from the destination ports. The frames Group messages are transmitted from the destination ports. The leave
on both the source and destination ports are sent with the timestamps delay is the difference in time from when the IGMP leave is sent
inserted. The Group Leave Delay is the difference in the value of the (timestamp A) and the last frame is forwarded to a receiving member
timestamp A of the first IGMP Leave Group frame sent and the timestamp port (timestamp B).
B of the last frame that is received on that destination port.
Group Leave delay = timestamp B - timestamp A Group Leave delay = timestamp B - timestamp A
Traffic should be sent at the throughput rate determined by the One of the keys is to transmit at the fastest rate the DUT/SUT can
forwarding tests of section 4. handle multicast frames. This is to get the best resolution and
least margin of error in the Leave Delay. However, you do not want
to transmit the frames too fast that frames are dropped by the
DUT/SUT. Traffic should be sent at the throughput rate determined by
the forwarding tests of section 4.
Result Result
Group Leave Delay values for each multicast group address on each The parameter to be measured is the leave delay time for each
destination port. Also, the number of frames transmitted and received, multicast group address per destination port. In addition, the number
and percent loss may be displayed. of frames transmitted and received and percent loss may be reported.
7 Capacity
7. Capacity
This section offers terms relating to the identification of multicast This section offers terms relating to the identification of multicast
group limits of a DUT/SUT. group limits of a DUT/SUT.
7.1 Multicast Group Capacity 7.1 Multicast Group Capacity
Definition: Definition
The maximum number of multicast groups a DUT/SUT can support while The maximum number of multicast groups a DUT/SUT can support while
maintaining the ability to forward multicast frames to all maintaining the ability to forward multicast frames to all multicast
multicast groups registered to that DUT/SUT. groups registered to that DUT/SUT.
Procedure: Procedure
One or more receiving ports of DUT/SUT will join an initial number of One or more receiving ports of DUT/SUT will join an initial number of
groups. Then after a delay the source port will transmit to each group groups.
at a transmission rate that the DUT/SUT can handle. If all frames
sent are forwarded and received the receiving ports will join an
incremental value of groups. Then after a delay the source port will
transmit to all groups at a transmission rate that the DUT/SUT can
handle. If all frames sent are forwarded and received the receiving
ports will continuing joining and testing until a frame is not forwarded
nor received.
The group capacity resolution will be the incremental value. So the Then after a delay (enough time for all ports to join) the source
capacity could be greater then last capacity passed but less then the port will transmit to each group at a transmission rate that the
one that failed. DUT/SUT can handle without dropping IP Multicast frames.
Once a capacity is determined the test should be re run with greater If all frames sent are forwarded by the DUT/SUT and received the test
delays after the JOIN and a slower transmission rate. And the initial iteration is said to pass at the current capacity.
group level should be raised to about five less then the previous
capacity and incremental value should be set to one.
Results: If the iteration passes at the capacity the test will add an user
defined incremental value of groups to each receive port.
The number of groups passed vs the number of groups failed. The The iteration is to run again at the new group level and capacity
results SHOULD give details when the frame fails to be forwarded tested as stated above.
about how many frames did and did not get forwarded. Which groups
DID and DID NOT get forwarded. Also, the frame rate MAY be reported. Once the test fails at a capacity the capacity is stated to be the
last Iteration that pass at a giving capacity.
Results
The parameter to be measured is the total number of group addresses
that were successfully forwarded with no loss.
8 Interaction
8. Interaction
Network forwarding devices are generally required to provide more Network forwarding devices are generally required to provide more
functionality than than the forwarding of traffic. Moreover, network functionality than just the forwarding of traffic. Moreover, network
forwarding devices may be asked to provide those functions in a forwarding devices may be asked to provide those functions in a variety
variety of environments. This section offers terms to assist in the of environments. This section offers terms to assist in the
charaterization of DUT/SUT behavior in consideration of potentially characterization of DUT/SUT behavior in consideration of potentially
interacting factors. interacting factors.
8.1 Forwarding Burdened Multicast Latency 8.1 Forwarding Burdened Multicast Latency
The Multicast Latency metrics can be influenced by forcing the The Multicast Latency metrics can be influenced by forcing the
DUT/SUT to perform extra processing of packets while multicast traffic is DUT/SUT to perform extra processing of packets while multicast
being forwarded for latency measurements. In this test, a set of traffic is being forwarded for latency measurements. In this test, a
ports on the tester will be designated to be source and destination set of ports on the tester will be designated to be source and
similar to the generic IP Multicast test setup. In addition to this destination similar to the generic IP Multicast test setup. In
setup, another set of ports will be selected to transmit some multicast addition to this setup, another set of ports will be selected to
traffic that is destined to multicast group addresses that have not transmit some multicast traffic that is destined to multicast group
been joined by these additional set of ports. For example, if ports 1, addresses that have not been joined by these additional set of ports.
2, 3, and 4 form the burdened response setup (setup A) which is used to
obtain the latency metrics and ports 5, 6, 7, and 8 form the non-burdened For example, if ports 1,2, 3, and 4 form the burdened response setup
response setup (setup B) which will afflict the burdened response setup, (setup A) which is used to obtain the latency metrics and ports 5, 6,
then setup B traffic will join multicast group addresses not joined by 7, and 8 form the non-burdened response setup (setup B) which will
the ports in this setup. By sending such multicast traffic, the DUT/SUT afflict the burdened response setup, then setup B traffic will join
will perform a lookup on the packets that will affect the processing of multicast group addresses not joined by the ports in this setup. By
setup A traffic. sending such multicast traffic, the DUT/SUT will perform a lookup on
the packets that will affect the processing of setup A traffic.
8.2 Forwarding Burdened Group Join Delay 8.2 Forwarding Burdened Group Join Delay
The port configuration in this test is similar to the one described in The port configuration in this test is similar to the one described
section 8.1, but in this test, the multicast traffic is not sent by the in section 8.1, but in this test, the multicast traffic is not sent
ports in setup B. In this test, the setup A traffic must be influenced by the ports in setup B. In this test, the setup A traffic must be
in such a way that will affect the DUT's/SUT's ability to process Group influenced in such a way that will affect the DUT's/SUT's ability to
Join messages. Therefore, in this test, the ports in setup B will send process Group Join messages. Therefore, in this test, the ports in
a set of IGMP Group Join messages while the ports in setup A are also setup B will send a set of IGMP Group Join messages while the ports
joining its own set of group addresses. Since the two sets of group in setup A are also joining its own set of group addresses. Since the
addresses are independent of each other, the group join delay for two sets of group addresses are independent of each other, the group
setup A may be different than in the case when there were no other join delay for setup A may be different than in the case when there
group addresses being joined. were no other group addresses being joined.
9 Security Considerations
As this document is solely for the purpose of providing metric
methodology and describes neither a protocol nor a protocol's
implementation, there are no security considerations associated with
this document.
10
References
[Br91] Bradner, S., "Benchmarking Terminology for Network
Interconnection Devices", RFC 1242, July 1991.
[Br96] Bradner, S., and J. McQuaid, "Benchmarking Methodology for
Network Interconnect Devices", RFC 2544, March 1999.
[Br97] Bradner, S. "Use of Keywords in RFCs to Reflect Requirement
Levels, RFC 2119, March 1997
[Du98] Dubray, K., "Terminology for IP Multicast Benchmarking", RFC
2432, October 1998.
[Hu95] Huitema, C. "Routing in the Internet." Prentice-Hall, 1995.
[Ka98] Kosiur, D., "IP Multicasting: the Complete Guide to Interactive
Corporate Networks", John Wiley & Sons, Inc, 1998.
[Ma98] Mandeville, R., "Benchmarking Terminology for LAN Switching
Devices", RFC 2285, February 1998.
[Mt98] Maufer, T. "Deploying IP Multicast in the Enterprise." Prentice-
Hall, 1998.
[Se98] Semeria, C. and Maufer, T. "Introduction to IP Multicast
Routing." http://www.3com.com/nsc/501303.html 3Com Corp., 1998.
11
Author's Addresses
Hardev Soor
Ixia Communications
4505 Las Virgenes Road, Suite 209
Calabasas, CA 91302
USA
Phone: 818 871 1800
EMail: hardev@ixia.com
Debra Stopp
Ixia Communications
4505 Las Virgenes Road, Suite 209
Calabasas, CA 91302
USA
Phone: 818 871 1800
EMail: debby@ixia.com
Ralph Daniels
Netcom Systems
948 Loop Road
Clayton, NC 27520
USA
Phone: 919 550 9475
EMail: Ralph_Daniels@NetcomSystems.com
Appendix A: Determining an even distribution Appendix A: Determining an even distribution
A.1 Scope Of This Appendix A.1 Scope Of This Appendix
This appendix discusses the suggested approach to configuring the This appendix discusses the suggested approach to configuring the
deterministic distribution methodology for tests that involve both deterministic distribution methodology for tests that involve both
multicast and unicast traffic classes in an aggregated traffic stream. multicast and unicast traffic classes in an aggregated traffic stream.
As such, this appendix MUST not be read as an amendment to the As such, this appendix MUST not be read as an amendment to the
methodology described in the body of this document but as a guide methodology described in the body of this document but as a guide to
to testing practice. testing practice.
It is important to understand and fully define the distribution of It is important to understand and fully define the distribution of
frames among all multicast and unicast destinations. If the frames among all multicast and unicast destinations. If the
distribution is not well defined or understood, the throughput and distribution is not well defined or understood, the throughput and
forwarding metrics are not meaningful. forwarding metrics are not meaningful.
In a homogeneous environment, a large, single burst of multicast In a homogeneous environment, a large, single burst of multicast frames
frames may be followed by a large burst of unicast frames. This is a may be followed by a large burst of unicast frames. This is a very
very different distribution than that of a non-homogeneous different distribution than that of a non-homogeneous environment, where
environment, where the multicast and unicast frames are intermingled the multicast and unicast frames are intermingled
throughout the entire transmission. throughout the entire transmission.
The recommended distribution is that of the non-homogeneous The recommended distribution is that of the non-homogeneous environment
environment because it more closely represents a real-world because it more closely represents a real-world scenario. The
scenario. The distribution is modeled by calculating the number of distribution is modeled by calculating the number of multicast frames
multicast frames per destination port as a burst, then calculating per destination port as a burst, then calculating the number of unicast
the number of unicast frames to transmit as a percentage of the total frames to transmit as a percentage of the total frames transmitted. The
frames transmitted. The overall effect of the distribution is small overall effect of the distribution is small bursts of multicast frames
bursts of multicast frames intermingled with small bursts of unicast intermingled with small bursts of unicast frames.
frames.
Example Example
This example illustrates the distribution algorithm for a 100 Mbps rate. This example illustrates the distribution algorithm for a 100 Mbps rate.
Frame size = 64 Frame size = 64
Duration of test = 30 seconds Duration of test = 30 seconds
Intended Load (ILOAD) = 100% of maximum rate Intended Load (ILOAD) = 100% of maximum rate
Mapping for unicast traffic: Port 1 to Port 2 Mapping for unicast traffic: Port 1 to Port 2
Port 3 to port 4 Port 3 to port 4
skipping to change at page 15, line 27 skipping to change at page 17, line 23
Loop counter = 4464300 / 900 = 4960.3333 (round it off to 4960) Loop counter = 4464300 / 900 = 4960.3333 (round it off to 4960)
Therefore, the actual number of frames that will be transmitted: Therefore, the actual number of frames that will be transmitted:
Unicast frames = 4960 * 180 = 892800 frames Unicast frames = 4960 * 180 = 892800 frames
Multicast frames = 4960 * 720 = 3571200 frames Multicast frames = 4960 * 720 = 3571200 frames
The following pattern will be established: The following pattern will be established:
UUUMMMMMMMMMMMMUUUMMMMMMMMMMMMUUUMMMMMMMMMMMMUUUMMMMMMMMMMMM UUUMMMMMMMMMMMMUUUMMMMMMMMMMMMUUUMMMMMMMMMMMMUUUMMMMMMMMMMMM
where U represents 60 Unicast frames (UUU = 180 frames) where U represents 60 Unicast frames (U = 180 frames)
M represents 60 Multicast frames (MMMMMMMMMMMM = 720 frames) M represents 60 Multicast frames (M = 720 frames)
8. Security Considerations.
As this document is solely for the purpose of providing metric methodology
and describes neither a protocol nor a protocol's implementation, there
are no security considerations associated with this document.
9. References
[Br91] Bradner, S., "Benchmarking Terminology for Network
Interconnection Devices", RFC 1242, July 1991.
[Br96] Bradner, S., and J. McQuaid, "Benchmarking Methodology for
Network Interconnect Devices", RFC 2544, March 1999.
[Br97] Bradner, S. "Use of Keywords in RFCs to Reflect Requirement
Levels, RFC 2119, March 1997
[Du98] Dubray, K., "Terminology for IP Multicast Benchmarking",
RFC 2432, October 1998.
[Hu95] Huitema, C. "Routing in the Internet." Prentice-Hall, 1995.
[Ka98] Kosiur, D., "IP Multicasting: the Complete Guide to Interactive
Corporate Networks", John Wiley & Sons, Inc, 1998.
[Ma98] Mandeville, R., "Benchmarking Terminology for LAN Switching
Devices", RFC 2285, February 1998.
[Mt98] Maufer, T. "Deploying IP Multicast in the Enterprise."
Prentice-Hall, 1998.
[Se98] Semeria, C. and Maufer, T. "Introduction to IP Multicast
Routing." http://www.3com.com/nsc/501303.html 3Com Corp.,
1998.
6. Author's Address
Hardev Soor
Ixia Communications
4505 Las Virgenes Road, Suite 209
Calabasas, CA 91302
USA
Phone: 818 871 1800
EMail: hardev@ixiacom.com
Debra Stopp
Ixia Communications
4505 Las Virgenes Road, Suite 209
Calabasas, CA 91302
USA
Phone: 818 871 1800
EMail: debby@ixiacom.com
Ralph Daniels
Netcom Systems
948 Loop Road
Clayton, NC 27520
USA
Phone: 919 550 9475 12
Full Copyright Statement
EMail: Ralph_Daniels@NetcomSystems.com "Copyright (C) The Internet Society (date). All Rights Reserved. This
document and translations of it may be copied and furnished to others,
and derivative works that comment on or otherwise explain it or assist
in its implementation may be prepared, copied, published and
distributed, in whole or in part, without restriction of any kind,
provided that the above copyright notice and this paragraph are
included on all such copies and derivative works. However, this
document itself may not be modified in any way, such as by removing the
copyright notice or references to the Internet Society or other
Internet organizations, except as needed for the purpose of developing
Internet standards in which case the procedures for copyrights defined
in the Internet Standards process must be followed, or as required to
translate it into.
 End of changes. 

This html diff was produced by rfcdiff 1.23, available from http://www.levkowetz.com/ietf/tools/rfcdiff/