draft-ietf-bmwg-ipflow-meth-09.txt   draft-ietf-bmwg-ipflow-meth-10.txt 
Internet Engineering Task Force Jan Novak Internet Engineering Task Force Jan Novak
Internet-Draft Cisco Systems, Inc. Internet-Draft Cisco Systems, Inc.
Intended status: Informational Intended status: Informational
Expires: 12 September, 2012 12 March 2012 Expires: 23 October, 2012 23 April 2012
IP Flow Information Accounting and Export Benchmarking IP Flow Information Accounting and Export Benchmarking
Methodology Methodology
draft-ietf-bmwg-ipflow-meth-09.txt draft-ietf-bmwg-ipflow-meth-10.txt
Abstract Abstract
This document provides a methodology and framework for quantifying This document provides a methodology and framework for quantifying
the performance impact of monitoring of IP flows on a network device the performance impact of monitoring of IP flows on a network device
and export of this information to a collector. It identifies the rate and export of this information to a collector. It identifies the rate
at which the IP flows are created, expired, and successfully exported at which the IP flows are created, expired, and successfully exported
as a new performance metric in combination with traditional as a new performance metric in combination with traditional
throughput. The metric is only applicable to the devices compliant throughput. The metric is only applicable to the devices compliant
with the Architecture for IP Flow Information Export [RFC5470]. with the Architecture for IP Flow Information Export [RFC5470]. The
methodology quantifies the impact of the IP flow monitoring process
on the network equipment.
Status of this Memo Status of this Memo
This Internet-Draft is submitted to IETF in full conformance with the This Internet-Draft is submitted to IETF in full conformance with the
provisions of BCP 78 and BCP 79. provisions of BCP 78 and BCP 79.
Internet-Drafts are working documents of the Internet Engineering Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF), its areas, and its working groups. Note that Task Force (IETF), its areas, and its working groups. Note that
other groups may also distribute working documents as Internet- other groups may also distribute working documents as Internet-
Drafts. Drafts.
Internet-Drafts are draft documents valid for a maximum of six Internet-Drafts are draft documents valid for a maximum of six
months and may be updated, replaced, or obsoleted by other months and may be updated, replaced, or obsoleted by other
documents at any time. It is inappropriate to use Internet-Drafts documents at any time. It is inappropriate to use Internet-Drafts
as reference material or to cite them other than as "work in as reference material or to cite them other than as "work in
progress." progress."
The list of current Internet-Drafts can be accessed at The list of current Internet-Drafts can be accessed at
http://www.ietf.org/ietf/1id-abstracts.txt. http://www.ietf.org/ietf/1id-abstracts.txt.
The list of Internet-Draft Shadow Directories can be accessed at The list of Internet-Draft Shadow Directories can be accessed at
http://www.ietf.org/shadow.html. http://www.ietf.org/shadow.html.
This Internet-Draft will expire on 12 September, 2012. This Internet-Draft will expire on 23 October, 2012.
Copyright Notice Copyright Notice
Copyright (c) 2012 IETF Trust and the persons identified as the Copyright (c) 2012 IETF Trust and the persons identified as the
document authors. All rights reserved. document authors. All rights reserved.
This document is subject to BCP 78 and the IETF Trust's Legal This document is subject to BCP 78 and the IETF Trust's Legal
Provisions Relating to IETF Documents Provisions Relating to IETF Documents
(http://trustee.ietf.org/license-info) in effect on the date of (http://trustee.ietf.org/license-info) in effect on the date of
publication of this document. Please review these documents publication of this document. Please review these documents
carefully, as they describe your rights and restrictions with respect carefully, as they describe your rights and restrictions with respect
to this document. Code Components extracted from this document must to this document. Code Components extracted from this document must
include Simplified BSD License text as described in Section 4.e of include Simplified BSD License text as described in Section 4.e of
the Trust Legal Provisions and are provided without warranty as the Trust Legal Provisions and are provided without warranty as
described in the Simplified BSD License. described in the Simplified BSD License.
Novak Expires October, 2012
Novak Expires September, 2012
Conventions used in this document Conventions used in this document
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL
NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and
"OPTIONAL" in this document are to be interpreted as described "OPTIONAL" in this document are to be interpreted as described
in RFC 2119 [RFC2119]. in RFC 2119 [RFC2119].
Table of Contents Table of Contents
1. Introduction. . . . . . . . . . . . . . . . . . . . . . . . . 3 1. Introduction. . . . . . . . . . . . . . . . . . . . . . . . . 3
2. Terminology . . . . . . . . . . . . . . . . . . . . . . . . . 4 2. Terminology . . . . . . . . . . . . . . . . . . . . . . . . . 4
2.1 Existing Terminology. . . . . . . . . . . . . . . . . . . 4 2.1 Existing Terminology. . . . . . . . . . . . . . . . . . . 4
2.2 New Terminology . . . . . . . . . . . . . . . . . . . . . 4 2.2 New Terminology . . . . . . . . . . . . . . . . . . . . . 4
3. Flow Monitoring Performance Benchmark . . . . . . . . . . . . 6 3. Flow Monitoring Performance Benchmark . . . . . . . . . . . . 6
3.1 Definition. . . . . . . . . . . . . . . . . . . . . . . . 6 3.1 Definition. . . . . . . . . . . . . . . . . . . . . . . . 6
3.2 Device Applicability. . . . . . . . . . . . . . . . . . . 6 3.2 Device Applicability. . . . . . . . . . . . . . . . . . . 6
3.3 Measurement Concept . . . . . . . . . . . . . . . . . . . 7 3.3 Measurement Concept . . . . . . . . . . . . . . . . . . . 7
3.4 The Measurement Procedure Overview. . . . . . . . . . . . 8 3.4 The Measurement Procedure Overview. . . . . . . . . . . . 8
4. Measurement Set-Up. . . . . . . . . . . . . . . . . . . . . . 9 4. Measurement Set-Up. . . . . . . . . . . . . . . . . . . . . . 9
4.1 Measurement Topology. . . . . . . . . . . . . . . . . . . 9 4.1 Measurement Topology. . . . . . . . . . . . . . . . . . . 9
4.2 Base DUT Set Up. . . . . . . . . . . . . . . . . . . . . 11 4.2 Baseline DUT Set Up. . . . . . . . . . . . . . . . . . . 11
4.3 Flow Monitoring Configuration. . . . . . . . . . . . . . 11 4.3 Flow Monitoring Configuration. . . . . . . . . . . . . . 11
4.4 Collector. . . . . . . . . . . . . . . . . . . . . . . . 16 4.4 Collector. . . . . . . . . . . . . . . . . . . . . . . . 16
4.5 Sampling . . . . . . . . . . . . . . . . . . . . . . . . 16 4.5 Sampling . . . . . . . . . . . . . . . . . . . . . . . . 16
4.6 Frame Formats. . . . . . . . . . . . . . . . . . . . . . 16 4.6 Frame Formats. . . . . . . . . . . . . . . . . . . . . . 16
4.7 Frame Sizes. . . . . . . . . . . . . . . . . . . . . . . 16 4.7 Frame Sizes. . . . . . . . . . . . . . . . . . . . . . . 17
4.8 Flow Export Data Packet Sizes. . . . . . . . . . . . . . 16 4.8 Flow Export Data Packet Sizes. . . . . . . . . . . . . . 17
4.9 Illustrative Test Set-up Examples. . . . . . . . . . . . 17 4.9 Illustrative Test Set-up Examples. . . . . . . . . . . . 17
5. Flow Monitoring Throughput Measurement Methodology . . . . . 18 5. Flow Monitoring Throughput Measurement Methodology . . . . . 19
5.1 Flow Monitoring Configuration. . . . . . . . . . . . . . 19 5.1 Flow Monitoring Configuration. . . . . . . . . . . . . . 19
5.2 Traffic Configuration. . . . . . . . . . . . . . . . . . 20 5.2 Traffic Configuration. . . . . . . . . . . . . . . . . . 20
5.3 Cache Population . . . . . . . . . . . . . . . . . . . . 20 5.3 Cache Population . . . . . . . . . . . . . . . . . . . . 21
5.4 Measurement Time Interval. . . . . . . . . . . . . . . . 20 5.4 Measurement Time Interval. . . . . . . . . . . . . . . . 21
5.5 Flow Export Rate Measurement . . . . . . . . . . . . . . 21 5.5 Flow Export Rate Measurement . . . . . . . . . . . . . . 22
5.6 The Measurement Procedure. . . . . . . . . . . . . . . . 22 5.6 The Measurement Procedure. . . . . . . . . . . . . . . . 23
6. RFC2544 Measurements . . . . . . . . . . . . . . . . . . . . 23 6. RFC2544 Measurements . . . . . . . . . . . . . . . . . . . . 24
6.1 Flow Monitoring Configuration. . . . . . . . . . . . . . 23 6.1 Flow Monitoring Configuration. . . . . . . . . . . . . . 24
6.2 Measurements With the Flow Monitoring Throughput Set-up. 24 6.2 Measurements With the Flow Monitoring Throughput Set-up. 25
6.3 Measurements With Fixed Flow Export Rate . . . . . . . . 24 6.3 Measurements With Fixed Flow Export Rate . . . . . . . . 25
7. Flow Monitoring Accuracy . . . . . . . . . . . . . . . . . . 25 7. Flow Monitoring Accuracy . . . . . . . . . . . . . . . . . . 26
8. Evaluating Flow Monitoring Applicability . . . . . . . . . . 26 8. Evaluating Flow Monitoring Applicability . . . . . . . . . . 27
9. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 26 9. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 27
10. Security Considerations . . . . . . . . . . . . . . . . . . 27 10. Security Considerations . . . . . . . . . . . . . . . . . . 27
11. IANA Considerations . . . . . . . . . . . . . . . . . . . . 27 11. IANA Considerations . . . . . . . . . . . . . . . . . . . . 28
12. References. . . . . . . . . . . . . . . . . . . . . . . . . 27 12. References. . . . . . . . . . . . . . . . . . . . . . . . . 28
12.1 Normative References. . . . . . . . . . . . . . . . . . 27 12.1 Normative References. . . . . . . . . . . . . . . . . . 28
12.2 Informative References. . . . . . . . . . . . . . . . . 27 12.2 Informative References. . . . . . . . . . . . . . . . . 28
Appendix A: Recommended Report Format . . . . . . . . . . . . . 29 Appendix A: Recommended Report Format . . . . . . . . . . . . . 30
Appendix B: Miscellaneous Tests . . . . . . . . . . . . . . . . 30 Appendix B: Miscellaneous Tests . . . . . . . . . . . . . . . . 31
B.1 DUT Under Traffic Load . . . . . . . . . . . . . . . . . 30 B.1 DUT Under Traffic Load . . . . . . . . . . . . . . . . . 31
B.2 In-band Flow Export. . . . . . . . . . . . . . . . . . . 30 B.2 In-band Flow Export. . . . . . . . . . . . . . . . . . . 31
B.3 Variable Packet Rate . . . . . . . . . . . . . . . . . . 30 B.3 Variable Packet Rate . . . . . . . . . . . . . . . . . . 32
B.4 Bursty Traffic . . . . . . . . . . . . . . . . . . . . . 31 B.4 Bursty Traffic . . . . . . . . . . . . . . . . . . . . . 32
Novak Expires September, 2012 Novak Expires October, 2012
B.5 Various Flow Monitoring Configurations . . . . . . . . . 31 B.5 Various Flow Monitoring Configurations . . . . . . . . . 32
B.6 Tests With Bidirectional Traffic . . . . . . . . . . . . 32 B.6 Tests With Bidirectional Traffic . . . . . . . . . . . . 33
B.7 Instantaneous Flow Export Rate . . . . . . . . . . . . . 32 B.7 Instantaneous Flow Export Rate . . . . . . . . . . . . . 33
1. Introduction 1. Introduction
Monitoring of IP flows (Flow monitoring) is defined in the Monitoring of IP flows (Flow monitoring) is defined in the
Architecture for IP Flow Information Export [RFC5470] and related Architecture for IP Flow Information Export [RFC5470] and related
IPFIX documents. It analyses the traffic using predefined fields IPFIX documents specified in section 1.2 of [RFC5470]. It
from the packet header as keys and stores the traffic and analyses the traffic using predefined fields from the packet
other internal information in the DUT (Device Under Test) memory. header as keys and stores the traffic and other internal
This cached flow information is then formatted into records (see information in the DUT (Device Under Test) memory. This cached
section 2.1 for term definitions) and exported from the DUT to an flow information is then formatted into records (see section 2.1
external data collector for analysis. More details on the for term definitions) and exported from the DUT to an external
measurement architecture is provided in section 3.3. data collector for analysis. More details on the measurement
architecture is provided in section 3.3.
Flow monitoring on network devices is widely deployed and has Flow monitoring on network devices is widely deployed and has
numerous uses in both service provider and enterprise segments as numerous uses in both service provider and enterprise segments as
detailed in the Requirements for IP Flow Information Export detailed in the Requirements for IP Flow Information Export
[RFC3917]. This document provides a methodology for measuring Flow [RFC3917]. This document provides a methodology for measuring Flow
monitoring performance so that network operators have a framework monitoring performance so that network operators have a framework
for measurements of impact on the network and network equipment. for measurements of impact on the network and network equipment.
This document's goal is a series of methodology specifications for This document's goal is a series of methodology specifications for
the measurement of Flow monitoring performance, in a way that is the measurement of Flow monitoring performance, in a way that is
comparable amongst various implementations, platforms, and comparable amongst various implementations, platforms, and
vendor's devices. vendor's devices.
Flow monitoring is in most cases run on network devices also Flow monitoring is in most cases run on network devices also
forwarding packets. This document therefore provides also the forwarding packets. This document therefore provides also the
methodology for [RFC2544] measurements in the presence of Flow methodology for [RFC2544] measurements in the presence of Flow
monitoring. It is applicable to IPv6 and MPLS traffic with their monitoring. It is applicable to IPv6 and MPLS traffic with their
specifics defined in [RFC5180] and [RFC5695] respectively. specifics defined in [RFC5180] and [RFC5695] respectively.
The most significant performance parameter is the rate at which IP This document specifies a methodology to measure the maximum IP
flows are created and expired in the network device's memory and flow export rate that a network device can sustain without
exported to a collector. Therefore, this document specifies a impacting the forwarding plane, without losing any IP flow
methodology to measure the maximum IP flow rate that a network information, and without compromising the IP flow accuracy (see
device can sustain without impacting the forwarding plane, without section 7 for details).
losing any IP flow information, and without compromising the IP flow
accuracy (see section 7 for details).
[RFC2544], [RFC5180] and [RFC5695] specify benchmarking of network [RFC2544], [RFC5180] and [RFC5695] specify benchmarking of network
devices forwarding IPv4, IPv6 and MPLS [RFC3031] traffic, devices forwarding IPv4, IPv6 and MPLS [RFC3031] traffic,
respectively. The methodology specified in this document stays the respectively. The methodology specified in this document stays the
same for any traffic type. The only restriction may be the DUT's same for any traffic type. The only restriction may be the DUT's
lack of support for Flow monitoring of the particular traffic type. lack of support for Flow monitoring of the particular traffic type.
A variety of different network device architectures exist that are A variety of different DUT architectures exist that are capable of
capable of Flow monitoring and export. As such, this document does Flow monitoring and export. As such, this document does not attempt
not attempt to list the various white box variables (CPU load, to list the various white box variables (CPU load, memory
memory utilization, hardware resources utilization etc) that could utilization, hardware resources utilization etc) that could be
be gathered as they always help in comparison evaluations. A more gathered as they always help in comparison evaluations. A more
Novak Expires September, 2012 Novak Expires October, 2012
complete understanding of the stress points of a particular device complete understanding of the stress points of a particular device
can be attained using this internal information and the tester MAY can be attained using this internal information and the tester MAY
choose to gather this information during the measurement iterations. choose to gather this information during the measurement iterations.
2. Terminology 2. Terminology
The terminology used in this document is based on [RFC5470], The terminology used in this document is based on [RFC5470],
[RFC2285] and [RFC1242] as summarised in section 2.1. The only new [RFC2285] and [RFC1242] as summarized in section 2.1. The only new
terms needed for this methodology are defined in section 2.2. terms needed for this methodology are defined in section 2.2.
2.1 Existing Terminology 2.1 Existing Terminology
Device Under Test (DUT) [RFC2285, section 3.1.1] Device Under Test (DUT) [RFC2285, section 3.1.1]
Flow [RFC5470, section 2] Flow [RFC5101, section 2]
Flow Key [RFC5470, section 2] Flow Key [RFC5101, section 2]
Flow Record [RFC5470, section 2] Flow Record [RFC5101, section 2]
Template Record [RFC5101, section 2]
Observation Point [RFC5470, section 2] Observation Point [RFC5470, section 2]
Metering Process [RFC5470, section 2] Metering Process [RFC5470, section 2]
Exporting Process [RFC5470, section 2] Exporting Process [RFC5470, section 2]
Exporter [RFC5470, section 2] Exporter [RFC5470, section 2]
Collector [RFC5470, section 2] Collector [RFC5470, section 2]
skipping to change at page 4, line 52 skipping to change at page 4, line 54
Throughput [RFC1242, section 3.17] Throughput [RFC1242, section 3.17]
2.2 New Terminology 2.2 New Terminology
2.2.1 Cache 2.2.1 Cache
Definition: Definition:
Memory area held and dedicated by the DUT to store Flow Memory area held and dedicated by the DUT to store Flow
information prior to the Flow Expiration. information prior to the Flow Expiration.
Novak Expires October, 2012
2.2.2 Cache Size 2.2.2 Cache Size
Definition: Definition:
The size of the Cache in terms of how many entries the Cache can The size of the Cache in terms of how many entries the Cache can
hold. hold.
Novak Expires September, 2012
Discussion: Discussion:
This term is typically represented as a configurable option in This term is typically represented as a configurable option in
the particular Flow monitoring implementation. Its highest value the particular Flow monitoring implementation. Its highest value
will depend on the memory available in the network device. will depend on the memory available in the network device.
Measurement units: Measurement units:
Number of Cache entries Number of Cache entries
2.2.3 Active Timeout 2.2.3 Active Timeout
skipping to change at page 5, line 33 skipping to change at page 5, line 39
Flows are considered long-running when they last longer than Flows are considered long-running when they last longer than
several multiples of the Active Timeout. If the Active Timeout is several multiples of the Active Timeout. If the Active Timeout is
zero, then Flows are considered long-running if they contain many zero, then Flows are considered long-running if they contain many
more packets (tens of packets) than usually observed in a single more packets (tens of packets) than usually observed in a single
transaction. transaction.
Measurement units: Measurement units:
Seconds Seconds
2.2.4 Inactive Timeout 2.2.4 Idle Timeout
Definition: Definition:
The time interval used by the Metering Process to expire an entry The time interval used by the Metering Process to expire an entry
from the Cache, when no more packets belonging to that specific from the Cache, when no more packets belonging to that specific
Cache entry have been observed during the interval. Cache entry have been observed during the interval.
Discussion: Discussion:
This term is typically represented as a configurable option in the This term is typically represented as a configurable option in the
particular Flow monitoring implementation. See section 5.1.1 of particular Flow monitoring implementation. See section 5.1.1 of
[RFC5470] for more detailed discussion. [RFC5470] for more detailed discussion. Note that some documents
in the industry refer to this "Idle Timeout" as the
"inactive timeout".
Measurement units: Measurement units:
Seconds Seconds
Novak Expires October, 2012
2.2.5 Flow Export Rate 2.2.5 Flow Export Rate
Definition: Definition:
The number of Cache entries that expire from the Cache (as defined The number of Cache entries that expire from the Cache (as defined
by the Flow Expiration term) and are exported to the Collector by the Flow Expiration term) and are exported to the Collector
within a measurement time interval. There SHOULD NOT be any export within a measurement time interval. There SHOULD NOT be any export
filtering, so that all the expired cache entries are exported. If filtering, so that all the expired cache entries are exported. If
there is export filtering and it can't be disabled, this needs to there is export filtering and it can't be disabled, this MUST be
be noted. indicated in the measurement report.
Novak Expires September, 2012
The measured Flow Export Rate MUST include both the Data Stream The measured Flow Export Rate MUST include both the Data Stream
and the Control Information, as defined in section 2 of [RFC5470]. and the Control Information, as defined in section 2 of [RFC5470].
Discussion: Discussion:
The Flow Export Rate is measured using Flow Export data observed The Flow Export Rate is measured using Flow Export data observed
at the Collector by counting the exported Flow Records during the at the Collector by counting the exported Flow Records during the
measurement time interval (see section 5.4). The value obtained is measurement time interval (see section 5.4). The value obtained is
an average of the instantaneous export rates observed during the an average of the instantaneous export rates observed during the
measurement time interval. The smallest possible measurement measurement time interval. The smallest possible measurement
interval (if attempting to measure nearly instantaneous export interval (if attempting to measure nearly instantaneous export
skipping to change at page 6, line 48 skipping to change at page 6, line 58
The losses of Cache entries or forwarded packets in this The losses of Cache entries or forwarded packets in this
definition are assumed to happen due to the lack of DUT resources definition are assumed to happen due to the lack of DUT resources
to process any additional traffic information or lack of resources to process any additional traffic information or lack of resources
to process Flow Export data. The physical layer issues, like to process Flow Export data. The physical layer issues, like
insufficient bandwidth from the DUT to the Collector or lack of insufficient bandwidth from the DUT to the Collector or lack of
Collector resources MUST be excluded as detailed in section 4. Collector resources MUST be excluded as detailed in section 4.
3.2 Device Applicability 3.2 Device Applicability
The Flow monitoring performance metric is applicable to network The Flow monitoring performance metric is applicable to network
devices that implement [RFC5470] architecture. These devices can be devices that deploy [RFC5470] architecture. These devices can be
Novak Expires October, 2012
network packet forwarding devices or appliances which analyze the network packet forwarding devices or appliances which analyze the
traffic but do not forward traffic (probes, sniffers, replicators). traffic but do not forward traffic (probes, sniffers, replicators).
This document does not intend to measure Collector performance, it This document does not intend to measure Collector performance, it
only requires sufficient Collector resources (as specified in section only requires sufficient Collector resources (as specified in section
4.4) in order to measure the DUT characteristics. 4.4) in order to measure the DUT characteristics.
Novak Expires September, 2012
3.3 Measurement Concept 3.3 Measurement Concept
Figure 1 below presents the functional block diagram of the DUT. The Figure 1 below presents the functional block diagram of the DUT. The
traffic in the figure represents the test traffic sent to the traffic in the figure represents the test traffic sent to the
DUT and forwarded by the DUT, if possible. When testing devices which DUT and forwarded by the DUT, if possible. When testing devices which
do not act as network packet forwarding devices (such as probes, do not act as network packet forwarding devices (such as probes,
sniffers and replicators) the forwarding plane is simply an sniffers and replicators) the forwarding plane is simply an
Observation Point as defined in section 2 of [RFC5470]. The [RFC2544] Observation Point as defined in section 2 of [RFC5470]. The [RFC2544]
Throughput of such devices will always be zero and the only Throughput of such devices will always be zero and the only
applicable performance metric is the Flow Monitoring Throughput. applicable performance metric is the Flow Monitoring Throughput.
Netflow is specified by [RFC3954].
+------------------------- + +------------------------- +
| IPFIX | NetFlow | Others | | IPFIX | NetFlow | Others |
+------------------------- + +------------------------- +
| ^ | | ^ |
| Flow Export | | Flow Export |
| ^ | | ^ |
| +-------------+ | | +-------------+ |
| | Monitoring | | | | Monitoring | |
| | Plane | | | | Plane | |
skipping to change at page 7, line 45 skipping to change at page 7, line 53
+------------------------- + +------------------------- +
Figure 1. The functional block diagram of the DUT Figure 1. The functional block diagram of the DUT
Flow monitoring is represented in the figure 1 by the Monitoring Flow monitoring is represented in the figure 1 by the Monitoring
Plane. It is enabled as specified in section 4.3. It uses the Plane. It is enabled as specified in section 4.3. It uses the
traffic information provided by the Forwarding Plane and configured traffic information provided by the Forwarding Plane and configured
Flow Keys to create Cache entries representing the traffic Flow Keys to create Cache entries representing the traffic
forwarded (or observed) by the DUT in the DUT Cache. The Cache forwarded (or observed) by the DUT in the DUT Cache. The Cache
entries are expired from the Cache depending on the Cache entries are expired from the Cache depending on the Cache
configuration (ie, the Active and Inactive Timeouts, number of Cache configuration (the Active and Idle Timeouts, the Cache Size),
entries and the Cache Size) and the traffic pattern. The Cache number of Cache entries and the traffic pattern. The Cache
entries are used by the Exporting Process to format the Flow Records entries are used by the Exporting Process to format the Flow Records
which are then exported from the DUT to the Collector (see figure 2 which are then exported from the DUT to the Collector (see figure 2
in section 4). in section 4).
Novak Expires October, 2012
The Forwarding Plane and Monitoring Plane represent two separate The Forwarding Plane and Monitoring Plane represent two separate
functional blocks, each with its own performance capability. The functional blocks, each with its own performance capability. The
Forwarding Plane handles user data packets and is fully characterised Forwarding Plane handles user data packets and is fully characterized
by the metrics defined by [RFC2544]. by the metrics defined by [RFC2544].
Novak Expires September, 2012 The Monitoring Plane handles Flows which reflect the analyzed
The Monitoring Plane handles Flows which reflect the analysed
traffic. The metric for Monitoring Plane performance is Flow Export traffic. The metric for Monitoring Plane performance is Flow Export
Rate, and the benchmark is the Flow Monitoring Throughput. Rate, and the benchmark is the Flow Monitoring Throughput.
3.4 The Measurement Procedure Overview 3.4 The Measurement Procedure Overview
The measurement procedure is fully specified in sections 4, 5 and 6. The measurement procedure is fully specified in sections 4, 5 and 6.
This section provides an overview of principles for the measurements. This section provides an overview of principles for the measurements.
The basic measurement procedure of performance characteristics of a The basic measurement procedure of performance characteristics of a
DUT with Flow monitoring enabled is a conventional Throughput DUT with Flow monitoring enabled is a conventional Throughput
skipping to change at page 8, line 52 skipping to change at page 8, line 57
The exit criteria for the Flow Monitoring Throughput measurement are The exit criteria for the Flow Monitoring Throughput measurement are
one of the following (e.g. if any of the conditions is reached): one of the following (e.g. if any of the conditions is reached):
a. The Flow Export Rate at which the DUT starts to lose Flow a. The Flow Export Rate at which the DUT starts to lose Flow
information or the Flow information gets corrupted information or the Flow information gets corrupted
b. The Flow Export Rate at which the Forwarding Plane starts to drop b. The Flow Export Rate at which the Forwarding Plane starts to drop
or corrupt packets (if the Forwarding Plane is present) or corrupt packets (if the Forwarding Plane is present)
A corrupted packet here means the packet header corruption (resulting A corrupted packet here means the packet header corruption (resulting
in the cyclic redundancy check failure on the transmission level and in the cyclic redundancy check failure on the transmission level and
Novak Expires October, 2012
consequent packet drop) or the packet payload corruption leading to consequent packet drop) or the packet payload corruption leading to
the lost application level data. the lost application level data.
Novak Expires September, 2012
3.4.2 Forwarding Plane Performance Measurement 3.4.2 Forwarding Plane Performance Measurement
The Forwarding Plane (see figure 1) performance metrics are fully The Forwarding Plane (see figure 1) performance metrics are fully
specified by [RFC2544] and MUST be measured accordingly. A detailed specified by [RFC2544] and MUST be measured accordingly. A detailed
traffic analysis (see below) with relation to Flow monitoring MUST be traffic analysis (see below) with relation to Flow monitoring MUST be
performed prior of any [RFC2544] measurements. Mainly the Flow Export performed prior of any [RFC2544] measurements. Most importantly the
Rate caused by the test traffic during an [RFC2544] measurement MUST Flow Export Rate caused by the test traffic during an [RFC2544]
be known and reported. measurement MUST be known and reported.
The required test traffic analysis mainly involves the following: The required test traffic analysis mainly involves the following:
a. Which packet header parameters are incremented or changed during a. Which packet header parameters are incremented or changed during
traffic generation traffic generation
b. Which Flow Keys the Flow monitoring configuration uses to generate b. Which Flow Keys the Flow monitoring configuration uses to generate
Flow Records Flow Records
The RFC2544 performance metrics can be measured in one of the three The RFC2544 performance metrics can be measured in one of the three
modes: modes:
skipping to change at page 9, line 55 skipping to change at page 9, line 58
possible architectures and implementation of the traffic monitoring possible architectures and implementation of the traffic monitoring
appliances (see section 3.2) are too various to be covered in this appliances (see section 3.2) are too various to be covered in this
document. Instead of the Forwarding Plane, these appliances generally document. Instead of the Forwarding Plane, these appliances generally
have some kind of feed (an optical splitter, an interface sniffing have some kind of feed (an optical splitter, an interface sniffing
traffic on a shared media or an internal channel on the DUT providing traffic on a shared media or an internal channel on the DUT providing
a copy of the traffic) providing the information about the traffic a copy of the traffic) providing the information about the traffic
necessary for Flow monitoring analysis. The measurement topology then necessary for Flow monitoring analysis. The measurement topology then
needs to be adjusted to the appliance architecture, and MUST be part needs to be adjusted to the appliance architecture, and MUST be part
of the measurement report. of the measurement report.
Novak Expires October, 2012
The measurement set-up is identical to that used by [RFC2544], with The measurement set-up is identical to that used by [RFC2544], with
the addition of a Collector to analyze the Flow Export(see figure 2). the addition of a Collector to analyze the Flow Export(see figure 2).
Novak Expires September, 2012
In the measurement topology with unidirectional traffic, the traffic In the measurement topology with unidirectional traffic, the traffic
is transmitted from the sender to the receiver through the DUT. The is transmitted from the sender to the receiver through the DUT. The
received traffic is analyzed to check it is identical to the received traffic is analyzed to check it is identical to the
generated traffic. generated traffic.
The ideal way to implement the measurement is by using a single The ideal way to implement the measurement is by using a single
device to provide the sender and receiver capabilities with one device to provide the sender and receiver capabilities with one
sending port and one receiving port. This allows for an easy check sending port and one receiving port. This allows for an easy check
whether all the traffic sent by the sender was re-transmitted by the whether all the traffic sent by the sender was re-transmitted by the
DUT and received at the receiver. DUT and received at the receiver.
skipping to change at page 10, line 55 skipping to change at page 10, line 58
test traffic transferred successfully by the DUT. This may be test traffic transferred successfully by the DUT. This may be
checked through measurements with and without the DUT. checked through measurements with and without the DUT.
Note that more complex topologies might be required. For example, if Note that more complex topologies might be required. For example, if
the effects of enabling Flow monitoring on several interfaces are of the effects of enabling Flow monitoring on several interfaces are of
concern or the media maximum speed is less than the DUT throughput, concern or the media maximum speed is less than the DUT throughput,
the topology can be expanded with several input and output ports. the topology can be expanded with several input and output ports.
However, the topology MUST be clearly written in the measurement However, the topology MUST be clearly written in the measurement
report. report.
Novak Expires September, 2012 Novak Expires October, 2012
4.2 Baseline DUT Set Up 4.2 Baseline DUT Set Up
The baseline DUT set-up and the way the set-up is reported in the The baseline DUT set-up and the way the set-up is reported in the
measurement results is fully specified in section 7 of [RFC2544]. measurement results is fully specified in section 7 of [RFC2544].
The baseline DUT configuration might include other features like The baseline DUT configuration might include other features like
packet filters or quality of service on the input and/or output packet filters or quality of service on the input and/or output
interfaces if there is the need to study Flow monitoring in the interfaces if there is the need to study Flow monitoring in the
presence of those features. The Flow monitoring measurement presence of those features. The Flow monitoring measurement
procedures do not change in this case. Consideration needs to be made procedures do not change in this case. Consideration needs to be made
skipping to change at page 11, line 28 skipping to change at page 11, line 28
report. report.
The DUT export interface (see figure 2) SHOULD be configured with The DUT export interface (see figure 2) SHOULD be configured with
sufficient output buffers to avoid dropping the Flow Export data due sufficient output buffers to avoid dropping the Flow Export data due
to a simple lack of resources in the interface hardware. The applied to a simple lack of resources in the interface hardware. The applied
configuration MUST be part of the measurement report. configuration MUST be part of the measurement report.
The test designer has the freedom to run tests in multiple The test designer has the freedom to run tests in multiple
configurations. It is therefore possible to run both non-production configurations. It is therefore possible to run both non-production
and real deployment configurations in the laboratory, according to and real deployment configurations in the laboratory, according to
the needs of the tester. All configurations MUST be fully documented. the needs of the tester. All configurations MUST be part of the
measurement report.
4.3 Flow Monitoring Configuration 4.3 Flow Monitoring Configuration
This section covers all the aspects of the Flow monitoring This section covers all the aspects of the Flow monitoring
configuration necessary on the DUT in order to perform the Flow configuration necessary on the DUT in order to perform the Flow
monitoring performance measurement. The necessary configuration has monitoring performance measurement. The necessary configuration has
a number of components (see [RFC5470]), namely Observation Points, a number of components (see [RFC5470]), namely Observation Points,
Metering Process and Exporting Process as detailed below. Metering Process and Exporting Process as detailed below.
The DUT MUST support the Flow monitoring architecture as specified by The DUT MUST support the Flow monitoring architecture as specified by
[RFC5470]. The DUT SHOULD support IPFIX [RFC5101] to allow meaningful [RFC5470]. The DUT SHOULD support IPFIX [RFC5101] to allow meaningful
results comparison due to the standardized export protocol. results comparison due to the standardized export protocol.
The DUT configuration and any existing Cache MUST be erased before The DUT configuration and any existing Cache and Cache entries MUST
application of any new configuration for the currently executed be erased before application of any new configuration for the
measurement. currently executed measurement.
4.3.1 Observation Points 4.3.1 Observation Points
The Observation Points specify the interfaces and direction where the The Observation Points specify the interfaces and direction where the
Flow monitoring traffic analysis is to be performed. Flow monitoring traffic analysis is to be performed.
The (*) in Figure 2 designates the Observation Points in the default The (*) in Figure 2 designates the Observation Points in the default
configuration. Other DUT Observation Points might be configured configuration. Other DUT Observation Points might be configured
depending on the specific measurement needs as follows: depending on the specific measurement needs as follows:
Novak Expires October, 2012
a. ingress port/ports only a. ingress port/ports only
b. egress port/ports only b. egress port/ports only
c. both ingress and egress c. both ingress and egress
Novak Expires September, 2012
This test topology corresponds to unidirectional traffic only with
traffic analysis performed on the input and/or output interface.
Testing with Bidirectional traffic is discussed in Appendix B.
Generally, the placement of Observation Points depends upon the Generally, the placement of Observation Points depends upon the
position of the DUT in the deployed network and the purpose of Flow position of the DUT in the deployed network and the purpose of Flow
monitoring. See [RFC3917] for detailed discussion. The measurement monitoring. See [RFC3917] for detailed discussion. The measurement
procedures are otherwise the same for all these possible procedures are otherwise the same for all these possible
configurations. configurations.
In the case when both ingress and egress Flow monitoring is enabled In the case when both ingress and egress Flow monitoring is enabled
on one DUT the results analysis needs to take into account that each on one DUT the results analysis needs to take into account that each
Flow will be represented in the DUT Cache by two Flow Records (one Flow will be represented in the DUT Cache by two Flow Records (one
for each direction). Therefore also the Flow Export will contain for each direction). Therefore also the Flow Export will contain
skipping to change at page 12, line 34 skipping to change at page 12, line 42
The specific Observation Points and associated monitoring direction The specific Observation Points and associated monitoring direction
MUST be included as part of the measurement report. MUST be included as part of the measurement report.
4.3.2 Metering Process 4.3.2 Metering Process
The Metering Process MUST be enabled in order to create the Cache in The Metering Process MUST be enabled in order to create the Cache in
the DUT and configure the Cache related parameters. the DUT and configure the Cache related parameters.
The Cache Size available to the DUT MUST be known and taken into The Cache Size available to the DUT MUST be known and taken into
account when designing the measurement as specified in section 5. account when designing the measurement as specified in section 5.
Typically Cache Size will be present in the "show" commands of the
Flow monitoring process, in the actual configuration or in the
product documentation from the DUT vendor. The Cache Size MUST have
a fixed value for the entire duration of the measurement. This
method is not applicable to benchmarking any Flow monitoring
applications which dynamically change their Cache Size.
The configuration of the Metering Process MUST be recorded. For The configuration of the Metering Process MUST be included as part
example, when a Flow monitoring implementation uses timeouts to of the measurement report. For example, when a Flow monitoring
expire entries from the Cache, the Cache's Inactive and Active implementation uses timeouts to expire entries from the Cache, the
Timeouts MUST be known and taken into account when designing the Cache's Idle and Active Timeouts MUST be known and taken into
measurement as specified in section 5. If the Flow monitoring account when designing the measurement as specified in section 5.
implementation allows only timeouts equal to zero (e.g. immediate If the Flow monitoring implementation allows only timeouts equal to
timeout or non-existent Cache) then the measurement conditions in zero (e.g. immediate timeout or non-existent Cache) then the
section 5 are fulfilled inherently without any additional measurement conditions in section 5 are fulfilllled inherently
configuration. The DUT simply exports information about every packet
immediately, subject to the Flow Export Rate definition in section Novak Expires October, 2012
2.2.5. without any additional configuration. The DUT simply exports
information about every packet immediately, subject to the Flow
Export Rate definition in section 2.2.5.
If the Flow monitoring implementation allows configuration of If the Flow monitoring implementation allows configuration of
multiple Metering Processes on a single DUT, the exact configuration multiple Metering Processes on a single DUT, the exact configuration
of each process MUST be included in the measurement report. Only of each process MUST be included in the measurement report. Only
measurements with the same number of Metering Processes can be measurements with the same number of Metering Processes can be
compared. compared.
The Cache Size, the Inactive and Active Timeouts MUST be included in The Cache Size, the Idle and Active Timeouts MUST be included in
the measurement report. the measurement report.
Novak Expires September, 2012
4.3.3 Exporting Process 4.3.3 Exporting Process
The Exporting Process MUST be configured in order to export the Flow The Exporting Process MUST be configured in order to export the Flow
Record data to the Collector. Record data to the Collector.
The Exporting Process MUST be configured in such a way that all Flow The Exporting Process MUST be configured in such a way that all Flow
Records from all configured Observation Points are exported towards Records from all configured Observation Points are exported towards
the Collector, after the expiration policy composed of the Inactive the Collector, after the expiration policy composed of the Idle
and Active Timeouts and Cache Size. and Active Timeouts and Cache Size.
The Exporting Process SHOULD be configured with IPFIX [RFC5101] as The Exporting Process SHOULD be configured with IPFIX [RFC5101] as
the protocol to use to format the Flow Export data. If the Flow the protocol to use to format the Flow Export data. If the Flow
monitoring implementation does not support IPFIX, proprietary monitoring implementation does not support IPFIX, proprietary
protocols MAY be used. Only measurements with same export protocol protocols MAY be used. Only measurements with same export protocol
SHOULD be compared since the protocols may differ in their export SHOULD be compared since the protocols may differ in their export
efficiency. The export efficiency might also be influenced by used efficiency. The export efficiency might also be influenced by used
template layout and ordering of the individual export fields within Template Record and ordering of the individual export fields within
the template. The templates used by the tested implementations SHOULD the template. The Template Records used by the tested
be analysed and reported as part of the measurement report. Ideally implementations SHOULD be analyzed and documented as part of the
only tests with same templates layout should be compared. measurement report. Ideally only tests with same Template Records
should be compared.
Various Flow monitoring implementations might use different default Various Flow monitoring implementations might use different default
values regarding the export of Control Information [RFC5470] and values regarding the export of Control Information [RFC5470] and
therefore Flow Export corresponding to Control Information SHOULD therefore Flow Export corresponding to Control Information SHOULD
be analyzed and reported as a separate item on the measurement be analyzed and reported as a separate item on the measurement
report. The export of Control Information SHOULD always be report. The export of Control Information SHOULD always be
configured consistently across all testing and configured to the configured consistently across all testing and configured to the
minimal possible value. Ideally just one set of Control Information minimal possible value. Ideally just one set of Control Information
should be exported during each measurement. Note that Control should be exported during each measurement. Note that Control
Information includes IPFIX Options and Templates [RFC5101]. Information includes options and Template Records [RFC5470].
Section 10 of [RFC5101] and section 8.1 of [RFC5470] discuss the Section 10 of [RFC5101] and section 8.1 of [RFC5470] discuss the
possibility of deploying various transport layer protocols to deliver possibility of deploying various transport layer protocols to deliver
Flow Export data from the DUT to the Collector. The selected protocol Flow Export data from the DUT to the Collector. The selected protocol
MUST be included in the measurement report. Only benchmarks with the MUST be included in the measurement report. Only benchmarks with the
same transport layer protocol should be compared. If the Flow same transport layer protocol SHOULD be compared. If the Flow
monitoring implementation allows the use of multiple the transport monitoring implementation allows the use of multiple the transport
layer protocols, each of the protocols SHOULD be measured in a layer protocols, each of the protocols SHOULD be measured in a
separate measurement run and the results reported independently in separate measurement run and the results reported independently in
the measurement report. the measurement report.
Novak Expires October, 2012
If a reliable transport protocol is used for the transmission of If a reliable transport protocol is used for the transmission of
the Flow Export data from the DUT, the configuration of the the Flow Export data from the DUT, the configuration of the
Transport session MUST allow for non-blocking data transmission. Transport session MUST allow for non-blocking data transmission.
An example of parameters to look at would be TCP window size and An example of parameters to look at would be TCP window size and
maximum segment size (MSS). The most substantial transport layer maximum segment size (MSS). The most substantial transport layer
parameters should be included in the measurement report. parameters should be included in the measurement report.
4.3.4 Flow Records 4.3.4 Flow Records
A Flow Record contains information about a specific Flow that was A Flow Record contains information about a specific Flow that was
observed at an Observation Point. A Flow Record contains measured observed at an Observation Point. A Flow Record contains measured
properties of the Flow (e.g., the total number of bytes for all the properties of the Flow (e.g., the total number of bytes for all the
Novak Expires September, 2012
Flow packets) and usually characteristic properties of the Flow Flow packets) and usually characteristic properties of the Flow
(e.g., source IP address). (e.g., source IP address).
The Flow Record definition is implementation specific. A Flow The Flow Record definition is implementation specific. A Flow
monitoring implementation might allow for only a fixed Flow Record monitoring implementation might allow for only a fixed Flow Record
definition, based on the most common IP parameters in the IPv4 or definition, based on the most common IP parameters in the IPv4 or
IPv6 headers - for example source and destination IP addresses, IP IPv6 headers - for example source and destination IP addresses, IP
protocol numbers or transport level port numbers. Another protocol numbers or transport level port numbers. Another
implementation might allow the user to define their own arbitrary implementation might allow the user to define their own arbitrary
Flow Record to monitor the traffic. The requirement for the Flow Record to monitor the traffic. The requirement for the
measurements defined in this document is only the need for a large measurements defined in this document is only the need for a large
number of Cache entries in the Cache. The Flow Keys needed to number of Cache entries in the Cache. The Flow Keys needed to
achieve that will typically be source and destination IP addresses achieve that will typically be source and destination IP addresses
and transport level port numbers. and transport level port numbers.
The recommended full IPv4, IPv6 or MPLS Flow Record is shown The recommended full IPv4, IPv6 or MPLS Flow Record is shown
below: below. Where IP address is indicated, it means either IPv4 or IPv6
depending on the traffic type being tested. The Flow Record
configuration is Flow monitoring implementation-specific and the
examples below can not therefore provide an exact specification
of individual entries in each Flow Record. The best key/field set
to use is left to the test designer using the capabilities of the
specific Flow monitoring implementation.
Flow Keys: Flow Keys:
Source IP address Source IP address
Destination IP address Destination IP address
MPLS label (for MPLS traffic type only) MPLS label (for MPLS traffic type only)
Transport layer source port Transport layer source port
Transport layer destination port Transport layer destination port
IP protocol number (IPv6 next header) IP protocol number (IPv6 next header)
IP type of service (IPv6 traffic class) IP type of service (IPv6 traffic class)
Other fields: Other fields:
Packet counter Packet counter
Byte counter Byte counter
Table 1: Recommended Configuration Table 1: Recommended Configuration
Novak Expires October, 2012
If the Flow monitoring allows for user defined Flow Records, the If the Flow monitoring allows for user defined Flow Records, the
minimal Flow Record configurations allowing large numbers of Cache minimal Flow Record configurations allowing large numbers of Cache
entries are for example: entries are for example:
Flow Keys: Flow Keys:
Source IP address Source IP address
Destination IP address Destination IP address
Other fields: Other fields:
Packet counter Packet counter
skipping to change at page 14, line 58 skipping to change at page 15, line 25
or: or:
Flow Keys: Flow Keys:
Transport layer source port Transport layer source port
Transport layer destination port Transport layer destination port
Other fields: Other fields:
Packet counter Packet counter
Table 2: User-defined Configuration Table 2: User-defined Configuration
Novak Expires September, 2012
The Flow Record configuration MUST be clearly noted in the The Flow Record configuration MUST be clearly noted in the
measurement report. The Flow Monitoring Throughput measurements on measurement report. The Flow Monitoring Throughput measurements on
different DUTs or different Flow monitoring implementations MUST be different DUTs or different Flow monitoring implementations MUST be
compared only for exactly same Flow Record configuration. compared only for exactly same Flow Record configuration.
4.3.5 Flow Monitoring With Multiple Configurations 4.3.5 Flow Monitoring With Multiple Configurations
The Flow monitoring architecture as specified in [RFC5470] allows for The Flow monitoring architecture as specified in [RFC5470] allows for
more complicated configurations with multiple Metering and Exporting more complicated configurations with multiple Metering and Exporting
Processes on a single DUT. Depending on the particular Flow Processes on a single DUT. Depending on the particular Flow
skipping to change at page 15, line 31 skipping to change at page 15, line 53
a. Several Observation Points with a single Metering Process and a a. Several Observation Points with a single Metering Process and a
single Exporting Process single Exporting Process
b. Several Observation Points, each with one Metering Process but b. Several Observation Points, each with one Metering Process but
all using just one instance of Exporting Process all using just one instance of Exporting Process
c. Several Observation Points with per Observation Point Metering c. Several Observation Points with per Observation Point Metering
Process and Exporting Process Process and Exporting Process
4.3.6 MPLS Measurement Specifics 4.3.6 MPLS Measurement Specifics
The Flow Record configuration for measurements with MPLS encapsulated The Flow Record configuration for measurements with MPLS encapsulated
traffic SHOULD contain the MPLS label. traffic SHOULD contain the MPLS label. For this document's purposes,
"MPLS Label" is the entire 4 byte MPLS header. Typically the label of
the interest will be at the top of the label stack, but this depends
on the details of the MPLS test set-up.
Novak Expires October, 2012
The tester SHOULD ensure that the data received by the Collector The tester SHOULD ensure that the data received by the Collector
contains the expected MPLS labels. contains the expected MPLS labels.
The MPLS forwarding performance document [RFC5695] specifies a number The MPLS forwarding performance document [RFC5695] specifies a number
of possible MPLS label operations to test. The Observation Points of possible MPLS label operations to test. The Observation Points
MUST be placed on all the DUT test interfaces where the particular MUST be placed on all the DUT test interfaces where the particular
MPLS label operation takes place. The performance measurements SHOULD MPLS label operation takes place. The performance measurements SHOULD
be performed with only one MPLS label operation at the time. be performed with only one MPLS label operation at the time.
The DUT MUST be configured in such a way that all the traffic is The DUT MUST be configured in such a way that all the traffic is
skipping to change at page 15, line 57 skipping to change at page 16, line 28
The Collector is needed in order to capture the Flow Export data The Collector is needed in order to capture the Flow Export data
which allows the Flow Monitoring Throughput to be measured. which allows the Flow Monitoring Throughput to be measured.
The Collector can be used as exclusively capture device providing The Collector can be used as exclusively capture device providing
just hexadecimal format of the Flow Export data. In such a case it just hexadecimal format of the Flow Export data. In such a case it
does not need to have any additional Flow Export decoding does not need to have any additional Flow Export decoding
capabilities and all the decoding is done off line. capabilities and all the decoding is done off line.
However if the Collector is also used to decode the Flow Export data However if the Collector is also used to decode the Flow Export data
then it SHOULD support IPFIX [RFC5101] for meaningful results then it SHOULD support IPFIX [RFC5101] for meaningful results
Novak Expires September, 2012
analysis. If proprietary Flow Export is deployed, the Collector MUST analysis. If proprietary Flow Export is deployed, the Collector MUST
support it otherwise the Flow Export data analysis is not possible. support it otherwise the Flow Export data analysis is not possible.
The Collector MUST be capable of capturing the export packets sent The Collector MUST be capable of capturing the export packets sent
from the DUT at the full rate without losing any of them. In the from the DUT at the full rate without losing any of them. In the
case of the use of reliable transport protocols (see also section case of the use of reliable transport protocols (see also section
4.3.3) to transmit Flow Export data, the Collector MUST have 4.3.3) to transmit Flow Export data, the Collector MUST have
sufficient resources to guarantee non-blocking data transmission on sufficient resources to guarantee non-blocking data transmission on
the transport layer session. the transport layer session.
skipping to change at page 16, line 30 skipping to change at page 16, line 54
4.5 Sampling 4.5 Sampling
Packet sampling and flow sampling is out of scope of this document. Packet sampling and flow sampling is out of scope of this document.
This document applies to situations without packet, flow, or export This document applies to situations without packet, flow, or export
sampling. sampling.
4.6 Frame Formats 4.6 Frame Formats
Flow monitoring itself is not dependent in any way on the media used Flow monitoring itself is not dependent in any way on the media used
on the input and output ports. Any media can be used as supported by on the input and output ports. Any media can be used as supported by
the DUT and the test equipment. the DUT and the test equipment. This applies both to data forwarding
interfaces and to the export interface (see Figure 2).
Novak Expires October, 2012
At the time of writing the most common transmission media and At the time of writing the most common transmission media and
corresponding frame formats (Ethernet, Packet over SONET) for IPv4, corresponding frame formats (Ethernet, Packet over SONET) for IPv4,
IPv6 and MPLS traffic are specified within [RFC2544], [RFC5180] and IPv6 and MPLS traffic are specified within [RFC2544], [RFC5180] and
[RFC5695]. [RFC5695].
The presented frame formats MUST be recorded in the measurement The presented frame formats MUST be recorded in the measurement
report. report.
4.7 Frame Sizes 4.7 Frame Sizes
skipping to change at page 16, line 57 skipping to change at page 17, line 29
When measuring with large frame sizes, care needs to be taken to When measuring with large frame sizes, care needs to be taken to
avoid any packet fragmentation on the DUT interfaces which could avoid any packet fragmentation on the DUT interfaces which could
negatively affect measured performance values. negatively affect measured performance values.
The presented frame sizes MUST be recorded in the measurement report. The presented frame sizes MUST be recorded in the measurement report.
4.8 Flow Export Data Packet Sizes 4.8 Flow Export Data Packet Sizes
The Flow monitoring performance will be affected by the packet size The Flow monitoring performance will be affected by the packet size
the particular implementation uses to transmit Flow Export data to the particular implementation uses to transmit Flow Export data to
the Collector. The used packet size MUST be part of the measurement
Novak Expires September, 2012
the Collector. The used packet size SHOULD be part of the measurement
report and only measurements with same packet sizes SHOULD be report and only measurements with same packet sizes SHOULD be
compared. compared.
The DUT export interface (see figure 2) maximum transmission unit The DUT export interface (see figure 2) maximum transmission unit
(MTU) SHOULD be configured to the largest available value for the (MTU) SHOULD be configured to the largest available value for the
media. The Flow Export MTU MUST be recorded in the measurement media. The Flow Export MTU MUST be recorded in the measurement
report. report.
4.9 Illustrative Test Set-up Examples 4.9 Illustrative Test Set-up Examples
The below examples represent a hypothetical test set-up to clarify The below examples represent a hypothetical test set-up to clarify
the use of Flow monitoring parameters and configuration, together the use of Flow monitoring parameters and configuration, together
with traffic parameters to test Flow monitoring. The actual with traffic parameters to test Flow monitoring. The actual
benchmarking specifications are in sections 5 and 6. benchmarking specifications are in sections 5 and 6.
4.9.1 Example 1 - Inactive Timeout Flow Expiration 4.9.1 Example 1 - Idle Timeout Flow Expiration
The traffic generator sends 1000 packets per second in 10000 defined The traffic generator sends 1000 packets per second in 10000 defined
streams, each stream identified by an unique destination IP address. streams, each stream identified by an unique destination IP address.
Therefore each stream has a packet rate of 0.1 packets per second. Therefore each stream has a packet rate of 0.1 packets per second.
The packets are sent in a round robin fashion (stream 1 to 10000) The packets are sent in a round robin fashion (stream 1 to 10000)
while incrementing the destination IP address for each sent packet. while incrementing the destination IP address for each sent packet.
After a packet for stream 10000 is sent, the next packet destination
IP address corresponds to stream 1's address again.
The configured Cache Size is 20000 Flow Records. The configured The configured Cache Size is 20000 Flow Records. The configured
Active Timeout is 100 seconds, the Inactive Timeout is 5 seconds. Active Timeout is 100 seconds, the Idle Timeout is 5 seconds.
Novak Expires October, 2012
Flow monitoring on the DUT uses the destination IP address as the Flow monitoring on the DUT uses the destination IP address as the
Flow Key. Flow Key.
A packet with destination IP address equal to A is sent every 10 A packet with destination IP address equal to A is sent every 10
seconds, so the Cache entry would be refreshed in the Cache every 10 seconds, so the Cache entry would be refreshed in the Cache every 10
seconds. However, the Inactive Timeout is 5 seconds, so the Cache seconds. However, the Idle Timeout is 5 seconds, so the Cache
entries will expire from the Cache due to the Inactive Timeout and entries will expire from the Cache due to the Idle Timeout and
when a new packet is sent with the same IP address A it will create a when a new packet is sent with the same IP address A it will create a
new entry in the Cache. This behaviour depends upon the design an new entry in the Cache. This behavior depends upon the design an
efficiency of the cache ager, and incidences of multi-packet flows efficiency of the cache ager, and incidences of multi-packet flows
observed during this test should be noted. observed during this test should be noted.
The measured Flow Export Rate in this case will be 1000 Flow The measured Flow Export Rate in this case will be 1000 Flow
Records per second since every single sent packet will always Records per second since every single sent packet will always
create a new Cache entry and 1000 packets per second is sent. create a new Cache entry and 1000 packets per second is sent.
The expected number of Cache entries in the Cache during the whole The expected number of Cache entries in the Cache during the whole
measurement is around 5000. It corresponds to the Inactive Timeout measurement is around 5000. It corresponds to the Idle Timeout
being 5 seconds and during those five seconds 5000 entries are being 5 seconds and during those five seconds 5000 entries are
created. This expectation might change in real measurement set-ups created. This expectation might change in real measurement set-ups
with large Cache Sizes and high packet rate where the DUT's actual with large Cache Sizes and high packet rate where the DUT's actual
export rate might be limited and lower than the Flow Expiration export rate might be limited and lower than the Flow Expiration
activity caused by the traffic offered to the DUT. This behaviour is activity caused by the traffic offered to the DUT. This behavior is
entirely implementation specific. entirely implementation specific.
Novak Expires September, 2012
4.9.2 Example 2 - Active Timeout Flow Expiration 4.9.2 Example 2 - Active Timeout Flow Expiration
The traffic generator sends 1000 packets per second in 100 defined The traffic generator sends 1000 packets per second in 100 defined
streams, each stream identified by an unique destination IP address. streams, each stream identified by an unique destination IP address.
Each stream has a packet rate of 10 packets per second. The packets Each stream has a packet rate of 10 packets per second. The packets
are sent in a round robin fashion (stream 1 to 100) while are sent in a round robin fashion (stream 1 to 100) while
incrementing the destination IP address for each sent packet. incrementing the destination IP address for each sent packet. After
a packet for stream 100 is sent, the next packet destination IP
address corresponds to stream 1's address again.
The configured Cache Size is 1000 Flow Records. The configured The configured Cache Size is 1000 Flow Records. The configured
Active Timeout is 100 seconds. The Inactive Timeout is 10 seconds. Active Timeout is 100 seconds. The Idle Timeout is 10 seconds.
Flow monitoring on the DUT uses the destination IP address as the Flow monitoring on the DUT uses the destination IP address as the
Flow Key. Flow Key.
After the first 100 packets are sent, 100 Cache entries will have After the first 100 packets are sent, 100 Cache entries will have
been created in the Flow monitoring Cache. The subsequent packets been created in the Flow monitoring Cache. The subsequent packets
will be counted against the already created Cache entries since the will be counted against the already created Cache entries since the
destination IP address (Flow Key) has already been seen by the DUT destination IP address (Flow Key) has already been seen by the DUT
(provided the Cache entries did not expire yet as described below). (provided the Cache entries did not expire yet as described below).
A packet with destination IP address equal to A is sent every 0.1 A packet with destination IP address equal to A is sent every 0.1
second, so the Cache entry is refreshed in the Cache every 0.1 second, so the Cache entry is refreshed in the Cache every 0.1
second, while the Inactive Timeout is 10 seconds. In this case the second, while the Idle Timeout is 10 seconds. In this case the
Cache entries will not expire until the Active Timeout, e.g. they Cache entries will not expire until the Active Timeout, e.g. they
will expire every 100 seconds and then the Cache entries will be will expire every 100 seconds and then the Cache entries will be
created again. created again.
Novak Expires October, 2012
If the test measurement time is 50 seconds from the start of the If the test measurement time is 50 seconds from the start of the
traffic generator then the measured Flow Export Rate is 0 since traffic generator then the measured Flow Export Rate is 0 since
during this period nothing expired from the Cache. during this period nothing expired from the Cache.
If the test measurement time is 100 seconds from the start of the If the test measurement time is 100 seconds from the start of the
traffic generator then the measured Flow Export Rate is 1 Flow Record traffic generator then the measured Flow Export Rate is 1 Flow Record
per second. per second.
If the test measurement time is 290 seconds from the start of the If the test measurement time is 290 seconds from the start of the
traffic generator then the measured Flow Export Rate is 2/3 of Flow traffic generator then the measured Flow Export Rate is 2/3 of Flow
skipping to change at page 18, line 55 skipping to change at page 19, line 28
Objective: Objective:
To measure the Flow monitoring performance in a manner comparable To measure the Flow monitoring performance in a manner comparable
between different Flow monitoring implementations. between different Flow monitoring implementations.
Metric definition: Metric definition:
Flow Monitoring Throughput - see section 3. Flow Monitoring Throughput - see section 3.
Novak Expires September, 2012
Discussion: Discussion:
Different Flow monitoring implementations might chose to handle Different Flow monitoring implementations might chose to handle
Flow Export from a partially empty Cache differently than in the Flow Export from a partially empty Cache differently than in the
case when the Cache is fully occupied. Similarly software and case when the Cache is fully occupied. Similarly software and
hardware based DUTs can handle the same situation as stated above hardware based DUTs can handle the same situation as stated above
differently. The purpose of the benchmark measurement in this differently. The purpose of the benchmark measurement in this
section is to abstract from all the possible behaviours and define section is to abstract from all the possible behaviors and define
one measurement procedure covering all the possibilities. The only one measurement procedure covering all the possibilities. The only
criteria is to measure as defined here until Flow Record or packet criteria is to measure as defined here until Flow Record or packet
losses are seen. The decision whether to dive deeper into the losses are seen. The decision whether to dive deeper into the
conditions under which the packet losses happen is left to the conditions under which the packet losses happen is left to the
tester. tester.
5.1 Flow Monitoring Configuration 5.1 Flow Monitoring Configuration
Cache Size Cache Size
Cache Size configuration is dictated by the expected position of Cache Size configuration is dictated by the expected position of
the DUT in the network and by the chosen Flow Keys of the Flow the DUT in the network and by the chosen Flow Keys of the Flow
Record. The number of unique Flow Keys sets that the traffic Record. The number of unique Flow Keys sets that the traffic
generator (sender) provides should be multiple times larger than generator (sender) provides should be multiple times larger than
the Cache Size. This ensures that the existing Cache entries are the Cache Size. This ensures that the existing Cache entries are
never updated before Flow Expiration and Flow Export. The Cache never updated by a packet from the sender before the particular
Size MUST be known in order to define the measurement Flow Expiration and Flow Export. This condition is simple to
circumstances properly. fullfill with linearly incremented Flow Keys (for example IP
addresses or transport layer ports) where the range of values
must be larger than Cache Size. When randomized traffic
generation is in use the generator must ensure that same Flow Keys
are not repeated within a range of randomly generated values.
Inactive Timeout Novak Expires October, 2012
Inactive Timeout is set (if configurable) to the minimum possible The Cache Size MUST be known in order to define the measurement
circumstances properly. Typically Cache Size will be found using
the "show" commands of the Flow monitoring implementation, in the
actual configuration, or in the product documentation from the
vendor.
Idle Timeout
Idle Timeout is set (if configurable) to the minimum possible
value on the DUT. This ensures that the Cache entries are expired value on the DUT. This ensures that the Cache entries are expired
as soon as possible and exported out of the DUT Cache. It MUST be as soon as possible and exported out of the DUT Cache. It MUST be
known in order to define the measurement circumstances completely known in order to define the measurement circumstances completely
and equally across implementations. and equally across implementations.
Active Timeout Active Timeout
Active Timeout is set (if configurable) to a value equal to or Active Timeout is set (if configurable) to a value equal to or
higher than the Inactive Timeout. It MUST be known in order to higher than the Idle Timeout. It MUST be known in order to
define the measurement circumstances completely and equally define the measurement circumstances completely and equally
across implementations. across implementations.
Flow Keys Definition: Flow Keys Definition:
The test needs large numbers of unique Cache entries to be created The test needs large numbers of unique Cache entries to be created
by incrementing values of one or several Flow Keys. The number of by incrementing values of one or several Flow Keys. The number of
unique combinations of Flow Keys values SHOULD be several times unique combinations of Flow Keys values SHOULD be several times
larger than the DUT Cache Size. This makes sure that any incoming larger than the DUT Cache Size. This makes sure that any incoming
packet will never refresh any already existing Cache entry. packet will never refresh any already existing Cache entry.
The availability of Cache Size, Inactive Timeout, Active Timeout as The availability of Cache Size, Idle Timeout, Active Timeout as
configuration parameters is implementation specific. If the Flow configuration parameters is implementation specific. If the Flow
monitoring implementation does not support these parameters, the test monitoring implementation does not support these parameters, the test
possibilities as specified by this document are restricted. Some possibilities as specified by this document are restricted. Some
testing might be viable if the implementation follows the testing might be viable if the implementation follows the
[IPFIX-CONFIG] document and needs to be considered on the case by [IPFIX-CONFIG] document and needs to be considered on the case by
by case basis. by case basis.
Novak Expires September, 2012
5.2 Traffic Configuration 5.2 Traffic Configuration
Traffic Generation Traffic Generation
The traffic generator needs to increment the Flow Keys values with The traffic generator needs to increment the Flow Keys values with
each sent packet. This way each packet represents one Cache entry each sent packet. This way each packet represents one Cache entry
in the DUT Cache. in the DUT Cache.
If the test traffic rate is below the maximum media rate for A particular Flow monitoring implementation might choose to deploy
the particular packet size the traffic generator MUST send the a hashing mechanism to match incoming data packets to certain Flow.
packets in equidistant time intervals. Traffic generators which do In such a case the combination of how the traffic is constructed
not fulfil this condition MUST NOT and cannot be used for the Flow and the hashing might influence the DUT Flow monitoring
Monitoring Throughput measurement. An example of this behaviour is performance. For example, if IP addresses are used as Flow Keys
if the test traffic rate is one half of the media rate and the this means there could be a performance difference for linearly
traffic generator achieves this by sending each half of the second incremented addresses (in ascending or descending order) as opposed
at the full media rate and then sending nothing for the second to IP addresses randomized in certain range. If randomized IP
half of the second. In such conditions it would be impossible to address sequences are used, then the traffic generator needs to be
distinguish if the DUT failed to handle the Flows due to the input able to reproduce the randomization (e.g. same set of IP addresses
buffers shortage during the burst or due to the limits in the Flow sent in same order in different test runs) in order to compare
Monitoring performance. various DUTs and Flow monitoring implementations.
Novak Expires October, 2012
If the test traffic rate is below the maximum media rate for
the particular packet size the traffic generator MUST send the
packets in equidistant time intervals. Traffic generators which do
not fulfilll this condition MUST NOT and cannot be used for the
Flow Monitoring Throughput measurement. An example of this behavior
is if the test traffic rate is one half of the media rate and the
traffic generator achieves this by sending each half of the second
at the full media rate and then sending nothing for the second
half of the second. In such conditions it would be impossible to
distinguish if the DUT failed to handle the Flows due to the input
buffers shortage during the burst or due to the limits in the Flow
Monitoring performance.
Measurement Duration Measurement Duration
The measurement duration (e.g. how long the test traffic is sent The measurement duration (e.g. how long the test traffic is sent
to the DUT) MUST be at least two times longer than the Inactive to the DUT) MUST be at least two times longer than the Idle
Timeout otherwise no Flow Export would be seen. The measurement Timeout otherwise no Flow Export would be seen. The measurement
duration SHOULD guarantee that the number of Cache entries created duration SHOULD guarantee that the number of Cache entries created
during the measurement exceeds the available Cache Size. during the measurement exceeds the available Cache Size.
5.3 Cache Population 5.3 Cache Population
The product of Inactive Timeout and the packet rate offered to the The product of Idle Timeout and the packet rate offered to the
DUT (cache population) during one measurement determines the total DUT (cache population) during one measurement determines the total
number of Cache entries in the DUT Cache during the measurement number of Cache entries in the DUT Cache during the measurement
(while taking into account some margin for dynamic behaviour during (while taking into account some margin for dynamic behavior during
high DUT loads when processing the Flows). high DUT loads when processing the Flows).
The Flow monitoring implementation might behave differently depending The Flow monitoring implementation might behave differently depending
on the relation of cache population to the available Cache Size on the relation of cache population to the available Cache Size
during the measurement. This behaviour is fully implementation during the measurement. This behavior is fully implementation
specific and will also be influenced if the DUT is software based or specific and will also be influenced if the DUT is software based or
hardware based architecture. hardware based architecture.
The cache population (if it is lower or higher than the available The cache population (if it is lower or higher than the available
Cache Size) during a particular benchmark measurement SHOULD be Cache Size) during a particular benchmark measurement SHOULD be
noted and mainly only measurements with same cache population SHOULD noted and mainly only measurements with same cache population SHOULD
be compared. be compared.
5.4 Measurement Time Interval 5.4 Measurement Time Interval
The measurement time interval is the time value which is used to The measurement time interval is the time value which is used to
calculate the measured Flow Export Rate from the captured Flow Export calculate the measured Flow Export Rate from the captured Flow Export
data. It is obtained as specified below. data. It is obtained as specified below.
RFC2544 specifies with the precision of the packet beginning and end RFC2544 specifies with the precision of the packet beginning and end
Novak Expires September, 2012
the time intervals to be used to measure the DUT time the time intervals to be used to measure the DUT time
characteristics. In the case of a Flow Monitoring Throughput characteristics. In the case of a Flow Monitoring Throughput
measurement the start and stop time needs to be clearly defined but measurement the start and stop time needs to be clearly defined but
the granularity of this definition can be limited to just marking the the granularity of this definition can be limited to just marking the
start and stop time with the start and stop of the traffic generator. start and stop time with the start and stop of the traffic generator.
This assumes that the traffic generator and DUT are collocated and This assumes that the traffic generator and DUT are collocated and
the variance in transmission delay from the generator to the DUT is the variance in transmission delay from the generator to the DUT is
Novak Expires October, 2012
negligible as compared to the total time of traffic generation. negligible as compared to the total time of traffic generation.
The measurement start time: the time when the traffic generator is The measurement start time: the time when the traffic generator is
started started
The measurement stop time: the time when the traffic generator is The measurement stop time: the time when the traffic generator is
stopped stopped
The measurement time interval is then calculated as the difference The measurement time interval is then calculated as the difference
(stop time) - (start time) - (Inactive Timeout). (stop time) - (start time) - (Idle Timeout).
This supposes that the Cache Size is large enough so that the time to This supposes that the Cache Size is large enough so that the time to
fill it up with Cache entries is longer than Inactive Timeout. fill it up with Cache entries is longer than Idle Timeout.
Otherwise the time to fill up the Cache needs to be used for Otherwise the time to fill up the Cache needs to be used for
calculation of the measurement time interval in the place of the calculation of the measurement time interval in the place of the
Inactive Timeout. Idle Timeout.
Instead of measuring the absolute values of stop and start time it is Instead of measuring the absolute values of stop and start time it is
possible to setup the traffic generator to send traffic for a certain possible to setup the traffic generator to send traffic for a certain
pre-defined time interval which is then used in the above definition pre-defined time interval which is then used in the above definition
instead of the difference (stop time) - (start time). instead of the difference (stop time) - (start time).
The Collector MUST stop collecting the Flow Export data at the The Collector MUST stop collecting the Flow Export data at the
measurement stop time. measurement stop time.
The Inactive Timeout (or the time needed to fill up the Cache) causes The Idle Timeout (or the time needed to fill up the Cache) causes
delay of the Flow Export data behind the test traffic which is delay of the Flow Export data behind the test traffic which is
analysed by the DUT. E.g. if the traffic starts at time point X Flow analyzed by the DUT. E.g. if the traffic starts at time point X Flow
Export will start only at the time point X + Inactive Timeout (or X + Export will start only at the time point X + Idle Timeout (or X +
time to fill up the Cache). Since Flow Export capture needs to stop time to fill up the Cache). Since Flow Export capture needs to stop
with the traffic (because that's when the DUT stops processing the with the traffic (because that's when the DUT stops processing the
Flows at the given rate) the time interval during which the DUT kept Flows at the given rate) the time interval during which the DUT kept
exporting data is shorter by the Inactive Timeout than the Time exporting data is shorter by the Idle Timeout than the Time
interval when the test traffic was sent from the traffic generator to interval when the test traffic was sent from the traffic generator to
the DUT. the DUT.
5.5 Flow Export Rate Measurement 5.5 Flow Export Rate Measurement
The Flow Export Rate needs to be measured in two consequent steps. The Flow Export Rate needs to be measured in two consequent steps.
The purpose of the first step (point a. below) is to gain the actual The purpose of the first step (point a. below) is to gain the actual
value for the rate, the second step (point b. below) needs to be done value for the rate, the second step (point b. below) needs to be done
in order to verify Flow Record drops during the measurement: in order to verify Flow Record drops during the measurement:
a. In the first step the captured Flow Export data MUST be analyzed a. In the first step the captured Flow Export data MUST be analyzed
only for the capturing interval (measurement time interval) as only for the capturing interval (measurement time interval) as
specified in section 5.4. During this period the DUT is forced to specified in section 5.4. During this period the DUT is forced to
Novak Expires September, 2012
process Cache entries at the rate the packets are sent. When process Cache entries at the rate the packets are sent. When
traffic generation finishes, the behaviour when emptying the Cache traffic generation finishes, the behavior when emptying the Cache
is completely implementation specific and the Flow Export data is completely implementation specific and the Flow Export data
from this period cannot be therefore used for the benchmarking. from this period cannot be therefore used for the benchmarking.
b. In the second step all the Flow Export data from the DUT MUST be b. In the second step all the Flow Export data from the DUT MUST be
captured in order to be capable to determine the Flow Record captured in order to be capable to determine the Flow Record
Novak Expires October, 2012
losses. It needs to be taken into account that especially when losses. It needs to be taken into account that especially when
large Cache Sizes (in order of magnitude of hundreds of thousands large Cache Sizes (in order of magnitude of hundreds of thousands
of entries and higher) are in use the Flow Export can take many of entries and higher) are in use the Flow Export can take many
multiples of Inactive Timeout to empty the Cache after the multiples of Idle Timeout to empty the Cache after the
measurement. This behaviour is completely implementation specific. measurement. This behavior is completely implementation specific.
If the Collector has the capability to redirect the Flow Export data If the Collector has the capability to redirect the Flow Export data
after the measurement time interval into different capture buffer after the measurement time interval into different capture buffer
(or time stamp the received Flow Export data after that) this can be (or time stamp the received Flow Export data after that) this can be
done in one step. Otherwise each Flow Monitoring Throughput done in one step. Otherwise each Flow Monitoring Throughput
measurement at certain packet rate needs to be executed twice - once measurement at certain packet rate needs to be executed twice - once
to capture the Flow Export data just for the measurement time to capture the Flow Export data just for the measurement time
interval (to determine the actual Flow Export Rate) and second time interval (to determine the actual Flow Export Rate) and second time
to capture all Flow Export data in order to determine Flow Record to capture all Flow Export data in order to determine Flow Record
losses at that packet rate. losses at that packet rate.
skipping to change at page 22, line 58 skipping to change at page 23, line 52
The Flow Export performance can be significantly affected by the way The Flow Export performance can be significantly affected by the way
the Flow monitoring implementation formats the Flow Records into the the Flow monitoring implementation formats the Flow Records into the
Flow Export packets. The ordering and frequency of Control Information Flow Export packets. The ordering and frequency of Control Information
export and mainly the number of Flow Records in one Flow Export packet export and mainly the number of Flow Records in one Flow Export packet
is of interest. The worst case scenario here is just one Flow Record is of interest. The worst case scenario here is just one Flow Record
in every Flow Export packet. in every Flow Export packet.
Flow Export data should be sanity checked during the benchmark Flow Export data should be sanity checked during the benchmark
measurement for: measurement for:
Novak Expires September, 2012
a. the number of Flow Records per packet, by simply calculating the a. the number of Flow Records per packet, by simply calculating the
ratio of exported Flow Records to the number of Flow Export ratio of exported Flow Records to the number of Flow Export
packets captured during the measurement (which should be available packets captured during the measurement (which should be available
as a counter on the Collector capture buffer) as a counter on the Collector capture buffer)
b. the number of Flow Records corresponding to the export of Control b. the number of Flow Records corresponding to the export of Control
Novak Expires October, 2012
Information per Flow Export packet (calculated as the ratio of the Information per Flow Export packet (calculated as the ratio of the
total number of such Flow Records in the Flow Export data and the total number of such Flow Records in the Flow Export data and the
number of Flow Export packets). number of Flow Export packets).
6. RFC2544 Measurements 6. RFC2544 Measurements
RFC2544 measurements can be performed under two Flow Monitoring set- RFC2544 measurements can be performed under two Flow Monitoring set-
ups (see also section 3.4.2). This section details both of them and ups (see also section 3.4.2). This section details both of them and
specifies ways to construct the test traffic so that RFC2544 specifies ways to construct the test traffic so that RFC2544
measurements can be performed in a controlled environment from the measurements can be performed in a controlled environment from the
skipping to change at page 23, line 57 skipping to change at page 24, line 52
transmitted and therefore the RFC2544 Throughput in most cases will transmitted and therefore the RFC2544 Throughput in most cases will
be much lower when Flow monitoring is enabled on the DUT than without be much lower when Flow monitoring is enabled on the DUT than without
it. it.
6.1 Flow Monitoring Configuration 6.1 Flow Monitoring Configuration
Flow monitoring configuration (as detailed in section 4.3) needs Flow monitoring configuration (as detailed in section 4.3) needs
to be applied the same way as discussed in section 5 with the to be applied the same way as discussed in section 5 with the
exception of the Active Timeout configuration. exception of the Active Timeout configuration.
Novak Expires September, 2012
The Active Timeout SHOULD be configured to exceed several times the The Active Timeout SHOULD be configured to exceed several times the
measurement time interval (see section 5.4). This makes sure that if measurement time interval (see section 5.4). This makes sure that if
measurements with two traffic components are performed (see section measurements with two traffic components are performed (see section
6.3.2) there is no Flow monitoring activity related to the second 6.3.2) there is no Flow monitoring activity related to the second
traffic component. traffic component.
Novak Expires October, 2012
The Flow monitoring configuration does not change in any other way The Flow monitoring configuration does not change in any other way
for the measurement performed in this section. What changes and makes for the measurement performed in this section. What changes and makes
the difference is the traffic configurations as specified in the the difference is the traffic configurations as specified in the
sections below. sections below.
6.2 Measurements with the Flow Monitoring Throughput Set-up 6.2 Measurements with the Flow Monitoring Throughput Set-up
The major requirement to perform a measurement with Flow Monitoring The major requirement to perform a measurement with Flow Monitoring
Throughput set-up is that the traffic and Flow monitoring is Throughput set-up is that the traffic and Flow monitoring is
configured in such a way that each sent packet creates one entry in configured in such a way that each sent packet creates one entry in
skipping to change at page 24, line 57 skipping to change at page 25, line 51
starting at 1 and incremented in small steps) packets with source starting at 1 and incremented in small steps) packets with source
IP address A and destination IP address B before changing both IP IP address A and destination IP address B before changing both IP
addresses to the next value addresses to the next value
This test traffic definition allows execution of the Flow monitoring This test traffic definition allows execution of the Flow monitoring
measurements with fixed Flow Export Rate while measuring the DUT measurements with fixed Flow Export Rate while measuring the DUT
RFC2544 characteristics. This set-up is the better option since it RFC2544 characteristics. This set-up is the better option since it
best simulates the live network traffic scenario with Flows best simulates the live network traffic scenario with Flows
containing more than just one packet. containing more than just one packet.
Novak Expires September, 2012
The initial packet rate at N equal to 1 defines the Flow Export Rate The initial packet rate at N equal to 1 defines the Flow Export Rate
for the whole measurement procedure. Subsequent increases of N will for the whole measurement procedure. Subsequent increases of N will
not change the Flow Export Rate as the time and Cache not change the Flow Export Rate as the time and Cache
characteristics of the test traffic stay the same. This set-up is characteristics of the test traffic stay the same. This set-up is
suitable for measurements with Flow Export Rates below the Flow suitable for measurements with Flow Export Rates below the Flow
Monitoring Throughput. Monitoring Throughput.
Novak Expires October, 2012
6.3.2 Measurements With Two Traffic Components 6.3.2 Measurements With Two Traffic Components
The test traffic set-up in section 6.3.1 might be difficult to The test traffic set-up in section 6.3.1 might be difficult to
achieve with commercial traffic generators or the granularity of the achieve with commercial traffic generators or the granularity of the
traffic rates as defined by the initial packet rate at N equal to 1 traffic rates as defined by the initial packet rate at N equal to 1
might not be suitable for the required measurement. An alternative might not be suitable for the required measurement. An alternative
mechanism is to define two traffic components in the test traffic. mechanism is to define two traffic components in the test traffic.
One to populate Flow monitoring Cache and the second one to execute One to populate Flow monitoring Cache and the second one to execute
the RFC2544 measurements. the RFC2544 measurements.
skipping to change at page 25, line 57 skipping to change at page 26, line 50
count in the exported data) and each set of parameters from each Flow count in the exported data) and each set of parameters from each Flow
Record can be checked against the parameters as configured on the Record can be checked against the parameters as configured on the
traffic generator and set in packets sent to the DUT. The exported traffic generator and set in packets sent to the DUT. The exported
Flow Record is considered accurate if: Flow Record is considered accurate if:
a. all the Flow Record fields are present in each exported Flow a. all the Flow Record fields are present in each exported Flow
Record Record
b. all the Flow Record fields values match the value ranges as set by b. all the Flow Record fields values match the value ranges as set by
the traffic generator (for example an IP address falls within the the traffic generator (for example an IP address falls within the
range of the IP addresses increments on the traffic generator) range of the IP addresses increments on the traffic generator)
Novak Expires September, 2012
c. all the possible Flow Record field values as defined at the c. all the possible Flow Record field values as defined at the
traffic generator have been found in the captured export data on traffic generator have been found in the captured export data on
the Collector. This check needs to be offset against detected the Collector. This check needs to be offset against detected
packet losses at the DUT during the measurement packet losses at the DUT during the measurement
For a DUT with packet forwarding, the Flow monitoring accuracy also
involves data checks on the received traffic, as already discussed
in section 4.
Novak Expires October, 2012
8. Evaluating Flow Monitoring Applicability 8. Evaluating Flow Monitoring Applicability
The measurement results as discussed in this document and obtained The measurement results as discussed in this document and obtained
for certain DUTs allow for a preliminary analysis of a Flow for certain DUTs allow for a preliminary analysis of a Flow
monitoring deployment based on the traffic analysis data from the monitoring deployment based on the traffic analysis data from the
providers network. providers network.
An example of such traffic analysis in the Internet is provided by An example of such traffic analysis in the Internet is provided by
[CAIDA] and the way it can be used is discussed below. The data [CAIDA] and the way it can be used is discussed below. The data
needed to make an estimate if a certain network device can manage the needed to make an estimate if a certain network device can manage the
particular amount of live traffic with Flow monitoring enabled is: particular amount of live traffic with Flow monitoring enabled is:
skipping to change at page 26, line 55 skipping to change at page 27, line 50
This work could have been performed thanks to the patience and This work could have been performed thanks to the patience and
support of Cisco Systems NetFlow development team, namely Paul support of Cisco Systems NetFlow development team, namely Paul
Aitken, Paul Atkins and Andrew Johnson. Thanks belong to Benoit Aitken, Paul Atkins and Andrew Johnson. Thanks belong to Benoit
Claise for numerous detailed reviews and presentations of the Claise for numerous detailed reviews and presentations of the
document and Aamer Akhter for initiating this work. A special document and Aamer Akhter for initiating this work. A special
acknowledgment needs to go to the whole of the working group and acknowledgment needs to go to the whole of the working group and
especially to the chair Al Morton for the support and work on especially to the chair Al Morton for the support and work on
this draft and Paul Aitken for a very detailed technical review. this draft and Paul Aitken for a very detailed technical review.
Novak Expires September, 2012
10. Security Considerations 10. Security Considerations
Documents of this type do not directly affect the security of Documents of this type do not directly affect the security of
the Internet or corporate networks as long as benchmarking the Internet or corporate networks as long as benchmarking
is not performed on devices or systems connected to operating is not performed on devices or systems connected to operating
networks. networks.
Novak Expires October, 2012
Benchmarking activities as described in this memo are limited to Benchmarking activities as described in this memo are limited to
technology characterization using controlled stimuli in a laboratory technology characterization using controlled stimuli in a laboratory
environment, with dedicated address space and the constraints environment, with dedicated address space and the constraints
specified in sections above. specified in sections above.
The benchmarking network topology will be an independent test setup The benchmarking network topology will be an independent test setup
and MUST NOT be connected to devices that may forward the test and MUST NOT be connected to devices that may forward the test
traffic into a production network, or misroute traffic to the test traffic into a production network, or misroute traffic to the test
management network. management network.
skipping to change at page 27, line 56 skipping to change at page 28, line 49
Interconnection Devices", RFC 1242, July 1991 Interconnection Devices", RFC 1242, July 1991
[RFC2285] Mandeville R., "Benchmarking Terminology for LAN Switching [RFC2285] Mandeville R., "Benchmarking Terminology for LAN Switching
Devices", Informational, RFC 2285, November 1998 Devices", Informational, RFC 2285, November 1998
[RFC3031] E. Rosen, A. Viswanathan, R. Callon, "Multiprotocol Label [RFC3031] E. Rosen, A. Viswanathan, R. Callon, "Multiprotocol Label
Switching Architecture", Standards Track, RFC 3031, Switching Architecture", Standards Track, RFC 3031,
January 2001 January 2001
[RFC3917] Quittek J., "Requirements for IP Flow Information Export [RFC3917] Quittek J., "Requirements for IP Flow Information Export
(IPFIX)", Informational, RFC 3917, October 2004. (IPFIX)", Informational, RFC 3917, October 2004
Novak Expires September, 2012 [RFC3954] Claise B., "Cisco Systems NetFlow Services Export
Version 9", Informational, RFC3954, October 2004
[RFC5101] Claise B., "Specification of the IP Flow Information [RFC5101] Claise B., "Specification of the IP Flow Information
Export (IPFIX) Protocol for the Exchange of IP Traffic Export (IPFIX) Protocol for the Exchange of IP Traffic
Flow Information", Standards Track, RFC 5101, January 2008 Flow Information", Standards Track, RFC 5101, January 2008
Novak Expires October, 2012
[RFC5180] C. Popoviciu, A. Hamza, D. Dugatkin, G. Van de Velde, [RFC5180] C. Popoviciu, A. Hamza, D. Dugatkin, G. Van de Velde,
"IPv6 Benchmarking Methodology for Network Interconnect "IPv6 Benchmarking Methodology for Network Interconnect
Devices", Informational, RFC 5180, May 2008 Devices", Informational, RFC 5180, May 2008
[RFC5470] Sadasivan, G., Brownlee, N., Claise, B., and J. Quittek, [RFC5470] Sadasivan, G., Brownlee, N., Claise, B., and J. Quittek,
"Architecture Model for IP Flow Information Export", "Architecture Model for IP Flow Information Export",
RFC 5470, October 2011 RFC 5470, October 2011
[RFC5695] Akhter A. "MPLS Forwarding Benchmarking Methodology", [RFC5695] Akhter A. "MPLS Forwarding Benchmarking Methodology",
RFC 5695, November 2009 RFC 5695, November 2009
[CAIDA] Claffy, K., "The nature of the beast: recent traffic [CAIDA] Claffy, K., "The nature of the beast: recent traffic
measurements from an Internet backbone", measurements from an Internet backbone",
http://www.caida.org/publications/papers/1998/Inet98/ http://www.caida.org/publications/papers/1998/Inet98/
Inet98.html Inet98.html
[IPFIX-CONFIG] Configuration Data Model for IPFIX and PSAMP, G. Muenz [IPFIX-CONFIG] Configuration Data Model for IPFIX and PSAMP, G. Muenz
et al, Work in Progress, et al, Work in Progress,
draft-ietf-ipfix-configuration-model-10 draft-ietf-ipfix-configuration-model-10
[PSAMP-MIB] Dietz, T., Claise, B., and J. Quittek, "Definitions of
Managed Objects for Packet Sampling",
draft-ietf-ipfix-psamp-mib-04 (work in progress),
October 2011
[IPFIX-MIB] Dietz, T., A. Kobayashi, Claise, B., and G. Muenz,
"Definitions of Managed Objects for IP Flow Information
Export",
draft-ietf-ipfix-rfc5815bis-03.txt (work in progress),
April 2012
Author's Addresses Author's Addresses
Jan Novak (editor) Jan Novak (editor)
Cisco Systems Cisco Systems
Edinburgh, Edinburgh,
United Kingdom United Kingdom
Email: janovak@cisco.com Email: janovak@cisco.com
Novak Expires September, 2012 Novak Expires October, 2012
Appendix A: Recommended Report Format Appendix A: (Informative) Recommended Report Format
Parameter Units Parameter Units
----------------------------------- ------------------------------------ ----------------------------------- ------------------------------------
Test Case test case name (section 5 and 6) Test Case test case name (section 5 and 6)
Test Topology Figure 2, other Test Topology Figure 2, other
Traffic Type IPv4, IPv6, MPLS, other Traffic Type IPv4, IPv6, MPLS, other
Test Results Test Results
Flow Monitoring Throughput Flow Records per second or Not Flow Monitoring Throughput Flow Records per second or Not
Applicable Applicable
Flow Export Rate Flow Records per second or Not Flow Export Rate Flow Records per second or Not
Applicable Applicable
Control Information Export Rate Flow Records per second Control Information Export Rate Flow Records per second
RFC2544 Throughput packets per second RFC2544 Throughput packets per second
(Other RFC2544 Metrics) (as appropriate) (Other RFC2544 Metrics) (as appropriate)
General Parameters General Parameters
Traffic Direction unidirectional, bidirectional
DUT Interface Type Ethernet, POS, ATM, other DUT Interface Type Ethernet, POS, ATM, other
DUT Interface Bandwidth MegaBits per second DUT Interface Bandwidth MegaBits per second
Traffic Specifications Traffic Specifications
Number of Traffic Components (see section 6.3.1 and 6.3.2) Number of Traffic Components (see section 6.3.1 and 6.3.2)
For each traffic component: For each traffic component:
Packet Size bytes Packet Size bytes
Traffic Packet Rate packets per second Traffic Packet Rate packets per second
Traffic Bit Rate MegaBits per second Traffic Bit Rate MegaBits per second
Number of Packets Sent number of entries Number of Packets Sent number of entries
Incremented Packet Header Fields list of fields Incremented Packet Header Fields list of fields
Number of Unique Header Values number of entries Number of Unique Header Values number of entries
Number of Packets per Flow number of entries Number of Packets per Flow number of entries
Traffic Generation linearly incremented or
randomized
Flow monitoring Specifications Flow monitoring Specifications
Direction ingress, egress, both Direction ingress, egress, both
Observation Points DUT interface names Observation Points DUT interface names
Cache Size number of entries Cache Size number of entries
Active Timeout seconds Active Timeout seconds
Inactive Timeout seconds Idle Timeout seconds
Flow Keys list of fields Flow Keys list of fields
Flow Record Fields total number of fields Flow Record Fields total number of fields
Number of Flows Created number of entries Number of Flows Created number of entries
Flow Export Transport Protocol UDP, TCP, SCTP, other Flow Export Transport Protocol UDP, TCP, SCTP, other
Flow Export Protocol IPFIX, NetFlow, other Flow Export Protocol IPFIX, NetFlow, other
Flow Export data packet size bytes Flow Export data packet size bytes
Flow Export MTU bytes Flow Export MTU bytes
MPLS Specifications (for traffic type MPLS only) MPLS Specifications (for traffic type MPLS only)
Tested Label Operation imposition, swap, disposition Tested Label Operation imposition, swap, disposition
Novak xpires September, 2012 The format of the report as documented in this appendix is informative
Appendix B: Miscellaneous Tests but the entries in the contents of it are required as specified in the
corresponding sections of this document.
Novak Expires October, 2012
Many of the configuration parameters required by the measurement
report can be retrieved from the [IPFIX-MIB] and [PSAMP-MIB] MIB
modules, and from [IPFIX-CONFIG] YANG module or other general MIBs.
Therefore, querying those modules from the DUT would be beneficial:
first of all, to help in populating the measurement report required
entries, but also to document all the other configuration parameters
from the DUT.
Appendix B: (Informative) Miscellaneous Tests
This section lists the tests which could be useful to asses a proper This section lists the tests which could be useful to asses a proper
Flow monitoring operation under various operational or stress Flow monitoring operation under various operational or stress
conditions. These tests are not deemed suitable for any benchmarking conditions. These tests are not deemed suitable for any benchmarking
for various reasons. for various reasons.
B.1 DUT Under Traffic Load B.1 DUT Under Traffic Load
The Flow Monitoring Throughput SHOULD be measured under different The Flow Monitoring Throughput should be measured under different
levels of static traffic load through the DUT. This can be achieved levels of static traffic load through the DUT. This can be achieved
only by using two traffic components as discussed in section 6.3.2. only by using two traffic components as discussed in section 6.3.2.
One traffic component exercises the Flow Monitoring Plane. The second One traffic component exercises the Flow Monitoring Plane. The second
traffic component loads only the Forwarding Plane without affecting traffic component loads only the Forwarding Plane without affecting
Flow monitoring (e.g. it creates just a certain amount of permanent Flow monitoring (e.g. it creates just a certain amount of permanent
Cache entries). Cache entries).
The variance in Flow Monitoring Throughput as function of the traffic The variance in Flow Monitoring Throughput as function of the traffic
load should be noted for comparison purposes between two DUTs of load should be noted for comparison purposes between two DUTs of
similar architecture and capability. similar architecture and capability.
skipping to change at page 30, line 49 skipping to change at page 31, line 57
Export configuration while following these test steps: Export configuration while following these test steps:
a. Perform benchmark test as specified in section 5 a. Perform benchmark test as specified in section 5
b. One of the results will be how much bandwidth Flow Export used b. One of the results will be how much bandwidth Flow Export used
on the dedicated Flow Export interface on the dedicated Flow Export interface
c. Change Flow Export configuration to use the test interface c. Change Flow Export configuration to use the test interface
d. Repeat the benchmark test while the receiver filters out the d. Repeat the benchmark test while the receiver filters out the
Flow Export data from analysis Flow Export data from analysis
The expected result is that the RFC2544 Throughput achieved in step The expected result is that the RFC2544 Throughput achieved in step
Novak Expires October, 2012
a. is same as the Throughput achieved in step d. provided that the a. is same as the Throughput achieved in step d. provided that the
bandwidth of the output DUT interface is not the bottleneck (in bandwidth of the output DUT interface is not the bottleneck (in
other words it must have enough capacity to forward both test and other words it must have enough capacity to forward both test and
Flow Export traffic). Flow Export traffic).
B.3 Variable Packet Size B.3 Variable Packet Size
The Flow monitoring measurements specified in this document would be The Flow monitoring measurements specified in this document would be
Novak Expires September, 2012
interesting to repeat with variable packet sizes within one interesting to repeat with variable packet sizes within one
particular test (e.g. test traffic containing mix of packet sizes). particular test (e.g. test traffic containing mix of packet sizes).
The packet forwarding tests specified mainly in [RFC2544] do not The packet forwarding tests specified mainly in [RFC2544] do not
recommend and perform such tests. Flow monitoring is not dependent recommend and perform such tests. Flow monitoring is not dependent
on packet sizes so such a test could be performed during the Flow on packet sizes so such a test could be performed during the Flow
Monitoring Throughput measurement and verify its value does not Monitoring Throughput measurement and verify its value does not
depend on the offered traffic packet sizes. The tests must be depend on the offered traffic packet sizes. The tests must be
carefully designed in order to avoid measurement errors due to the carefully designed in order to avoid measurement errors due to the
physical bandwidth limitations and changes of the base forwarding physical bandwidth limitations and changes of the base forwarding
performance with packet size. performance with packet size.
skipping to change at page 31, line 49 skipping to change at page 32, line 57
monitoring Cache exists and no Flow analysis occurs. The performance monitoring Cache exists and no Flow analysis occurs. The performance
measurement of the DUT in such a case is just pure [RFC2544] measurement of the DUT in such a case is just pure [RFC2544]
measurement. measurement.
B.5.2 RFC2544 Throughput with Metering Process B.5.2 RFC2544 Throughput with Metering Process
If only Metering Process is enabled it means that Flow analysis on If only Metering Process is enabled it means that Flow analysis on
the DUT is enabled and operational but no Flow Export happens. The the DUT is enabled and operational but no Flow Export happens. The
performance measurement of a DUT in such a configuration represents performance measurement of a DUT in such a configuration represents
an useful test of the DUT capabilities (this corresponds to the case an useful test of the DUT capabilities (this corresponds to the case
Novak Expires October, 2012
when the network operator uses Flow monitoring for example for manual when the network operator uses Flow monitoring for example for manual
denial of service attacks detection and does not wish to use Flow denial of service attacks detection and does not wish to use Flow
Export). Export).
The performance testing on this DUT can be performed as discussed in The performance testing on this DUT can be performed as discussed in
this document but it is not possible to verify the operation and this document but it is not possible to verify the operation and
results without interrogating the DUT. results without interrogating the DUT.
Novak Expires September, 2012
B.5.3 RFC2544 Throughput with Metering and Exporting Process B.5.3 RFC2544 Throughput with Metering and Exporting Process
This test represents the performance testing as discussed in This test represents the performance testing as discussed in
section 6. section 6.
B.6 Tests With Bidirectional Traffic B.6 Tests With Bidirectional Traffic
The test topology in figure 2 can be expanded to verify Flow Bidirectional traffic is not part of the normative benchmarking tests
monitoring functionality with bidirectional traffic in two possible based on discussion and recommendation of the Benchmarking working
ways: group. The experienced participants stated that this kind of traffic
did not provide reproducible results.
a. use two sets of interfaces, one for Flow monitoring for ingress
traffic and one for Flow monitoring egress traffic
b. use exactly same set-up as in figure 2 but use the interfaces in
full duplex mode e.g. sending and receiving simultaneously on each
of them
The set-up in point a. above is in fact equivalent to the set-up with The test topology in figure 2 can be expanded to verify Flow
several Observation Points as already discussed in section 4.1 monitoring functionality with bidirectional traffic using the
and 4.3.1. interfaces in full duplex mode e.g. sending and receiving
simultaneously on each of them.
For the set-up in point b. same rules should be applied (as per Same rules should be applied for Flow creation in the DUT Cache
section 4.1 and 4.3.1) - traffic passing through each Observation (as per section 4.1 and 4.3.1) - traffic passing through each
Point SHOULD always create a new Cache entry in the Cache e.g. the Observation Point should always create a new Cache entry in the Cache
same traffic SHOULD NOT be just looped back on the receiving e.g. the same traffic should not be just looped back on the receiving
interfaces to create the bidirectional traffic flow. interfaces to create the bidirectional traffic flow.
B.7 Instantaneous Flow Export Rate B.7 Instantaneous Flow Export Rate
An additional useful information when analysing the Flow Export data An additional useful information when analysing the Flow Export data
is the time distribution of the instantaneous Flow Export Rate. It is the time distribution of the instantaneous Flow Export Rate. It
can be derived during the measurements in two ways: can be derived during the measurements in two ways:
a. The Collector might provide the capability to decode Flow Export a. The Collector might provide the capability to decode Flow Export
during capturing and at the same time counting the Flow Records during capturing and at the same time counting the Flow Records
skipping to change at page 32, line 50 skipping to change at page 33, line 54
time interval than specified in section 5.4) Flow Export Rate time interval than specified in section 5.4) Flow Export Rate
b. The Flow Export protocol (like IPFIX [RFC5101]) can provide time b. The Flow Export protocol (like IPFIX [RFC5101]) can provide time
stamps in the Flow Export packets which would allow time based stamps in the Flow Export packets which would allow time based
analysis and calculate the Flow Export Rate as an average over analysis and calculate the Flow Export Rate as an average over
much shorter time interval than specified in section 5.4 much shorter time interval than specified in section 5.4
The accuracy and shortest time average will always be limited by the The accuracy and shortest time average will always be limited by the
precision of the time stamps (1 second for IPFIX) or by the precision of the time stamps (1 second for IPFIX) or by the
capabilities of the DUT and the Collector. capabilities of the DUT and the Collector.
Novak Expires September, 2012 Novak Expires October, 2012
 End of changes. 128 change blocks. 
208 lines changed or deleted 290 lines changed or added

This html diff was produced by rfcdiff 1.41. The latest version is available from http://tools.ietf.org/tools/rfcdiff/