[Docs] [txt|pdf] [Tracker] [Email] [Nits]
Versions: 00 01
IPFIX Working Group A. Kobayashi
Internet-Draft H. Nishida
Intended status: Informational NTT PF Lab.
Expires: April 18, 2008 C. Sommer
F. Dressler
Univ. Erlangen
E. Stephan
France Telecom
October 16, 2007
Problems with Flow Collection in Large-Scale Networks
draft-kobayashi-ipfix-large-ps-00.txt
Status of this Memo
By submitting this Internet-Draft, each author represents that any
applicable patent or other IPR claims of which he or she is aware
have been or will be disclosed, and any of which he or she becomes
aware will be disclosed, in accordance with Section 6 of BCP 79.
Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF), its areas, and its working groups. Note that
other groups may also distribute working documents as Internet-
Drafts.
Internet-Drafts are draft documents valid for a maximum of six months
and may be updated, replaced, or obsoleted by other documents at any
time. It is inappropriate to use Internet-Drafts as reference
material or to cite them other than as "work in progress."
The list of current Internet-Drafts can be accessed at
http://www.ietf.org/ietf/1id-abstracts.txt.
The list of Internet-Draft Shadow Directories can be accessed at
http://www.ietf.org/shadow.html.
This Internet-Draft will expire on April 18, 2008.
Copyright Notice
Copyright (C) The IETF Trust (2007).
Kobayashi, et al. Expires April 18, 2008 [Page 1]
Internet-Draft Problem Statement in large NW October 2007
Abstract
Flow measurement techniques have been developed in order to cope with
increasing bandwiths. Flow measurement is currently being used as a
popular method for monitoring large networks, e.g. at Internet
Service Providers or multinational corporations, for accounting,
management, and security purposes. To construct the measurement
system for such networks, the biggest challenge is scalability.
Whereas packet sampling functions in current flow measurement
solutions such as NetFlow/sFlow/IPFIX help to approach this issue,
some problems still remain. This document describes such open
issues, which a flow-based measurement system might encounter in
large-scale networks. The results are intended to be used to define
and to develope an IPFIX mediator device.
Table of Contents
1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 3
2. Problem Statement . . . . . . . . . . . . . . . . . . . . . . 4
2.1. Traffic Measurement in Service Provider Networks . . . . . 4
2.2. Flow-Based Measurement in Integrated Networks . . . . . . 4
2.3. Approaches to Scalability . . . . . . . . . . . . . . . . 5
2.3.1. Allocating Multiple Collectors in Parallel . . . . . . 5
2.3.2. Adjusting Sampling Rates . . . . . . . . . . . . . . . 6
2.3.3. Exporting Aggregated Flows from Original Exporters . . 6
2.3.4. Using a Hierarchy of IPFIX Concentrators . . . . . . . 6
3. Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . 8
4. Security Considerations . . . . . . . . . . . . . . . . . . . 9
5. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 10
6. References . . . . . . . . . . . . . . . . . . . . . . . . . . 11
6.1. Normative References . . . . . . . . . . . . . . . . . . . 11
6.2. Informative References . . . . . . . . . . . . . . . . . . 11
Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 12
Intellectual Property and Copyright Statements . . . . . . . . . . 14
Kobayashi, et al. Expires April 18, 2008 [Page 2]
Internet-Draft Problem Statement in large NW October 2007
1. Introduction
Flow-based measurement has dramatically improved the quality of
traffic measurement methods available for service provider networks.
It is quickly becoming one of the essential operation tools in many
networks. Especially monitoring operations in large-scale networks
need flow measurement techniques to ensure stable and efficient
operations. However, network providers might soon encounter
problematic situations because of scalability issues. With traffic
volume ever increasing in large networks, the number of exported Flow
Records becomes huge. Also, Flow information is required for
multiple purposes and it is difficult to maintain the scalability of
a flow-based measurement system and at the same time offer Flows in
the way required for purposes such as Traffic Engineering,
Troubleshooting, Accounting, and Security. This document provides
the problem description and, thus, the requirements for an extension
to IPFIX, the IPFIX mediator.
The IPFIX applicability draft [I-D.ietf-ipfix-as] describes a variety
of applications for the IPFIX protocol [I-D.ietf-ipfix-protocol]. In
most networks, the flexibility of the IPFIX protocol could meet the
requirements of a flow-based measurement system. But, from the
viewpoint of large-scale and fast-growing networks, there are still
some unresolved problems. This document describes such problematic
cases in large-scale networks.
Kobayashi, et al. Expires April 18, 2008 [Page 3]
Internet-Draft Problem Statement in large NW October 2007
2. Problem Statement
2.1. Traffic Measurement in Service Provider Networks
Flow measurement methods are one of the most popular measurement
methods of network service providers. Usually, an operator measures
traffic on several observation points for a specific purpose, using a
certain level of sampling rate such as from 1/10,000 to 1/1,000.
This value depends on the several factors, such as the capacity of
the management network, the available storage and speed of the
Collector, or the load of the routers/switches. This section
describes some of use cases of flow-based measurement on the service
provider networks.
o In gateway routers, exchange traffic between neighboring AS is
monitored using flow-based measurement. The result helps with the
planing of peering policies and with traffic engineering to
optimize the network resources.
o In server side routers, the traffic toward a specific customer's
server is measured. The collected data is used to detect traffic
anomalies, such as distributed denial-of-service attacks and
packet scans. Furthermore, this data is valuable for network
forensics, when customers ask to investigate their server traffic.
o In access routers connected toward the broadband customers, the
traffic of each customer is measured to profile them. Operators
pay special attention to heavy users that abuse network bandwidth
by using P2P file exchange applications. Results give good
feedback about the implemented billing policy.
As described above, the network providers are making heavy use of
Flow measurements. The number of Observation Points in the networks
can even be increased to improve on the effectiveness of these
methods. In the near future, we anticipate that the advanced
features of IPFIX, such as monitoring wide-area traffic exchange
behavior and QoS performance will accelerate the utilization. The
increasing amount of the traffic that is brought about by broadband
users might have a impact on the present measurement parameters, such
as the sampling rate or the granularity of Flows.
2.2. Flow-Based Measurement in Integrated Networks
Recently several networks seem to shift towards Integrated Networks,
such as the Internet and MPLS, which include IPv4, IPv6 and VPN
traffic. These sorts of Flow Records need to be analyzed separately
and from different perspectives. However, handling them separately
without improving the capability of the Collector is difficult. If
Kobayashi, et al. Expires April 18, 2008 [Page 4]
Internet-Draft Problem Statement in large NW October 2007
the original Exporter can not distribute specific Flow Records based
on the Flow contents, a single Collector needs to be able to handle
several kinds of Flow Records. Except for the original Exporter,
other devices assisting in this function would thus be necessary.
2.3. Approaches to Scalability
The volume of traffic on ordinary service provider networks has been
increasing year by year. As the sizes of networks become larger, the
amount of Flow Records becomes greater. The huge amount of Flow
Records has been burdening management networks and Collecting
Processes. Maintaining scalability is difficult as a particular
network grows.
Generally, large-scale networks already have multiple 10Gb/s links,
their total traffic exceeding 100Gb/s. In the near future, broadband
users' traffic will increase by approximately 50% per year according
to [TRAFGRW]. When operators monitor traffic of 500Gb/s with a
sampling rate of 1/1000, the amount of exported Flow Records from
Exporters could exceed 50k f/s. This value reaches beyond the
ability of a single Collector. It should be noted that the current
sampling rate might not be able to be applied to the Exporter within
the next five years. We consider the problem to be expected to
encounter within the next five years in large-scale networks.
In addition, network operators need to monitor the overall wide-scale
traffic exchange, and investigate detailed traffic behavior when
traffic incidents happen. Achieving this seems to be difficult for a
single Collector because of the huge amount of Flow Records.
To that end, the network operators can consider several solutions.
The next sections describe how operators can cope with such a huge
amount of Flow Records using available solutions of NetFlow/sFlow/
IPFIX. However, each solution also presents new problems.
2.3.1. Allocating Multiple Collectors in Parallel
Generally speaking, allocating multiple Collectors to cope with the
amount of Flow Records can resolve many scalability problems. But,
in that case, network operators will have difficulties to get an
overview of traffic exchange behavior in whole network. Most of
Flow-based applications require the observation of flows from the
whole network under observation. For example, they correspond to the
wide-area traffic matrices that are the router-by-router or PoP-by-
PoP traffic matrix. To get the wide-area traffic matrices, single
Collector device needs to gather the Flow Records from the whole
networks.
Kobayashi, et al. Expires April 18, 2008 [Page 5]
Internet-Draft Problem Statement in large NW October 2007
2.3.2. Adjusting Sampling Rates
Adjusting the sampling rate definitely reduces the amount of Flow
Records, and a flow-based measurement system can thus easily adapt to
the ability of the Collecting and Exporting Processes. But, in that
case, Flows with small traffic volume could easily get lost. When
the traffic incidents happen, network operators can no longer
investigate traffic behavior. While traffic volume on networks
continues to increase, network operators will not be able to maintain
the sampling rates currently used. In the near future, it is
possible that flow-based measurement systems will not be able to
detect traffic anomalies which can currently be detected.
2.3.3. Exporting Aggregated Flows from Original Exporters
When using Flow aggregation, the Flow Records are generally
aggregated based on network prefix, peering AS (autonomous system)
number, or BGP Next-Hop. This solution is especially useful for
measurements of wide-area traffic exchange. However, network
operators can no longer monitor traffic behavior in-depth, the
simplest type of Flows being those comprised of all packets having
the same 5-tuple of protocol, source and destination IP addresses and
port numbers.
As one of solutions that resolve this problem, if the Exporter allows
aggregation with multiple granularities, network operators can
flexibly adapt the granularity of aggregation according to their
requirements. An Exporter might, for example, by default export
aggregated Flow Records. If the traffic incident happens, network
operators could then change the Flow Key to measure fine-grained
Flows. To implement this solution, a system that dynamically changes
the Flow Key in cooperation with Exporters and Collectors is
required. It is, however, difficult to replace all Exporters in the
whole network, as all Exporters in the network will need to implement
this solution.
2.3.4. Using a Hierarchy of IPFIX Concentrators
In large-scale networks, creating a hierarchical measurement system
by using IPFIX Concentrators, as described in [RFC3917], can prove to
be very useful. If IPFIX Concentrators store the received Flow
Records, and then the stored Flow Records are allowed to be retrieved
by other devices, this architecture might actually become a most
useful distributed collection system. As described in
[I-D.dressler-ipfix-aggregation], in the case of a measurement system
consisting of both aggregating Exporters and non-aggregating
Exporters, an IPFIX Concentrator that can assist the latter by
aggregating received Flow Records to any granularity.
Kobayashi, et al. Expires April 18, 2008 [Page 6]
Internet-Draft Problem Statement in large NW October 2007
However, IPFIX header information, such as Exporter IP address,
Observation Domain ID and Export Time would likely be lost in the
process of mediation that is performed by the Concentrator. Exporter
IP address and Observation Domain ID indicate the observation point
from the viewpoint of entire network domain. Export Time indicates
the reference of time for flow information. Such information is
necessary for guaranteeing the continuity of the work of the top
level Collector. Additionally in some cases the information that is
reported by Option Templates, such as the sampling rate, could be
lost. It depends on the implementation of the IPFIX Concentrator.
It should be noted that the information communicated by IPFIX
Concentrator needs to be defined.
Kobayashi, et al. Expires April 18, 2008 [Page 7]
Internet-Draft Problem Statement in large NW October 2007
3. Conclusion
This document has covered the following problems related to flow-
based measurement systems in large-scale networks.
o Adjusting the sampling rate and granularity of Flow causes small
Flows to become invisible. Large-scale networks need another
solution that can handle the huge amounts of Flow Records when
using current sampling rates, which range from 1/10,000 to 1/1000.
o Using IPFIX Concentrators might lose data that should be
communicated by the original Exporter.
o In Integrated Networks, which mix MPLS VPN with other tunnel
traffic and IPv4/IPv6, flow-based measurement might not work well.
These problems have their roots in constructing scalable flow-based
measurement systems. Although network providers can resolve these
problems by utilizing advanced features of Exporters and Collectors,
it is difficult to perform an atomic migration of the whole
measurement system while the amount of traffic grows. To assist the
ability of the Exporters and the Collectors, it should be noted that
there are some IPFIX devices for the network providers to select.
They correspond to Flow mediation devices. Especially standardized
IPFIX Concentrator could have been useful devices.
Kobayashi, et al. Expires April 18, 2008 [Page 8]
Internet-Draft Problem Statement in large NW October 2007
4. Security Considerations
A flow-based measurement system might lead to privacy violation
problems, such as the export of flows to an unexpected address, if
the system is not confined to the large-scale network under
observation. General security issues of the IPFIX protocol are
covered by the security considerations section in
[I-D.ietf-ipfix-protocol]. Security MUST be considered, if different
networks exchange the flow information. As the security of the
exchange relies mostly on the protocol used, UDP does not look
appropriate to exchange information between networks.
Kobayashi, et al. Expires April 18, 2008 [Page 9]
Internet-Draft Problem Statement in large NW October 2007
5. IANA Considerations
This document has no actions for IANA.
Kobayashi, et al. Expires April 18, 2008 [Page 10]
Internet-Draft Problem Statement in large NW October 2007
6. References
6.1. Normative References
[RFC2119] Bradner, S., "Key words for use in RFCs to Indicate
Requirement Levels", BCP 14, RFC 2119, March 1997.
6.2. Informative References
[I-D.dressler-ipfix-aggregation]
Dressler, F., Sommer, C., and G. Munz, "IPFIX
Aggregation", draft-dressler-ipfix-aggregation-03.txt
(work in progress) , June 2006.
[I-D.ietf-ipfix-as]
Zseby, T., Boschi, E., Brownlee, N., and B. Claise, "IPFIX
Applicability", draft-dressler-ipfix-as-10.txt (work in
progress) , June 2006.
[I-D.ietf-ipfix-protocol]
Claise, B., "IPFIX Protocol Specification",
draft-ietf-ipfix-protocol-23 (work in progress) ,
October 2006.
[RFC3917] Quittek, J., Zseby, T., Claise, B., and S. Zander,
"Requirements for IP Flow Information Export(IPFIX)",
October 2004.
[TRAFGRW] Cho, K., Fukuda, K., Esaki, H., and A. Kato, "The Impact
and Implications of the Growth in Residential User-to-User
Traffic", SIGCOMM2006, pp. 207-218, Pisa, Italy, September
2006. , October 2006.
Kobayashi, et al. Expires April 18, 2008 [Page 11]
Internet-Draft Problem Statement in large NW October 2007
Authors' Addresses
Atsushi Kobayashi
NTT Information Sharing Platform Laboratories
3-9-11 Midori-cho
Musashino-shi, Tokyo 180-8585
Japan
Phone: +81-422-59-3978
Email: akoba@nttv6.net
Haruhiko Nishida
NTT Information Sharing Platform Laboratories
3-9-11 Midori-cho
Musashino-shi, Tokyo 180-8585
Japan
Phone: +81-422-59-3978
Email: nishida.haruhiko@lab.ntt.co.jp
Christoph Sommer
University of Erlangen-Nuremberg
Department of Computer Science 7
Martensstr. 3
Erlangen 91058
Germany
Phone: +49 9131 85-27993
Email: christoph.sommer@informatik.uni-erlangen.de
URI: http://www7.informatik.uni-erlangen.de/~sommer/
Falko Dressler
University of Erlangen-Nuremberg
Department of Computer Science 7
Martensstr. 3
Erlangen 91058
Germany
Phone: +49 9131 85-27914
Email: dressler@informatik.uni-erlangen.de
URI: http://www7.informatik.uni-erlangen.de/~dressler/
Kobayashi, et al. Expires April 18, 2008 [Page 12]
Internet-Draft Problem Statement in large NW October 2007
Emile Stephan
France Telecom Division R&D
2 avenue Pierre Marzin
Lannion F-22307
France
Phone: +33 2 96 05 18 52
Email: emile.stephan@orange-ftgroup.com
Kobayashi, et al. Expires April 18, 2008 [Page 13]
Internet-Draft Problem Statement in large NW October 2007
Full Copyright Statement
Copyright (C) The IETF Trust (2007).
This document is subject to the rights, licenses and restrictions
contained in BCP 78, and except as set forth therein, the authors
retain all their rights.
This document and the information contained herein are provided on an
"AS IS" basis and THE CONTRIBUTOR, THE ORGANIZATION HE/SHE REPRESENTS
OR IS SPONSORED BY (IF ANY), THE INTERNET SOCIETY, THE IETF TRUST AND
THE INTERNET ENGINEERING TASK FORCE DISCLAIM ALL WARRANTIES, EXPRESS
OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF
THE INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED
WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE.
Intellectual Property
The IETF takes no position regarding the validity or scope of any
Intellectual Property Rights or other rights that might be claimed to
pertain to the implementation or use of the technology described in
this document or the extent to which any license under such rights
might or might not be available; nor does it represent that it has
made any independent effort to identify any such rights. Information
on the procedures with respect to rights in RFC documents can be
found in BCP 78 and BCP 79.
Copies of IPR disclosures made to the IETF Secretariat and any
assurances of licenses to be made available, or the result of an
attempt made to obtain a general license or permission for the use of
such proprietary rights by implementers or users of this
specification can be obtained from the IETF on-line IPR repository at
http://www.ietf.org/ipr.
The IETF invites any interested party to bring to its attention any
copyrights, patents or patent applications, or other proprietary
rights that may cover technology that may be required to implement
this standard. Please address the information to the IETF at
ietf-ipr@ietf.org.
Acknowledgment
Funding for the RFC Editor function is provided by the IETF
Administrative Support Activity (IASA).
Kobayashi, et al. Expires April 18, 2008 [Page 14]
Html markup produced by rfcmarkup 1.129d, available from
https://tools.ietf.org/tools/rfcmarkup/