[Docs] [txt|pdf] [Tracker] [WG] [Email] [Diff1] [Diff2] [Nits]

Versions: 00 01 02 03 04 05 06 07 08 09 RFC 5982

IPFIX Working Group                                    A. Kobayashi, Ed.
Internet-Draft                                               NTT PF Lab.
Intended status: Informational                            B. Claise, Ed.
Expires: November 1, 2009                            Cisco Systems, Inc.
                                                          April 30, 2009







                   IPFIX Mediation: Problem Statement
            draft-ietf-ipfix-mediators-problem-statement-03

Status of this Memo

   This Internet-Draft is submitted to IETF in full conformance with the
   provisions of BCP 78 and BCP 79.  This document may contain material
   from IETF Documents or IETF Contributions published or made publicly
   available before November 10, 2008.  The person(s) controlling the
   copyright in some of this material may not have granted the IETF
   Trust the right to allow modifications of such material outside the
   IETF Standards Process.  Without obtaining an adequate license from
   the person(s) controlling the copyright in such materials, this
   document may not be modified outside the IETF Standards Process, and
   derivative works of it may not be created outside the IETF Standards
   Process, except to format it for publication as an RFC or to
   translate it into languages other than English.

   Internet-Drafts are working documents of the Internet Engineering
   Task Force (IETF), its areas, and its working groups.  Note that
   other groups may also distribute working documents as Internet-
   Drafts.

   Internet-Drafts are draft documents valid for a maximum of six months
   and may be updated, replaced, or obsoleted by other documents at any
   time.  It is inappropriate to use Internet-Drafts as reference
   material or to cite them other than as "work in progress."

   The list of current Internet-Drafts can be accessed at
   http://www.ietf.org/ietf/1id-abstracts.txt.

   The list of Internet-Draft Shadow Directories can be accessed at
   http://www.ietf.org/shadow.html.

   This Internet-Draft will expire on November 1, 2009.




Kobayashi, et al.       Expires November 1, 2009                [Page 1]


Internet-Draft         Mediation Problem Statement            April 2009


Copyright Notice

   Copyright (c) 2009 IETF Trust and the persons identified as the
   document authors.  All rights reserved.

   This document is subject to BCP 78 and the IETF Trust's Legal
   Provisions Relating to IETF Documents in effect on the date of
   publication of this document (http://trustee.ietf.org/license-info).
   Please review these documents carefully, as they describe your rights
   and restrictions with respect to this document.









































Kobayashi, et al.       Expires November 1, 2009                [Page 2]


Internet-Draft         Mediation Problem Statement            April 2009


Abstract

   Flow-based measurement is a popular method for various network
   monitoring usages.  The sharing of flow-based information for
   monitoring applications having different requirements raises some
   open issues in terms of measurement system scalability, flow-based
   measurement flexibility, and export reliability that IPFIX Mediation
   may help resolve.  IPFIX Mediation covers two classes of mediation:
   context mediation for traffic data and transport mediation for
   transport protocols.  This document describes the IPFIX Mediation
   applicability examples, along with some problems that network
   administrators have been facing.







































Kobayashi, et al.       Expires November 1, 2009                [Page 3]


Internet-Draft         Mediation Problem Statement            April 2009


Table of Contents

   1.  Introduction . . . . . . . . . . . . . . . . . . . . . . . . .  5
   2.  Terminology and Definition . . . . . . . . . . . . . . . . . .  6
   3.  IPFIX/PSAMP Documents Overview . . . . . . . . . . . . . . . .  9
     3.1.  IPFIX Documents Overview . . . . . . . . . . . . . . . . .  9
     3.2.  PSAMP Documents Overview . . . . . . . . . . . . . . . . .  9
   4.  Problem Statement  . . . . . . . . . . . . . . . . . . . . . . 10
     4.1.  Coping with IP Traffic Growth  . . . . . . . . . . . . . . 10
     4.2.  Coping with Multipurpose Traffic Measurement . . . . . . . 11
     4.3.  Coping with Heterogeneous Environments . . . . . . . . . . 11
     4.4.  Summary  . . . . . . . . . . . . . . . . . . . . . . . . . 11
   5.  Mediation Applicability Examples . . . . . . . . . . . . . . . 12
     5.1.  Adjusting Flow Granularity . . . . . . . . . . . . . . . . 12
     5.2.  Hierarchical Collecting Infrastructure . . . . . . . . . . 12
     5.3.  Correlation for Data Records . . . . . . . . . . . . . . . 13
     5.4.  Time Composition . . . . . . . . . . . . . . . . . . . . . 13
     5.5.  Spatial Composition  . . . . . . . . . . . . . . . . . . . 14
     5.6.  Data Record Anonymization  . . . . . . . . . . . . . . . . 15
     5.7.  Data Retention . . . . . . . . . . . . . . . . . . . . . . 15
     5.8.  IPFIX Export from a Branch Office  . . . . . . . . . . . . 16
     5.9.  Distributing Data Records  . . . . . . . . . . . . . . . . 17
     5.10. Flow-based Sampling and Selection  . . . . . . . . . . . . 18
     5.11. Interoperability between Legacy Protocols and IPFIX  . . . 19
   6.  IPFIX Mediators Implementation Specific Problems . . . . . . . 20
     6.1.  Loss of Original Exporter Information  . . . . . . . . . . 20
     6.2.  Loss of Base Time Information  . . . . . . . . . . . . . . 20
     6.3.  Transport Sessions Management  . . . . . . . . . . . . . . 21
     6.4.  Loss of Option Template Information  . . . . . . . . . . . 21
     6.5.  Template ID Management . . . . . . . . . . . . . . . . . . 21
     6.6.  Consideration for Network Topology . . . . . . . . . . . . 22
     6.7.  Exporting the Function Item  . . . . . . . . . . . . . . . 22
     6.8.  Consideration for Aggregation  . . . . . . . . . . . . . . 23
   7.  Summary and Conclusion . . . . . . . . . . . . . . . . . . . . 24
   8.  Security Considerations  . . . . . . . . . . . . . . . . . . . 26
   9.  IANA Considerations  . . . . . . . . . . . . . . . . . . . . . 27
   10. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . 28
   11. References . . . . . . . . . . . . . . . . . . . . . . . . . . 29
     11.1. Normative References . . . . . . . . . . . . . . . . . . . 29
     11.2. Informative References . . . . . . . . . . . . . . . . . . 29
   Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 31










Kobayashi, et al.       Expires November 1, 2009                [Page 4]


Internet-Draft         Mediation Problem Statement            April 2009


1.  Introduction

   While the IPFIX requirements defined in [RFC3917] mention an
   intermediate function, such as an IPFIX Proxy or an IPFIX
   Concentrator, there are no documents defining the function called
   IPFIX Mediation.  IPFIX Mediation is a generic function that covers
   the manipulation of IPFIX Flow Records, PSAMP Packet Reports, entire
   IPFIX Messages, or their transmission.  This document describes
   general problems, applicability examples, and defines the terminology
   (IPFIX Proxy, Concentrator, etc.) for referring to different use
   cases for IPFIX Mediation.  Furthermore, some specific problems
   related to the IPFIX protocol [RFC5101] when applying IPFIX Mediation
   are addressed.

   This document is structured as follows: section 2 describes the
   terminology used in this document, section 3 gives an IPFIX/PSAMP
   document overview, section 4 introduces general problems related to
   flow-based measurement, section 5 describes some applicability
   examples where IPFIX Mediations would be beneficial, and, finally,
   section 6 describes some problems an IPFIX Mediation implementation
   might face.






























Kobayashi, et al.       Expires November 1, 2009                [Page 5]


Internet-Draft         Mediation Problem Statement            April 2009


2.  Terminology and Definition

   The terms in this section are in line with those in the IPFIX
   Protocol specifications [RFC5101] and the PSAMP specification
   document [RFC5476].  The terms Observation Point, Observation Domain,
   Flow Key, Flow Record, Data Record, Exporting Process, Exporter,
   IPFIX Device, Collecting Process, Collector, IPFIX Message, Metering
   Process, Transport Session, Information Element, and Template
   Withdrawal Message, are defined in the IPFIX protocol specifications
   [RFC5101].  The terms Packet Report, Sampling, Filtering, PSAMP
   Device, and Configured Selection Fraction are defined in the PSAMP
   specification document [RFC5476].  Furthermore, new terminology to be
   used in the context of IPFIX Mediation is defined in this section.
   All these terms have an initial capital letter in this document.

   While IPFIX Mediation can process both Flow Records and Packet
   Reports, this document prefers the more generic "Data Record" term as
   this is a more generic term, unless the reference to the IPFIX Flow
   Record or PSAMP Packet Report is required.

   IPFIX Mediation

      IPFIX Mediation is a generic function that covers the manipulation
      of IPFIX Flow Records, PSAMP Packet Reports, or entire IPFIX
      Messages, or their transmission.  The IPFIX Mediation offers one
      or multiple of the following capabilities:

      *  content mediation that changes Flow Records/Packet Reports
         information

         +  aggregating Flow Records/Packet Reports based on a new set
            of Flow Key fields

         +  correlating a set of Flow Records/Packet Reports

         +  filtering and selecting Flow Records/Packet Reports

         +  modifying Flow Records/Packet Reports, including:

            -  changing the value of specified Information Elements

            -  adding new Information Elements by deriving further Flow
               or packet properties from existing fields (for example:
               calculating new metrics or new counters)

            -  deleting specified Information Elements





Kobayashi, et al.       Expires November 1, 2009                [Page 6]


Internet-Draft         Mediation Problem Statement            April 2009


      *  transport mediation

         +  changing the transport protocol that carries IPFIX Messages

         +  rerouting entire IPFIX Messages to an appropriate Collecting
            Process

         +  replicating Flow Records/Packet Reports (or the entire IPFIX
            Messages)

   IPFIX Mediator

      An IPFIX Mediator is an IPFIX Device that implements one or more
      IPFIX Mediation capabilities.  Examples are devices such as
      routers, switches, network management systems (NMS), etc.

   Original Exporter

      An Original Exporter is an IPFIX Device that hosts the Observation
      Points where the metered IP packets are observed.

   IPFIX Proxy

      An IPFIX Proxy is a type of IPFIX Mediation that relays incoming
      Transport Sessions to one or multiple Collectors.  The protocols
      used at the input and the output can be different, which implies
      that IPFIX Messages, Data Records, and Template Records need to be
      encoded, e.g., for converting from a legacy protocol to IPFIX.  An
      IPFIX Proxy is not implemented on the Original Exporter, but as a
      separate Mediator.

   IPFIX Concentrator

      An IPFIX Concentrator is a type of IPFIX Mediation that receives
      Flow Records/Packet Reports, correlates them, aggregates them, or
      modifies them, then exports the new Data Records.

   IPFIX Distributor

      An IPFIX Distributor is a type of IPFIX Mediation that distributes
      Data Records to one or multiple IPFIX Collectors.  The decision as
      to which IPFIX Collector a Data Record is exported can be
      determined by filtering certain field values or other properties
      derived from the Data Record.







Kobayashi, et al.       Expires November 1, 2009                [Page 7]


Internet-Draft         Mediation Problem Statement            April 2009


   IPFIX Masquerading Proxy

      An IPFIX Masquerading Proxy is a type of IPFIX Mediation that
      screens out parts of input Flow Records/Packet Reports according
      to configured policies.  It can thus, for example, hide the
      network topology information or customers' IP addresses.













































Kobayashi, et al.       Expires November 1, 2009                [Page 8]


Internet-Draft         Mediation Problem Statement            April 2009


3.  IPFIX/PSAMP Documents Overview

3.1.  IPFIX Documents Overview

   The IPFIX protocol [RFC5101] provides network administrators with
   access to IP flow information.  The architecture for the export of
   measured IP flow information out of an IPFIX Exporting Process to a
   Collecting Process is defined in [RFC5470], per the requirements
   defined in [RFC3917].  The IPFIX protocol [RFC5101] specifies how
   IPFIX Data Records and Templates are carried via a number of
   transport protocols from IPFIX Exporting Processes to IPFIX
   Collecting Processes.  IPFIX has a formal description of IPFIX
   Information Elements, their names, types, and additional semantic
   information, as specified in [RFC5102].  [I-D.ietf-ipfix-mib]
   specifies the IPFIX Management Information Base.  Finally, [RFC5472]
   describes what types of applications can use the IPFIX protocol and
   how they can use the information provided.  It furthermore shows how
   the IPFIX framework relates to other architectures and frameworks.
   The storage of IPFIX Messages in a file is specified in
   [I-D.ietf-ipfix-file].

3.2.  PSAMP Documents Overview

   The framework for packet selection and reporting [RFC5474] enables
   network elements to select subsets of packets by statistical and
   other methods and to export a stream of reports on the selected
   packets to a Collector.  The set of packet selection techniques
   (Sampling, Filtering, and Hashing) standardized by PSAMP are
   described in [RFC5475].  The PSAMP protocol [RFC5476] specifies the
   export of packet information from a PSAMP Exporting Process to a
   Collector.  Like IPFIX, PSAMP has a formal description of its
   Information Elements, their names, types and additional semantic
   information.  The PSAMP information model is defined in [RFC5477].
   [I-D.ietf-psamp-mib] describes the PSAMP Management Information Base.

















Kobayashi, et al.       Expires November 1, 2009                [Page 9]


Internet-Draft         Mediation Problem Statement            April 2009


4.  Problem Statement

   Network administrators generally face the problems of measurement
   system scalability, flow-based measurement flexibility, and export
   reliability, even if some techniques, such as Sampling, Filtering,
   Data Records aggregation and export replication, have already been
   developed.  The problems consist of optimizing the resources of the
   measurement system while pursuing appropriate conditions: data
   accuracy, flow granularity, and export reliability.  These conditions
   depend on two factors.

   o  measurement systems capacity:
      This consists of the bandwidth of the management network, the
      storage capacity, and the performances of the collecting devices
      and exporting devices.

   o  applications requirements:
      Different applications, such as traffic engineering, detecting
      anomaly traffic, and accounting, etc., impose different Flow
      Record granularity, and data accuracy.

   The recent continued IP traffic growth has been overwhelming the
   measurement system capacity, and multi-purposing applications, along
   with the heterogeneous environments, have further been contributing
   to a complex situation.  The following sub-sections explain different
   problems in more details.

4.1.  Coping with IP Traffic Growth

   Enterprise or service provider networks already have multiple 10 Gb/s
   links, their total traffic exceeding 100 Gb/s.  In the near future,
   broadband users' traffic will increase by approximately 40% every
   year according to [TRAFGRW].  When operators monitor traffic of 500
   Gb/s with a Sampling rate of 1/1000, the amount of exported Flow
   Records from Exporters could exceed 50 kFlows/s.  This value is
   beyond the ability of a single Collector.

   To deal with this problem, current data reduction techniques, such as
   Sampling, Filtering, Data Records aggregation have been generally
   implemented on Exporters.  Note that Sampling technique leads to
   potential loss of small Flows.  With both Sampling and aggregation
   techniques, administrators might no longer be able to investigate
   very granular traffic change and anomaly detection, both of which can
   currently be detected.  With Filtering, only a subset of the Flow
   Records are exported.

   Considering the potential drawbacks of Sampling, Filtering, and Data
   Records aggregation, there is a need for a large-scale collecting



Kobayashi, et al.       Expires November 1, 2009               [Page 10]


Internet-Draft         Mediation Problem Statement            April 2009


   infrastructure that does not rely on data reduction techniques.

4.2.  Coping with Multipurpose Traffic Measurement

   A set of conditions (flow granularity and data accuracy) may meet the
   requirements of some applications, such as traffic engineering, but
   may not meet the requirements of other applications, such as
   accounting, QoS performance, or even security.  Therefore, with a
   single set of conditions, multipurpose traffic measurements cannot be
   accomplished.

   To cope with the issue, an Exporter needs to export traffic data with
   strictest condition (fine flow granularity and high data accuracy)
   required by the set of applications.  However, this implies an
   increased load on both the Exporter and Collector.

4.3.  Coping with Heterogeneous Environments

   Network administrators use IPFIX Devices and PSAMP Devices from
   various vendors, various software versions, various device types
   (router, switch, or probe) in a single network domain.  Even legacy
   flow export protocols are still deployed in current network.  This
   heterogeneous environment leads to differences in Metering Process
   capability, Exporting Process capacity (export rate, cache memory,
   etc.), and data format.  For example, probes and switches cannot
   retrieve packet property information from a route table.

   To deal with this problem, the collecting infrastructure needs to
   absorb the differences.  However, equipping all collecting devices
   with this absorption function is difficult.

4.4.  Summary

   In optimizing the resources of a measurement system, it is important
   to use traffic data reduction techniques at the possible initial
   phase, e.g., at the Exporter.  However, this implementation is made
   difficult by heterogeneous environment of exporting devices.

   This implies that a new Mediation functional block is required in
   typical Exporter-Collector architectures.  Based on some
   applicability examples, the next section shows the limitation of the
   typical Exporter-Collector architecture model and the IPFIX Mediation
   benefits.








Kobayashi, et al.       Expires November 1, 2009               [Page 11]


Internet-Draft         Mediation Problem Statement            April 2009


5.  Mediation Applicability Examples

5.1.  Adjusting Flow Granularity

   The simplest types of Flows are those comprised of packets all having
   a fixed IP-quintuple of protocol, source and destination IP
   addresses, and source and destination port numbers.  However, a
   shorter set of Flow Keys, such as a triple, a double, or a single
   Flow Key, (for example network prefix, peering AS number, or BGP
   Next-Hop fields), creates more aggregated Flow Records.  This is
   especially useful for measuring traffic exchange in an entire network
   domain and for easily adjusting the performance of Exporters and
   Collectors.

   Implementation analysis:

      Implementations for this case depend on where Flow granularity is
      adjusted.  More suitable implementations use configurable Metering
      Processes in Original Exporters.  The cache in the Metering
      Process can specify its own set of Flow Keys and extra fields.
      The Original Exporter thus creates directly aggregated Flow
      Records.

      In the case where the Original Exporter contains a Metering
      Process that creates fixed tuple Flow Records (no possibility to
      change the Flow Keys), an IPFIX Concentrator can adjust the Flow
      Keys by aggregation Flow Records.  Even if the case where the
      Original Exporter contains a Metering Process for which the Flow
      Keys can be configured, an IPFIX Concentrator can further
      aggregate the Flow Records.

5.2.  Hierarchical Collecting Infrastructure

   The increase of IPFIX Exporters, the increase of the traffic over
   large-scale networks, and the variety of treatments expected to be
   performed over the Data Records is more and more difficult to handle
   within a single Collector.  Hence to increase the collecting (e.g.
   the bandwidth capacity) and processing capacity must be distributed
   over several IPFIX entities.  As a possible approach, a hierarchical
   structure is useful for increasing the measurement systems capacity,
   both in export bandwidth capacity and in collecting capacity.

   Implementation analysis:

      To cope with the increase of IPFIX Exporters and traffic, one
      possible implementation uses IPFIX Concentrators to build a
      hierarchical collection system.  To cope with the variety of
      treatments, one possible implementation uses IPFIX Distributors to



Kobayashi, et al.       Expires November 1, 2009               [Page 12]


Internet-Draft         Mediation Problem Statement            April 2009


      build a distributed collection system.  More specific cases are
      described in section 5.9.

5.3.  Correlation for Data Records

   The correlation amongst Data Records or between Data record and meta
   data provides new metrics or information, including the following.

   o  One-to-one correlation between Data Records

      *  One way delay and packet arrival interval time etc.  One way
         delay from the correlation of Packet Reports from different
         Exporters along a specific path.

      *  Treatment from the correlation of Data Records with the same
         Flow Key(s) observed at incoming/outgoing interfaces.  Examples
         are the rate-limiting ratio, the compression ratio, the
         optimization ratio, etc.

   o  Correlation amongst Data Records

      Average/maximum/minimum values from correlating multiple Data
      Records.  Examples are the average/maximum/minimum packets of
      Flow, the average/maximum/minimum one way delay, the average/
      maximum/minimum packet loss, etc.

   o  Correlation between Data Record and other meta data

      Examples are some BGP attributes associated with Data Record by
      looking up routing table.

   Implementation analysis:

      One possible implementation for the case uses an IPFIX
      Concentrator located between the Metering Processes and Exporting
      Processes on the Original Exporter, or alternatively a separate
      IPFIX Concentrator located between the Original Exporters and
      IPFIX Collectors.

5.4.  Time Composition

   Time composition is defined as the aggregation of consecutive Data
   Records with identical Flow Keys.  It leads to the same output as
   setting a longer active interval timer on Original Exporters with one
   advantage: the creation of new metrics such as average, maximum and
   minimum values from Flow Records with a shorter time interval enables
   administrators to keep track of changes that might have happened
   during the time interval.



Kobayashi, et al.       Expires November 1, 2009               [Page 13]


Internet-Draft         Mediation Problem Statement            April 2009


   Implementation analysis:

      One possible implementation for this case uses an IPFIX
      Concentrator located between the Metering Processes and Exporting
      Processes on the Original Exporter, or alternatively as a separate
      IPFIX Concentrator located between the Original Exporters and
      IPFIX Collectors.

5.5.  Spatial Composition

   Spatial composition is defined as the aggregation of Data Records in
   a set of Observation Points within an Observation Domain, across
   multiple Observation Domains from a single Exporter, or even across
   multiple Exporters.  The spatial composition is divided into four
   types.

   o  Case 1: Spatial Composition within one Observation Domain

      For example, in the case where a link aggregation exists, Data
      Records observed at physical interfaces belonging to the same
      trunk can be merged.

   o  Case 2: Spatial Composition across Observation Domains, but within
      a single Exporter

      For example, in the case where a link aggregation exists, Data
      Records observed at physical interfaces belonging to a same trunk
      grouping beyond the line interface module can be merged.

   o  Case 3: Spatial Composition across Exporters

      Data Records observed within an administrative domain, such as the
      west area and east area of an ISP network, can be merged.

   o  Case 4: Spatial Composition across administrative domains

      Data Records observed across administrative domains, such as
      across different customer networks or different ISP networks, can
      be merged.

   Implementation analysis:

      One possible implementation for the case 1 and 2 uses an IPFIX
      Concentrator located between the Metering Processes and Exporting
      Processes on the Original Exporter.  A separate IPFIX Concentrator
      located between the Original Exporters and IPFIX Collector is a
      valid solution for the case 1, 2, 3, and 4.




Kobayashi, et al.       Expires November 1, 2009               [Page 14]


Internet-Draft         Mediation Problem Statement            April 2009


5.6.  Data Record Anonymization

   IPFIX exports across administrative domains can be used to measure
   traffic for wide-area traffic engineering or to analyze Internet
   traffic trends, as described in the Spatial Composition across
   administrative domains in the previous subsection.
   In such case, administrators need to adhere to privacy protection
   policies and prevent access to confidential traffic measurements by
   other people.  Typically, anonymization techniques enables the
   provision of traffic data to other people without violating these
   policies.

   Generally, anonymization modifies a data set to protect the identity
   of the people or entities described by the data set from being
   disclosed.  It also attempts to preserve sets of network traffic
   properties useful for a given analysis while ensuring the data cannot
   be traced back to the specific networks, hosts, or users generating
   the traffic.  For example, IP address anonymization is particularly
   important for avoiding the identification of the users, hosts, and
   routers.

   Implementation analysis:

      One possible implementation for this case uses an anonymization
      function at the Original Exporter.  However, this increases the
      load on the Original Exporter.  A more flexible implementation
      uses a separate IPFIX Masquerading Proxy between the Original
      Exporter and Collector.

5.7.  Data Retention

   Data retention refers to the storage of traffic data by service
   providers and commercial organizations.  Network administrators
   should retain both IP and voice traffic data, in wired and wireless
   networks, generated by end users while using a service provider's
   services.  The traffic data is required for the purpose of the
   investigation, detection and prosecution of serious crime, if
   necessary.  Data retention services examples are the following:

   o  Fixed telephony (includes fixed voice calls, voicemail, and
      conference and data calls)

   o  Mobile telephony (includes mobile voice calls, voicemail,
      conference and data calls, SMS, and MMS)

   o  Internet telephony (includes every multimedia session associated
      with IP multimedia services)




Kobayashi, et al.       Expires November 1, 2009               [Page 15]


Internet-Draft         Mediation Problem Statement            April 2009


   o  Internet e-mail

   o  Internet access

   Data retention for Internet access services in particular requires a
   measurement system with reliable export and huge storage as the data
   must be available for a long period of time, typically a couple of
   years.

   Implementation analysis:

      Regarding export reliability requirement, the most suitable
      implementation uses the SCTP transport protocol between the
      Original Exporter and Collector.  If a unreliable transport
      protocol such as UDP is used, a legacy exporting device exports
      Data Records to a nearby IPFIX Proxy through UDP, and then an
      IPFIX Proxy could reliably export them to the top IPFIX Collector
      through SCTP.  If a unreliable transport protocol such as UDP is
      used and if there is no IPFIX Proxy, the legacy exporting device
      must duplicate the exports to several Collectors.

      Regarding huge storage requirement, one possible implementation
      uses a decentralized set of Collectors.  If administrators need to
      retrieve specific Data Records, these Collectors would need to be
      equipped with IPFIX Mediations.

5.8.  IPFIX Export from a Branch Office

   Generally, in large enterprise networks, Data Records from branch
   offices are gathered in a central office.  However, in the long
   distance branch office case, the bandwidth for transport IPFIX is
   limited.  Therefore, even if multiple Flow Records type should be of
   interest to the Collector (Flow Records in both directions, Flow
   Records before and after WAN optimization techniques, performance
   metrics associated with the Flow Records exported on regular
   interval), the export bandwidth limitation is an important factor to
   pay attention to.

   Implementation analysis:

      One possible implementation for the case uses an IPFIX
      Concentrator located in a branch office.  The IPFIX Concentrator
      would aggregate and correlate Flow Records to cope with the export
      bandwidth limitation.







Kobayashi, et al.       Expires November 1, 2009               [Page 16]


Internet-Draft         Mediation Problem Statement            April 2009


5.9.  Distributing Data Records

   Recently, several networks have shifted towards integrated networks,
   such as the pure IP and MPLS, which includes IPv4, IPv6, and VPN
   traffic.  Data Record types (IPv4, IPv6, MPLS, and VPN) need to be
   analyzed separately and from different perspectives for different
   organizations.  A single Collector handling all Data Record types
   might become a bottleneck in the collecting infrastructure.  Data
   Records distributed based on their respective types can be exported
   to the appropriate Collector, resulting in the load distribution
   amongst multiple Collectors.

   Implementation analysis:

      One possible implementation for this case uses the replications of
      the IPFIX Message in an Original Exporter for multiple IPFIX
      Collectors.  Each Collector then extracts the Data Record required
      by its own applications.  However, the replication increases the
      load of the Exporting Process and the waste of the bandwidth
      between the Exporter and Collector.

      A more sophisticated implementation uses an IPFIX Distributor
      located between the Metering Processes and Exporting Processes in
      an Original Exporter.  The IPFIX Distributor determines
      respectively to which Collector a Data Record is exported by
      filtering certain field values.  If a Original Exporter does not
      have IPFIX Distributor capability, it exports Data Records to a
      nearby separate IPFIX Distributor, and then the IPFIX Distributor
      could distribute them to the appropriate IPFIX Collectors.

      For example, in the case of distributing a specific customer's
      Data Records, an IPFIX Distributor needs to identify the customer
      networks.  The Route Distinguisher (RD), ingress interface,
      peering AS number, or BGP Next-Hop, or simply the network prefix
      may be evaluated to distinguish different customer networks.  In
      the following figure, the IPFIX Distributor reroutes Data Records
      on the basis of the RD value.  This system enables each customer's
      traffic to be inspected independently.













Kobayashi, et al.       Expires November 1, 2009               [Page 17]


Internet-Draft         Mediation Problem Statement            April 2009


                                               .---------.
                                               |Traffic  |
                                         .---->|Collector|<==>Customer#A
                                         |     |#1       |
                                         |     '---------'
                                      RD=100:1
    .----------.        .-----------.    |
    |IPFIX     |        |IPFIX      |----'     .---------.
    |Exporter#1|        |Distributor| RD=100:2 |Traffic  |
    |          |------->|           |--------->|Collector|<==>Customer#B
    |          |        |           |          |#2       |
    |          |        |           |----.     '---------'
    '----------'        '-----------'    |
                                      RD=100:3
                                         |     .---------.
                                         |     |Traffic  |
                                         '---->|Collector|<==>Customer#C
                                               |#3       |
                                               '---------'

      Figure A: Distributing Data Records to Collectors using IPFIX
      Distributor

5.10.  Flow-based Sampling and Selection

   Generally, the distribution of the number of packets per Flow seems
   to be heavy-tailed.  Most types of Flow Records are likely to be
   small Flows consisting of a small number of packets.  The measurement
   system is overwhelmed with a huge amount of these small Flows.  If
   statistics information of small Flows is exported as merged data by
   applying a policy or threshold, the load on the Exporter is reduced.
   Furthermore, if the flow distribution is known, exporting only a
   subset of the Data Records might be sufficient.

   Implementation analysis:

      One possible implementation for this case uses an IPFIX
      Concentrator located between the Metering Processes and Exporting
      Processes on the Original Exporter, or alternatively as a separate
      IPFIX Concentrator located between the Original Exporters and
      IPFIX Collectors.  A set of IPFIX Mediation functions, such as
      filtering, selecting and aggregation is used in the IPFIX
      Concentrator.








Kobayashi, et al.       Expires November 1, 2009               [Page 18]


Internet-Draft         Mediation Problem Statement            April 2009


5.11.  Interoperability between Legacy Protocols and IPFIX

   During the migration process from a legacy protocol such as NetFlow
   [RFC3954] to IPFIX, both NetFlow exporting devices and IPFIX
   Exporters are likely to coexist in the same network.  Operators need
   to continue measuring the traffic data from legacy exporting devices,
   even after introducing IPFIX Collectors.

   Implementation analysis:

      One possible implementation for this case uses an IPFIX Proxy that
      converts a legacy protocol to IPFIX.







































Kobayashi, et al.       Expires November 1, 2009               [Page 19]


Internet-Draft         Mediation Problem Statement            April 2009


6.  IPFIX Mediators Implementation Specific Problems

6.1.  Loss of Original Exporter Information

   Both the Exporter IP address indicated by the source IP address of
   the IPFIX Transport Session and the Observation Domain ID included in
   the IPFIX Message header are likely to be lost by an IPFIX Mediator
   such as IPFIX Concentrator too.  In some cases, a IPFIX Masquerading
   Proxy might drop the information.  In other cases, the Collector must
   recognize the Original Exporter (and potentially the Observation
   Domain and Observation Point as well) whether Data Records go through
   an IPFIX Mediator or not.  Note that, if the Mediator can not
   communicate the Original Exporter IP address, then the top level
   Collector will wrongly deduce that the IP address of the IPFIX
   Mediator is that of the Original Exporter.

   In the following figure, a Collector can identify two IP addresses:
   10.1.1.3 (IPFIX Mediator) and 10.1.1.2 (Exporter#2), respectively.
   The Collector, however, needs to somehow recognize both Exporter#1
   and Exporter#2, which are the Original Exporters.  The IPFIX Mediator
   must have a specific way to the Original Exporter IP address to the
   IPFIX Collector.

   .----------.          .--------.
   |IPFIX     |          |IPFIX   |
   |Exporter#1|--------->|Mediator|---+
   |          |          |        |   |
   '----------'          '--------'   |      .---------.
   IP:10.1.1.1         IP:10.1.1.3    '----->|IPFIX    |
   ODID:10             ODID:0                |Collector|
                                      +----->|         |
   .----------.                       |      '---------'
   |IPFIX     |                       |
   |Exporter#2|-----------------------'
   |          |
   '----------'
   IP:10.1.1.2
   ODID:20

   Figure B: Loss of Original Exporter Information.

6.2.  Loss of Base Time Information

   The Export Time field included in the IPFIX Message header indicates
   the base time for Data Records.  IPFIX Information Elements,
   described in [RFC5102], have delta time fields that indicate the time
   difference from the value of the Export Time field.  If the Data
   Records include any delta time fields and the IPFIX Mediator



Kobayashi, et al.       Expires November 1, 2009               [Page 20]


Internet-Draft         Mediation Problem Statement            April 2009


   overwrites the Export Time field when sending IPFIX Messages, the
   delta time fields become meaningless and, because Collectors cannot
   recognize this situation, wrong time values are propagated.

6.3.  Transport Sessions Management

   Maintaining relationships between the incoming Transport Sessions and
   the outgoing ones depends on the Mediator's implementation.  If
   multiple incoming Transport Sessions are relayed to a single outgoing
   Transport Session, and if the IPFIX Mediators shuts down its outgoing
   Transport Session, Data Records on other incoming Transport Sessions
   would not be relayed at all.  In the case of resetting of an incoming
   session, the behavior of the IPFIX Mediator needs to be specified.

6.4.  Loss of Option Template Information

   In some cases, depending on the implementation of the IPFIX
   Mediators, the information that is reported by the Option Templates
   could also be lost.  If, for example, the Sampling rate is not
   communicated from the Mediator to the Collector, the Collector would
   miscalculate the traffic volume.  This might lead to crucial
   problems.  Even if an IPFIX Mediator was to simply relay received
   Option Template Information, the values of its scope fields could
   become meaningless in the context of a different Transport Sessions.
   The minimal information to be communicated by an IPFIX Mediator must
   be specified.

6.5.  Template ID Management

   The Template ID is unique on the basis of the Transport Session and
   Observation Domain ID.  If Mediations are not able to manage the
   relations amongst the Template IDs and the incoming Transport Session
   information, and if the Template ID is used in the Options Template
   scope, the Mediators would, for example, relay wrong values in the
   scope field and in the Template Withdrawal Message.  The Collector
   would thus not be able to interpret the Template ID in the Template
   Withdrawal Message and in the Options Template scope.  As a
   consequence, there is a risk that the Collector would then shut down
   the IPFIX Transport Session.

   For example, an IPFIX Distributor must maintain the state of the
   incoming Transport Sessions in order to manage the Template ID on its
   outgoing Transport Session correctly.  In the following figure, even
   if the Transport Session from Exporter re-initializes, the IPFIX
   Distributor must manage the association of Template IDs in specific
   Transport Session.  Typically, when the Exporter#1 Transport Session
   re-initialized, the Template ID 256 replaced the previous Template ID
   258, while the IPFIX Distributor will keep exporting the Template ID



Kobayashi, et al.       Expires November 1, 2009               [Page 21]


Internet-Draft         Mediation Problem Statement            April 2009


   256 to the Collector.


   .----------. OLD: Template ID 258
   |IPFIX     | NEW: Template ID 256
   |Exporter#1|----+
   |          |    |
   '----------'    X
   .----------.    |           .-----------.               .----------.
   |IPFIX     |    '---------->|           |               |          |
   |Exporter#2|--------------->|IPFIX      |-------X------>|IPFIX     |
   |          |Template ID 257 |Distributor|Template ID 256| Collector|
   '----------'    +---------->|           |               |          |
   .----------.    |           '-----------'               '----------'
   |IPFIX     |    |
   |Exporter#3|----'
   |          | Template ID 256
   '----------'

   Figure C: Relaying from Multiple Transport Sessions to Single
   Transport Session.

6.6.  Consideration for Network Topology

   While IPFIX Mediation can be applied anywhere, caution should be
   taken as how to aggregate the counters, as there is a potential risk
   of double-counting.  For example, if three Exporters export Flow
   Records related to the same Flow, the one-way delay can be
   calculated, while the summing up the number of packets and bytes does
   not make sense.  Alternatively, if three Exporters export Flow
   Records entering an administrative domain, then the sum of the
   packets and bytes is a valid operation.  Therefore, the possible
   function to be applied to Flow Records must take into consideration
   the measurement topology.  The information such as the network
   topology, or at least the Observation Point and measurement
   direction, is required on the IPFIX Mediation.

6.7.  Exporting the Function Item

   In some case, the top IPFIX Collector needs to recognize which
   specific function(s) the IPFIX Mediation has executed on the Data
   Records.  The IPFIX Collector cannot distinguish between Time
   Composition, Spatial Composition, and Flow Key aggregation, if the
   IPFIX Mediator does not export the applied function.  Some parameters
   related to the function also would need to be exported.  For example,
   in case of Time Composition, the active time of original Flow Records
   is required to interpret the minimum/maximum counter correctly.  In
   case of Spatial Composition, spatial area information on which Data



Kobayashi, et al.       Expires November 1, 2009               [Page 22]


Internet-Draft         Mediation Problem Statement            April 2009


   Records is aggregated is required.

6.8.  Consideration for Aggregation

   In case of Flow Key aggregation, Time Composition, and Spatial
   Composition, there are the following considerations:

   o  Aggregation rule for non Flow Keys

      There are no obvious rules of non Flow Keys.  For example, if an
      IPFIX Mediation receives two Flow Records with different DSCP
      values, and this DSCP field is not a Flow Key, those two Flow
      Records can be aggregated based on the Flow Keys value.  However,
      there is no rules for what the DSCP value should be for the
      aggregated Data Record.  Potential solutions are: the value of
      single of the two DSCP, the value 0 (in this case, the value 0 is
      a valid DSCP value), or removing a DSCP field in its Data Record.

   o  Configured Selection Fraction on aggregation

      There is no obvious rules of how to compute Configured Selection
      Fraction, and whether a Mediator should report Configured
      Selection Fraction, when aggregation resulting from Sampling.  For
      example, special care must be taken in the following: aggregation
      resulting from the different Configured Selection Fraction,
      aggregation resulting from different Sampling techniques, such as
      Systematic Count-Based Sampling and Random n-out-of-N Sampling
      etc.























Kobayashi, et al.       Expires November 1, 2009               [Page 23]


Internet-Draft         Mediation Problem Statement            April 2009


7.  Summary and Conclusion

   This document described the problems that network administrators have
   been facing, the applicability of IPFIX Mediation to these problems,
   and the problems related to the implementation of IPFIX Mediators.
   To assist the operations of the Exporters and Collectors, there are
   various IPFIX Mediations from which the administrators may select.
   Examples of the applicability of IPFIX Mediation are as follows.

   o  Regarding large-scale measurement system, IPFIX Concentrators or
      IPFIX Distributors help to achieve traffic analysis with high data
      accuracy and fine flow granularity even as IP traffic grows.  As
      IPFIX Mediation capabilities, Flow selection Sampling,
      aggregation, and composition are effective.

   o  Regarding data retention, IPFIX Mediators enhance the export
      reliability, and the storage of the measurement system.

   o  Regarding the distribution of Data Records, IPFIX Distributors
      help to achieve multipurpose traffic analysis for different
      organizations, or help to achieve respective traffic analysis
      based on Data Record types(IPv4, IPv6, MPLS, and VPN).

   o  Regarding IPFIX Exporting across domains, IPFIX Masquerading
      Proxies help administrators to anonymize or filter Flow Records/
      Packet Reports, preventing privacy violations.

   o  Regarding interoperability, IPFIX Proxies provide interoperability
      between legacy protocols and IPFIX, even during the migration
      period to IPFIX.

   As a result, the IPFIX Mediation benefits become apparent.  However,
   there are still some open issues with the use of IPFIX Mediators.

   o  Both Observation Point and IPFIX Message header information, such
      as the Exporter IP address, Observation Domain ID, and Export Time
      field, might be lost.  This data should therefore be communicated
      between the Original Exporter and Collector via the IPFIX
      Mediator.

   o  IPFIX Mediators are required to manage Transport Sessions,
      Template IDs, and Observation Domain IDs.  Otherwise, anomalous
      IPFIX messages could be created.

   o  Data advertised by Option Templates from the Original Exporter,
      such as the Sampling rate and Sampling algorithm used, might be
      lost.  If a Collector is not informed of current Sampling rates,
      traffic information might become worthless.



Kobayashi, et al.       Expires November 1, 2009               [Page 24]


Internet-Draft         Mediation Problem Statement            April 2009


   These problems stem from the fact that no standards regarding IPFIX
   Mediation have been set.  In particular, the minimum set of
   information that should be communicated between Original Exporters
   and Collectors, the management between different IPFIX Transport
   Sessions, and the internal components of IPFIX Mediators should be
   standardized.













































Kobayashi, et al.       Expires November 1, 2009               [Page 25]


Internet-Draft         Mediation Problem Statement            April 2009


8.  Security Considerations

   A flow-based measurement system must prevent potential security
   threats: the disclosure of confidential traffic data, injection of
   incorrect data, and unauthorized access to traffic data.  These
   security threats of the IPFIX protocol are covered by the security
   considerations section in [RFC5101] and are still valid for IPFIX
   Mediators.

   And a measurement system must also prevent following security threats
   related to IPFIX Mediation:

   o  Attacks against IPFIX Mediator

      IPFIX Mediators can be considered as a prime target for attacks,
      as an alternative to IPFIX Exporters and Collectors.  IPFIX
      Proxies or Masquerading Proxies need to prevent unauthorized
      access or denial-of-service (DoS) attacks from untrusted public
      networks.

   o  Man-in-the-middle attack by untrusted IPFIX Mediator

      The Collector-Mediator-Exporter structure model would increase the
      risk of the man-in-the-middle attack.

   o  Configuration on IPFIX Mediation

      In the case of IPFIX Distributors and IPFIX Masquerading Proxies,
      an accidental misconfiguration and unauthorized access to
      configuration data could lead to the crucial problem of disclosure
      of confidential traffic data.




















Kobayashi, et al.       Expires November 1, 2009               [Page 26]


Internet-Draft         Mediation Problem Statement            April 2009


9.  IANA Considerations

   This document has no actions for IANA.
















































Kobayashi, et al.       Expires November 1, 2009               [Page 27]


Internet-Draft         Mediation Problem Statement            April 2009


10.  Acknowledgements

   The authors would like to thank Gerhard Muenz, Keisuke Ishibashi and
   Nevil Brownlee for providing valuable comments and suggestions.















































Kobayashi, et al.       Expires November 1, 2009               [Page 28]


Internet-Draft         Mediation Problem Statement            April 2009


11.  References

11.1.  Normative References

   [RFC5101]  Claise, B., "Specification of the IP Flow Information
              Export (IPFIX) Protocol for the Exchange of IP Traffic
              Flow Information", January 2008.

   [RFC5476]  Claise, B., "Packet Sampling (PSAMP) Protocol
              Specifications", March 2009.

11.2.  Informative References

   [I-D.ietf-ipfix-file]
              Trammell, B., Boschi, E., Mark, L., Zseby, T., and A.
              Wagner, "Specification of the IPFIX File Format",
              draft-ietf-ipfix-file-05 (work in progress) ,
              November 2007.

   [I-D.ietf-ipfix-mib]
              Dietz, T., Claise, B., and A. Kobayashi, "Definitions of
              Managed Objects for IP Flow Information Export",
              draft-ietf-ipfix-mib-06 (work in progress) , March 2009.

   [I-D.ietf-psamp-mib]
              Dietz, T. and B. Claise, "Definitions of Managed Objects
              for Packet Sampling", draft-ietf-psamp-mib-06 (work in
              progress) , June 2006.

   [RFC3917]  Quittek, J., Zseby, T., Claise, B., and S. Zander,
              "Requirements for IP Flow Information Export(IPFIX)",
              October 2004.

   [RFC3954]  Claise, B., "Cisco Systems NetFlow Services Export Version
              9", October 2004.

   [RFC5102]  Quittek, J., Bryant, S., Claise, B., Aitken, P., and J.
              Meyer, "Information Model for IP Flow Information Export",
              January 2008.

   [RFC5470]  Sadasivan, G., Brownlee, N., Claise, B., and J. Quittek,
              "Architecture for IP Flow Information Export", March 2009.

   [RFC5472]  Zseby, T., Boschi, E., Brownlee, N., and B. Claise, "IP
              Flow Information Export (IPFIX) Applicability",
              March 2009.

   [RFC5474]  Duffield, N., "A Framework for Packet Selection and



Kobayashi, et al.       Expires November 1, 2009               [Page 29]


Internet-Draft         Mediation Problem Statement            April 2009


              Reporting", March 2009.

   [RFC5475]  Zseby, T., Molina, M., Duffield, N., Niccolini, S., and F.
              Raspall, "Sampling and Filtering Techniques for IP Packet
              Selection", March 2009.

   [RFC5477]  Dietz, T., Claise, B., Aitken, P., Dressler, F., and G.
              Carle, "Information Model for Packet Sampling Exports",
              March 2009.

   [TRAFGRW]  Cho, K., Fukuda, K., Esaki, H., and A. Kato, "The Impact
              and Implications of the Growth in Residential User-to-User
              Traffic", SIGCOMM2006, pp. 207-218, Pisa, Italy, September
              2006. .





































Kobayashi, et al.       Expires November 1, 2009               [Page 30]


Internet-Draft         Mediation Problem Statement            April 2009


Authors' Addresses

   Atsushi Kobayashi
   NTT Information Sharing Platform Laboratories
   3-9-11 Midori-cho
   Musashino-shi, Tokyo  180-8585
   Japan

   Phone: +81-422-59-3978
   Email: akoba@nttv6.net
   URI:   http://www3.plala.or.jp/akoba/


   Benoit Claise
   Cisco Systems, Inc.
   De Kleetlaan 6a b1
   Diegem  1831
   Belgium

   Phone: +32 2 704 5622
   Email: bclaise@cisco.com


   Haruhiko Nishida
   NTT Information Sharing Platform Laboratories
   3-9-11 Midori-cho
   Musashino-shi, Tokyo  180-8585
   Japan

   Phone: +81-422-59-3978
   Email: nishida.haruhiko@lab.ntt.co.jp


   Christoph Sommer
   University of Erlangen-Nuremberg
   Department of Computer Science 7
   Martensstr. 3
   Erlangen  91058
   Germany

   Phone: +49 9131 85-27993
   Email: christoph.sommer@informatik.uni-erlangen.de
   URI:   http://www7.informatik.uni-erlangen.de/~sommer/








Kobayashi, et al.       Expires November 1, 2009               [Page 31]


Internet-Draft         Mediation Problem Statement            April 2009


   Falko Dressler
   University of Erlangen-Nuremberg
   Department of Computer Science 7
   Martensstr. 3
   Erlangen  91058
   Germany

   Phone: +49 9131 85-27914
   Email: dressler@informatik.uni-erlangen.de
   URI:   http://www7.informatik.uni-erlangen.de/~dressler/


   Stephan Emile
   France Telecom
   2 avenue Pierre Marzin
   Lannion,   F-22307

   Fax:   +33 2 96 05 18 52
   Email: emile.stephan@orange-ftgroup.com































Kobayashi, et al.       Expires November 1, 2009               [Page 32]


Html markup produced by rfcmarkup 1.129b, available from https://tools.ietf.org/tools/rfcmarkup/