draft-ietf-rtfm-acct-arch-report-00.txt | rfc2063.txt | |||
---|---|---|---|---|
Internet Engineering Task Force Nevil Brownlee | Network Working Group N. Brownlee | |||
INTERNET-DRAFT The University of Auckland | Request for Comments: 2063 The University of Auckland | |||
Cyndi Mills | Category: Experimental C. Mills | |||
BBN Systems and Technologies | BBN Systems and Technologies | |||
Greg Ruth | G. Ruth | |||
GTE Laboratories, Inc | GTE Laboratories, Inc. | |||
January 1997 | ||||
Feb 1996 | ||||
Traffic Flow Measurement: Architecture | Traffic Flow Measurement: Architecture | |||
<draft-ietf-rtfm-acct-arch-report-00.txt> | ||||
Status of this Memo | Status of this Memo | |||
This document is an Internet Draft. Internet Drafts are working | This memo defines an Experimental Protocol for the Internet | |||
documents of the Internet Engineering Task Force (IETF), its Areas, and | community. This memo does not specify an Internet standard of any | |||
its Working Groups. Note that other groups may also distribute working | kind. Discussion and suggestions for improvement are requested. | |||
documents as Internet Drafts. This Internet Draft is a product of the | Distribution of this memo is unlimited. | |||
Internet Accounting Working Group of the IETF. | ||||
Internet Drafts are draft documents valid for a maximum of six months. | ||||
Internet Drafts may be updated, replaced, or obsoleted by other | ||||
documents at any time. It is not appropriate to use Internet Drafts as | ||||
reference material or to cite them other than as a "working draft" or | ||||
"work in progress." | ||||
Please check the I-D abstract listing contained in the Internet-drafts | ||||
Shadow Directories on nic.ddn.mil, nnsc.nsf.net, nic.nordu.net, | ||||
ftp.nisc.sri.com or munnari.oz.au to learn the current status of this or | ||||
any other Internet Draft. | ||||
Abstract | Abstract | |||
This document describes an architecture for the measurement and | This document describes an architecture for the measurement and | |||
reporting of network traffic flows, discusses how this relates to an | reporting of network traffic flows, discusses how this relates to an | |||
overall network traffic flow architecture, and describes how it can be | overall network traffic flow architecture, and describes how it can | |||
used within the Internet. It is intended to provide a starting point | be used within the Internet. It is intended to provide a starting | |||
for the Realtime Traffic Flow Measurement Working Group. | point for the Realtime Traffic Flow Measurement Working Group. | |||
Contents | ||||
1 Statement of Purpose and Scope 3 | ||||
2 Traffic Flow Measurement Architecture 4 | ||||
2.1 Meters and Traffic Flows . . . . . . . . . . . . . . . . . . . 4 | ||||
2.2 Interaction Between METER and METER READER . . . . . . . . . . 7 | ||||
2.3 Interaction Between MANAGER and METER . . . . . . . . . . . . 7 | ||||
2.4 Interaction Between MANAGER and METER READER . . . . . . . . . 8 | ||||
2.5 Multiple METERs or METER READERs . . . . . . . . . . . . . . . 8 | ||||
2.6 Interaction Between MANAGERs (MANAGER - MANAGER) . . . . . . . 9 | ||||
2.7 Between METER READER and APPLICATION . . . . . . . . . . . . . 9 | ||||
3 Traffic Flows and Reporting Granularity 10 | ||||
3.1 Flows and their Attributes . . . . . . . . . . . . . . . . . . 10 | ||||
3.2 Granularity of Flow Measurements . . . . . . . . . . . . . . . 12 | ||||
3.3 Rolling Counters, Timestamps, Report-in-One-Bucket-Only . . . 14 | ||||
4 Meters 15 | ||||
4.1 Meter Structure . . . . . . . . . . . . . . . . . . . . . . . 16 | ||||
4.2 Flow Table . . . . . . . . . . . . . . . . . . . . . . . . . . 17 | ||||
4.3 Packet Handling, Packet Matching . . . . . . . . . . . . . . . 18 | ||||
4.4 Rules and Rule Sets . . . . . . . . . . . . . . . . . . . . . 21 | ||||
4.5 Maintaining the Flow Table . . . . . . . . . . . . . . . . . . 25 | ||||
4.6 Handling Increasing Traffic Levels . . . . . . . . . . . . . . 26 | ||||
5 Meter Readers 26 | ||||
5.1 Identifying Flows in Flow Records . . . . . . . . . . . . . . 26 | ||||
5.2 Usage Records, Flow Data Files . . . . . . . . . . . . . . . . 27 | ||||
5.3 Meter to Meter Reader: Usage Record Transmission . . . . . . 27 | ||||
6 Managers 28 | ||||
6.1 Between Manager and Meter: Control Functions . . . . . . . . 28 | ||||
6.2 Between Manager and Meter Reader: Control Functions . . . . . 29 | ||||
6.3 Exception Conditions . . . . . . . . . . . . . . . . . . . . . 31 | ||||
6.4 Standard Rule Sets . . . . . . . . . . . . . . . . . . . . . . 32 | ||||
7 APPENDICES 33 | ||||
7.1 Appendix A: Network Characterisation . . . . . . . . . . . . 33 | ||||
7.2 Appendix B: Recommended Traffic Flow Measurement Capabilities 34 | ||||
7.3 Appendix C: List of Defined Flow Attributes . . . . . . . . . 35 | ||||
7.4 Appendix D: List of Meter Control Variables . . . . . . . . . 36 | ||||
8 Acknowledgments 36 | Table of Contents | |||
9 References 37 | 1 Statement of Purpose and Scope 2 | |||
2 Traffic Flow Measurement Architecture 4 | ||||
2.1 Meters and Traffic Flows . . . . . . . . . . . . . . . . . . 4 | ||||
2.2 Interaction Between METER and METER READER . . . . . . . . . 6 | ||||
2.3 Interaction Between MANAGER and METER . . . . . . . . . . . 6 | ||||
2.4 Interaction Between MANAGER and METER READER . . . . . . . . 7 | ||||
2.5 Multiple METERs or METER READERs . . . . . . . . . . . . . . 7 | ||||
2.6 Interaction Between MANAGERs (MANAGER - MANAGER) . . . . . . 8 | ||||
2.7 METER READERs and APPLICATIONs . . . . . . . . . . . . . . . 8 | ||||
3 Traffic Flows and Reporting Granularity 9 | ||||
3.1 Flows and their Attributes . . . . . . . . . . . . . . . . . 9 | ||||
3.2 Granularity of Flow Measurements . . . . . . . . . . . . . . 11 | ||||
3.3 Rolling Counters, Timestamps, Report-in-One-Bucket-Only . . 13 | ||||
4 Meters 15 | ||||
4.1 Meter Structure . . . . . . . . . . . . . . . . . . . . . . 15 | ||||
4.2 Flow Table . . . . . . . . . . . . . . . . . . . . . . . . . 17 | ||||
4.3 Packet Handling, Packet Matching . . . . . . . . . . . . . . 17 | ||||
4.4 Rules and Rule Sets . . . . . . . . . . . . . . . . . . . . 21 | ||||
4.5 Maintaining the Flow Table . . . . . . . . . . . . . . . . . 24 | ||||
4.6 Handling Increasing Traffic Levels . . . . . . . . . . . . . 25 | ||||
10 Security Considerations 37 | 5 Meter Readers 26 | |||
5.1 Identifying Flows in Flow Records . . . . . . . . . . . . . 26 | ||||
5.2 Usage Records, Flow Data Files . . . . . . . . . . . . . . . 27 | ||||
5.3 Meter to Meter Reader: Usage Record Transmission. . . . . . 27 | ||||
6 Managers 28 | ||||
6.1 Between Manager and Meter: Control Functions . . . . . . . 28 | ||||
6.2 Between Manager and Meter Reader: Control Functions . . . 29 | ||||
6.3 Exception Conditions . . . . . . . . . . . . . . . . . . . . 31 | ||||
6.4 Standard Rule Sets . . . . . . . . . . . . . . . . . . . . 32 | ||||
7 APPENDICES 33 | ||||
7.1 Appendix A: Network Characterisation . . . . . . . . . . . . 33 | ||||
7.2 Appendix B: Recommended Traffic Flow Measurement Capabilities 34 | ||||
7.3 Appendix C: List of Defined Flow Attributes . . . . . . . . 35 | ||||
7.4 Appendix D: List of Meter Control Variables . . . . . . . . 36 | ||||
8 Acknowledgments 36 | ||||
9 References 37 | ||||
10 Security Considerations 37 | ||||
11 Authors' Addresses 37 | ||||
11 Author's Addresses 37 | ||||
1 Statement of Purpose and Scope | 1 Statement of Purpose and Scope | |||
This document describes an architecture for traffic flow measurement and | This document describes an architecture for traffic flow measurement | |||
reporting for data networks which has the following characteristics: | and reporting for data networks which has the following | |||
characteristics: | ||||
- The traffic flow model can be consistently applied to any | - The traffic flow model can be consistently applied to any | |||
protocol/application at any network layer (e.g. network, | protocol/application at any network layer (e.g. network, | |||
transport, application layers). | transport, application layers). | |||
- Traffic flow attributes are defined in such a way that they are | - Traffic flow attributes are defined in such a way that they are | |||
valid for multiple networking protocol stacks, and that traffic | valid for multiple networking protocol stacks, and that traffic | |||
flow measurement implementations are useful in MULTI-PROTOCOL | flow measurement implementations are useful in MULTI-PROTOCOL | |||
environments. | environments. | |||
- Users may specify their traffic flow measurement requirements in a | - Users may specify their traffic flow measurement requirements | |||
simple manner, allowing them to collect the flow data they need | in a simple manner, allowing them to collect the flow data they | |||
while ignoring other traffic. | need while ignoring other traffic. | |||
- The data reduction effort to produce requested traffic flow | - The data reduction effort to produce requested traffic flow | |||
information is placed as near as possible to the network | information is placed as near as possible to the network | |||
measurement point. This reduces the volume of data to be obtained | measurement point. This reduces the volume of data to be | |||
(and transmitted across the network for storage), and minimises the | obtained (and transmitted across the network for storage), | |||
amount of processing required in traffic flow analysis | and minimises the amount of processing required in traffic | |||
applications. | flow analysis applications. | |||
The architecture specifies common metrics for measuring traffic flows. | The architecture specifies common metrics for measuring traffic | |||
By using the same metrics, traffic flow data can be exchanged and | flows. By using the same metrics, traffic flow data can be exchanged | |||
compared across multiple platforms. Such data is useful for: | and compared across multiple platforms. Such data is useful for: | |||
- Understanding the behaviour of existing networks, | - Understanding the behaviour of existing networks, | |||
- Planning for network development and expansion, | - Planning for network development and expansion, | |||
- Quantification of network performance, | - Quantification of network performance, | |||
- Verifying the quality of network service, and | - Verifying the quality of network service, and | |||
- Attribution of network usage to users. | - Attribution of network usage to users. | |||
The traffic flow measurement architecture is deliberately structured so | The traffic flow measurement architecture is deliberately structured | |||
that specific protocol implementations may extend coverage to | so that specific protocol implementations may extend coverage to | |||
multi-protocol environments and to other protocol layers, such as usage | multi-protocol environments and to other protocol layers, such as | |||
measurement for application-level services. Use of the same model for | usage measurement for application-level services. Use of the same | |||
both network- and application-level measurement may simplify the | model for both network- and application-level measurement may | |||
development of generic analysis applications which process and/or | simplify the development of generic analysis applications which | |||
correlate any or all levels of traffic and usage information. Within | process and/or correlate any or all levels of traffic and usage | |||
this docuemt the term 'usage data' is used as a generic term for the | information. Within this docuemt the term 'usage data' is used as a | |||
data obtained using the traffic flow measurement architecture. | generic term for the data obtained using the traffic flow measurement | |||
architecture. | ||||
This document is not a protocol specification. It specifies and | This document is not a protocol specification. It specifies and | |||
structures the information that a traffic flow measurement system needs | structures the information that a traffic flow measurement system | |||
to collect, describes requirements that such a system must meet, and | needs to collect, describes requirements that such a system must | |||
outlines tradeoffs which may be made by an implementor. | meet, and outlines tradeoffs which may be made by an implementor. | |||
For performance reasons, it may be desirable to use traffic information | For performance reasons, it may be desirable to use traffic | |||
gathered through traffic flow measurement in lieu of network statistics | information gathered through traffic flow measurement in lieu of | |||
obtained in other ways. Although the quantification of network | network statistics obtained in other ways. Although the | |||
performance is not the primary purpose of this architecture, the | quantification of network performance is not the primary purpose of | |||
measured traffic flow data may be used as an indication of network | this architecture, the measured traffic flow data may be used as an | |||
performance. | indication of network performance. | |||
A cost recovery structure decides "who pays for what." The major issue | A cost recovery structure decides "who pays for what." The major | |||
here is how to construct a tariff (who gets billed, how much, for which | issue here is how to construct a tariff (who gets billed, how much, | |||
things, based on what information, etc). Tariff issues include | for which things, based on what information, etc). Tariff issues | |||
fairness, predictability (how well can subscribers forecast their | include fairness, predictability (how well can subscribers forecast | |||
network charges), practicality (of gathering the data and administering | their network charges), practicality (of gathering the data and | |||
the tariff), incentives (e.g. encouraging off-peak use), and cost | administering the tariff), incentives (e.g. encouraging off-peak | |||
recovery goals (100% recovery, subsidisation, profit making). Issues | use), and cost recovery goals (100% recovery, subsidisation, profit | |||
such as these are not covered here. | making). Issues such as these are not covered here. | |||
Background information explaining why this approach was selected is | Background information explaining why this approach was selected is | |||
provided by 'Traffic Flow Measurement: Background' RFC [1]. | provided by 'Traffic Flow Measurement: Background' RFC [1]. | |||
2 Traffic Flow Measurement Architecture | 2 Traffic Flow Measurement Architecture | |||
A traffic flow measurement system is used by network Operations | A traffic flow measurement system is used by network Operations | |||
personnel for managing and developing a network. It provides a tool for | personnel for managing and developing a network. It provides a tool | |||
measuring and understanding the network's traffic flows. This | for measuring and understanding the network's traffic flows. This | |||
information is useful for many purposes, as mentioned in section 1 | information is useful for many purposes, as mentioned in section 1 | |||
(above). | (above). | |||
The following sections outline a model for traffic flow measurement, | The following sections outline a model for traffic flow measurement, | |||
which draws from working drafts of the OSI accounting model [2]. Future | which draws from working drafts of the OSI accounting model [2]. | |||
extensions are anticipated as the model is refined to address additional | Future extensions are anticipated as the model is refined to address | |||
protocol layers. | additional protocol layers. | |||
2.1 Meters and Traffic Flows | 2.1 Meters and Traffic Flows | |||
At the heart of the traffic measurement model are network entities | At the heart of the traffic measurement model are network entities | |||
called traffic METERS. Meters count certain attributes (such as numbers | called traffic METERS. Meters count certain attributes (such as | |||
of packets and bytes) and classify them as belonging to ACCOUNTABLE | numbers of packets and bytes) and classify them as belonging to | |||
ENTITIES using other attributes (such as source and destination | ACCOUNTABLE ENTITIES using other attributes (such as source and | |||
addresses). An accountable entity is someone who (or something which) | destination addresses). An accountable entity is someone who (or | |||
is responsible for some activitiy on the network. It may be a user, a | something which) is responsible for some activitiy on the network. | |||
host system, a network, a group of networks, etc, depending on the | It may be a user, a host system, a network, a group of networks, etc, | |||
granularity specified by the meter's configuration. | depending on the granularity specified by the meter's configuration. | |||
We assume that routers or traffic monitors throughout a network are | We assume that routers or traffic monitors throughout a network are | |||
instrumented with meters to measure traffic. Issues surrounding the | instrumented with meters to measure traffic. Issues surrounding the | |||
choice of meter placement are discussed in the 'Traffic Flow | choice of meter placement are discussed in the 'Traffic Flow | |||
Measurement: Background' RFC [1]. An important aspect of meters is | Measurement: Background' RFC [1]. An important aspect of meters is | |||
that they provide a way of succinctly aggregating entity usage | that they provide a way of succinctly aggregating entity usage | |||
information. | information. | |||
For the purpose of traffic flow measurement we define the concept of a | For the purpose of traffic flow measurement we define the concept of | |||
TRAFFIC FLOW, which is an artificial logical equivalent to a call or | a TRAFFIC FLOW, which is an artificial logical equivalent to a call | |||
connection. A flow is a portion of traffic, delimited by a start and | or connection. A flow is a portion of traffic, delimited by a start | |||
stop time, that was generated by a particular accountable entity. | and stop time, that was generated by a particular accountable entity. | |||
Attribute values (source/destination addresses, packet counts, byte | Attribute values (source/destination addresses, packet counts, byte | |||
counts, etc.) associated with a flow are aggregate quantities | counts, etc.) associated with a flow are aggregate quantities | |||
reflecting events which take place in the DURATION between the start and | reflecting events which take place in the DURATION between the start | |||
stop times. The start time of a flow is fixed for a given flow; the end | and stop times. The start time of a flow is fixed for a given flow; | |||
time may increase with the age of the flow. | the end time may increase with the age of the flow. | |||
For connectionless network protocols such as IP there is by definition | For connectionless network protocols such as IP there is by | |||
no way to tell whether a packet with a particular source/destination | definition no way to tell whether a packet with a particular | |||
combination is part of a stream of packets or not - each packet is | source/destination combination is part of a stream of packets or not | |||
completely independent. A traffic meter has, as part of its | - each packet is completely independent. A traffic meter has, as | |||
configuration, a set of 'rules' which specify the flows of interest, in | part of its configuration, a set of 'rules' which specify the flows | |||
terms of the values of their attributes. It derives attribute values | of interest, in terms of the values of their attributes. It derives | |||
from each observed packet, and uses these to decide which flow they | attribute values from each observed packet, and uses these to decide | |||
belong to. Classifying packets into 'flows' in this way provides an | which flow they belong to. Classifying packets into 'flows' in this | |||
economical and practical way to measure network traffic and ascribe it | way provides an economical and practical way to measure network | |||
to accountable entities. | traffic and ascribe it to accountable entities. | |||
Usage information which is not deriveable from traffic flows may also be | Usage information which is not deriveable from traffic flows may also | |||
of interest. For example, an application may wish to record accesses to | be of interest. For example, an application may wish to record | |||
various different information resources or a host may wish to record the | accesses to various different information resources or a host may | |||
username (subscriber id) for a particular network session. Provision is | wish to record the username (subscriber id) for a particular network | |||
made in the traffic flow architecture to do this. In the future the | session. Provision is made in the traffic flow architecture to do | |||
measurement model will be extended to gather such information from | this. In the future the measurement model will be extended to gather | |||
applications and hosts so as to provide values for higher-layer flow | such information from applications and hosts so as to provide values | |||
attributes. | for higher-layer flow attributes. | |||
As well as FLOWS and METERS, the traffic flow measurement model includes | As well as FLOWS and METERS, the traffic flow measurement model | |||
MANAGERS, METER READERS and ANALYSIS APPLICAIONS, which are explained in | includes MANAGERS, METER READERS and ANALYSIS APPLICAIONS, which are | |||
following sections. The relationships between them are shown by the | explained in following sections. The relationships between them are | |||
diagram below. Numbers on the diagram refer to sections in this | shown by the diagram below. Numbers on the diagram refer to sections | |||
document. | in this document. | |||
MANAGER | MANAGER | |||
/ \ | / \ | |||
2.3 / \ 2.4 | 2.3 / \ 2.4 | |||
/ \ | / \ | |||
/ \ ANALYSIS | / \ ANALYSIS | |||
METER <-----> METER READER <-----> APPLICATION | METER <-----> METER READER <-----> APPLICATION | |||
2.2 2.7 | 2.2 2.7 | |||
- MANAGER: A traffic measurement manager is an application which | - MANAGER: A traffic measurement manager is an application which | |||
skipping to change at page 6, line 28 | skipping to change at page 5, line 46 | |||
combine the functions of meter reader and manager within a single | combine the functions of meter reader and manager within a single | |||
network entity. | network entity. | |||
- METER: Meters are placed at measurement points determined by | - METER: Meters are placed at measurement points determined by | |||
network Operations personnel. Each meter selectively records | network Operations personnel. Each meter selectively records | |||
network activity as directed by its configuration settings. It can | network activity as directed by its configuration settings. It can | |||
also aggregate, transform and further process the recorded activity | also aggregate, transform and further process the recorded activity | |||
before the data is stored. The processed and stored results are | before the data is stored. The processed and stored results are | |||
called the 'usage data.' | called the 'usage data.' | |||
- METER READER: A meter reader reliaably transports usage data from | - METER READER: A meter reader reliably transports usage data from | |||
meters so that it is available to analysis applications. | meters so that it is available to analysis applications. | |||
- ANALYSIS APPLICATION: An analysis application processes the usage | - ANALYSIS APPLICATION: An analysis application processes the usage | |||
data so as to provide information and reports which are useful for | data so as to provide information and reports which are useful for | |||
network engineering and management purposes. Examples include: | network engineering and management purposes. Examples include: | |||
- TRAFFIC FLOW MATRICES, showing the total flow rates for many of | - TRAFFIC FLOW MATRICES, showing the total flow rates for | |||
the possible paths within an internet. | many of the possible paths within an internet. | |||
- FLOW RATE FREQUENCY DISTRIBUTIONS, indicating how flow rates | - FLOW RATE FREQUENCY DISTRIBUTIONS, indicating how flow | |||
vary with time. | rates vary with time. | |||
- USAGE DATA showing the total traffic volumes sent and received | - USAGE DATA showing the total traffic volumes sent and | |||
by particular hosts. | received by particular hosts. | |||
The operation of the traffic measurement system as a whole is best | The operation of the traffic measurement system as a whole is best | |||
understood by considering the interactions between its components. | understood by considering the interactions between its components. | |||
These are described in the following sections. | These are described in the following sections. | |||
2.2 Interaction Between METER and METER READER | 2.2 Interaction Between METER and METER READER | |||
The information which travels along this path is the usage data itself. | The information which travels along this path is the usage data | |||
A meter holds usage data in an array of flow data records known as the | itself. A meter holds usage data in an array of flow data records | |||
FLOW TABLE. A meter reader may collect the data in any suitable manner, | known as the FLOW TABLE. A meter reader may collect the data in any | |||
for example it might upload a copy of the whole flow table using a file | suitable manner. For example it might upload a copy of the whole | |||
transfer protocol, read the records in the current flow set one at a | flow table using a file transfer protocol, or read the records in the | |||
time using a suitable data transfer protocol. Note that the meter | current flow set one at a time using a suitable data transfer | |||
reader need not read complete flow data records, a subset of their | protocol. Note that the meter reader need not read complete flow | |||
attribute values may well be sufficient. | data records, a subset of their attribute values may well be | |||
sufficient. | ||||
A meter reader may collect usage data from one or more meters. Data may | A meter reader may collect usage data from one or more meters. Data | |||
be collected from the meters at any time. There is no requirement for | may be collected from the meters at any time. There is no | |||
collections to be synchronized in any way. | requirement for collections to be synchronized in any way. | |||
2.3 Interaction Between MANAGER and METER | 2.3 Interaction Between MANAGER and METER | |||
A manager is responsible for configuring and controlling one or more | A manager is responsible for configuring and controlling one or more | |||
meters. At the time of writing a meter can only be controlled by a | meters. At the time of writing a meter can only be controlled by a | |||
single manager; in the future this restriction may be relaxed. Each | single manager; in the future this restriction may be relaxed. Each | |||
meter's configuration includes information such as: | meter's configuration includes information such as: | |||
- Flow specifications, e.g. which traffic flows are to be measured, | - Flow specifications, e.g. which traffic flows are to be measured, | |||
how they are to be aggregated, and any data the meter is required | how they are to be aggregated, and any data the meter is required | |||
to compute for each flow being measured. | to compute for each flow being measured. | |||
- Meter control parameters, e.g. the maximum size of its flow table, | - Meter control parameters, e.g. the maximum size of its flow table, | |||
the 'inactivity' time for flows (if no packets belonging to a flow | the 'inactivity' time for flows (if no packets belonging to a flow | |||
are seen for this time the flow is considered to have ended, i.e. | are seen for this time the flow is considered to have ended, i.e. | |||
to have become idle). | to have become idle). | |||
- Sampling rate. Normally every packet will be observed, may | - Sampling rate. Normally every packet will be observed. It may | |||
sometimes be necessary to use sampling techniques to observe only | sometimes be necessary to use sampling techniques to observe only | |||
some of the packets. (Sampling algorithms are not prescribed by | some of the packets. (Sampling algorithms are not prescribed by | |||
the architecture; it should be noted that before using sampling one | the architecture; it should be noted that before using sampling one | |||
should verify the statistical validity of the algorithm used). | should verify the statistical validity of the algorithm used). | |||
Current experience with the measurement architecture shows that a | Current experience with the measurement architecture shows that a | |||
carefully-designed and implemented meter compresses the data such | carefully-designed and implemented meter compresses the data such | |||
that in normal LANs and WANs of today sampling is really not | that in normal LANs and WANs of today sampling is really not | |||
needed. | needed. | |||
2.4 Interaction Between MANAGER and METER READER | 2.4 Interaction Between MANAGER and METER READER | |||
A manager is responsible for configuring and controlling one or more | A manager is responsible for configuring and controlling one or more | |||
meter readers. A meter reader may only be controlled by a single | meter readers. A meter reader may only be controlled by a single | |||
manager. A meter reader needs to know at least the following for every | manager. A meter reader needs to know at least the following for | |||
meter is is collecting usage data from: | every meter is is collecting usage data from: | |||
- The meter's unique identity, i.e. its network name or address. | - The meter's unique identity, i.e. its network name or address. | |||
- How often usage data is to be collected from the meter. | - How often usage data is to be collected from the meter. | |||
- Which flow records are to be collected (e.g. all active flows, the | - Which flow records are to be collected (e.g. all active flows, the | |||
whole flow table, flows seen since a given time, etc.). | whole flow table, flows seen since a given time, etc.). | |||
- Which attribute values are to be collected for the required flow | - Which attribute values are to be collected for the required flow | |||
records (e.g. all attributes, or a small subset of them) | records (e.g. all attributes, or a small subset of them) | |||
Since redundant reporting may be used in order to increase the | Since redundant reporting may be used in order to increase the | |||
reliability of usage data, exchanges among multiple entities must be | reliability of usage data, exchanges among multiple entities must be | |||
considered as well. These are discussed below. | considered as well. These are discussed below. | |||
2.5 Multiple METERs or METER READERs | 2.5 Multiple METERs or METER READERs | |||
-- METER READER A -- | -- METER READER A -- | |||
/ | \ | / | \ | |||
/ | \ | / | \ | |||
=====METER 1 METER 2=====METER 3 METER 4===== | =====METER 1 METER 2=====METER 3 METER 4===== | |||
\ | / | \ | / | |||
\ | / | \ | / | |||
-- METER READER B -- | -- METER READER B -- | |||
Several uniquely identified meters may report to one or more meter | Several uniquely identified meters may report to one or more meter | |||
readers. The diagram above gives an example of how multiple meters and | readers. The diagram above gives an example of how multiple meters | |||
meter readers could be used. | and meter readers could be used. | |||
In the diagram above meter 1 is read by meter reader A, and meter 4 is | In the diagram above meter 1 is read by meter reader A, and meter 4 | |||
read by meter reader B. Meters 1 and 4 have no redundancy; if either | is read by meter reader B. Meters 1 and 4 have no redundancy; if | |||
fails, usage data for their network segments will be lost. | either fails, usage data for their network segments will be lost. | |||
Meters 2 and 3, however, measure traffic on the same network segment. | Meters 2 and 3, however, measure traffic on the same network segment. | |||
One of them may fail leaving the other collecting the segment's usage | One of them may fail leaving the other collecting the segment's usage | |||
data. Meters 2 and 3 are read by meter reader A and by meter reader B. | data. Meters 2 and 3 are read by meter reader A and by meter reader | |||
If one meter reader fails, the other will continue collecting usage | B. If one meter reader fails, the other will continue collecting | |||
data. | usage data. | |||
The architecture does not require multiple meter readers to be | The architecture does not require multiple meter readers to be | |||
synchronized. In the situation above meter readers A and B could both | synchronized. In the situation above meter readers A and B could | |||
collect usage data at the same intervals, but not neccesarily at the | both collect usage data at the same intervals, but not neccesarily at | |||
same times. Note that because collections are asynchronous it is | the same times. Note that because collections are asynchronous it is | |||
unlikely that usage records from two different meter readers will agree | unlikely that usage records from two different meter readers will | |||
exactly. | agree exactly. | |||
If precisely synchronized collections are required this can be achieved | If precisely synchronized collections are required this can be | |||
by having one meter reader request each meter to begin collecting a new | achieved by having one manager request each meter to begin collecting | |||
set of flows, then allowing all meter readers to collect the usage data | a new set of flows, then allowing all meter readers to collect the | |||
from the old sets of flows. | usage data from the old sets of flows. | |||
If there is only one meter reader and it fails, the meters continue to | If there is only one meter reader and it fails, the meters continue | |||
run. When the meter reader is restarted it can collect all of the | to run. When the meter reader is restarted it can collect all of the | |||
accumulated flow data. Should this happen, time resolution will be lost | accumulated flow data. Should this happen, time resolution will be | |||
(because of the missed collections) but overall traffic flow information | lost (because of the missed collections) but overall traffic flow | |||
will not. The only exception to this would occur if the traffic volume | information will not. The only exception to this would occur if the | |||
was sufficient to 'roll over' counters for some flows during the | traffic volume was sufficient to 'roll over' counters for some flows | |||
failure; this is addressed in the section on 'Rolling Counters.' | during the failure; this is addressed in the section on 'Rolling | |||
Counters.' | ||||
2.6 Interaction Between MANAGERs (MANAGER - MANAGER) | 2.6 Interaction Between MANAGERs (MANAGER - MANAGER) | |||
Synchronization between multiple management systems is the province of | Synchronization between multiple management systems is the province | |||
network management protocols. This traffic flow measurement | of network management protocols. This traffic flow measurement | |||
architecture specifies only the network management controls necessary to | architecture specifies only the network management controls necessary | |||
perform the traffic flow measurement function and does not address the | to perform the traffic flow measurement function and does not address | |||
more global issues of simultaneous or interleaved (possibly conflicting) | the more global issues of simultaneous or interleaved (possibly | |||
commands from multiple network management stations or the process of | conflicting) commands from multiple network management stations or | |||
transferring control from one network management station to another. | the process of transferring control from one network management | |||
station to another. | ||||
2.7 Between METER READER and APPLICATION | 2.7 METER READERs and APPLICATIONs | |||
Once a collection of usage data has been assembled by a meter reader it | Once a collection of usage data has been assembled by a meter reader | |||
can be processed by an analysis application. Details of analysis | it can be processed by an analysis application. Details of analysis | |||
applications - such as the reports they produce and the data they | applications - such as the reports they produce and the data they | |||
require - are outside the scope of this architecture. | require - are outside the scope of this architecture. | |||
It should be noted, however, that analysis applications will often | It should be noted, however, that analysis applications will often | |||
require considerable amounts of input data. An important part of | require considerable amounts of input data. An important part of | |||
running a traffic flow measurement system is the storage and regular | running a traffic flow measurement system is the storage and regular | |||
reduction of flow data so as to produce daily, weekly or monthly summary | reduction of flow data so as to produce daily, weekly or monthly | |||
files for further analysis. Again, details of such data handling are | summary files for further analysis. Again, details of such data | |||
outside the scope of this architecture. | handling are outside the scope of this architecture. | |||
3 Traffic Flows and Reporting Granularity | 3 Traffic Flows and Reporting Granularity | |||
A flow was defined in section 2.1 above in abstract terms as follows: | A flow was defined in section 2.1 above in abstract terms as follows: | |||
"A TRAFFIC FLOW is an artifical logical equivalent to a call or | "A TRAFFIC FLOW is an artifical logical equivalent to a call or | |||
connection, belonging to an ACCOUNTABLE ENTITY." | connection, belonging to an ACCOUNTABLE ENTITY." | |||
In practical terms, a flow is a stream of packets passing across a | In practical terms, a flow is a stream of packets passing across a | |||
network between two end points (or being sent from a single end point), | network between two end points (or being sent from a single end | |||
which have been summarized by a traffic meter for analysis purposes. | point), which have been summarized by a traffic meter for analysis | |||
purposes. | ||||
3.1 Flows and their Attributes | 3.1 Flows and their Attributes | |||
Every traffic meter maintains a table of 'flow records' for flows seen | Every traffic meter maintains a table of 'flow records' for flows | |||
by the meter. A flow record holds the values of the ATTRIBUTES of | seen by the meter. A flow record holds the values of the ATTRIBUTES | |||
interest for its flow. These attributes might include: | of interest for its flow. These attributes might include: | |||
- ADDRESSES for the flow's source and destination. These comprise | - ADDRESSES for the flow's source and destination. These comprise | |||
the protocol type, the source and destination addresses at various | the protocol type, the source and destination addresses at various | |||
network layers (extracted from the packet), and the number of the | network layers (extracted from the packet), and the number of the | |||
interface on which the packet was observed. | interface on which the packet was observed. | |||
- First and last TIMES when packets were seen for this flow, i.e. | - First and last TIMES when packets were seen for this flow, i.e. | |||
the 'creation' and 'last activity' times for the flow. | the 'creation' and 'last activity' times for the flow. | |||
- COUNTS for 'forward' (source to destination) and 'backward' | - COUNTS for 'forward' (source to destination) and 'backward' | |||
(destination to source) components (e.g. packets and bytes) of the | (destination to source) components (e.g. packets and bytes) of the | |||
flow's traffic. The specifying of 'source' and 'destination' for | flow's traffic. The specifying of 'source' and 'destination' for | |||
flows is discussed in the section on packet matching below. | flows is discussed in the section on packet matching below. | |||
- OTHER attributes, e.g. information computed by the meter. | - OTHER attributes, e.g. information computed by the meter. | |||
A flow's ACCOUNTABLE ENTITY is specified by the values of its ADDRESS | A flow's ACCOUNTABLE ENTITY is specified by the values of its ADDRESS | |||
attributes. For example, if a flow's address attributes specified only | attributes. For example, if a flow's address attributes specified | |||
that "source address = IP address 10.1.0.1," then all IP packets from | only that "source address = IP address 10.1.0.1," then all IP packets | |||
and to that address would be counted in that flow. If a flow's address | from and to that address would be counted in that flow. If a flow's | |||
list were specified as "source address = IP address 10.1.0.1, | address list were specified as "source address = IP address 10.1.0.1, | |||
destination address = IP address 26.1.0.1" then only IP packets between | destination address = IP address 26.1.0.1" then only IP packets | |||
10.1.0.1 and 26.1.0.1 would be counted in that flow. | between 10.1.0.1 and 26.1.0.1 would be counted in that flow. | |||
The addresses specifying a flow's address attributes may include one or | The addresses specifying a flow's address attributes may include one | |||
more of the following types: | or more of the following types: | |||
- The INTERFACE NUMBER for the flow, i.e. the interface on which the | - The INTERFACE NUMBER for the flow, i.e. the interface on which the | |||
meter measured the traffic. Together with a unique address for the | meter measured the traffic. Together with a unique address for the | |||
meter this uniquely identifies a particular physical-level port. | meter this uniquely identifies a particular physical-level port. | |||
- The ADJACENT ADDRESS, i.e. the [n-1] layer address of the | - The ADJACENT ADDRESS, i.e. the [n-1] layer address of the | |||
immediate source or destination on the path of the packet. For | immediate source or destination on the path of the packet. For | |||
example, if flow measurement is being performed at the IP layer on | example, if flow measurement is being performed at the IP layer on | |||
an Ethernet LAN [3], an adjacent address is a six-octet Media | an Ethernet LAN [3], an adjacent address is a six-octet Media | |||
Access Control (MAC) address. For a host connected to the same LAN | Access Control (MAC) address. For a host connected to the same LAN | |||
skipping to change at page 11, line 29 | skipping to change at page 10, line 32 | |||
- The PEER ADDRESS, which identifies the source or destination of the | - The PEER ADDRESS, which identifies the source or destination of the | |||
PEER-LEVEL packet. The form of a peer address will depend on the | PEER-LEVEL packet. The form of a peer address will depend on the | |||
network-layer protocol in use, and the network layer [n] at which | network-layer protocol in use, and the network layer [n] at which | |||
traffic measurement is being performed. | traffic measurement is being performed. | |||
- The TRANSPORT ADDRESS, which identifies the source or destination | - The TRANSPORT ADDRESS, which identifies the source or destination | |||
port for the packet, i.e. its [n+1] layer address. For example, | port for the packet, i.e. its [n+1] layer address. For example, | |||
if flow measurement is being performed at the IP layer a transport | if flow measurement is being performed at the IP layer a transport | |||
address is a two-octet UDP or TCP port number. | address is a two-octet UDP or TCP port number. | |||
The four definitions above specify addresses for each of the four lowest | The four definitions above specify addresses for each of the four | |||
layers of the OSI reference model, i.e. Physical layer, Link layer, | lowest layers of the OSI reference model, i.e. Physical layer, Link | |||
Network layer and Transport layer. A FLOW RECORD stores both the VALUE | layer, Network layer and Transport layer. A FLOW RECORD stores both | |||
for each of its addresses (as described above) and a MASK specifying | the VALUE for each of its addresses (as described above) and a MASK | |||
which bits of the address value are being used and which are ignored. | specifying which bits of the address value are being used and which | |||
Note that if address bits are being ignored the meter will set them to | are ignored. Note that if address bits are being ignored the meter | |||
zero, however their actual values are undefined. | will set them to zero, however their actual values are undefined. | |||
One of the key features of the traffic measurement architecture is that | One of the key features of the traffic measurement architecture is | |||
attributes have essentially the same meaning for different protocols, so | that attributes have essentially the same meaning for different | |||
that analysis applications can use the same reporting formats for all | protocols, so that analysis applications can use the same reporting | |||
protocols. This is straightforward for peer addresses; although the | formats for all protocols. This is straightforward for peer | |||
form of addresses differs for the various protocols, the meaning of a | addresses; although the form of addresses differs for the various | |||
'peer address' remains the same. It becomes harder to maintain this | protocols, the meaning of a 'peer address' remains the same. It | |||
correspondence at higher layers - for example, at the Network layer IP, | becomes harder to maintain this correspondence at higher layers - for | |||
Novell IPX and AppleTalk all use port numbers as a 'transport address,' | example, at the Network layer IP, Novell IPX and AppleTalk all use | |||
but CLNP and DECnet have no notion of ports. Further work is needed | port numbers as a 'transport address,' but CLNP and DECnet have no | |||
here, particularly in selecting attributes which will be suitable for | notion of ports. Further work is needed here, particularly in | |||
the higher layers of the OSI reference model. | selecting attributes which will be suitable for the higher layers of | |||
the OSI reference model. | ||||
Reporting by adjacent intermediate sources and destinations or simply by | Reporting by adjacent intermediate sources and destinations or simply | |||
meter interface (most useful when the meter is embedded in a router) | by meter interface (most useful when the meter is embedded in a | |||
supports hierarchical Internet reporting schemes as described in the | router) supports hierarchical Internet reporting schemes as described | |||
'Traffic Flow Measurement: Background' RFC [1]. That is, it allows | in the 'Traffic Flow Measurement: Background' RFC [1]. That is, it | |||
backbone and regional networks to measure usage to just the next lower | allows backbone and regional networks to measure usage to just the | |||
level of granularity (i.e. to the regional and stub/enterprise levels, | next lower level of granularity (i.e. to the regional and | |||
respectively), with the final breakdown according to end user (e.g. to | stub/enterprise levels, respectively), with the final breakdown | |||
source IP address) performed by the stub/enterprise networks. | according to end user (e.g. to source IP address) performed by the | |||
stub/enterprise networks. | ||||
In cases where network addresses are dynamically allocated (e.g. mobile | In cases where network addresses are dynamically allocated (e.g. | |||
subscribers), further subscriber identification will be necessary if | mobile subscribers), further subscriber identification will be | |||
flows are to ascribed to individual users. Provision is made to further | necessary if flows are to ascribed to individual users. Provision is | |||
specify the accountable entity through the use of an optional SUBSCRIBER | made to further specify the accountable entity through the use of an | |||
ID as part of the flow id. A subscriber ID may be associated with a | optional SUBSCRIBER ID as part of the flow id. A subscriber ID may | |||
particular flow either through the current rule set or by proprietary | be associated with a particular flow either through the current rule | |||
means within a meter, for example via protocol exchanges with one or | set or by proprietary means within a meter, for example via protocol | |||
more (multi-user) hosts. At this time a subscriber ID is an arbitrary | exchanges with one or more (multi-user) hosts. At this time a | |||
text string; later versions of the architecture may specify its contents | subscriber ID is an arbitrary text string; later versions of the | |||
on more detail. | architecture may specify its contents on more detail. | |||
3.2 Granularity of Flow Measurements | 3.2 Granularity of Flow Measurements | |||
GRANULARITY is the 'control knob' by which an application and/or the | GRANULARITY is the 'control knob' by which an application and/or the | |||
meter can trade off the overhead associated with performing usage | meter can trade off the overhead associated with performing usage | |||
reporting against the level of detail supplied. A coarser granularity | reporting against the level of detail supplied. A coarser | |||
means a greater level of aggregation; finer granularity means a greater | granularity means a greater level of aggregation; finer granularity | |||
level of detail. Thus, the number of flows measured at a meter can be | means a greater level of detail. Thus, the number of flows measured | |||
regulated by changing the granularity of the accountable entity, the | (and stored) at a meter can be regulated by changing the granularity | |||
attributes, or the time intervals. Flows are like an adjustable pipe - | of the accountable entity, the attributes, or the time intervals. | |||
many fine-granularity streams can carry the data with each stream | Flows are like an adjustable pipe - many fine-granularity streams can | |||
measured individually, or data can be bundled in one coarse-granularity | carry the data with each stream measured individually, or data can be | |||
pipe. | bundled in one coarse-granularity pipe. | |||
Flow granularity is controlled by adjusting the level of detail at which | Flow granularity is controlled by adjusting the level of detail at | |||
the following are reported: | which the following are reported: | |||
- The accountable entity (address attributes, discussed above) | - The accountable entity (address attributes, discussed above). | |||
- The categorisation of packets (other attributes, discussed below) | - The categorisation of packets (other attributes, discussed below). | |||
- The lifetime/duration of a flow (the reporting interval) | - The lifetime/duration of flows (the reporting interval needs to be | |||
short enough to measure them with sufficient precision). | ||||
The set of rules controlling the determination of each packet's | The set of rules controlling the determination of each packet's | |||
accountable entity is known as the meter's CURRENT RULE SET. As will be | accountable entity is known as the meter's CURRENT RULE SET. As will | |||
shown, the meter's current rule set forms an integral part of the | be shown, the meter's current rule set forms an integral part of the | |||
reported information, i.e. the recorded usage information cannot be | reported information, i.e. the recorded usage information cannot be | |||
properly interpreted without a definition of the rules used to collect | properly interpreted without a definition of the rules used to | |||
that information. | collect that information. | |||
Settings for these granularity factors may vary from meter to meter. | Settings for these granularity factors may vary from meter to meter. | |||
They are determined by the meter's current rule set, so they will change | They are determined by the meter's current rule set, so they will | |||
if network Operations personnel reconfigure the meter to use a new rule | change if network Operations personnel reconfigure the meter to use a | |||
set. It is expected that the collection rules will change rather | new rule set. It is expected that the collection rules will change | |||
infrequently; nonetheless, the rule set in effect at any time must be | rather infrequently; nonetheless, the rule set in effect at any time | |||
identifiable via a RULE SET ID. Granularity of accountable entities is | must be identifiable via a RULE SET ID. Granularity of accountable | |||
further specified by additional ATTRIBUTES. These attributes include: | entities is further specified by additional ATTRIBUTES. These | |||
attributes include: | ||||
- Meter variables such as the index of the flow's record in the flow | - Meter variables such as the index of the flow's record in the flow | |||
table and the rule set id for the rules which the meter was running | table and the rule set id for the rules which the meter was running | |||
while the flow was observed. The values of these attributes | while the flow was observed. The values of these attributes | |||
provide a way of distinguishing flows observed by a meter at | provide a way of distinguishing flows observed by a meter at | |||
different times. | different times. | |||
- Attributes which record information derived from other attribute | - Attributes which record information derived from other attribute | |||
values. Six of these are defined (SourceClass, DestClass, | values. Six of these are defined (SourceClass, DestClass, | |||
FlowClass, SourceKind, DestKind, FlowKind), and their meaning is | FlowClass, SourceKind, DestKind, FlowKind), and their meaning is | |||
determined by the meter's rule set. For example, one could have a | determined by the meter's rule set. For example, one could have a | |||
subroutine in the rule set which determined whether a source or | subroutine in the rule set which determined whether a source or | |||
destination peer address was a member of an arbitrary list of | destination peer address was a member of an arbitrary list of | |||
networks, and set SourceClass/DestClass to one if the source/dest | networks, and set SourceClass/DestClass to one if the source/dest | |||
peer address was in the list or to zero otherwise. | peer address was in the list or to zero otherwise. | |||
- Administratively specified attributes such as Quality Of Service | - Administratively specified attributes such as Quality Of Service | |||
and Priority, etc. These are not defined at this time. | and Priority, etc. These are not defined at this time. | |||
- Higher-layer (especially application-level) attributes. These are | - Higher-layer (especially application-level) attributes. These are | |||
not defined at this time. | not defined at this time. | |||
Settings for these granularity factors may vary from meter to meter. | Settings for these granularity factors may vary from meter to meter. | |||
They are determined by the meter's current rule set, so they will change | They are determined by the meter's current rule set, so they will | |||
if network Operations personnel reconfigure the meter to use a new rule | change if network Operations personnel reconfigure the meter to use a | |||
set. | new rule set. | |||
The LIFETIME of a flow is the time interval which began when the meter | The LIFETIME of a flow is the time interval which began when the | |||
observed the first packet belonging to the flow and ended when it saw | meter observed the first packet belonging to the flow and ended when | |||
the last packet. Flow lifetimes are very variable, but many - if not | it saw the last packet. Flow lifetimes are very variable, but many - | |||
most - are rather short. A meter cannot measure lifetimes directly; | if not most - are rather short. A meter cannot measure lifetimes | |||
instead a meter reader collects usage data for flows which have been | directly; instead a meter reader collects usage data for flows which | |||
active since the last collection, and an analysis application may | have been active since the last collection, and an analysis | |||
compare the data from each collection so as to determine when each flow | application may compare the data from each collection so as to | |||
actually stopped. | determine when each flow actually stopped. | |||
The meter does, however, need to reclaim memory (i.e. records in the | The meter does, however, need to reclaim memory (i.e. records in the | |||
flow table) being held by idle flows. The meter configuration includes | flow table) being held by idle flows. The meter configuration | |||
a variable called InactivityTimeout, which specifies the minimum time a | includes a variable called InactivityTimeout, which specifies the | |||
meter must wait before recovering the flow's record. In addition, | minimum time a meter must wait before recovering the flow's record. | |||
before recovering a flow record the meter must be sure that the flow's | In addition, before recovering a flow record the meter must be sure | |||
data has been collected by at least one meter reader. | that the flow's data has been collected by at least one meter reader. | |||
These 'lifetime' issues are considered further in the section on meter | These 'lifetime' issues are considered further in the section on | |||
readers (below). A complete list of the attributes currently defined is | meter readers (below). A complete list of the attributes currently | |||
given in Appendix C later in this document. | defined is given in Appendix C later in this document. | |||
3.3 Rolling Counters, Timestamps, Report-in-One-Bucket-Only | 3.3 Rolling Counters, Timestamps, Report-in-One-Bucket-Only | |||
Once an usage record is sent the decision needs to be made whether to | Once an usage record is sent, the decision needs to be made whether | |||
clear any existing flow records or to maintain them and add to their | to clear any existing flow records or to maintain them and add to | |||
counts when recording subsequent traffic on the same flow. The second | their counts when recording subsequent traffic on the same flow. The | |||
method, called rolling counters, is recommended and has several | second method, called rolling counters, is recommended and has | |||
advantages. Its primary advantage is that it provides greater | several advantages. Its primary advantage is that it provides | |||
reliability - the system can now often survive the loss of some usage | greater reliability - the system can now often survive the loss of | |||
records, such as might occur if a meter reader failed and later | some usage records, such as might occur if a meter reader failed and | |||
restarted. The next usage record will very often contain yet another | later restarted. The next usage record will very often contain yet | |||
reading of many of the same flow buckets which were in the lost usage | another reading of many of the same flow buckets which were in the | |||
record. The 'continuity' of data provided by rolling counters can also | lost usage record. The 'continuity' of data provided by rolling | |||
supply information used for "sanity" checks on the data itself, to guard | counters can also supply information used for "sanity" checks on the | |||
against errors in calculations. | data itself, to guard against errors in calculations. | |||
The use of rolling counters does introduce a new problem: how to | The use of rolling counters does introduce a new problem: how to | |||
distinguish a follow-on flow record from a new flow record. Consider | distinguish a follow-on flow record from a new flow record. Consider | |||
the following example. | the following example. | |||
CONTINUING FLOW OLD FLOW, then NEW FLOW | CONTINUING FLOW OLD FLOW, then NEW FLOW | |||
start time = 1 start time = 1 | start time = 1 start time = 1 | |||
Usage record N: flow count = 2000 flow count = 2000 (done) | Usage record N: flow count = 2000 flow count = 2000 (done) | |||
start time = 1 start time = 5 | start time = 1 start time = 5 | |||
Usage record N+1: flow count = 3000 new flow count = 1000 | Usage record N+1: flow count = 3000 new flow count = 1000 | |||
Total count: 3000 3000 | Total count: 3000 3000 | |||
In the continuing flow case, the same flow was reported when its count | In the continuing flow case, the same flow was reported when its | |||
was 2000, and again at 3000: the total count to date is 3000. In the | count was 2000, and again at 3000: the total count to date is 3000. | |||
OLD/NEW case, the old flow had a count of 2000. Its record was then | In the OLD/NEW case, the old flow had a count of 2000. Its record | |||
stopped (perhaps because of temporary idleness, or MAX LIFETIME policy), | was then stopped (perhaps because of temporary idleness, or MAX | |||
but then more traffic with the same characteristics arrived so a new | LIFETIME policy), but then more traffic with the same characteristics | |||
flow record was started and it quickly reached a count of 1000. The | arrived so a new flow record was started and it quickly reached a | |||
total flow count from both the old and new records is 3000. | count of 1000. The total flow count from both the old and new | |||
records is 3000. | ||||
The flow START TIMESTAMP attribute is sufficient to resolve this. In | The flow START TIMESTAMP attribute is sufficient to resolve this. In | |||
the example above, the CONTINUING FLOW flow record in the second usage | the example above, the CONTINUING FLOW flow record in the second | |||
record has an old FLOW START timestamp, while the NEW FLOW contains a | usage record has an old FLOW START timestamp, while the NEW FLOW | |||
recent FLOW START timestamp. | contains a recent FLOW START timestamp. | |||
Each packet is counted in one and only one flow, so as to avoid multiple | Each packet is counted in one and only one flow, so as to avoid | |||
counting of a single packet (prevent double billing). The record of a | multiple counting of a single packet. The record of a single flow is | |||
single flow is informally called a "bucket." If multiple, sometimes | informally called a "bucket." If multiple, sometimes overlapping, | |||
overlapping, records of usage information are required (aggregate, | records of usage information are required (aggregate, individual, | |||
individual, etc), the network manager should collect the counts in | etc), the network manager should collect the counts in sufficiently | |||
sufficiently detailed granularity so that aggregate and combination | detailed granularity so that aggregate and combination counts can be | |||
counts can be reconstructed in post-processing of the raw usage data. | reconstructed in post-processing of the raw usage data. | |||
For example, consider a meter from which it is required to record both | For example, consider a meter from which it is required to record | |||
'total packets coming in interface #1' and 'total packets arriving from | both 'total packets coming in interface #1' and 'total packets | |||
any interface sourced by IP address = a.b.c.d.' Although a bucket can | arriving from any interface sourced by IP address = a.b.c.d.' | |||
be declared for each case, it is not clear how to handle a packet which | Although a bucket can be declared for each case, it is not clear how | |||
satisfies both criteria. It must only be counted once. By default it | to handle a packet which satisfies both criteria. It must only be | |||
will be counted in the first bucket for which it qualifies, and not in | counted once. By default it will be counted in the first bucket for | |||
the other bucket. Further, it is not possible to reconstruct this | which it qualifies, and not in the other bucket. Further, it is not | |||
information by post-processing. The solution in this case is to define | possible to reconstruct this information by post-processing. The | |||
not two, but THREE buckets, each one collecting a unique combination of | solution in this case is to define not two, but THREE buckets, each | |||
the two criteria: | one collecting a unique combination of the two criteria: | |||
Bucket 1: Packets which came in interface 1, | Bucket 1: Packets which came in interface 1, | |||
AND were sourced by IP address a.b.c.d | AND were sourced by IP address a.b.c.d | |||
Bucket 2: Packets which came in interface 1, | Bucket 2: Packets which came in interface 1, | |||
AND were NOT sourced by IP address a.b.c.d | AND were NOT sourced by IP address a.b.c.d | |||
Bucket 3: Packets which did NOT come in interface 1, | Bucket 3: Packets which did NOT come in interface 1, | |||
AND were sourced by IP address a.b.c.d | AND were sourced by IP address a.b.c.d | |||
(Bucket 4: Packets which did NOT come in interface 1, | (Bucket 4: Packets which did NOT come in interface 1, | |||
AND NOT sourced by IP address a.b.c.d) | AND NOT sourced by IP address a.b.c.d) | |||
The desired information can now be reconstructed by post-processing. | The desired information can now be reconstructed by post-processing. | |||
"Total packets coming in interface 1" can be found by adding buckets 1 & | "Total packets coming in interface 1" can be found by adding buckets | |||
2, and "Total packets sourced by IP address a.b.c.d" can be found by | 1 & 2, and "Total packets sourced by IP address a.b.c.d" can be found | |||
adding buckets 1 & 3. Note that in this case bucket 4 is not explicitly | by adding buckets 1 & 3. Note that in this case bucket 4 is not | |||
required since its information is not of interest, but it is supplied | explicitly required since its information is not of interest, but it | |||
here in parentheses for completeness. | is supplied here in parentheses for completeness. | |||
4 Meters | 4 Meters | |||
A traffic flow meter is a device for collecting data about traffic flows | A traffic flow meter is a device for collecting data about traffic | |||
at a given point within a network; we will call this the METERING POINT. | flows at a given point within a network; we will call this the | |||
The header of every packet passing the network metering point is offered | METERING POINT. The header of every packet passing the network | |||
to the traffic meter program. | metering point is offered to the traffic meter program. | |||
A meter could be implemented in various ways, including: | A meter could be implemented in various ways, including: | |||
- A dedicated small host, connected to a LAN (so that it can see all | - A dedicated small host, connected to a LAN (so that it can see all | |||
packets as they pass by) and running a 'traffic meter' program. | packets as they pass by) and running a 'traffic meter' program. | |||
The metering point is the LAN segment to which the meter is | The metering point is the LAN segment to which the meter is | |||
attached. | attached. | |||
- A multiprocessing system with one or more network interfaces, with | - A multiprocessing system with one or more network interfaces, with | |||
drivers enabling a traffic meter program to see packets. In this | drivers enabling a traffic meter program to see packets. In this | |||
case the system provides multiple metering points - traffic flows | case the system provides multiple metering points - traffic flows | |||
on any subset of its network interfaces can be measured. | on any subset of its network interfaces can be measured. | |||
- A packet-forwarding device such as a router or switch. This is | - A packet-forwarding device such as a router or switch. This is | |||
similar to (b) except that every received packet should also be | similar to (b) except that every received packet should also be | |||
forwarded, usually on a different interface. | forwarded, usually on a different interface. | |||
The discussion in the following sections assumes that a meter may | ||||
only run a single rule set. It is, however, possible for a meter to | ||||
run several rule sets concurrently, matching each packet against | ||||
every active rule set and producing a single flow table with flows | ||||
from all the active rule sets. The overall effect of doing this | ||||
would be similar to running several independent meters, one for each | ||||
rule set. | ||||
4.1 Meter Structure | 4.1 Meter Structure | |||
An outline of the meter's structure is given in the following diagram. | An outline of the meter's structure is given in the following | |||
diagram. | ||||
Briefly, the meter works as follows: | Briefly, the meter works as follows: | |||
- Incoming packet headers arrive at the top left of the diagram and | - Incoming packet headers arrive at the top left of the diagram and | |||
are passed to the PACKET PROCESSOR. | are passed to the PACKET PROCESSOR. | |||
- The packet processor passes them to the Packet Matching Engine | - The packet processor passes them to the Packet Matching Engine | |||
(PME) where they are classified. | (PME) where they are classified. | |||
- The PME is a Virtual Machine running a pattern matching program | - The PME is a Virtual Machine running a pattern matching program | |||
contained in the CURRENT RULE SET. It is invoked by the Packet | contained in the CURRENT RULE SET. It is invoked by the Packet | |||
Processor, and returns instructions on what to do with the packet. | Processor, and returns instructions on what to do with the packet. | |||
skipping to change at page 17, line 34 | skipping to change at page 17, line 7 | |||
| | | | |||
+--------*--------+ | +--------*--------+ | |||
| 'Collect' index | | | 'Collect' index | | |||
+--------+--------+ | +--------+--------+ | |||
| | | | |||
* | * | |||
Meter Reader | Meter Reader | |||
4.2 Flow Table | 4.2 Flow Table | |||
Every traffic meter maintains a table of TRAFFIC FLOW RECORDS for flows | Every traffic meter maintains a table of TRAFFIC FLOW RECORDS for | |||
seen by the meter. A flow record contains attribute values for its | flows seen by the meter. A flow record contains attribute values for | |||
flow, including: | its flow, including: | |||
- Addresses for the flow's source and destination. These include | - Addresses for the flow's source and destination. These include | |||
addresses and masks for various network layers (extracted from the | addresses and masks for various network layers (extracted from the | |||
packet), and the number of the interface on which the packet was | packet), and the number of the interface on which the packet was | |||
observed. | observed. | |||
- First and last times when packets were seen for this flow. | - First and last times when packets were seen for this flow. | |||
- Counts for 'forward' (source to destination) and 'backward' | - Counts for 'forward' (source to destination) and 'backward' | |||
(destination to source) components of the flow's traffic. | (destination to source) components of the flow's traffic. | |||
- Other attributes, e.g. state of the flow record (discussed below). | - Other attributes, e.g. state of the flow record (discussed below). | |||
The state of a flow record may be: | The state of a flow record may be: | |||
- INACTIVE: The flow record is not being used by the meter. | - INACTIVE: The flow record is not being used by the meter. | |||
- CURRENT: The record is in use and describes a flow which belongs to | - CURRENT: The record is in use and describes a flow which belongs to | |||
the 'current flow set,' i.e. the set of flows recently seen by the | the 'current flow set,' i.e. the set of flows recently seen by the | |||
meter. | meter. | |||
- IDLE: The record is in use and the flow which it describes is part | - IDLE: The record is in use and the flow which it describes is part | |||
of the current flow set. In addition, no packets belonging to this | of the current flow set. In addition, no packets belonging to this | |||
flow have been seen for a period specified by the meter's | flow have been seen for a period specified by the meter's | |||
InactivityTime variable. | InactivityTime variable. | |||
4.3 Packet Handling, Packet Matching | 4.3 Packet Handling, Packet Matching | |||
Each packet header received by the traffic meter program is processed as | Each packet header received by the traffic meter program is processed | |||
follows: | as follows: | |||
- Extract attribute values from the packet header and use them to | - Extract attribute values from the packet header and use them to | |||
create a MATCH KEY for the packet. | create a MATCH KEY for the packet. | |||
- Match the packet's key against the current rule set, as explained | - Match the packet's key against the current rule set, as explained | |||
in detail below. | in detail below. | |||
The rule set specifies whether the packet is to be counted or ignored. | The rule set specifies whether the packet is to be counted or | |||
If it is to be counted the matching process produces a FLOW KEY for the | ignored. If it is to be counted the matching process produces a FLOW | |||
flow to which the packet belongs. This flow key is used to find the | KEY for the flow to which the packet belongs. This flow key is used | |||
flow's record in the flow table; if a record does not yet exist for this | to find the flow's record in the flow table; if a record does not yet | |||
flow, a new flow record may be created. The counts for the matching | exist for this flow, a new flow record may be created. The counts | |||
flow record can then be incremented. | for the matching flow record can then be incremented. | |||
For example, the rule set could specify that packets to or from any host | For example, the rule set could specify that packets to or from any | |||
in IP network 130.216 are to be counted. It could also specify that | host in IP network 130.216 are to be counted. It could also specify | |||
flow records are to be created for every pair of 24-bit (Class C) | that flow records are to be created for every pair of 24-bit (Class | |||
subnets within network 130.216. | C) subnets within network 130.216. | |||
Each packet's match key is passed to the meter's PATTERN MATCHING ENGINE | Each packet's match key is passed to the meter's PATTERN MATCHING | |||
(PME) for matching. The PME is a Virtual Machine which uses a set of | ENGINE (PME) for matching. The PME is a Virtual Machine which uses a | |||
instructions called RULES, i.e. a RULE SET is a program for the PME. A | set of instructions called RULES, i.e. a RULE SET is a program for | |||
packet's match key contains an interface number, source address (S) and | the PME. A packet's match key contains an interface number, source | |||
destination address (D) values. It does not, however, contain any | address (S) and destination address (D) values. It does not, | |||
attribute masks for its attributes, only their values. | however, contain any attribute masks for its attributes, only their | |||
values. | ||||
If measured flows were unidirectional, i.e. only counted packets | If measured flows were unidirectional, i.e. only counted packets | |||
travelling in one direction, the matching process would be simple. The | travelling in one direction, the matching process would be simple. | |||
PME would be called once to match the packet. Any flow key produced by | The PME would be called once to match the packet. Any flow key | |||
a successful match would be used to find the flow's record in the flow | produced by a successful match would be used to find the flow's | |||
table, and that flow's counters would be updated. | record in the flow table, and that flow's counters would be updated. | |||
Flows are, however, bidirectional, reflecting the forward and reverse | Flows are, however, bidirectional, reflecting the forward and reverse | |||
packets of a protocol interchange or 'session.' Maintaining two sets of | packets of a protocol interchange or 'session.' Maintaining two sets | |||
counters in the meter's flow record makes the resulting flow data much | of counters in the meter's flow record makes the resulting flow data | |||
simpler to handle, since analysis programs do not have to gather | much simpler to handle, since analysis programs do not have to gather | |||
together the 'forward' and 'reverse' components of sessions. | together the 'forward' and 'reverse' components of sessions. | |||
Implementing bi-directional flows is, of course, more difficult for the | Implementing bi-directional flows is, of course, more difficult for | |||
meter, since it must decide whether a packet is a 'forward' packet or a | the meter, since it must decide whether a packet is a 'forward' | |||
'reverse' one. To make this decision the meter will often need to | packet or a 'reverse' one. To make this decision the meter will | |||
invoke the PME twice, once for each possible packet direction. | often need to invoke the PME twice, once for each possible packet | |||
direction. | ||||
The diagram below describes the algorithm used by the traffic meter to | The diagram below describes the algorithm used by the traffic meter | |||
process each packet. Flow through the diagram is from left to right and | to process each packet. Flow through the diagram is from left to | |||
top to bottom, i.e. from the top left corner to the bottom right | right and top to bottom, i.e. from the top left corner to the bottom | |||
corner. S indicates the flow's source address (i.e. its set of source | right corner. S indicates the flow's source address (i.e. its set | |||
address attribute values) from the packet, and D indicates its | of source address attribute values) from the packet, and D indicates | |||
destination address. | its destination address. | |||
There are several cases to consider. These are: | There are several cases to consider. These are: | |||
- The packet is recognised as one which is TO BE IGNORED. | - The packet is recognised as one which is TO BE IGNORED. | |||
- The packet MATCHES IN BOTH DIRECTIONS. One situation in which this | - The packet MATCHES IN BOTH DIRECTIONS. One situation in which this | |||
could happen would be a rule set which matches flows within network | could happen would be a rule set which matches flows within network | |||
X (Source = X, Dest = X) but specifies that flows are to be created | X (Source = X, Dest = X) but specifies that flows are to be created | |||
for each subnet within network X, say subnets y and z. If, for | for each subnet within network X, say subnets y and z. If, for | |||
example a packet is seen for y->z, the meter must check that flow | example a packet is seen for y->z, the meter must check that flow | |||
z->y is not already current before creating y->z. | z->y is not already current before creating y->z. | |||
- The packet MATCHES IN ONE DIRECTION ONLY. If its flow is already | - The packet MATCHES IN ONE DIRECTION ONLY. If its flow is already | |||
current, its forward or reverse counters are incremented. | current, its forward or reverse counters are incremented. | |||
Otherwise it is added to the flow table and then counted. | Otherwise it is added to the flow table and then counted. | |||
The algorithm uses four functions, as follows: | The algorithm uses four functions, as follows: | |||
match(A->B) implements the PME. It uses the meter's current rule set | match(A->B) implements the PME. It uses the meter's current rule set | |||
to match the attribute values in the packet's match key. A->B means | to match the attribute values in the packet's match key. A->B means | |||
that the assumed source address is A and destination address B, i.e. | that the assumed source address is A and destination address B, i.e. | |||
that the packet was travelling from A to B. match() returns one of | that the packet was travelling from A to B. match() returns one of | |||
three results: | three results: | |||
'Ignore' means that the packet was matched but this flow is not | 'Ignore' means that the packet was matched but this flow is not | |||
to be counted. | to be counted. | |||
skipping to change at page 20, line 44 | skipping to change at page 20, line 31 | |||
current(S->D) ------------------ count(S->D,f) --------------+ | current(S->D) ------------------ count(S->D,f) --------------+ | |||
| Fail | | | Fail | | |||
| Suc | | | Suc | | |||
current(D->S) ------------------ count(D->S,r) --------------+ | current(D->S) ------------------ count(D->S,r) --------------+ | |||
| Fail | | | Fail | | |||
| | | | | | |||
create(S->D) ------------------- count(S->D,f) --------------+ | create(S->D) ------------------- count(S->D,f) --------------+ | |||
| | | | |||
* | * | |||
When writing rule sets one must remember that the meter will normally | When writing rule sets one must remember that the meter will normally | |||
try to match each packet in both directions. It is particularly | try to match each packet in both directions. It is particularly | |||
important that the rule set does not contain inconsistencies which will | important that the rule set does not contain inconsistencies which | |||
upset this process. | will upset this process. | |||
Consider, for example, a rule set which counts packets from source | Consider, for example, a rule set which counts packets from source | |||
network A to destination network B, but which ignores packets from | network A to destination network B, but which ignores packets from | |||
source network B. This is an obvious example of an inconsistent rule | source network B. This is an obvious example of an inconsistent rule | |||
set, since packets from network B should be counted as reverse packets | set, since packets from network B should be counted as reverse | |||
for the A-to-B flow. | packets for the A-to-B flow. | |||
This problem could be avoided by devising a language for specifying rule | This problem could be avoided by devising a language for specifying | |||
files and writing a compiler for it, thus making it much easier to | rule files and writing a compiler for it, thus making it much easier | |||
produce correct rule sets. Another approach would be to write a 'rule | to produce correct rule sets. Another approach would be to write a | |||
set consistency checker' program, which could detect problems in | 'rule set consistency checker' program, which could detect problems | |||
hand-written rule sets. | in hand-written rule sets. | |||
In the short term the best way to avoid these problems is to write rule | In the short term the best way to avoid these problems is to write | |||
sets which only clasify flows in the forward direction, and rely on the | rule sets which only clasify flows in the forward direction, and rely | |||
meter to handle reverse-travelling packets. | on the meter to handle reverse-travelling packets. | |||
4.4 Rules and Rule Sets | 4.4 Rules and Rule Sets | |||
A rule set is an array of rules. Rule sets are held within a meter as | A rule set is an array of rules. Rule sets are held within a meter | |||
entries in an array of rule sets. One member of this array is the | as entries in an array of rule sets. One member of this array is the | |||
CURRENT RULE SET, in that it is the one which is currently being used by | CURRENT RULE SET, in that it is the one which is currently being used | |||
the meter to classify incoming packets. | by the meter to classify incoming packets. | |||
Rule set 1 is built in to the meter and cannot be changed. It is run | Rule set 1 is built in to the meter and cannot be changed. It is run | |||
when the meter is started up, and provides a very coarse reporting | when the meter is started up, and provides a very coarse reporting | |||
granularity; it is mainly useful for verifying that the meter is | granularity; it is mainly useful for verifying that the meter is | |||
running, before a 'useful' rule set is downloaded to it. | running, before a 'useful' rule set is downloaded to it. | |||
If the meter is instructed to use rule set 0, it will cease measuring; | If the meter is instructed to use rule set 0, it will cease | |||
all packets will be ignored until another (non-zero) rule set is made | measuring; all packets will be ignored until another (non-zero) rule | |||
current. | set is made current. | |||
Each rule in a rule set is structured as follows: | Each rule in a rule set is structured as follows: | |||
+-------- test ---------+ +---- action -----+ | +-------- test ---------+ +---- action -----+ | |||
attribute & mask = value: opcode, parameter; | attribute & mask = value: opcode, parameter; | |||
Opcodes contain two flags: 'goto' and 'test.' The PME maintains a | Opcodes contain two flags: 'goto' and 'test.' The PME maintains a | |||
Boolean indicator called the 'test indicator,' which is initially set | Boolean indicator called the 'test indicator,' which is initially set | |||
(on). Execution begins with rule 1, the first in the rule set. It | (on). Execution begins with rule 1, the first in the rule set. It | |||
proceeds as follows: | proceeds as follows: | |||
If the test indicator is on: | If the test indicator is on: | |||
Perform the test, i.e. AND the attribute value with the | Perform the test, i.e. AND the attribute value with the | |||
mask and compare it with the value. | mask and compare it with the value. | |||
If these are equal the test has succeeded; perform the | If these are equal the test has succeeded; perform the | |||
rule's action (below). | rule's action (below). | |||
If the test fails execute the next rule in the rule set. | If the test fails execute the next rule in the rule set. | |||
If there are no more rules in the rule set, return from the | If there are no more rules in the rule set, return from the | |||
match() function indicating failure. | match() function indicating failure. | |||
skipping to change at page 22, line 16 | skipping to change at page 22, line 5 | |||
Set the test indicator to this rule's test flag value. | Set the test indicator to this rule's test flag value. | |||
Determine the next rule to execute. | Determine the next rule to execute. | |||
If the opcode has its goto flag set, its parameter value | If the opcode has its goto flag set, its parameter value | |||
specifies the number of the next rule. | specifies the number of the next rule. | |||
Opcodes which don't have their goto flags set either | Opcodes which don't have their goto flags set either | |||
determine the next rule in special ways (Return), | determine the next rule in special ways (Return), | |||
or they terminate execution (Ignore, Fail, Count, | or they terminate execution (Ignore, Fail, Count, | |||
CountPkt). | CountPkt). | |||
Perform the action. | Perform the action. | |||
The PME maintains two 'history' data structures. The first, the | The PME maintains two 'history' data structures. The first, the | |||
'return' stack, simply records the index (i.e. 1-origin rule number) of | 'return' stack, simply records the index (i.e. 1-origin rule number) | |||
each Gosub rule as it is executed; Return rules pop their Gosub rule | of each Gosub rule as it is executed; Return rules pop their Gosub | |||
index. The second, the 'pattern' queue, is used to save information for | rule index. The second, the 'pattern' queue, is used to save | |||
later use in building a flow key. A flow key is built by zeroing all | information for later use in building a flow key. A flow key is | |||
its attribute values, then copying attribute and mask information from | built by zeroing all its attribute values, then copying attribute and | |||
the pattern stack in the order it was enqueued. | mask information from the pattern stack in the order it was enqueued. | |||
The opcodes are: | The opcodes are: | |||
opcode goto test | opcode goto test | |||
1 Ignore 0 - | 1 Ignore 0 - | |||
2 Fail 0 - | 2 Fail 0 - | |||
3 Count 0 - | 3 Count 0 - | |||
4 CountPkt 0 - | 4 CountPkt 0 - | |||
5 Return 0 0 | 5 Return 0 0 | |||
6 Gosub 1 1 | 6 Gosub 1 1 | |||
7 GosubAct 1 0 | 7 GosubAct 1 0 | |||
8 Assign 1 1 | 8 Assign 1 1 | |||
9 AssignAct 1 0 | 9 AssignAct 1 0 | |||
10 Goto 1 1 | 10 Goto 1 1 | |||
11 GotoAct 1 0 | 11 GotoAct 1 0 | |||
12 PushRuleTo 1 1 | 12 PushRuleTo 1 1 | |||
13 PushRuleToAct 1 0 | 13 PushRuleToAct 1 0 | |||
14 PushPktTo 1 1 | 14 PushPktTo 1 1 | |||
15 PushPktToAct 1 0 | 15 PushPktToAct 1 0 | |||
The actions they perform are: | The actions they perform are: | |||
Ignore: Stop matching, return from the match() function | Ignore: Stop matching, return from the match() function | |||
indicating that the packet is to be ignored. | indicating that the packet is to be ignored. | |||
Fail: Stop matching, return from the match() function | Fail: Stop matching, return from the match() function | |||
indicating failure. | indicating failure. | |||
Count: Stop matching. Save this rule's attribute name, | Count: Stop matching. Save this rule's attribute name, | |||
mask and value in the PME's pattern queue, then | mask and value in the PME's pattern queue, then | |||
construct a flow key for the flow to which this | construct a flow key for the flow to which this | |||
skipping to change at page 24, line 25 | skipping to change at page 24, line 17 | |||
PME's pattern queue. SET the test indicator then | PME's pattern queue. SET the test indicator then | |||
goto the specified rule. | goto the specified rule. | |||
PushPktToAct: Same as PushPktTo, except that the test indicator | PushPktToAct: Same as PushPktTo, except that the test indicator | |||
is cleared before going to the specified rule. | is cleared before going to the specified rule. | |||
PushPktTo actions may be used to save a value from | PushPktTo actions may be used to save a value from | |||
the packet using a specified mask. The test in | the packet using a specified mask. The test in | |||
PushPktTo rules will almost never be executed. | PushPktTo rules will almost never be executed. | |||
As well as the attributes applying directly to packets (such as | As well as the attributes applying directly to packets (such as | |||
SourcePeerAddress, DestTransAddress, etc.) the PME implements several | SourcePeerAddress, DestTransAddress, etc.) the PME implements | |||
further attribtes. These are: | several further attribtes. These are: | |||
Null: Tests performed on the Null attribute always succeed. | Null: Tests performed on the Null attribute always succeed. | |||
v1 .. v5: v1, v2, v3, v4 and v5 are 'meter variables.' They | v1 .. v5: v1, v2, v3, v4 and v5 are 'meter variables.' They | |||
provide a way to pass parameters into rule-matching | provide a way to pass parameters into rule-matching | |||
subroutines. Each may hold the name of a normal | subroutines. Each may hold the name of a normal | |||
attribute; its value is set by an Assign action. | attribute; its value is set by an Assign action. | |||
When a meter variable appears as the attribute of a | When a meter variable appears as the attribute of a | |||
rule, its value specifies the actual attribute to be | rule, its value specifies the actual attribute to be | |||
tested. For example, if v1 had been assigned | tested. For example, if v1 had been assigned | |||
skipping to change at page 25, line 7 | skipping to change at page 24, line 44 | |||
SourceKind, DestKind, FlowKind: | SourceKind, DestKind, FlowKind: | |||
These six attributes may be set by executing PushRuleto | These six attributes may be set by executing PushRuleto | |||
actions. They allow the PME to save (in flow records) | actions. They allow the PME to save (in flow records) | |||
information which has been built up during matching. | information which has been built up during matching. | |||
Since their values are only defined when matching is | Since their values are only defined when matching is | |||
complete (and the flow key is built) their values may | complete (and the flow key is built) their values may | |||
not be tested in rules. | not be tested in rules. | |||
4.5 Maintaining the Flow Table | 4.5 Maintaining the Flow Table | |||
The flow table may be thought of as a 1-origin array of flow records. | The flow table may be thought of as a 1-origin array of flow records. | |||
(A particular implementation may, of course, use whatever data structure | (A particular implementation may, of course, use whatever data | |||
is most suitable). When the meter starts up there are no known flows; | structure is most suitable). When the meter starts up there are no | |||
all the flow records are in the 'inactive' state. | known flows; all the flow records are in the 'inactive' state. | |||
Each time a packet is seen for a flow which is not in the current flow | Each time a packet is seen for a flow which is not in the current | |||
set a flow record is set up for it; the state of such a record is | flow set a flow record is set up for it; the state of such a record | |||
'current.' When selecting a record for the new flow the meter searches | is 'current.' When selecting a record for the new flow the meter | |||
the flow table for a 'inactive' record - there is no particular | searches the flow table for a 'inactive' record - there is no | |||
significance in the ordering of records within the table. | particular significance in the ordering of records within the table. | |||
Flow data may be collected by a 'meter reader' at any time. There is no | Flow data may be collected by a 'meter reader' at any time. There is | |||
requirement for collections to be synchronized. The reader may collect | no requirement for collections to be synchronized. The reader may | |||
the data in any suitable manner, for example it could upload a copy of | collect the data in any suitable manner, for example it could upload | |||
the whole flow table using a file transfer protocol, or it could read | a copy of the whole flow table using a file transfer protocol, or it | |||
the records in the current flow set row by row using a suitable data | could read the records in the current flow set row by row using a | |||
transfer protocol. | suitable data transfer protocol. | |||
The meter keeps information about collections, in particular it | The meter keeps information about collections, in particular it | |||
maintains a LastCollectTime variable which remembers the time the last | maintains a LastCollectTime variable which remembers the time the | |||
collection was made. A second variable, InactivityTime, specifies the | last collection was made. A second variable, InactivityTime, | |||
minimum time the meter will wait before considering that a flow is idle. | specifies the minimum time the meter will wait before considering | |||
that a flow is idle. | ||||
The meter must recover records used for idle flows, if only to prevent | The meter must recover records used for idle flows, if only to | |||
it running out of flow records. Recovered flow records are returned to | prevent it running out of flow records. Recovered flow records are | |||
the 'inactive' state. A variety of recovery strategies are possible, | returned to the 'inactive' state. A variety of recovery strategies | |||
including the following: | are possible, including the following: | |||
One possible recovery strategy is to recover idle flow records as soon | One possible recovery strategy is to recover idle flow records as | |||
as possible after their data has been collected. To implement this the | soon as possible after their data has been collected. To implement | |||
meter could run a background process which scans the flow table looking | this the meter could run a background process which scans the flow | |||
for 'current' flows whose 'last packet' time is earlier than the meter's | table looking for 'current' flows whose 'last packet' time is earlier | |||
LastCollectTime. This would be suitable for use when one was interested | than the meter's LastCollectTime. This would be suitable for use | |||
in measuring flow lifetimes. | when one was interested in measuring flow lifetimes. | |||
Another recovery strategy is to leave idle flows alone as long as | Another recovery strategy is to leave idle flows alone as long as | |||
possible, which would be suitable if one was only interested in | possible, which would be suitable if one was only interested in | |||
measuring total traffic volumes. It could be implemented by having the | measuring total traffic volumes. It could be implemented by having | |||
meter search for collected idle flows only when it ran out of 'inactive' | the meter search for collected idle flows only when it ran out of | |||
flow records. | 'inactive' flow records. | |||
One further factor a meter should consider before recovering a flow is | One further factor a meter should consider before recovering a flow | |||
the number of meter readers which have collected the flow's data. If | is the number of meter readers which have collected the flow's data. | |||
there are multiple meter readers operating, network Operations personnel | If there are multiple meter readers operating, network Operations | |||
should be able to specify the minimum number of meters - or perhaps a | personnel should be able to specify the minimum number of meters - or | |||
specific list of meters - which should collect a flow's data before its | perhaps a specific list of meters - which should collect a flow's | |||
memory can be recovered. This issue will be further developed in the | data before its memory can be recovered. This issue will be further | |||
future. | developed in the future. | |||
4.6 Handling Increasing Traffic Levels | 4.6 Handling Increasing Traffic Levels | |||
Under normal conditions the meter reader specifies which set of usage | Under normal conditions the meter reader specifies which set of usage | |||
records it wants to collect, and the meter provides them. | records it wants to collect, and the meter provides them. | |||
If memory usage rises above the high-water mark the meter should switch | If memory usage rises above the high-water mark the meter should | |||
to a STANDBY RULE SET so as to increase the granularity of flow | switch to a STANDBY RULE SET so as to increase the granularity of | |||
collection and decrease the rate at which new flows are created. When | flow collection and decrease the rate at which new flows are created. | |||
the manager, usually as part of a regular poll, becomes aware that the | When the manager, usually as part of a regular poll, becomes aware | |||
meter is using its standby rule set, it could decrease the interval | that the meter is using its standby rule set, it could decrease the | |||
between collections. The meter should also increase its efforts to | interval between collections. The meter should also increase its | |||
recover flow memory so as to reduce the number of idle flows in memory. | efforts to recover flow memory so as to reduce the number of idle | |||
When the situation returns to normal, the manager may request the meter | flows in memory. When the situation returns to normal, the manager | |||
to switch back to its normal rule set. | may request the meter to switch back to its normal rule set. | |||
5 Meter Readers | 5 Meter Readers | |||
Usage data is accumulated by a meter (e.g. in a router) as memory | Usage data is accumulated by a meter (e.g. in a router) as memory | |||
permits. It is collected at regular reporting intervals by meter | permits. It is collected at regular reporting intervals by meter | |||
readers, as specified by a manager. The collected data is recorded in a | readers, as specified by a manager. The collected data is recorded | |||
disk file called a FLOW DATA FILE, as a sequence of USAGE RECORDS. | in a disk file called a FLOW DATA FILE, as a sequence of USAGE | |||
RECORDS. | ||||
The following sections describe the contents of usage records and flow | The following sections describe the contents of usage records and | |||
data files. Note, however, that at this stage the details of such | flow data files. Note, however, that at this stage the details of | |||
records and files is not specified in the architecture. Specifying a | such records and files is not specified in the architecture. | |||
common format for them would be a worthwhile future development. | Specifying a common format for them would be a worthwhile future | |||
development. | ||||
5.1 Identifying Flows in Flow Records | 5.1 Identifying Flows in Flow Records | |||
Once a packet has been classified and is ready to be counted, an | Once a packet has been classified and is ready to be counted, an | |||
appropriate flow data record must already exist in the flow table; | appropriate flow data record must already exist in the flow table; | |||
otherwise one must be created. The flow record has a flexible format | otherwise one must be created. The flow record has a flexible format | |||
where unnecessary identification attributes may be omitted. The | where unnecessary identification attributes may be omitted. The | |||
determination of which attributes of the flow record to use, and of what | determination of which attributes of the flow record to use, and of | |||
values to put in them, is specified by the current rule set. | what values to put in them, is specified by the current rule set. | |||
Note that the combination of start time, rule set id and subscript (row | Note that the combination of start time, rule set id and subscript | |||
number in the flow table) provide a unique flow identifier, regardless | (row number in the flow table) provide a unique flow identifier, | |||
of the values of its other attributes. | regardless of the values of its other attributes. | |||
The current rule set may specify additional information, e.g. a | The current rule set may specify additional information, e.g. a | |||
computed attribute value such as FlowKind, which is to be placed in the | computed attribute value such as FlowKind, which is to be placed in | |||
attribute section of the usage record. That is, if a particular flow is | the attribute section of the usage record. That is, if a particular | |||
matched by the rule set, then the corresponding flow record should be | flow is matched by the rule set, then the corresponding flow record | |||
marked not only with the qualifying identification attributes, but also | should be marked not only with the qualifying identification | |||
with the additional information. Using this feature, several flows may | attributes, but also with the additional information. Using this | |||
each carry the same FlowKind value, so that the resulting usage records | feature, several flows may each carry the same FlowKind value, so | |||
can be used in post-processing or between meter reader and meter as a | that the resulting usage records can be used in post-processing or | |||
criterion for collection. | between meter reader and meter as a criterion for collection. | |||
5.2 Usage Records, Flow Data Files | 5.2 Usage Records, Flow Data Files | |||
The collected usage data will be stored in flow data files on the meter | The collected usage data will be stored in flow data files on the | |||
reader, one file for each meter. As well as containing the measured | meter reader, one file for each meter. As well as containing the | |||
usage data, flow data files must contain information uniquely | measured usage data, flow data files must contain information | |||
identifiying the meter from which it was collected. | uniquely identifiying the meter from which it was collected. | |||
A USAGE RECORD contains the descriptions of and values for one or more | A USAGE RECORD contains the descriptions of and values for one or | |||
flows. Quantities are counted in terms of number of packets and number | more flows. Quantities are counted in terms of number of packets and | |||
of bytes per flow. Each usage record contains the entity identifier of | number of bytes per flow. Each usage record contains the entity | |||
the meter (a network address), a time stamp and a list of reported flows | identifier of the meter (a network address), a time stamp and a list | |||
(FLOW DATA RECORDS). A meter reader will build up a file of usage | of reported flows (FLOW DATA RECORDS). A meter reader will build up a | |||
records by regularly collecting flow data from a meter, using this data | file of usage records by regularly collecting flow data from a meter, | |||
to build usage records and concatenating them to the tail of a file. | using this data to build usage records and concatenating them to the | |||
Such a file is called a FLOW DATA FILE. | tail of a file. Such a file is called a FLOW DATA FILE. | |||
A usage record contains the following information in some form: | A usage record contains the following information in some form: | |||
+-------------------------------------------------------------------+ | +-------------------------------------------------------------------+ | |||
| RECORD IDENTIFIERS: | | | RECORD IDENTIFIERS: | | |||
| Meter Id (& digital signature if required) | | | Meter Id (& digital signature if required) | | |||
| Timestamp | | | Timestamp | | |||
| Collection Rules ID | | | Collection Rules ID | | |||
+-------------------------------------------------------------------+ | +-------------------------------------------------------------------+ | |||
| FLOW IDENTIFIERS: | COUNTERS | | | FLOW IDENTIFIERS: | COUNTERS | | |||
| Address List | Packet Count | | | Address List | Packet Count | | |||
| Subscriber ID (Optional) | Byte Count | | | Subscriber ID (Optional) | Byte Count | | |||
| Attributes (Optional) | Flow Start/Stop Time | | | Attributes (Optional) | Flow Start/Stop Time | | |||
+-------------------------------------------------------------------+ | +-------------------------------------------------------------------+ | |||
5.3 Meter to Meter Reader: Usage Record Transmission | 5.3 Meter to Meter Reader: Usage Record Transmission | |||
The usage record contents are the raison d'etre of the system. The | The usage record contents are the raison d'etre of the system. The | |||
accuracy, reliability, and security of transmission are the primary | accuracy, reliability, and security of transmission are the primary | |||
concerns of the meter/meter reader exchange. Since errors may occur on | concerns of the meter/meter reader exchange. Since errors may occur | |||
networks, and Internet packets may be dropped, some mechanism for | on networks, and Internet packets may be dropped, some mechanism for | |||
ensuring that the usage information is transmitted intact is needed. | ensuring that the usage information is transmitted intact is needed. | |||
Flow data is moved from meter to meter reader via a series of protocol | Flow data is moved from meter to meter reader via a series of | |||
exchanges between them. This may be carried out in various ways, moving | protocol exchanges between them. This may be carried out in various | |||
individual attribute values, complete flows, or the entire flow table | ways, moving individual attribute values, complete flows, or the | |||
(i.e. all the active flows). One possible method of achieving this | entire flow table (i.e. all the active flows). One possible method | |||
transfer is to use SNMP; the 'Traffic Flow Measurement: Meter MIB' | of achieving this transfer is to use SNMP; the 'Traffic Flow | |||
document [4] gives details. Note that this is simply one example; the | Measurement: Meter MIB' document [4] gives details. Note that this | |||
transfer of flow data from meter to meter reader is not specified in | is simply one example; the transfer of flow data from meter to meter | |||
this document. | reader is not specified in this document. | |||
The reliability of the data transfer method under light, normal, and | The reliability of the data transfer method under light, normal, and | |||
extreme network loads should be understood before selecting among | extreme network loads should be understood before selecting among | |||
collection methods. | collection methods. | |||
In normal operation the meter will be running a rule file which provides | In normal operation the meter will be running a rule file which | |||
the required degree of flow reporting granularity, and the meter | provides the required degree of flow reporting granularity, and the | |||
reader(s) will collect the flow data often enough to allow the meter's | meter reader(s) will collect the flow data often enough to allow the | |||
garbage collection mechanism to maintain a stable level of memory usage. | meter's garbage collection mechanism to maintain a stable level of | |||
memory usage. | ||||
In the worst case traffic may increase to the point where the meter is | In the worst case traffic may increase to the point where the meter | |||
in danger of running completely out of flow memory. The meter | is in danger of running completely out of flow memory. The meter | |||
implementor must decide how to handle this, for example by switching to | implementor must decide how to handle this, for example by switching | |||
a default (extremely coarse granularity) rule set, by sending a trap to | to a default (extremely coarse granularity) rule set, by sending a | |||
the manager, or by attempting to dump flow data to the meter reader. | trap to the manager, or by attempting to dump flow data to the meter | |||
reader. | ||||
Users of the Traffic Flow Measurement system should analyse their | Users of the Traffic Flow Measurement system should analyse their | |||
requirements carefully and assess for themselves whether it is more | requirements carefully and assess for themselves whether it is more | |||
important to attempt to collect flow data at normal granularity | important to attempt to collect flow data at normal granularity | |||
(increasing the collection frequency as needed to keep up with traffic | (increasing the collection frequency as needed to keep up with | |||
volumes), or to accept flow data with a coarser granularity. Similarly, | traffic volumes), or to accept flow data with a coarser granularity. | |||
it may be acceptable to lose flow data for a short time in return for | Similarly, it may be acceptable to lose flow data for a short time in | |||
being sure that the meter keeps running properly, i.e. is not | return for being sure that the meter keeps running properly, i.e. is | |||
overwhelmed by rising traffic levels. | not overwhelmed by rising traffic levels. | |||
6 Managers | 6 Managers | |||
A manager configures meters and controls meter readers. It does this | A manager configures meters and controls meter readers. It does this | |||
via the interactions described below. | via the interactions described below. | |||
6.1 Between Manager and Meter: Control Functions | 6.1 Between Manager and Meter: Control Functions | |||
- DOWNLOAD RULE SET: A meter may hold an array of rule sets. One of | - DOWNLOAD RULE SET: A meter may hold an array of rule sets. One of | |||
these, the 'default' rule set, is built in to the meter and cannot | these, the 'default' rule set, is built in to the meter and cannot | |||
be changed; the others must be downloaded by the manager. A | be changed; the others must be downloaded by the manager. A | |||
manager may use any suitable protocol exchange to achieve this, for | manager may use any suitable protocol exchange to achieve this, for | |||
example an FTP file transfer or a series of SNMP SETs, one for each | example an FTP file transfer or a series of SNMP SETs, one for each | |||
row of the rule set. | row of the rule set. | |||
skipping to change at page 29, line 40 | skipping to change at page 29, line 31 | |||
- Flows with the smallest number of unreported packets. | - Flows with the smallest number of unreported packets. | |||
- SET INACTIVITY TIMEOUT: This is a time in seconds since the last | - SET INACTIVITY TIMEOUT: This is a time in seconds since the last | |||
packet was seen for a flow. Flow records may be reclaimed if they | packet was seen for a flow. Flow records may be reclaimed if they | |||
have been idle for at least this amount of time, and have been | have been idle for at least this amount of time, and have been | |||
collected in accordance with the current collection criteria. | collected in accordance with the current collection criteria. | |||
6.2 Between Manager and Meter Reader: Control Functions | 6.2 Between Manager and Meter Reader: Control Functions | |||
Because there are a number of parameters that must be set for traffic | Because there are a number of parameters that must be set for traffic | |||
flow measurement to function properly, and viable settings may change as | flow measurement to function properly, and viable settings may change | |||
a result of network traffic characteristics, it is desirable to have | as a result of network traffic characteristics, it is desirable to | |||
dynamic network management as opposed to static meter configurations. | have dynamic network management as opposed to static meter | |||
Many of these operations have to do with space tradeoffs - if memory at | configurations. Many of these operations have to do with space | |||
the meter is exhausted, either the reporting interval must be decreased | tradeoffs - if memory at the meter is exhausted, either the reporting | |||
or a coarser granularity of aggregation must be used so that more data | interval must be decreased or a coarser granularity of aggregation | |||
fits into less space. | must be used so that more data fits into less space. | |||
Increasing the reporting interval effectively stores data in the meter; | Increasing the reporting interval effectively stores data in the | |||
usage data in transit is limited by the effective bandwidth of the | meter; usage data in transit is limited by the effective bandwidth of | |||
virtual link between the meter and the meter reader, and since these | the virtual link between the meter and the meter reader, and since | |||
limited network resources are usually also used to carry user data (the | these limited network resources are usually also used to carry user | |||
purpose of the network), the level of traffic flow measurement traffic | data (the purpose of the network), the level of traffic flow | |||
should be kept to an affordable fraction of the bandwidth. | measurement traffic should be kept to an affordable fraction of the | |||
("Affordable" is a policy decision made by the network Operations | bandwidth. ("Affordable" is a policy decision made by the network | |||
personnel). At any rate, it must be understood that the operations | Operations personnel). At any rate, it must be understood that the | |||
below do not represent the setting of independent variables; on the | operations below do not represent the setting of independent | |||
contrary, each of the values set has a direct and measurable effect on | variables; on the contrary, each of the values set has a direct and | |||
the behaviour of the other variables. | measurable effect on the behaviour of the other variables. | |||
Network management operations follow: | Network management operations follow: | |||
- MANAGER and METER READER IDENTIFICATION: The manager should ensure | - MANAGER and METER READER IDENTIFICATION: The manager should ensure | |||
that meters report to the correct set of collection stations, and | that meters report to the correct set of collection stations, and | |||
take steps to prevent unauthorised access to usage information. | take steps to prevent unauthorised access to usage information. | |||
The collection stations so identified should be prepared to poll if | The collection stations so identified should be prepared to poll if | |||
necessary and accept data from the appropriate meters. Alternate | necessary and accept data from the appropriate meters. Alternate | |||
collection stations may be identified in case both the primary | collection stations may be identified in case both the primary | |||
manager and the primary collection station are unavailable. | manager and the primary collection station are unavailable. | |||
Similarly, alternate managers may be identified. | Similarly, alternate managers may be identified. | |||
skipping to change at page 31, line 13 | skipping to change at page 31, line 7 | |||
initial configuration. | initial configuration. | |||
- FLOW LIFETIME CONTROL: Flow termination parameters include timeout | - FLOW LIFETIME CONTROL: Flow termination parameters include timeout | |||
parameters for obsoleting inactive flows and removing them from | parameters for obsoleting inactive flows and removing them from | |||
tables and maximum flow lifetimes. This is intertwined with | tables and maximum flow lifetimes. This is intertwined with | |||
reporting interval and granularity, and must be set in accordance | reporting interval and granularity, and must be set in accordance | |||
with the other parameters. | with the other parameters. | |||
6.3 Exception Conditions | 6.3 Exception Conditions | |||
Exception conditions must be handled, particularly occasions when the | Exception conditions must be handled, particularly occasions when the | |||
meter runs out of buffer space. Since, to prevent counting any packet | meter runs out of buffer space. Since, to prevent counting any | |||
twice, packets can only be counted in a single flow at any given time, | packet twice, packets can only be counted in a single flow at any | |||
discarding records will result in the loss of information. The | given time, discarding records will result in the loss of | |||
mechanisms to deal with this are as follows: | information. The mechanisms to deal with this are as follows: | |||
- METER OUTAGES: In case of impending meter outages (controlled | - METER OUTAGES: In case of impending meter outages (controlled | |||
crashes, etc.) the meter could send a trap to the manager. The | crashes, etc.) the meter could send a trap to the manager. The | |||
manager could then request one or more meter readers to pick up the | manager could then request one or more meter readers to pick up the | |||
usage record from the meter. | usage record from the meter. | |||
Following an uncontrolled meter outage such as a power failure, the | Following an uncontrolled meter outage such as a power failure, the | |||
meter could send a trap to the manager indicating that it has | meter could send a trap to the manager indicating that it has | |||
restarted. The manager could then download the meter's correct | restarted. The manager could then download the meter's correct | |||
rule set and advise the meter reader(s) that the meter is running | rule set and advise the meter reader(s) that the meter is running | |||
skipping to change at page 32, line 4 | skipping to change at page 31, line 44 | |||
should continue measuring and the meter reader(s) should keep | should continue measuring and the meter reader(s) should keep | |||
gathering usage records. | gathering usage records. | |||
- BUFFER PROBLEMS: The network manager may realise that there is a | - BUFFER PROBLEMS: The network manager may realise that there is a | |||
'low memory' condition in the meter. This can usually be | 'low memory' condition in the meter. This can usually be | |||
attributed to the interaction between the following controls: | attributed to the interaction between the following controls: | |||
- The reporting interval is too infrequent, | - The reporting interval is too infrequent, | |||
- The reporting granularity is too fine, or | - The reporting granularity is too fine, or | |||
- The throughput/bandwidth of circuits carrying the usage data is | ||||
too low. | - The throughput/bandwidth of circuits carrying the usage | |||
data is too low. | ||||
The manager may change any of these parameters in response to the | The manager may change any of these parameters in response to the | |||
meter (or meter reader's) plea for help. | meter (or meter reader's) plea for help. | |||
6.4 Standard Rule Sets | 6.4 Standard Rule Sets | |||
Although the rule table is a flexible tool, it can also become very | Although the rule table is a flexible tool, it can also become very | |||
complex. It may be helpful to develop some rule sets for common | complex. It may be helpful to develop some rule sets for common | |||
applications: | applications: | |||
- PROTOCOL TYPE: The meter records packets by protocol type. This | - PROTOCOL TYPE: The meter records packets by protocol type. This | |||
will be the default rule table for Traffic Flow Meters. | will be the default rule table for Traffic Flow Meters. | |||
- ADJACENT SYSTEMS: The meter records packets by the MAC address of | - ADJACENT SYSTEMS: The meter records packets by the MAC address of | |||
the Adjacent Systems (neighbouring originator or next-hop). | the Adjacent Systems (neighbouring originator or next-hop). | |||
(Variants on this table are "report source" or "report sink" only.) | (Variants on this table are "report source" or "report sink" only.) | |||
This strategy might be used by a regional or backbone network which | This strategy might be used by a regional or backbone network which | |||
wants to know how much aggregate traffic flows to or from its | wants to know how much aggregate traffic flows to or from its | |||
subscriber networks. | subscriber networks. | |||
skipping to change at page 33, line 9 | skipping to change at page 33, line 9 | |||
- HYBRID SYSTEMS: Combinations of the above, e.g. for one interface | - HYBRID SYSTEMS: Combinations of the above, e.g. for one interface | |||
report End Systems, for another interface report Adjacent Systems. | report End Systems, for another interface report Adjacent Systems. | |||
This strategy might be used by an enterprise network to learn | This strategy might be used by an enterprise network to learn | |||
detail about local usage and use an aggregate count for the shared | detail about local usage and use an aggregate count for the shared | |||
regional network. | regional network. | |||
7 APPENDICES | 7 APPENDICES | |||
7.1 Appendix A: Network Characterisation | 7.1 Appendix A: Network Characterisation | |||
Internet users have extraordinarily diverse requirements. Networks | Internet users have extraordinarily diverse requirements. Networks | |||
differ in size, speed, throughput, and processing power, among other | differ in size, speed, throughput, and processing power, among other | |||
factors. There is a range of traffic flow measurement capabilities and | factors. There is a range of traffic flow measurement capabilities | |||
requirements. For traffic flow measurement purposes, the Internet may | and requirements. For traffic flow measurement purposes, the | |||
be viewed as a continuum which changes in character as traffic passes | Internet may be viewed as a continuum which changes in character as | |||
through the following representative levels: | traffic passes through the following representative levels: | |||
International | | International | | |||
Backbones/National --------------- | Backbones/National --------------- | |||
/ \ | / \ | |||
Regional/MidLevel ---------- ---------- | Regional/MidLevel ---------- ---------- | |||
/ \ \ / / \ | / \ \ / / \ | |||
Stub/Enterprise --- --- --- ---- ---- | Stub/Enterprise --- --- --- ---- ---- | |||
||| ||| ||| |||| |||| | ||| ||| ||| |||| |||| | |||
End-Systems/Hosts xxx xxx xxx xxxx xxxx | End-Systems/Hosts xxx xxx xxx xxxx xxxx | |||
Note that mesh architectures can also be built out of these components, | Note that mesh architectures can also be built out of these | |||
and that these are merely descriptive terms. The nature of a single | components, and that these are merely descriptive terms. The nature | |||
network may encompass any or all of the descriptions below, although | of a single network may encompass any or all of the descriptions | |||
some networks can be clearly identified as a single type. | below, although some networks can be clearly identified as a single | |||
type. | ||||
BACKBONE networks are typically bulk carriers that connect other | BACKBONE networks are typically bulk carriers that connect other | |||
networks. Individual hosts (with the exception of network management | networks. Individual hosts (with the exception of network management | |||
devices and backbone service hosts) typically are not directly connected | devices and backbone service hosts) typically are not directly | |||
to backbones. | connected to backbones. | |||
REGIONAL networks are closely related to backbones, and differ only in | REGIONAL networks are closely related to backbones, and differ only | |||
size, the number of networks connected via each port, and geographical | in size, the number of networks connected via each port, and | |||
coverage. Regionals may have directly connected hosts, acting as hybrid | geographical coverage. Regionals may have directly connected hosts, | |||
backbone/stub networks. A regional network is a SUBSCRIBER to the | acting as hybrid backbone/stub networks. A regional network is a | |||
backbone. | SUBSCRIBER to the backbone. | |||
STUB/ENTERPRISE networks connect hosts and local area networks. | STUB/ENTERPRISE networks connect hosts and local area networks. | |||
STUB/ENTERPRISE networks are SUBSCRIBERS to regional and backbone | STUB/ENTERPRISE networks are SUBSCRIBERS to regional and backbone | |||
networks. | networks. | |||
END SYSTEMS, colloquially HOSTS, are SUBSCRIBERS to any of the above | END SYSTEMS, colloquially HOSTS, are SUBSCRIBERS to any of the above | |||
networks. | networks. | |||
Providing a uniform identification of the SUBSCRIBER in finer | Providing a uniform identification of the SUBSCRIBER in finer | |||
granularity than that of end-system, (e.g. user/account), is beyond the | granularity than that of end-system, (e.g. user/account), is beyond | |||
scope of the current architecture, although an optional attribute in the | the scope of the current architecture, although an optional attribute | |||
traffic flow measurement record may carry system-specific "accountable | in the traffic flow measurement record may carry system-specific | |||
(billable) party" labels so that meters can implement proprietary or | "accountable (billable) party" labels so that meters can implement | |||
non-standard schemes for the attribution of network traffic to | proprietary or non-standard schemes for the attribution of network | |||
responsible parties. | traffic to responsible parties. | |||
7.2 Appendix B: Recommended Traffic Flow Measurement Capabilities | 7.2 Appendix B: Recommended Traffic Flow Measurement Capabilities | |||
Initial recommended traffic flow measurement conventions are outlined | Initial recommended traffic flow measurement conventions are outlined | |||
here according to the following Internet building blocks. It is | here according to the following Internet building blocks. It is | |||
important to understand what complexity reporting introduces at each | important to understand what complexity reporting introduces at each | |||
network level. Whereas the hierarchy is described top-down in the | network level. Whereas the hierarchy is described top-down in the | |||
previous section, reporting requirements are more easily addressed | previous section, reporting requirements are more easily addressed | |||
bottom-up. | bottom-up. | |||
End-Systems | End-Systems | |||
Stub Networks | Stub Networks | |||
Enterprise Networks | Enterprise Networks | |||
Regional Networks | Regional Networks | |||
Backbone Networks | Backbone Networks | |||
END-SYSTEMS are currently responsible for allocating network usage to | END-SYSTEMS are currently responsible for allocating network usage to | |||
end-users, if this capability is desired. From the Internet Protocol | end-users, if this capability is desired. From the Internet Protocol | |||
perspective, end-systems are the finest granularity that can be | perspective, end-systems are the finest granularity that can be | |||
identified without protocol modifications. Even if a meter violated | identified without protocol modifications. Even if a meter violated | |||
protocol boundaries and tracked higher-level protocols, not all packets | protocol boundaries and tracked higher-level protocols, not all | |||
could be correctly allocated by user, and the definition of user itself | packets could be correctly allocated by user, and the definition of | |||
varies too widely from operating system to operating system (e.g. how | user itself varies too widely from operating system to operating | |||
to trace network usage back to users from shared processes). | system (e.g. how to trace network usage back to users from shared | |||
processes). | ||||
STUB and ENTERPRISE networks will usually collect traffic data either by | STUB and ENTERPRISE networks will usually collect traffic data either | |||
end- system network address or network address pair if detailed | by end- system network address or network address pair if detailed | |||
reporting is required in the local area network. If no local reporting | reporting is required in the local area network. If no local | |||
is required, they may record usage information in the exit router to | reporting is required, they may record usage information in the exit | |||
track external traffic only. (These are the only networks which | router to track external traffic only. (These are the only networks | |||
routinely use attributes to perform reporting at granularities finer | which routinely use attributes to perform reporting at granularities | |||
than end-system or intermediate-system network address.) | finer than end-system or intermediate-system network address.) | |||
REGIONAL networks are intermediate networks. In some cases, subscribers | REGIONAL networks are intermediate networks. In some cases, | |||
will be enterprise networks, in which case the intermediate system | subscribers will be enterprise networks, in which case the | |||
network address is sufficient to identify the regional's immediate | intermediate system network address is sufficient to identify the | |||
subscriber. In other cases, individual hosts or a disjoint group of | regional's immediate subscriber. In other cases, individual hosts or | |||
hosts may constitute a subscriber. Then end- system network address | a disjoint group of hosts may constitute a subscriber. Then end- | |||
pairs need to be tracked for those subscribers. When the source may be | system network address pairs need to be tracked for those | |||
an aggregate entity (such as a network, or adjacent router representing | subscribers. When the source may be an aggregate entity (such as a | |||
traffic from a world of hosts beyond) and the destination is a singular | network, or adjacent router representing traffic from a world of | |||
entity (or vice versa), the meter is said to be operating as a HYBRID | hosts beyond) and the destination is a singular entity (or vice | |||
system. | versa), the meter is said to be operating as a HYBRID system. | |||
At the regional level, if the overhead is tolerable it may be | At the regional level, if the overhead is tolerable it may be | |||
advantageous to report usage both by intermediate system network address | advantageous to report usage both by intermediate system network | |||
(e.g. adjacent router address) and by end-system network address or | address (e.g. adjacent router address) and by end-system network | |||
end-system network address pair. | address or end-system network address pair. | |||
BACKBONE networks are the highest level networks operating at higher | BACKBONE networks are the highest level networks operating at higher | |||
link speeds and traffic levels. The high volume of traffic will in most | link speeds and traffic levels. The high volume of traffic will in | |||
cases preclude detailed traffic flow measurement. Backbone networks | most cases preclude detailed traffic flow measurement. Backbone | |||
will usually account for traffic by adjacent routers' network addresses. | networks will usually account for traffic by adjacent routers' | |||
network addresses. | ||||
7.3 Appendix C: List of Defined Flow Attributes | 7.3 Appendix C: List of Defined Flow Attributes | |||
This Appendix provides a checklist of the attributes defined to date; | This Appendix provides a checklist of the attributes defined to date; | |||
others will be added later as the Traffic Measurement Architecture is | others will be added later as the Traffic Measurement Architecture is | |||
further developed. | further developed. | |||
0 Null | 0 Null | |||
1 Flow Subscript Integer Flow table info | 1 Flow Subscript Integer Flow table info | |||
2 Flow Status Integer | 2 Flow Status Integer | |||
3 Source Interface Integer Source Address | 4 Source Interface Integer Source Address | |||
4 Source Adjacent Type Integer | 5 Source Adjacent Type Integer | |||
5 Source Adjacent Address String | 6 Source Adjacent Address String | |||
6 Source Adjacent Mask String | 7 Source Adjacent Mask String | |||
7 Source Peer Type Integer | 8 Source Peer Type Integer | |||
8 Source Peer Address String | 9 Source Peer Address String | |||
9 Source Peer Mask String | 10 Source Peer Mask String | |||
10 Source Trans Type Integer | 11 Source Trans Type Integer | |||
11 Source Trans Address String | 12 Source Trans Address String | |||
12 Source Trans Mask String | 13 Source Trans Mask String | |||
13 Destination Interface Integer Destination Address | ||||
14 Destination Adjacent Type Integer | ||||
15 Destination Adjacent Address String | ||||
16 Destination AdjacentMask String | ||||
17 Destination PeerType Integer | ||||
18 Destination PeerAddress String | ||||
19 Destination PeerMask String | ||||
20 Destination TransType Integer | ||||
21 Destination TransAddress String | ||||
22 Destination TransMask String | ||||
23 Packet Scale Factor Integer 'Other' attributes | 14 Destination Interface Integer Destination Address | |||
24 Byte Scale Factor Integer | 15 Destination Adjacent Type Integer | |||
25 Rule Set Number Integer | 16 Destination Adjacent Address String | |||
26 Forward Bytes Counter Source-to-Dest counters | 17 Destination AdjacentMask String | |||
27 Forward Packets Counter | 18 Destination PeerType Integer | |||
28 Reverse Bytes Counter Dest-to-Source counters | 19 Destination PeerAddress String | |||
29 Reverse Packets Counter | 20 Destination PeerMask String | |||
30 First Time TimeTicks Activity times | 21 Destination TransType Integer | |||
31 Last Active Time TimeTicks | 22 Destination TransAddress String | |||
23 Destination TransMask String | ||||
32 Source Subscriber ID String Session attributes | 24 Packet Scale Factor Integer 'Other' attributes | |||
33 Destination Subscriber ID String | 25 Byte Scale Factor Integer | |||
34 Session ID String | 26 Rule Set Number Integer | |||
27 Forward Bytes Counter Source-to-Dest counters | ||||
28 Forward Packets Counter | ||||
29 Reverse Bytes Counter Dest-to-Source counters | ||||
30 Reverse Packets Counter | ||||
31 First Time TimeTicks Activity times | ||||
32 Last Active Time TimeTicks | ||||
33 Source Subscriber ID String Session attributes | ||||
34 Destination Subscriber ID String | ||||
35 Session ID String | ||||
35 Source Class Integer 'Computed' attributes | 36 Source Class Integer 'Computed' attributes | |||
36 Destination Class Integer | 37 Destination Class Integer | |||
37 Flow Class Integer | 38 Flow Class Integer | |||
38 Source Kind Integer | 39 Source Kind Integer | |||
39 Destination Kind Integer | 40 Destination Kind Integer | |||
40 Flow Kind Integer | 41 Flow Kind Integer | |||
51 V1 Integer Meter variables | 51 V1 Integer Meter variables | |||
52 V2 Integer | 52 V2 Integer | |||
53 V3 Integer | 53 V3 Integer | |||
54 V4 Integer | 54 V4 Integer | |||
55 V5 Integer | 55 V5 Integer | |||
7.4 Appendix D: List of Meter Control Variables | 7.4 Appendix D: List of Meter Control Variables | |||
Current Rule Set Number Integer | Current Rule Set Number Integer | |||
Standby Rule Set Number Integer | Standby Rule Set Number Integer | |||
High Water Mark Percentage | High Water Mark Percentage | |||
Flood Mark Percentage | Flood Mark Percentage | |||
Inactivity Timeout (seconds) Integer | Inactivity Timeout (seconds) Integer | |||
Last Collect Time TimeTicks | Last Collect Time TimeTicks | |||
8 Acknowledgments | 8 Acknowledgments | |||
An initial draft of this document was produced under the auspices of the | This document was initially produced under the auspices of the IETF's | |||
IETF's Internet Accounting Working Group with assistance from SNMP, RMON | Internet Accounting Working Group with assistance from SNMP, RMON and | |||
and SAAG working groups. This version documents the implementation work | SAAG working groups. This version documents the implementation work | |||
done by the Internet Accounting Working Group, and is intended to | done by the Internet Accounting Working Group, and is intended to | |||
provide a starting point for the Realtime Traffic Flow Measurement | provide a starting point for the Realtime Traffic Flow Measurement | |||
Working Group. Particular thanks are due to Stephen Stibler (IBM | Working Group. Particular thanks are due to Stephen Stibler (IBM | |||
Research) for his patient and careful comments during the preparation of | Research) for his patient and careful comments during the preparation | |||
this draft. | of this memo. | |||
9 References | 9 References | |||
[1] Mills, C., Hirsch, G. and Ruth, G., "Internet Accounting | [1] Mills, C., Hirsch, G. and G. Ruth, "Internet Accounting | |||
Background", RFC 1272, Bolt Beranek and Newman Inc., Meridian | Background", RFC 1272, Bolt Beranek and Newman Inc., Meridian | |||
Technology Corporation, November 1991. | Technology Corporation, November 1991. | |||
[2] International Standards Organisation (ISO), "Management | [2] International Standards Organisation (ISO), "Management | |||
Framework," Part 4 of Information Processing Systems Open | Framework," Part 4 of Information Processing Systems Open | |||
Systems Interconnection Basic Reference Model, ISO 7498-4, | Systems Interconnection Basic Reference Model, ISO 7498-4, | |||
1994. | 1994. | |||
[3] IEEE 802.3/ISO 8802-3 Information Processing Systems - | [3] IEEE 802.3/ISO 8802-3 Information Processing Systems - | |||
Local Area Networks - Part 3: Carrier sense multiple access | Local Area Networks - Part 3: Carrier sense multiple access | |||
with collision detection (CSMA/CD) access method and physical | with collision detection (CSMA/CD) access method and physical | |||
layer specifications, 2nd edition, September 21, 1990. | layer specifications, 2nd edition, September 21, 1990. | |||
[4] Brownlee, N., "Traffic Flow Measurement: Meter MIB," | [4] Brownlee, N., "Traffic Flow Measurement: Meter MIB", | |||
Internet Draft, 'Working draft' to become an experimental RFC. | RFC 2064, The University of Auckland, January 1997. | |||
10 Security Considerations | 10 Security Considerations | |||
Security issues are not discussed in detail in this document. The | Security issues are not discussed in detail in this document. The | |||
meter's management and collection protocols are responsible for | meter's management and collection protocols are responsible for | |||
providing sufficient data integrity and confidentiality. | providing sufficient data integrity and confidentiality. | |||
11 Author's Addresses | 11 Authors' Addresses | |||
Nevil Brownlee | Nevil Brownlee | |||
The University of Auckland | Information Technology Systems & Services | |||
The University of Auckland | ||||
Phone: +64 9 373 7599 x8941 | Phone: +64 9 373 7599 x8941 | |||
E-mail: n.brownlee @auckland.ac.nz | EMail: n.brownlee @auckland.ac.nz | |||
Cyndi Mills | Cyndi Mills | |||
BBN Systems and Technologies | BBN Systems and Technologies | |||
Phone: +1 617 873 4143 | Phone: +1 617 873 4143 | |||
E-mail: cmills@bbn.com | EMail: cmills@bbn.com | |||
Greg Ruth | Greg Ruth | |||
GTE Laboratories, Inc | GTE Laboratories, Inc | |||
Phone: +1 617 466 2448 | ||||
E-mail: gruth@gte.com | Phone: +1 617 466 2448 | |||
EMail: gruth@gte.com | ||||
End of changes. 182 change blocks. | ||||
848 lines changed or deleted | 857 lines changed or added | |||
This html diff was produced by rfcdiff 1.41. The latest version is available from http://tools.ietf.org/tools/rfcdiff/ |