--- 1/draft-ietf-bmwg-dsmterm-02.txt 2006-02-04 22:51:53.000000000 +0100 +++ 2/draft-ietf-bmwg-dsmterm-03.txt 2006-02-04 22:51:53.000000000 +0100 @@ -1,27 +1,27 @@ Network Working Group Jerry Perser INTERNET-DRAFT Spirent - Expires in: May 2002 David Newman + Expires in: December 2002 David Newman Network Test Sumit Khurana Telcordia Shobha Erramilli QNetworx Scott Poretsky Avici Systems - November 2001 + June 2002 Terminology for Benchmarking Network-layer Traffic Control Mechanisms - + Status of this Memo This document is an Internet-Draft and is in full conformance with all provisions of Section 10 of RFC2026. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF), its areas, and its working groups. Note that other groups may also distribute working documents as Internet- Drafts. @@ -41,76 +41,90 @@ Table of Contents 1. Introduction .............................................. 2 2. Existing definitions ...................................... 3 3. Term definitions ............................................3 3.1 Configuration Terms 3.1.1 Classification .........................................3 3.1.2 Codepoint Set ..........................................4 3.1.3 Congestion .............................................4 3.1.4 Congestion Management ..................................5 - 3.2 Vectors ....................................................6 - 3.2.1 Intended Vector ........................................6 + 3.1.5 Flow ...................................................6 + 3.2 Measurement Terms Network-layer Traffic Control Mechanisms - 3.2.2 Offered Vector .........................................6 - 3.2.3 Expected Vectors - 3.2.3.1 Expected Forwarding Vector ...........................7 - 3.2.3.2 Expected Loss Vector .................................8 - 3.2.3.3 Expected Sequence Vector .............................8 - 3.2.3.4 Expected Delay Vector ................................9 - 3.2.3.5 Expected Jitter Vector ..............................10 - 3.2.4 Output Vectors - 3.2.4.1 Forwarding Vector ...................................11 - 3.2.4.2 Loss Vector .........................................11 - 3.2.4.3 Sequence Vector .....................................12 - 3.2.4.4 Delay Vector ........................................13 - 3.2.4.5 Jitter Vector .......................................14 - 3.3 Measurement Terms - 3.3.1 Channel Capacity ......................................15 - 3.3.2 Conforming ............................................15 - 3.3.3 Nonconforming .........................................16 - 3.3.4 Delay .................................................16 - 3.3.5 Flow ..................................................17 - 3.3.6 Stream ................................................18 - 3.3.7 Test Sequence number ..................................19 - 3.3.8 Undifferentiated Response .............................19 - 4. Security Considerations ....................................20 - 5. References .................................................20 - 6. Author's Address ...........................................21 - 7. Full Copyright Statement ...................................22 + 3.2.1 Channel Capacity .......................................7 + 3.2.2 Conforming .............................................7 + 3.2.3 Nonconforming ..........................................8 + 3.2.4 Delay ..................................................8 + 3.2.6 Undifferentiated Response ................................9 + 3.3 Sequence Tracking + 3.3.1 In-sequence Packet .....................................9 + 3.3.2 Out-of-order Packet ...................................10 + 3.3.3 Duplicate Packet ......................................11 + 3.3.4 Stream ................................................11 + 3.3.5 Test Sequence number .................................12 + 3.4 Vectors ...................................................12 + 3.4.1 Intended Vector .......................................12 + 3.4.2 Offered Vector ........................................13 + 3.4.3 Expected Vectors + 3.4.3.1 Expected Forwarding Vector ........................13 + 3.4.3.2 Expected Loss Vector ..............................14 + 3.4.3.3 Expected Sequence Vector ..........................14 + 3.4.3.4 Expected Instantaneous Delay Vector ...............15 + 3.4.3.5 Expected Average Delay Vector .....................16 + 3.4.3.6 Expected Maximum Delay Vector .....................17 + 3.4.3.7 Expected Minimum Delay Vector .....................17 + 3.4.3.8 Expected Instantaneous Delay Variation Vector .....18 + 3.4.3.9 Expected Average Delay Variation Vector ...........19 + 3.4.3.10 Expected Peak-to-peak Delay Variation Vector .....19 + 3.4.4 Output Vectors + 3.4.4.1 Forwarding Vector .................................20 + 3.4.4.2 Loss Vector .......................................20 + 3.4.4.3 Sequence Vector ...................................21 + 3.4.4.4 Instantaneous Delay Vector ........................22 + 3.4.4.5 Average Delay Vector ..............................23 + 3.4.4.6 Maximum Delay Vector ..............................23 + 3.4.4.7 Minimum Delay Vector ..............................24 + 3.4.4.8 Instantaneous Delay Variation Vector ..............25 + 3.4.4.9 Average Delay Variation Vector ....................26 + 3.4.4.10 Peak-to-peak Delay Variation Vector ..............27 + 4. Security Considerations ....................................28 + 5. References .................................................28 + 6. Author's Address ...........................................29 + 7. Full Copyright Statement ...................................30 1. Introduction This document describes terminology for the benchmarking of devices that implement traffic control based on IP precedence or diff-serv code point criteria. New terminology is needed because most existing measurements assume the absence of congestion and only a single per-hop- behavior. This document introduces several new terms that will allow measurements to be taken during periods of congestion. + Network-layer Traffic Control Mechanisms + Another key difference from existing terminology is the definition of measurements as observed on egress as well as ingress of a device/system under test. Again, the existence of congestion requires the addition of egress measurements as well as those taken on ingress; without observing traffic leaving a device/system it is not possible to say whether traffic-control mechanisms effectively dealt with congestion. The principal measurements introduced in this document are vectors for rate, delay, and jitter, all of which can be observed with or without congestion of the DUT/SUT. - Network-layer Traffic Control Mechanisms - This document describes only those terms relevant to measuring behavior of a device or a group of devices using one of these two mechanisms. End-to-end and service-level measurements are beyond the scope of this document. 2. Existing definitions RFC 1242 "Benchmarking Terminology for Network Interconnect Devices" and RFC 2285 "Benchmarking Terminology for LAN Switching Devices" should be consulted before attempting to make use of this @@ -134,52 +148,49 @@ 3. Term definitions 3.1 Configuration Terms 3.1.1 Classification Definition: Selection of packets based on the contents of packet header according to defined rules. + Network-layer Traffic Control Mechanisms + Discussion: Packets can be selected based on the DS field or IP Precedence in the packet header. Classification can also be based on Multi-Field (MF) criteria such as IP Source and destination addresses, protocol and port number. Classification determines the per-hop behaviors and traffic conditioning functions such as shaping and dropping that are to be applied to the packet. Measurement units: - - Network-layer Traffic Control Mechanisms - n/a - Issues: - See Also: 3.1.2 Codepoint Set Definition: The set of all DS Code-points or IP precedence values used during the test duration. Discussion: Describes all the code-point markings associated with packets that are input to the DUT/SUT. For each entry in the codepoint set, there are associated vectors describing the - rate of traffic containing that particular DSCP or IP - precedence value. + rate of traffic, delay, loss, or jitter containing that + particular DSCP or IP precedence value. The treatment that a packet belonging to a particular code- point gets is subject to the DUT classifying packets to map to the correct PHB. Moreover, the forwarding treatment in general is also dependent on the complete set of offered vectors. Measurement Units: n/a @@ -187,31 +198,34 @@ 3.1.3 Congestion Definition: A condition in which one or more egress interfaces are offered more packets than are forwarded at any given instant. Discussion: This condition is a superset of the overload definition [2]. The overload definition assumes the congestion is introduced + Network-layer Traffic Control Mechanisms + strictly by the tester on ingress of a DUT/SUT. That may or may not be the case here. - Another difference is that with multiple-DUT measurements, - congestion may occur at multiple points. For example, - multiple edge devices collectively may congest a core device. - In contrast, overload [1] deals only with overload on + Another difference between congestion and overload occurs + when the SUT comprises multiple elements, in that congestion + may occur at multiple points. Consider an SUT comprising + multiple edge devices exchanging traffic with a single core + device. Depending on traffic patterns, the edge devices may + induce congestion on multiple egress interfaces on the core + device. In contrast, overload [1] deals only with overload on ingress. - Network-layer Traffic Control Mechanisms - Ingress observations alone are not sufficient to cover all cases in which congestion may occur. A device with an infinite amount of memory could buffer an infinite amount of packets, and eventually forward all of them. However, these packets may or may not be forwarded during the test duration. Even though ingress interfaces accept all packets without loss, this hypothetical device may still be congested. The definition presented here explicitly defines congestion as an event observable on egress interfaces. Regardless of @@ -212,343 +226,787 @@ packets may or may not be forwarded during the test duration. Even though ingress interfaces accept all packets without loss, this hypothetical device may still be congested. The definition presented here explicitly defines congestion as an event observable on egress interfaces. Regardless of internal architecture, any device that cannot forward packets on one or more egress interfaces is congested. Measurement units: - n/a - Issues: - See Also: 3.1.4 Congestion Management Definition: An implementation of one or more per-hop-behaviors to avoid or minimize the condition of congestion. Discussion: Congestion management may seek either to control congestion or avoid it altogether. Such mechanisms classify packets based upon IP Precedence or DSCP settings in a packet's IP header. Congestion avoidance mechanisms seek to prevent congestion before it actually occurs. - Congestion control mechanisms gives one or more service - classes preferential treatment over other classes during - periods of congestion. + Congestion control mechanisms gives one or flows (with a + discrete IP Precedence or DSCP value) preferential treatment + over other classes during periods of congestion. Measurement units: + n/a + Network-layer Traffic Control Mechanisms + + See Also: + + 3.1.5 Flow + + Definition: + A flow is a one or more of packets sharing a common intended + pair of source and destination interfaces. + + Discussion: + Packets are grouped by the ingress and egress interfaces they + use on a given DUT/SUT. + + A flow can contain multiple source IP addresses and/or + destination IP addresses. All packets in a flow must enter + on the same ingress interface and exit on the same egress + interface, and have some common network layer content. + Microflows [3] are a subset of flows. As defined in [3], + microflows require application-to-application measurement. In + contrast, flows use lower-layer classification criteria. + Since this document focuses on network-layer classification + criteria, we concentrate here on the use of network-layer + identifiers in describing a flow. Flow identifiers also may + reside at the data-link, transport, or application layers of + the ISO model. However, identifiers other than those at the + network layer are out of scope for this document. + + A flow may contain a single code point/IP precedence value or + may contain multiple values destined for a single egress + interface. This is determined by the test methodology. + + Measurement units: n/a - Issues: + See Also: + Microflow [3] + Streams + Network-layer Traffic Control Mechanisms + + 3.2 Measurement Terms + + 3.2.1 Channel Capacity + + Definition: + The maximum forwarding rate [2] at which none of the offered + packets are dropped by the DUT/SUT. + + Discussion: + Channel capacity measures the packet rate at the egress + interface(s) of the DUT/SUT. In contrast, throughput as + defined in RFC 1242 measures the packet rate is based on the + ingress interface(s) of the DUT/SUT. + + Ingress-based measurements do not account for congestion of + the DUT/SUT. Channel capacity, as an egress measurement, does + take congestion into account. + + Understanding channel capacity is a necessary precursor to + any measurement involving congestion. Throughput numbers can + be higher than channel capacity because of queueing. + + This measurement differs from forwarding rate at maximum + offered load (FRMOL) [2] in that it is intolerant of loss. + + Measurement units: + N-octet packets per second See Also: + Throughput [1] + Forwarding Rate at Maximum Offered Load [2] + + 3.2.2 Conforming + + Definition: + Packets which lie within specific rate, delay, or jitter + bounds. + + Discussion: + A DUT/SUT may be configured to allow a given traffic class to + consume a given amount of bandwidth, or to fall within + predefined delay or jitter boundaries. All packets that lie + within specified bounds are then said to be conforming, + whereas those outside the bounds are nonconforming. + Measurement units: + n/a + + See Also: + Expected Vector Network-layer Traffic Control Mechanisms - 3.2 Vectors + Forwarding Vector + Offered Vector + Nonconforming + + 3.2.3 Nonconforming + + Definition: + Packets that do not lie within specific rate, delay, or + jitter bounds. + + Discussion: + A DUT/SUT may be configured to allow a given traffic class to + consume a given amount of bandwidth, or to fall within + predefined delay or jitter boundaries. All packets that do + not lie within these bounds are then said to be + nonconforming. + + Measurement units: + n/a + + See Also: + Expected Vector + Forwarding Vector + Offered Vector + Conforming + + 3.2.4 Delay + + Definition: + The time interval starting when the last bit of the input IP + packet reaches the input port of the DUT/SUT and ending when + the last bit of the output IP packet is seen on the output + port of the DUT/SUT. + + Discussion: + Delay is measured the same regardless of the type of DUT/SUT. + Latency [1] require some knowledge of whether the DUT/SUT is + a "store and forward" or a "bit forwarding" device. The fact + that a DUT/SUT's technology has a lower delay than another + technology should be visible. + + By specifying the metric to be inside the Internet protocol, + the tester is relieved from specifying the start/end for + every data link layer protocol that IP runs on. This avoids + determining if the start/end delimiters are included in the + frame. Also heterogeneous data link protocol can be used in + a test. + + The measurement point at the end closely simulates the way an + internet datagram is processed. An internet datagram is not + Network-layer Traffic Control Mechanisms + + passed up or down the stack unless it is complete. + Completion occurs once the last bit of the IP packet has been + received. + + Delay can be run at any offered load. Recommend at or below + the channel capacity for non-congested delay. For congested + delay, run the offered load above the channel capacity. + + Measurement units: + Seconds. + + See Also: + Latency [1] + + 3.2.6 Undifferentiated Response + + Definition: + The vector(s) obtained when mechanisms used to support diff- + serv or IP precedence are disabled. + + Discussion: + Enabling diff-serv or IP precedence mechanisms may impose + additional processing overhead for packets. This overhead may + degrade performance even when traffic belonging to only one + class, the best-effort class, is offered to the device. + + Measurements with "undifferentiated response" should be made + to establish a baseline. + + The vector(s) obtained with DSCPs or IP precedence enabled + can be compared to the undifferentiated response to determine + the effect of differentiating traffic. + + Measurement units: + n/a + + 3.3 Sequence Tracking + + 3.3.1 In-sequence Packet + + Definition: + A received packet with the expected Test Sequence number. + + Discussion: + In-sequence is done on a stream level. As packets are + received on a stream, each packet's Test Sequence number is + compared with the previous packet. Only packets that match + the expected are considered in-sequence. + + Network-layer Traffic Control Mechanisms + + Packets that do not match the expected Test Sequence number + are counted as _not in-sequence_ or out-of-sequence. Every + packet that is received is either in-sequence or out-of- + sequence. Subtracting the in-sequence from the received + packets (for that stream) can derive the out-of-sequence + count. + + Two types of events will prevent the in-sequence from + incrementing: packet loss and reordered packets. + + Measurement units: + Packet count + + See Also: + Stream + Test Sequence number + + 3.3.2 Out-of-order Packet + + Definition: + A received packet with a Test Sequence number less that + expected. + + Discussion: + As a stream of packets enter a DUT/SUT, they include a Stream + Test Sequence number indicating the order the packets were + sent to the DUT/SUT. On exiting the DUT/SUT, these packets + may arrive in a different order. Each packet that was re- + ordered is counted as an Out-of-order Packet. + + Certain streaming protocol (such as TCP) require the packets + to be in a certain order. Packets outside this are dropped + by the streaming protocols even though there were properly + received by the IP layer. The type of reordering tolerated + by a streaming protocol varies from protocol to protocol, and + also by implementation. + + Out-of-order Packet count is based on the worst case + streaming protocol. It allows for no reordering. + + Packet loss does not affect the Out-of-order Packet count. + Only packets that were not received in the order that they + were transmitted. + + Measurement units: + Packet count + + See Also: + Stream + Test Sequence number + Network-layer Traffic Control Mechanisms + + 3.3.3 Duplicate Packet + + Definition: + A received packet with a Test Sequence number matching a + previously received packet. + + Discussion: + + Measurement units: + Packet count + + See Also: + Stream + Test Sequence number + + 3.3.4 Stream + + Definition: + A group of packets tracked as a single entity by the traffic + receiver. A stream may share a common content such as type + (IP, UDP), packet size, or payload. + + Discussion: + Streams are tracked by test sequence number or "unique + signature field" (RFC 2889). Streams define how individual + packet's statistics are grouped together to form an + intelligible summary. + + Common stream groupings would be by egress interface, + destination address, source address, DSCP, or IP precedence. + A stream using test sequence numbers can track the ordering + of packets as they transverse the DUT/SUT. + + Streams are not restricted to a pair of source and + destination interfaces as long as all packets are tracked as + a single entity. A mulitcast stream can be forward to + multiple destination interfaces. + + Measurement units: + n/a + + See Also: + Flow + MicroFlow [3] + Test sequence number + Network-layer Traffic Control Mechanisms + + 3.3.6 Test Sequence number + + Definition: + A field in the IP payload portion of the packet that is used + to verify the order of the packets on the egress of the + DUT/SUT. + + Discussion: + The traffic generator sets the test sequence number value and + the traffic receiver checks the value upon receipt of the + packet. The traffic generator changes the value on each + packet transmitted based on an algorithm agreed to by the + traffic receiver. + + The traffic receiver keeps track of the sequence numbers on a + per stream basis. In addition to number of received packets, + the traffic receiver may also report number of in-sequence + packets, number of out-sequence packets, number of duplicate + packets, and number of reordered packets. + + The recommended algorithm to use to change the sequence + number on sequential packets is an incrementing value. + + Measurement units: + n/a + + See Also: + Stream + + 3.4 Vectors A vector is a group of packets all containing a specific DSCP or IP precedence value. Vectors are expressed as a pair of numbers. The first is being the particular diff-serv value. The second is the metric expressed as a rate, loss percentage, delay, or jitter. - 3.2.1 Intended Vector + 3.4.1 Intended Vector Definition: A vector describing the rate at which packets having a specific code-point (or IP precedence) that an external source attempts to transmit to a DUT/SUT. Discussion: Intended loads across the different code-point classes determine the metrics associated with a specific code-point traffic class. + Network-layer Traffic Control Mechanisms + Measurement Units: N-octets packets per second - Issues: - See Also: Offered Vector Expected Forwarding Vector Expected Loss Vector Expected Sequence Vector Expected Delay Vector Expected Jitter Vector Forwarding Vector Loss Vector - 3.2.2 Offered Vector + 3.4.2 Offered Vector Definition: A vector describing the measured rate at which packets having a specific DSCP or IP precedence value are offered to the DUT/SUT. Discussion: Offered loads across the different code-point classes, constituting a code-point set, determine the metrics associated with a specific code-point traffic class. Measurement Units: N-octets packets per second - Network-layer Traffic Control Mechanisms - - Issues: - Packet size. See Also: Expected Forwarding Vector Expected Loss Vector Expected Sequence Vector Expected Delay Vector Expected Jitter Vector Forwarding Vector Codepoint Set - 3.2.3 Expected Vectors + 3.4.3 Expected Vectors - 3.2.3.1 Expected Forwarding Vector + 3.4.3.1 Expected Forwarding Vector Definition: A vector describing the expected output rate of packets having a specific DSCP or IP precedence value. The value is dependent on the set of offered vectors and configuration of the DUT. Discussion: + + Network-layer Traffic Control Mechanisms + The DUT is configured in a certain way in order that service - differentiation occurs for behavior aggregates when a - specific traffic mix consisting of multiple behavior - aggregates is applied. This term attempts to capture the - expected forwarding behavior, for which the device is - configured, when subjected to a certain offered load. + differentiation occurs for a particular DSCP or IP precedence + value when a specific traffic mix consisting of multiple + DSCPs or IP precedence values are applied. This term attempts + to capture the expected forwarding behavior when subjected to + a certain offered vectors. - The actual algorithms or mechanism, that the DUT uses to - achieve service differentiation, is not important in - describing the expected vector. + The actual algorithm or mechanism the DUT uses to achieve + service differentiation is not important in describing the + expected forwarding vector. Measurement units: N-octet packets per second - Issues: - See Also: Intended Vector Offered Vector Output Vectors Expected Loss Vector Expected Sequence Vector Expected Delay Vector Expected Jitter Vector - Network-layer Traffic Control Mechanisms - 3.2.3.2 Expected Loss Vector + 3.4.3.2 Expected Loss Vector Definition: A vector describing the percentage of packets, having a specific DSCP or IP precedence value, that should not be forwarded. The value is dependent on the set of offered vectors and configuration of the DUT. Discussion: The DUT is configured in a certain way in order that service - differentiation occurs for behavior aggregates when a - specific traffic mix consisting of multiple behavior - aggregates is applied. This term attempts to capture the - expected loss behavior, for which the device is configured, - when subjected to a certain offered load. + differentiation occurs for a particular DSCP or IP precedence + value when a specific traffic mix consisting of multiple + DSCPs or IP precedence values are applied. This term attempts + to capture the expected forwarding behavior when subjected to + a certain offered vectors. - The actual algorithms or mechanism, that the DUT uses to - achieve service differentiation, is not important in - describing the expected loss vector. + The actual algorithm or mechanism the DUT uses to achieve + service differentiation is not important in describing the + expected loss vector. Measurement Units: Percentage of intended packets that are expected to be dropped. - Issues: - See Also: Intended Vector Offered Vector Expected Forwarding Vector + Network-layer Traffic Control Mechanisms + Expected Sequence Vector Expected Delay Vector Expected Jitter Vector 3.2.3.3 Expected Sequence Vector Definition: - A vector describing the expected sequencing of packets having - a specific DSCP or IP precedence value. The value is - dependent on the set of offered vectors and configuration of - the DUT. + A vector describing the expected in-sequence packets having a + specific DSCP or IP precedence value. The value is dependent + on the set of offered vectors and configuration of the DUT. Discussion: The DUT is configured in a certain way in order that service - differentiation occurs for behavior aggregates when a - specific traffic mix consisting of multiple behavior - aggregates is applied. This term attempts to capture the - expected sequence behavior, for which the device is - configured, when subjected to a certain offered load. - - Network-layer Traffic Control Mechanisms + differentiation occurs for a particular DSCP or IP precedence + value when a specific traffic mix consisting of multiple + DSCPs or IP precedence values are applied. This term attempts + to capture the expected forwarding behavior when subjected to + a certain offered vectors. - The actual algorithms or mechanism, that the DUT uses to - achieve service differentiation, is not important in - describing the expected vector. + The actual algorithm or mechanism the DUT uses to achieve + service differentiation is not important in describing the + expected sequence vector. Measurement Units: N-octet packets per second - Issues: - See Also: Intended Vector Offered Vector Output Vectors Expected Loss Vector Expected Forwarding Vector Expected Delay Vector Expected Jitter Vector - 3.2.3.4 Expected Delay Vector + 3.4.3.4 Expected Instantaneous Delay Vector Definition: A vector describing the expected delay for packets having a specific DSCP or IP precedence value. The value is dependent on the set of offered vectors and configuration of the DUT. Discussion: The DUT is configured in a certain way in order that service - differentiation occurs for behavior aggregates when a - specific traffic mix consisting of multiple behavior - aggregates is applied. This term attempts to capture the - expected delay behavior, for which the device is configured, - when subjected to a certain offered load. + differentiation occurs for a particular DSCP or IP precedence + value when a specific traffic mix consisting of multiple + DSCPs or IP precedence values are applied. This term attempts + to capture the expected forwarding behavior when subjected to + a certain offered vectors. - The actual algorithms or mechanism, that the DUT uses to - achieve service differentiation, is not important in - describing the expected delay vector. + Network-layer Traffic Control Mechanisms - Measurement units: + The actual algorithm or mechanism the DUT uses to achieve + service differentiation is not important in describing the + expected delay vector. + Measurement units: Seconds. - Issues: + See Also: + Intended Vector + Offered Vector + Output Vectors + Expected Loss Vector + Expected Sequence Vector + Expected Forwarding Vector + Expected Jitter Vector + + 3.4.3.5 Expected Average Delay Vector + + Definition: + A vector describing the expected average delay for packets + having a specific DSCP or IP precedence value. The value is + dependent on the set of offered vectors and configuration of + the DUT. + + Discussion: + The DUT is configured in a certain way in order that service + differentiation occurs for a particular DSCP or IP precedence + value when a specific traffic mix consisting of multiple + DSCPs or IP precedence values are applied. This term attempts + to capture the expected forwarding behavior when subjected to + a certain offered vectors. + + The actual algorithm or mechanism the DUT uses to achieve + service differentiation is not important in describing the + expected average delay vector. + + Measurement units: + Seconds. See Also: Intended Vector Offered Vector Output Vectors Expected Loss Vector Expected Sequence Vector Expected Forwarding Vector Expected Jitter Vector Network-layer Traffic Control Mechanisms - 3.2.3.5 Expected Jitter Vector + 3.4.3.6 Expected Maximum Delay Vector Definition: - A vector describing the expected variation in the delay of - packet arrival times for packets having specific DSCP or IP - precedence value. The value is dependent on the set of - offered vectors and configuration of the DUT. + A vector describing the expected maximum delay for packets + having a specific DSCP or IP precedence value. The value is + dependent on the set of offered vectors and configuration of + the DUT. Discussion: - Jitter is the absolute value of the difference between the - delay measurement of two packets belonging to the same - stream. + The DUT is configured in a certain way in order that service + differentiation occurs for a particular DSCP or IP precedence + value when a specific traffic mix consisting of multiple + DSCPs or IP precedence values are applied. This term attempts + to capture the expected forwarding behavior when subjected to + a certain offered vectors. - The jitter between two consecutive packets in a stream is - reported as the "instantaneous jitter". Instantaneous jitter - can be expressed as |D(i) - D(i-1)| where D equals the delay - and i is the test sequence number. Packets lost are not - counted in the jitter measurement. + The actual algorithm or mechanism the DUT uses to achieve + service differentiation is not important in describing the + expected maximum delay vector. - Average Jitter is the average of the instantaneous jitter - measured during the test duration. + Measurement units: + Seconds. - Peak-to-peak jitter is the maximum delay minus the minimum - delay of the packets forwarded by the DUT/SUT. + See Also: + Intended Vector + Offered Vector + Output Vectors + Expected Loss Vector + Expected Sequence Vector + Expected Forwarding Vector + Expected Jitter Vector - Measurement units: - Seconds (instantaneous) - Seconds P-P (peak to peak) - Seconds Avg (average) + 3.4.3.7 Expected Minimum Delay Vector - Issues: + Definition: + A vector describing the expected minimum delay for packets + having a specific DSCP or IP precedence value. The value is + dependent on the set of offered vectors and configuration of + the DUT. + + Discussion: + The DUT is configured in a certain way in order that service + differentiation occurs for a particular DSCP or IP precedence + value when a specific traffic mix consisting of multiple + DSCPs or IP precedence values are applied. This term attempts + to capture the expected forwarding behavior when subjected to + a certain offered vectors. + + The actual algorithm or mechanism the DUT uses to achieve + service differentiation is not important in describing the + expected minimum delay vector. + + Network-layer Traffic Control Mechanisms + + Measurement units: + Seconds. See Also: Intended Vector Offered Vector Output Vectors Expected Loss Vector Expected Sequence Vector - Expected Delay Vector Expected Forwarding Vector + Expected Jitter Vector - 3.2.4 Output Vectors + 3.2.3.8 Expected Instantaneous Delay Variation Vector + + Definition: + A vector describing the expected variation in the delay of + two consecutive packets' arrival times having a specific DSCP + or IP precedence value. The value is dependent on the set of + offered vectors and configuration of the DUT. + + Discussion: + Instantaneous Delay Variation is the absolute value of the + difference between the delay measurement of two packets + belonging to the same stream. + + The delay fluctuation between two consecutive packets in a + stream is reported as the "Instantaneous Delay Variation". + Instantaneous Delay Variation can be expressed as |D(i) - + D(i-1)| where D equals the delay and i is the test sequence + number. Packets lost are not counted in the measurement. + + Forwarding Vector may contain several Instantaneous Delay + Variation Vectors. For n packets received in a Forwarding + Vector, there is n-1 several Instantaneous Delay Variation + Vectors. + + Measurement units: + Seconds + + See Also: + Delay + Offered Vector + Output Vectors + Expected Average Delay Variation Vector + Expected Peak-to-peak Delay Variation Vector + Stream Network-layer Traffic Control Mechanisms - 3.2.4.1 Forwarding Vector + 3.2.3.9 Expected Average Delay Variation Vector + + Definition: + A vector describing the expected average variation in the + delay of packet arrival times for packets having specific + DSCP or IP precedence value. The value is dependent on the + set of offered vectors and configuration of the DUT. + + Discussion: + Average Delay Variation is the average of all the + Instantaneous Delay Variation Vectors measured during the + test duration. + + Measurement units: + Seconds + + See Also: + Intended Vector + Offered Vector + Output Vectors + Expected Instantaneous Delay Variation Vector + Expected Peak-to-peak Delay Variation Vector + + 3.2.3.10 Expected Peak-to-peak Delay Variation Vector + + Definition: + A vector describing the expected maximum variation in the + delay of packet arrival times for packets having specific + DSCP or IP precedence value. The value is dependent on the + set of offered vectors and configuration of the DUT. + + Discussion: + Peak-to-peak Delay Variation Vector is the maximum delay + minus the minimum delay of the packets (in a vector) + forwarded by the DUT/SUT. + + Peak-to-peak Delay Variation is not derived from the + Instantaneous Delay Variation Vector. Peak-to-peak Delay + Variation is based upon all the packets during the test + duration, not just two consecutive packets. + + Measurement units: + Seconds + + See Also: + Intended Vector + Offered Vector + Output Vectors + Expected Instantaneous Delay Variation Vector + Expected Average Delay Variation Vector + Network-layer Traffic Control Mechanisms + + 3.4.4 Output Vectors + + 3.4.4.1 Forwarding Vector Definition: The number of packets per second for all packets containing a specific DSCP or IP precedence value that a device can be - observed to successfully transmit to the correct destination + observed to successfully forward to the correct destination interface in response to an offered vector. Discussion: Forwarding Vector is expressed as pair of numbers. Both the specific DSCP (or IP precedence) value AND the packets per second value combine to make a vector. The Forwarding Vector represents packet rate based on its specific DSCP (or IP precedence) value. It is not necessarily based on a stream or flow. The Forwarding Vector may be expressed as per port of the DUT/SUT. However, it must be consistent with the Expected Forwarding Vector. Forwarding Vector is a per-hop measurement. The DUT/SUT may change the specific DSCP (or IP precedence) value for a multiple-hop measurement. Measurement units: N-octet packets per second - Issues: - See Also: Intended Vector Offered Vector Expected Vectors Loss Vector Sequence Vector - Delay Vector - Jitter Vector + Delay Vectors - 3.2.4.2 Loss Vector + 3.4.4.2 Loss Vector Definition: The percentage of packets containing specific DSCP or IP precedence value that a DUT/SUT did not transmit to the correct destination interface in response to an offered vector. Discussion: Loss Vector is expressed as pair of numbers. Both the specific DSCP (or IP precedence) value AND the percentage @@ -562,38 +1020,35 @@ port of the DUT/SUT. However, it must be consistent with the Expected Loss Vector Loss Vector is a per-hop measurement. The DUT/SUT may change the specific DSCP or IP precedence value for a multiple-hop measurement. Measurement Units: Percentage of offered packets that are not forwarded. - Issues: - See Also: Intended Vector Offered Vector Expected Vectors Forwarding Vector Sequence Vector - Delay Vector - Jitter Vector + Delay Vectors - 3.2.4.3 Sequence Vector + 3.4.4.3 Sequence Vector Definition: The number of packets per second for all packets containing a specific DSCP or IP precedence value that a device can be - observed to transmit out of sequence to the correct - destination interface in response to an offered vector. + observed to transmit in sequence to the correct destination + interface in response to an offered vector. Discussion: Sequence Vector is expressed as pair of numbers. Both the specific DSCP (or IP precedence) value AND the packets per second value combine to make a vector. The Sequence Vector represents packet rate based on its specific DSCP or IP precedence value. It is not necessarily based on a stream or flow. The Sequence Vector may be expressed as per port of the DUT/SUT. However, it must be @@ -601,403 +1056,324 @@ Sequence Vector is a per-hop measurement. The DUT/SUT may change the specific DSCP or IP precedence value for a multiple-hop measurement. Measurement Units: N-octet packets per second Issues: + See Also: + In-sequence Packet Network-layer Traffic Control Mechanisms - See Also: Intended Vector Offered Vector Expected Vectors Loss Vector Forwarding Vector - Delay Vector - Jitter Vector + Delay Vectors - 3.2.4.4 Delay Vector + 3.4.4.4 Instantaneous Delay Vector Definition: - The delay for packets containing specific DSCP or IP + The delay for a packet containing specific DSCP or IP precedence value that a device can be observed to successfully transmit to the correct destination interface in response to an offered vector. Discussion: - Delay vector is expressed as pair of numbers. Both the - specific DSCP (or IP precedence) value AND delay value - combine to make a vector. + Instantaneous Delay vector is expressed as pair of numbers. + Both the specific DSCP (or IP precedence) value AND delay + value combine to make a vector. - The Delay Vector represents delay on its specific DSCP or IP - precedence value. It is not necessarily based on a stream or - flow. The Delay vector may be expressed as per port of the - DUT/SUT. However, it must be consistent with the Expected - Delay vector. + The Instantaneous Delay Vector represents delay on its + specific DSCP or IP precedence value. It is not necessarily + based on a stream or flow. The Delay vector may be expressed + as per port of the DUT/SUT. However, it must be consistent + with the Expected Delay vectors. - Delay vector is measured similarly regardless of the type of - DUT/SUT. Latency [1] require some knowledge of whether the - DUT/SUT is a "store and forward" or a "bit forwarding" - device. The fact that a DUT/SUT's technology has a lower - delay than another technology should be visible. + Instantaneous Delay Vector is a per-hop measurement. The + DUT/SUT may change the specific DSCP or IP precedence value + for a multiple-hop measurement. - Delay Vector is a per-hop measurement. The DUT/SUT may - change the specific DSCP or IP precedence value for a - multiple-hop measurement. + Instantaneous Delay vector can be obtained at any offered + load. Recommend at or below the channel capacity in the + absence of congestion. For congested delay, run the offered + load above the channel capacity. - Delay vector can be obtained at any offered load. Recommend - at or below the channel capacity in the absence of - congestion. For congested delay, run the offered load above - the channel capacity. + Forwarding Vector may contain several Instantaneous Delay + Vectors. For every packet received in a Forwarding Vector, + there is a corresponding Instantaneous Delay Vector. Measurement Units: - seconds - - Issues: + Seconds See Also: Delay - Network-layer Traffic Control Mechanisms - Intended Vector Offered Vector - Expected Delay Vector - Loss Vector - Forwarding Vector - Jitter Vector + Expected Delay Vectors + Average Delay Vector + Maximum Delay Vector + Minimum Delay Vector + Network-layer Traffic Control Mechanisms - 3.2.4.5 Jitter Vector + 3.4.4.5 Average Delay Vector Definition: - The variation in the delay for packets containing specific - DSCP or IP precedence value that a device can be observed to + The average delay for packets containing specific DSCP or IP + precedence value that a device can be observed to successfully transmit to the correct destination interface in response to an offered vector. Discussion: - Jitter is the absolute value of the difference between the - delay measurement of two packets belonging to the same - stream. - - Jitter vector is expressed as pair of numbers. Both the - specific DSCP (or IP precedence) value AND jitter value + Average Delay vector is expressed as pair of numbers. Both + the specific DSCP (or IP precedence) value AND delay value combine to make a vector. - The jitter between two consecutive packets in a stream is - reported as the "instantaneous jitter". Instantaneous jitter - can be expressed as |D(i) - D(i-1)| where D equals the delay - and i is the test sequence number. Packets lost are not - counted in the jitter measurement. - - Jitter vector is a per-hop measurement. The DUT/SUT may - change the specific DSCP or IP precedence value for a - multiple-hop measurement. + The Delay Vector represents delay on its specific DSCP or IP + precedence value. It is not necessarily based on a stream or + flow. The Delay vector may be expressed as per port of the + DUT/SUT. However, it must be consistent with the Expected + Delay vector. - Average Jitter is the average of the instantaneous jitter - measured during the test duration. + The Average Delay Vector is computed by averaging all the + Instantaneous Delay Vectors for a given vector. - Peak-to-peak Jitter is the maximum delay minus the minimum - delay of the packets forwarded by the DUT/SUT. + Average Delay Vector is a per-hop measurement. The DUT/SUT + may change the specific DSCP or IP precedence value for a + multiple-hop measurement. - Measurement units: - Seconds (instantaneous) - Seconds P-P (peak to peak) - Seconds Avg (average) + Average Delay vector can be obtained at any offered load. + Recommend at or below the channel capacity in the absence of + congestion. For congested delay, run the offered load above + the channel capacity. - Issues: + Measurement Units: + Seconds See Also: + Delay Intended Vector Offered Vector - Expected Vectors - Network-layer Traffic Control Mechanisms - - Loss Vector - Sequence Vector - Delay Vector - Forwarding Vector - - 3.3 Measurement Terms + Expected Delay Vectors + Instantaneous Delay Vector + Maximum Delay Vector + Minimum Delay Vector - 3.3.1 Channel Capacity + 3.4.4.6 Maximum Delay Vector Definition: - The maximum forwarding rate [2] at which none of the offered - packets are dropped by the DUT/SUT. - - Discussion: - Channel capacity measures the packet rate at the egress - interface(s) of the DUT/SUT. In contrast, throughput as - defined in RFC 1242 measures the packet rate is based on the - ingress interface(s) of the DUT/SUT. - - Ingress-based measurements do not account for congestion of - the DUT/SUT. Channel capacity, as an egress measurement, does - take congestion into account. - - Understanding channel capacity is a necessary precursor to - any measurement involving congestion. Throughput numbers can - be higher than channel capacity because of queueing. - - This measurement differs from forwarding rate at maximum - offered load (FRMOL) [2] in that it is intolerant of loss. - - Measurement units: - - N-octet packets per second - - Issues: - - See Also: - Throughput [1] - Forwarding Rate at Maximum Offered Load [2] - - 3.3.2 Conforming + The maximum delay from all packets containing specific DSCP + or IP precedence value that a device can be observed to + successfully transmit to the correct destination interface in + response to an offered vector. - Definition: - Packets which lie within specific rate, delay, or jitter - bounds. + Network-layer Traffic Control Mechanisms Discussion: - A DUT/SUT may be configured to allow a given traffic class to - consume a given amount of bandwidth, or to fall within - Network-layer Traffic Control Mechanisms + Maximum Delay vector is expressed as pair of numbers. Both + the specific DSCP (or IP precedence) value AND delay value + combine to make a vector. - predefined delay or jitter boundaries. All packets that lie - within specified bounds are then said to be conforming, - whereas those outside the bounds are nonconforming. + The Maximum Delay Vector represents delay on its specific + DSCP or IP precedence value. It is not necessarily based on + a stream or flow. The Maximum Delay vector may be expressed + as per port of the DUT/SUT. However, it must be consistent + with the Expected Delay vector. - Measurement units: + Maximum Delay Vector is based upon the maximum Instantaneous + Delay Vector of all packets in a Forwarding Vector. - n/a + Maximum Delay Vector is a per-hop measurement. The DUT/SUT + may change the specific DSCP or IP precedence value for a + multiple-hop measurement. - Issues: + Measurement Units: + Seconds See Also: - Expected Vector - Forwarding Vector + Delay + Intended Vector Offered Vector - Nonconforming - - 3.3.3 Nonconforming - - Definition: - Packets that do not lie within specific rate, delay, or - jitter bounds. - - Discussion: - A DUT/SUT may be configured to allow a given traffic class to - consume a given amount of bandwidth, or to fall within - predefined delay or jitter boundaries. All packets that do - not lie within these bounds are then said to be - nonconforming. - - Measurement units: - - n/a - - Issues: - - See Also: - Expected Vector + Expected Delay Vectors + Instantaneous Delay Vector Forwarding Vector - Offered Vector - Conforming + Average Delay Vector + Minimum Delay Vector - 3.3.4 Delay + 3.4.4.7 Minimum Delay Vector Definition: - The time interval starting when the last bit of the input IP - packets reaches the input port of the DUT/SUT and ending when - the last bit of the output IP packets is seen on the output - port of the DUT/SUT. + The minimum delay from all packets containing specific DSCP + or IP precedence value that a device can be observed to + successfully transmit to the correct destination interface in + response to an offered vector. Discussion: + Delay vector is expressed as pair of numbers. Both the + specific DSCP (or IP precedence) value AND delay value + combine to make a vector. - Network-layer Traffic Control Mechanisms - - Delay is measured the same regardless of the type of DUT/SUT. - Latency [1] require some knowledge of whether the DUT/SUT is - a "store and forward" or a "bit forwarding" device. The fact - that a DUT/SUT's technology has a lower delay than another - technology should be visible. - - By specifying the metric to be inside the Internet protocol, - the tester is relieved from specifying the start/end for - every data link layer protocol that IP runs on. This avoids - determining if the start/end delimiter are included in the - frame. Also heterogeneous data link protocol can be used in - a test. + The Minimum Delay Vector represents delay on its specific + DSCP or IP precedence value. It is not necessarily based on + a stream or flow. The Minimum Delay vector may be expressed + as per port of the DUT/SUT. However, it must be consistent + with the Expected Delay vector. - The measurement point at the end is closely simulates the way - an internet datagram is processed. An internet datagram is - not passed up or down the stack unless it is complete. - Completion occurs once the last bit of the IP packet has been - received. + Network-layer Traffic Control Mechanisms - Delay can be run at any offered load. Recommend at or below - the channel capacity for non-congested delay. For congested - delay, run the offered load above the channel capacity. + Minimum Delay Vector is based upon the minimum Instantaneous + Delay Vector of all packets in a Forwarding Vector. - Measurement units: + Minimum Delay Vector is a per-hop measurement. The DUT/SUT + may change the specific DSCP or IP precedence value for a + multiple-hop measurement. - Seconds. + Minimum Delay vector can be obtained at any offered load. + Recommend at or below the channel capacity in the absence of + congestion. For congested delay, run the offered load above + the channel capacity. - Issues: + Measurement Units: + Seconds See Also: - Latency [1] + Delay + Intended Vector + Offered Vector + Expected Delay Vectors + Instantaneous Delay Vector + Forwarding Vector + Average Delay Vector + Maximum Delay Vector - 3.3.5 Flow + 3.4.4.8 Instantaneous Delay Variation Vector Definition: - A flow is a one or more of packets sharing a common intended - pair of source and destination interfaces. + The variation in the delay for two consecutive packets + containing specific DSCP or IP precedence value that a device + can be observed to successfully transmit to the correct + destination interface in response to an offered vector. Discussion: - Packets are grouped by the ingress and egress interfaces they - use on a given DUT/SUT. + Instantaneous Delay Variation is the absolute value of the + difference between the delay measurement of two packets + belonging to the same stream. - A flow can contain multiple source IP addresses and/or - destination IP addresses. All packets in a flow must enter - on the same ingress interface and exit on the same egress - interface, and have some common network layer content. + Jitter vector is expressed as pair of numbers. Both the + specific DSCP (or IP precedence) value AND jitter value + combine to make a vector. + + The delay fluctuation between two consecutive packets in a + stream is reported as the "Instantaneous Delay Variation". + Instantaneous Delay Variation can be expressed as |D(i) - + D(i-1)| where D equals the delay and i is the test sequence + number. Packets lost are not counted in the measurement. - Microflows [3] are a subset of flows. As defined in [3], - microflows require application-to-application measurement. In - contrast, flows use lower-layer classification criteria. - Since this document focuses on network-layer classification Network-layer Traffic Control Mechanisms - criteria, we concentrate here on the use of network-layer - identifiers in describing a flow. Flow identifiers also may - reside at the data-link, transport, or application layers of - the ISO model. However, identifiers other than those at the - network layer are out of scope for this document. + Instantaneous Delay Variation Vector is a per-hop + measurement. The DUT/SUT may change the specific DSCP or IP + precedence value for a multiple-hop measurement. - A flow may contain a single code point/IP precedence value or - may contain multiple values destined for a single egress - interface. This is determined by the test methodology. + Forwarding Vector may contain several Instantaneous Delay + Variation Vectors. For n packets received in a Forwarding + Vector, there is n-1 several Instantaneous Delay Variation + Vectors. Measurement units: - - n/a - - Issues: + Seconds See Also: - Microflow [3] - Streams + Delay + Forwarding Vector + Stream + Expected Vectors + Average Delay Variation Vector + Peak-to-peak Delay Variation Vector - 3.3.6 Stream + 3.4.4.9 Average Delay Variation Vector Definition: - A group of packets tracked as a single entity by the traffic - receiver. A stream may share a common content such as type - (IP, UDP), packet size, or payload. + The average variation in the delay for packets containing + specific DSCP or IP precedence value that a device can be + observed to successfully transmit to the correct destination + interface in response to an offered vector. Discussion: - Streams are tracked by Test sequence number or "unique - signature field" (RFC 2889). Streams define how individual - packet's statistics are grouped together to form an - intelligible summary. + Average Delay Variation is the average of all the + Instantaneous Delay Variation Vectors measured during the + test duration. - Common stream groupings would be by egress interface, - destination address, source address, DSCP, or IP precedence. - A stream using Test sequence numbers can track the ordering - of packets as they transverse the DUT/SUT. + Average Delay Variation vector is expressed as pair of + numbers. Both the specific DSCP (or IP precedence) value AND + jitter value combine to make a vector. - Streams are not restricted to a pair of source and - destination interfaces as long as all packets are tracked as - a single entity. A mulitcast stream can be forward to - multiple destination interfaces. + Average Delay Variation vector is a per-hop measurement. The + DUT/SUT may change the specific DSCP or IP precedence value + for a multiple-hop measurement. Measurement units: - - n/a - - Issues: - - Network-layer Traffic Control Mechanisms + Seconds See Also: - Flow - MicroFlow [3] - Test sequence number + Delay + Forwarding Vector + Stream + Expected Vectors + Instantaneous Delay Variation Vector + Peak-to-peak Delay Variation Vector + Network-layer Traffic Control Mechanisms - 3.3.7 Test Sequence number + 3.4.4.10 Peak-to-peak Delay Variation Vector Definition: - A field in the IP payload portion of the packet that is used - to verify the order of the packets on the egress of the - DUT/SUT. + The maximum possible variation in the delay for packets + containing specific DSCP or IP precedence value that a device + can be observed to successfully transmit to the correct + destination interface in response to an offered vector. Discussion: - The traffic generator sets the Test sequence number value and - the traffic receiver checks the value upon receipt of the - packet. The traffic generator changes the value on each - packet transmitted based on an algorithm agreed to by the - traffic receiver. + Peak-to-peak Delay Variation Vector is the maximum delay + minus the minimum delay of the packets (in a vector) + forwarded by the DUT/SUT. - The traffic receiver keeps track of the sequence numbers on a - per stream basis. In addition to number of received packets, - the traffic receiver may also report number of in-sequence - packets, number of out-sequence packets, number of duplicate - packets, and number of reordered packets. + Delay Variation vector is expressed as pair of numbers. Both + the specific DSCP (or IP precedence) value AND jitter value + combine to make a vector. - The recommended algorithm to use to change the sequence - number on sequential packets is an incrementing value. + Peak-to-peak Delay Variation is not derived from the + Instantaneous Delay Variation Vector. Peak-to-peak Delay + Variation is based upon all the packets during the test + duration, not just two consecutive packets. Measurement units: - - n/a - - Issues: + Seconds See Also: + Delay + Forwarding Vector Stream - - 3.3.8 Undifferentiated Response - - Definition: - The vector(s) obtained when mechanisms used to support diff- - serv or IP precedence are disabled. - - Discussion: - Enabling diff-serv or IP precedence mechanisms may impose - additional processing overhead for packets. This overhead may - degrade performance even when traffic belonging to only one - class, the best-effort class, is offered to the device. - + Expected Vectors + Average Delay Variation Vector + Peak-to-peak Delay Variation Vector Network-layer Traffic Control Mechanisms - Measurements with "undifferentiated response" should be made - to establish a baseline. - - The vector(s) obtained with DSCPs or IP precedence enabled - can be compared to the undifferentiated response to determine - the effect of differentiating traffic. - - Measurement units: - - n/a - 4. Security Considerations Documents of this type do not directly affect the security of the Internet or of corporate networks as long as benchmarking - is not performed on devices or systems connected to operating - networks. + is not performed on devices or systems connected to + production networks. Packets with unintended and/or unauthorized DSCP or IP precedence values may present security issues. Determining the security consequences of such packets is out of scope for this document. 5. References [1] Bradner, S., Editor, "Benchmarking Terminology for Network Interconnection Devices", RFC 1242, July 1991.