[Docs] [txt|pdf] [Tracker] [Email] [Diff1] [Diff2] [Nits] [IPR]

Versions: 00 01 02 03

Network Working Group                                          A. Charny
Internet-Draft                                       Cisco Systems, Inc.
Intended status: Standards Track                                J. Zhang
Expires: May 21, 2008                    Cisco Systems, Inc. and Cornell
                                                              University
                                                          F. Le Faucheur
                                                              V. Liatsos
                                                     Cisco Systems, Inc.
                                                       November 18, 2007


   Pre-Congestion Notification Using Single Marking for Admission and
                              Termination
                 draft-charny-pcn-single-marking-03.txt

Status of this Memo

   By submitting this Internet-Draft, each author represents that any
   applicable patent or other IPR claims of which he or she is aware
   have been or will be disclosed, and any of which he or she becomes
   aware will be disclosed, in accordance with Section 6 of BCP 79.

   Internet-Drafts are working documents of the Internet Engineering
   Task Force (IETF), its areas, and its working groups.  Note that
   other groups may also distribute working documents as Internet-
   Drafts.

   Internet-Drafts are draft documents valid for a maximum of six months
   and may be updated, replaced, or obsoleted by other documents at any
   time.  It is inappropriate to use Internet-Drafts as reference
   material or to cite them other than as "work in progress."

   The list of current Internet-Drafts can be accessed at
   http://www.ietf.org/ietf/1id-abstracts.txt.

   The list of Internet-Draft Shadow Directories can be accessed at
   http://www.ietf.org/shadow.html.

   This Internet-Draft will expire on May 21, 2008.

Copyright Notice

   Copyright (C) The IETF Trust (2007).

Abstract

   Pre-Congestion Notification described in
   [I-D.eardley-pcn-architecture] and earlier in



Charny, et al.            Expires May 21, 2008                  [Page 1]


Internet-Draft           PCN with Single Marking           November 2007


   [I-D.briscoe-tsvwg-cl-architecture] approach proposes the use of an
   Admission Control mechanism to limit the amount of real-time PCN
   traffic to a configured level during the normal operating conditions,
   and the use of a Flow Termination mechanism to tear-down some of the
   flows to bring the PCN traffic level down to a desirable amount
   during unexpected events such as network failures, with the goal of
   maintaining the QoS assurances to the remaining flows.  In
   [I-D.eardley-pcn-architecture], Admission and Flow Termination use
   two different markings and two different metering mechanisms in the
   internal nodes of the PCN region.  This draft proposes a mechanism
   using a single marking and metering for both Admission and Flow
   Termination, and presents an analysis of the tradeoffs.  A side-
   effect of this proposal is that a different marking and metering
   Admission mechanism than that proposed in
   [I-D.eardley-pcn-architecture] may be also feasible, and may result
   in a number of benefits.  In addition, this draft proposes a
   migration path for incremental deployment of this approach as an
   intermediate step to the dual-marking approach.

Requirements Language

   The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
   "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
   document are to be interpreted as described in RFC 2119 [RFC2119].



























Charny, et al.            Expires May 21, 2008                  [Page 2]


Internet-Draft           PCN with Single Marking           November 2007


Table of Contents

   1.  Introduction . . . . . . . . . . . . . . . . . . . . . . . . .  5
     1.1.  Changes from -02 version . . . . . . . . . . . . . . . . .  5
     1.2.  Terminology  . . . . . . . . . . . . . . . . . . . . . . .  5
     1.3.  Background and Motivation  . . . . . . . . . . . . . . . .  5
   2.  The Single Marking Approach  . . . . . . . . . . . . . . . . .  7
     2.1.  High Level description . . . . . . . . . . . . . . . . . .  7
     2.2.  Operation at the PCN-interior-node . . . . . . . . . . . .  8
     2.3.  Operation at the PCN-egress-node . . . . . . . . . . . . .  8
     2.4.  Operation at the PCN-ingress-node  . . . . . . . . . . . .  8
       2.4.1.  Admission Decision . . . . . . . . . . . . . . . . . .  8
       2.4.2.  Flow Termination Decision  . . . . . . . . . . . . . .  9
   3.  Benefits of Allowing the Single Marking Approach . . . . . . . 10
   4.  Impact on PCN Architectural Framework  . . . . . . . . . . . . 11
     4.1.  Impact on the PCN-Internal-Node  . . . . . . . . . . . . . 11
     4.2.  Impact on the PCN-boundary nodes . . . . . . . . . . . . . 11
       4.2.1.  Impact on PCN-Egress-Node  . . . . . . . . . . . . . . 11
       4.2.2.  Impact on the PCN-Ingress-Node . . . . . . . . . . . . 12
     4.3.  Summary of Proposed Enhancements Required for Support
           of Single Marking Options  . . . . . . . . . . . . . . . . 13
     4.4.  Proposed Optional Renaming of the Marking and Marking
           Thresholds . . . . . . . . . . . . . . . . . . . . . . . . 14
     4.5.  An Optimization Using a Single Configuration Parameter
           for Single Marking . . . . . . . . . . . . . . . . . . . . 15
   5.  Incremental Deployment Considerations  . . . . . . . . . . . . 15
   6.  Tradeoffs, Issues and Limitations of Single Marking
       Approach . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
     6.1.  Global Configuration Requirements  . . . . . . . . . . . . 16
     6.2.  Assumptions on Loss  . . . . . . . . . . . . . . . . . . . 16
     6.3.  Effect of Reaction Timescale of Admission Mechanism  . . . 17
     6.4.  Performance Implications and Tradeoffs . . . . . . . . . . 17
     6.5.  Effect on Proposed Anti-Cheating Mechanisms  . . . . . . . 18
     6.6.  ECMP Handling  . . . . . . . . . . . . . . . . . . . . . . 18
     6.7.  Traffic Engineering Considerations . . . . . . . . . . . . 19
   7.  Performance Evaluation Comparison  . . . . . . . . . . . . . . 22
     7.1.  Relationship to other drafts . . . . . . . . . . . . . . . 22
     7.2.  Admission Control: High Level Conclusions  . . . . . . . . 23
     7.3.  Flow Termination Results . . . . . . . . . . . . . . . . . 24
       7.3.1.  Sensitivity to Low Ingress-Egress aggregation
               levels . . . . . . . . . . . . . . . . . . . . . . . . 24
       7.3.2.  Over-termination in the Multi-bottleneck Scenarios . . 25
     7.4.  Future work  . . . . . . . . . . . . . . . . . . . . . . . 26
   8.  Appendix A:  Simulation Details  . . . . . . . . . . . . . . . 26
     8.1.  Simulation Setup and Environment . . . . . . . . . . . . . 27
       8.1.1.  Network and Signaling Models . . . . . . . . . . . . . 27
       8.1.2.  Traffic Models . . . . . . . . . . . . . . . . . . . . 29
       8.1.3.  Performance Metrics  . . . . . . . . . . . . . . . . . 32



Charny, et al.            Expires May 21, 2008                  [Page 3]


Internet-Draft           PCN with Single Marking           November 2007


     8.2.  Admission Control  . . . . . . . . . . . . . . . . . . . . 33
       8.2.1.  Parameter Settings . . . . . . . . . . . . . . . . . . 33
       8.2.2.  Sensitivity to EWMA weight and CLE . . . . . . . . . . 33
       8.2.3.  Effect of Ingress-Egress Aggregation . . . . . . . . . 36
       8.2.4.  Effect of Multiple Bottlenecks . . . . . . . . . . . . 41
     8.3.  Termination Control  . . . . . . . . . . . . . . . . . . . 45
       8.3.1.  Ingress-Egress Aggregation Experiments . . . . . . . . 45
       8.3.2.  Multiple Bottlenecks Experiments . . . . . . . . . . . 48
   9.  Appendix B. Controlling The Single Marking Configuration
       with a Single Parameter  . . . . . . . . . . . . . . . . . . . 54
     9.1.  Assumption . . . . . . . . . . . . . . . . . . . . . . . . 54
     9.2.  Details of the Proposed Enhancements to PCN
           Architecture . . . . . . . . . . . . . . . . . . . . . . . 54
       9.2.1.  PCN-Internal-Node  . . . . . . . . . . . . . . . . . . 54
       9.2.2.  PCN-Egress-Node  . . . . . . . . . . . . . . . . . . . 55
       9.2.3.  PCN-Ingress-Node . . . . . . . . . . . . . . . . . . . 56
   10. Security Considerations  . . . . . . . . . . . . . . . . . . . 57
   11. References . . . . . . . . . . . . . . . . . . . . . . . . . . 57
     11.1. Normative References . . . . . . . . . . . . . . . . . . . 57
     11.2. Informative References . . . . . . . . . . . . . . . . . . 57
     11.3. References . . . . . . . . . . . . . . . . . . . . . . . . 58
   Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 58
   Intellectual Property and Copyright Statements . . . . . . . . . . 60




























Charny, et al.            Expires May 21, 2008                  [Page 4]


Internet-Draft           PCN with Single Marking           November 2007


1.  Introduction

1.1.  Changes from -02 version

   o  Added Flow Termination results (Section 7 and Section 8.3)

   o  Minor other edits

   o  Alignment with draft-charny-pcn-comparison

1.2.  Terminology

   This draft uses the terminology defined in
   [I-D.eardley-pcn-architecture]

1.3.  Background and Motivation

   Pre-Congestion Notification [I-D.eardley-pcn-architecture] approach
   proposes to use an Admission Control mechanism to limit the amount of
   real-time PCN traffic to a configured level during the normal
   operating conditions, and to use a Flow Termination mechanism to
   tear-down some of the flows to bring the PCN traffic level down to a
   desirable amount during unexpected events such as network failures,
   with the goal of maintaining the QoS assurances to the remaining
   flows.  In [I-D.eardley-pcn-architecture], Admission and Flow
   Termination use two different markings and two different metering
   mechanisms in the internal nodes of the PCN region.  Admission
   Control algorithms for variable-rate real-time traffic such as video
   have traditionally been based on the observation of the queue length,
   and hence re-using these techniques and ideas in the context of pre-
   congestion notification is highly attractive, and motivated the
   threshold- and ramp- marking and metering techniques based on the
   virtual queue implementation described in
   [I-D.briscoe-tsvwg-cl-architecture] for Admission.  On the other
   hand, for Flow Termination, it is desirable to know how many flows
   need to be terminated, and that in turn motivates excess-rate-based
   Flow Termination metering.  This provides some motivation for
   employing different metering algorithm for Admission and for Flow
   Termination.

   Furthermore, it is frequently desirable to trigger Flow Termination
   at a substantially higher traffic level than the level at which no
   new flows are to be admitted.  There are multiple reasons for the
   requirement to enforce a different configured-admissible-rate and
   configured-termination-rate.  These include, for example:

   o  End-users are typically more annoyed by their established call
      dying than by getting a busy tone at call establishment.  Hence



Charny, et al.            Expires May 21, 2008                  [Page 5]


Internet-Draft           PCN with Single Marking           November 2007


      decisions to terminate flows may need to be done at a higher load
      level than the decision to stop admitting.

   o  There are often very tight (possibly legal) obligations on network
      operators to not drop established calls.

   o  Voice Call Routing often has the ability to route/establish the
      call on another network (e.g., PSTN) if it is determined at call
      establishment that one network (e.g., packet network) can not
      accept the call.  Therefore, not admitting a call on the packet
      network at initial establishment may not impact the end-user.  In
      contrast, it is usually not possible to reroute an established
      call onto another network mid-call.  This means that call
      Termination can not be hidden to the end-user.

   o  Flow Termination is typically useful in failure situations where
      some loads get rerouted thereby increasing the load on remaining
      links.  Because the failure may only be temporary, the operator
      may be ready to tolerate a small degradation during the interim
      failure period.  This also argues for a higher configured-
      termination-rate than configured-admissible-rate

   o  A congestion notification based Admission scheme has some inherent
      inaccuracies because of its reactive nature and thus may
      potentially over admit in some situations (such as burst of calls
      arrival).  If the Flow Termination scheme reacted at the same rate
      threshold as the Admission , calls may get routinely dropped after
      establishment because of over admission, even under steady state
      conditions.

   These considerations argue for metering for Admission and Flow
   Termination at different traffic levels and hence, implicitly, for
   different markings and metering schemes.

   Different marking schemes require different codepoints.  Thus, such
   separate markings consume valuable real-estate in the packet header,
   especially scarce in the case of MPLS Pre-Congestion Notification
   [I-D.davie-ecn-mpls] .  Furthermore, two different metering
   techniques involve additional complexity in the data path of the
   internal routers of the PCN-domain.

   To this end, [I-D.briscoe-tsvwg-cl-architecture] proposes an
   approach, referred to as "implicit Preemption marking" in that draft,
   that does not require separate termination-marking.  However, it does
   require two separate measurement schemes: one measurement for
   Admission and another measurement for Flow Termination.  Furthermore,
   this approach mandates that the configured-termination-rate be equal
   to a drop rate.  This approach effectively uses dropping as the way



Charny, et al.            Expires May 21, 2008                  [Page 6]


Internet-Draft           PCN with Single Marking           November 2007


   to convey information about how much traffic can "fit" under the
   configured-termination-rate, instead of using a separate termination
   marking.  This is a significant restriction in that it results in
   flow termination only taking effect once packets actually get
   dropped.

   This document presents an approach that allows the use of a single
   PCN marking and a single metering technique at the internal devices
   without requiring that the dropping and flow termination thresholds
   be the same.  We argue that this approach can be used as intermediate
   step in implementation and deployment of a full-fledged dual-marking
   PCN implementation.  We also quantify performance tradeoffs that are
   associated with the choice of the Single Marking approach.


2.  The Single Marking Approach

2.1.  High Level description

   The proposed approach is based on several simple ideas:

   o  Replace virtual-queue-based threshold- or ramp-marking for
      Admission Control by excess-rate-marking:

      *  meter traffic exceeding the configured-admissible-rate and mark
         *excess* traffic (e.g. using a token bucket with the rate
         configured with the rate equal to configured-admissible-rate)

      *  at the PCN-boundary-node, stop admitting traffic when the
         fraction of marked traffic for a given edge-to-edge aggregate
         exceeds a configured threshold (e.g. stop admitting when 1% of
         all traffic in the edge-to-edge aggregate received at the
         ingress is marked)

   o  Impose a PCN-domain-wide constraint on the ratio U between the
      configured-admissible-rate on a link and level of the PCN load on
      the link at which Flow Termination needs to be triggered (but do
      not explicitly configure configured-termination-rate).  For
      example, one might impose a policy that Flow Termination is
      triggered when PCN traffic exceeds 120% of the configured-
      admissible-rate on any link of the PCN-domain).

   The remaining part of this section describes the possible operation
   of the system.







Charny, et al.            Expires May 21, 2008                  [Page 7]


Internet-Draft           PCN with Single Marking           November 2007


2.2.  Operation at the PCN-interior-node

   The PCN-interior-node meters the aggregate PCN traffic and marks the
   excess rate.  A number of implementations are possible to achieve
   that.  A token bucket implementation is particularly attractive
   because of its relative simplicity, and even more so because a token
   bucket implementation is readily available in the vast majority of
   existing equipment.  The rate of the token bucket is configured to
   correspond to the configured-admissible-rate, and the depth of the
   token bucket can be configured by an operator based on the desired
   tolerance to PCN traffic burstiness.

   Note that no configured-termination-rate is explicitly configured at
   the PCN-interior-node, and the PCN-interior-node does nothing at all
   to enforce it.  All marking is based on the single configured rate
   threshold (configured-admissible-rate).

2.3.  Operation at the PCN-egress-node

   The PCN-egress-node measures the rate of both marked and unmarked
   traffic on a per-ingress basis, and reports to the PCN-ingress-node
   two values: the rate of unmarked traffic from this ingress node,
   which we deem Sustainable Admission Rate (SAR) and the Congestion
   Level Estimate (CLE), which is the fraction of the marked traffic
   received from this ingress node.  Note that Sustainable Admission
   Rate is analogous to the sustainable termination rate of CL, except
   in this case it is based on the configured-admissible- rather than
   termination threshold, while the CLE is exactly the same as that of
   CL.  The details of the rate measurement are outside the scope of
   this draft.

2.4.  Operation at the PCN-ingress-node

2.4.1.  Admission Decision

   Just as in CL, the admission decision is based on the CLE.  The
   ingress node stops admission of new flows if the CLE is above a pre-
   defined threshold (e.g. 1%).  Note that although the logic of the
   decision is exactly the same as in the case of CL, the detailed
   semantics of the marking is different.  This is because the marking
   used for admission in this proposal reflects the excess rate over the
   configured-admissible-rate, while in CL, the marking is based on
   exceeding a virtual queue threshold.  Notably, in the current
   proposal, if the average sustained rate of admitted traffic is 5%
   over the admission threshold, then 5% of the traffic is expected to
   be marked, whereas in the context of CL a steady 5% overload should
   eventually result in 100% of all traffic being admission marked.  A
   consequence of this is that for "smooth" constant-rate traffic, the



Charny, et al.            Expires May 21, 2008                  [Page 8]


Internet-Draft           PCN with Single Marking           November 2007


   approach presented here will not mark any traffic at all until the
   rate of the traffic exceeds the configured admission threshold by the
   amount corresponding to the chosen CLE threshold.

   At first glance this may seem to result in a violation of the pre-
   congestion notification premise that attempts to stop admission
   before the desired traffic level is reached.  However, in reality one
   can simply embed the CLE level into the desired configuration of the
   admission threshold.  That is, if a certain rate X is the actual
   target admission threshold, then one should configure the rate of the
   metering device (e.g. the rate of the token bucket) to X-y where y
   corresponds to the level of CLE that would trigger admission blocking
   decision.

   A more important distinction is that the ramp- version of the
   virtual-queue based marking reacts to short-term burstiness of
   traffic, while the excess-rate based marking is only capable of
   reacting to rate violations at the timescale chosen for rate
   measurement.  Based on our investigation, it seems that this
   distinction is not crucial in the context of PCN when no actual
   queuing is expected even if the virtual queue is full.  More
   discussion on this is presented later in the draft.

2.4.2.  Flow Termination Decision

   When the ingress observes a non-zero CLE and Sustainable Admission
   Rate (SAR), it first computes the Sustainable Termination Rate (STR)
   by simply multiplying SAR by the system-wide constant U where U is
   the system-wide ratio between (implicit) termination and admission
   thresholds on all links in the PCN domain: STR = SAR*U. The PCN-
   ingress-node then performs exactly the same operation as in CL with
   respect to STR: it terminates the appropriate number of flows to
   ensure that the rate of traffic it sends to the corresponding egress
   node does not exceed STR.

   Note: In certain cases where ingress-egress aggregations are not
   sufficient, additional mechanism may be needed to improve the
   accuracy of algorithm.  One possibility is to guard/activate the
   termination control with a trigger computed from EWMA smoothed egress
   measurements (e.g. the termination should be triggered when the ratio
   of smoothed marked and smoothed unmarked traffic is greater than
   U-1).  Sections 7 and 8.3 provide additional discussion on this
   issue.  For sufficient levels of aggregation of IEA traffic, no
   smoothing of the termination trigger is required.

   Just as in the case of CL, an implementation may decide to slow down
   the termination process by preempting fewer flows than is necessary
   to cap its traffic to STR by employing a variety of techniques such



Charny, et al.            Expires May 21, 2008                  [Page 9]


Internet-Draft           PCN with Single Marking           November 2007


   as safety factors or hysteresis.  In summary, the operation of
   Termination at the ingress node is mostly identical to that of CL,
   with the only exception that the sustainable Termination rate is
   computed from the sustainable admission rate rather than derived from
   a separate marking.  As discussed earlier, this is enabled by
   imposing a system-wide restriction on the termination-to-admission
   thresholds ratio and changing the semantics of the admission marking
   from ramp- or threshold - to excess-rate-marking.


3.  Benefits of Allowing the Single Marking Approach

   The following is a summary of benefits associated with enabling the
   Single Marking (SM) approach.  Some tradeoffs will be discussed in
   section 7 below.

   o  Reduced implementation requirements on core routers due to a
      single metering implementation instead of two different ones.

   o  Ease of use on existing hardware: given that the proposed approach
      is particularly amenable to a token bucket implementation, the
      availability of token buckets on virtually all commercially
      available routers makes this approach especially attractive.

   o  Enabling incremental implementation and deployment of PCN (see
      section 4).

   o  Reduced number of codepoints which need to be conveyed in the
      packet header.  If the PCN-bits used in the packets header to
      convey the congestion notification information are the ECN-bits in
      an IP core and the EXP-bits in an MPLS core, those are very
      expensive real-estate.  The current proposals need 5 codepoints,
      which is especially important in the context of MPLS where there
      is only a total of 8 EXP codepoints which must also be shared with
      DiffServ.  Eliminating one codepoint considerably helps.

   o  A possibility of using a token-bucket-based, excess-rate-based
      implementation for admission provides extra flexibility for the
      choice of an admission mechanism, even if two separate markings
      and thresholds are used.

   Subsequent sections argue that these benefits can be achieved with a
   relatively minor enhancements to the proposed PCN architecture as
   defined in [I-D.eardley-pcn-architecture], allow simpler
   implementations at the PCN-interior nodes, and trivial modifications
   at the PCN- boundary nodes.  However, a number of tradeoffs need to
   be also considered, as discussed in section 7.




Charny, et al.            Expires May 21, 2008                 [Page 10]


Internet-Draft           PCN with Single Marking           November 2007


4.  Impact on PCN Architectural Framework

   The goal of this section is to propose several minor changes to the
   PCN architecture framework as currently described in
   [I-D.eardley-pcn-architecture] in order to enable the single marking
   approach.

4.1.  Impact on the PCN-Internal-Node

   No changes are required to the PCN-internal-node in architectural
   framework in [I-D.eardley-pcn-architecture] in order to support the
   Single Marking Proposal.  The current architecture
   [I-D.eardley-pcn-architecture] already allows only one marking and
   metering scheme rather than two by supporting either "admission only"
   or "termination only" functionality.  To support the SM proposal a
   single threshold (i.e.  Configured-termination-rate) must be
   configured at the PCN-internal-node, and excess-rate marking as
   described in should be used to mark packets as described in
   [I-D.briscoe-tsvwg-cl-architecture].

   The configuration parameter(s) at the PCN-ingress-nodes and PCN-
   egress-node (described in section 4.2) will determine how the marking
   should be interpreted by the PCN-boundary-nodes.

4.2.  Impact on the PCN-boundary nodes

   We propose an addition of one global configuration parameter
   MARKING_MODE to be used at all PCN boundary nodes.  If MARKING_MODE =
   DUAL_MARKING, the behavior of the appropriate PCN-boundary-node as
   described in the current version of [I-D.eardley-pcn-architecture].
   If MARKING_MODE = SINGLE_MARKING, the behavior of the appropriate
   boundary nodes is as described in the subsequent subsections.

4.2.1.  Impact on PCN-Egress-Node

   The exact operation of the PCN-Egress-node depends on whether it is
   admission marking (AM-marking) or termination-marking (TM-marking)
   that is used for SM.  An assumption made in
   draft-charny-pcn-comparison-00 is to use AM-marked packets for SM
   instead of TM-marked packets.  In that case the MARKING_MODE will
   signal that Sustainable-Rate must be measured against the AM-marked
   packets, while Congestion-Level-Estimate (CLE) will be measured
   against AM-marked packets just as in the case of CL.  If, however,
   TM-marking is used for SM, then CLE in SM will need to be measured
   against the TM-marked packets.

   In more detail, if the encoding used for SM is that of TM-marking,
   then the setting MARKING_MODE=SINGLE_MARKING indicates that the CLE



Charny, et al.            Expires May 21, 2008                 [Page 11]


Internet-Draft           PCN with Single Marking           November 2007


   is measured against termination-marked packets, while if
   MARKING_MODE=DUAL_MARKING, the CLE is measured against admission-
   marked packets.  The method of measurement of CLE does not depend on
   the choice of the marking against which the measurement is performed.

   If, however, the encoding used for SM is that of AM-marking, then the
   setting MARKING_MODE=SINGLE_MARKING indicates that the Sustainable-
   Rate is measured against AM-marked packets, while the setting of
   MARKING_MODE=DUAL_MARKING indicates that Sustainable-Rate should be
   measured against TM-marked packets

   We note that from the implementation point of view, the same two
   functions (measuring the CLE and measuring the Sustainable-Aggregate-
   Rate are required by both the SM approach and the approach in CL, so
   the difference in the implementation complexity of the PCN-egress-
   node is quite negligible and amounts to checking which encoding is
   used for which function based on the setting of a global parameter.
   If this checking is implemented, then switching the egress nodes from
   supporting SM to supporting CL amounts to changing the setting of the
   global parameter.

4.2.2.  Impact on the PCN-Ingress-Node

   If MARKING_MODE=DUAL_MARKING, the PCN-ingress-node behaves exactly as
   described in [I-D.eardley-pcn-architecture].  If MARKING_MODE =
   SINGLE_MARKING, then an additional global parameter U is defined.  U
   must be configured at all PCN-ingress-nodes and has the meaning of
   the desired ratio between the traffic level at which termination
   should occur and the desired admission threshold, as described in
   section 2.4 above.  The value of U must be greater than or equal to
   1.  The value of this constant U is used to multiply the Sustainable
   Aggregate Rate received from a given PCN-egress-node to compute the
   rate threshold used for flow termination decisions.

   In more detail, if MARKING_MODE=SINGLE_MARKING, then

   o  A PCN-ingress-node receives CLE and/or Sustainable Aggregate Rate
      from each PCN-egress-node it has traffic to.  This is fully
      compatible with PCN architecture as described in
      [I-D.eardley-pcn-architecture].

   o  A PCN-ingress-node bases its admission decisions on the value of
      CLE.  Specifically, once the value of CLE exceeds a configured
      threshold, the PCN-ingress-node stops admitting new flows.  It
      restarts admitting when the CLE value goes down below the
      specified threshold.  This is fully compatible with PCN
      architecture as described in [I-D.eardley-pcn-architecture].




Charny, et al.            Expires May 21, 2008                 [Page 12]


Internet-Draft           PCN with Single Marking           November 2007


   o  A PCN-ingress node receiving a Sustainable Rate from a particular
      PCN-egress node measures its traffic to that egress node.  This
      again is fully compatible with PCN architecture as described in
      draft-earley-pcn-architecture-00.

   o  The PCN-ingress-node computes the desired Termination Rate to a
      particular PCN-egress-node by multiplying the Sustainable
      Aggregate Rate from a given PCN-egress-node by the value of the
      configuration parameter U. This computation step represents a
      proposed change to the current version of
      [I-D.eardley-pcn-architecture].

   o  Once the Termination Rate is computed, it is used for the flow
      termination decision in a manner fully compatible with
      [I-D.eardley-pcn-architecture].  Namely the PCN-ingress-node
      compares the measured traffic rate destined to the given PCN-
      egress-node with the computed Termination rate for that egress
      node, and terminates a set of traffic flows to reduce the rate
      exceeding that Termination rate.  This is fully compatible with
      [I-D.eardley-pcn-architecture].

   We note that as in the case of the PCN-egress-node, the change in the
   implementation of the PCN-ingress-node to support SM is quite
   negligible (a single multiplication per ingress rate measurement
   interval for each egress node).  [Note: If additional smoothing of
   the termination signal is required to deal with low IE aggregation as
   mentioned in section 2.4.2, this smoothing constitutes an additional
   requirement on the PCN-ingress-node.]

4.3.  Summary of Proposed Enhancements Required for Support of Single
      Marking Options

   The enhancements to the PCN architecture as defined in
   [I-D.eardley-pcn-architecture], in summary, amount to:

   o  defining a global (within the PCN domain) configuration parameter
      MARKING_MODE at PCN-boundary nodes

   o  Defining a global (within the PCN domain) configuration parameter
      U at the PCN-ingress-nodes.  This parameter signifies the implicit
      ratio between the termination and admission thresholds at all
      links

   o  Multiplication of Sustainable-Aggregate-Rate by the constant U at
      the PCN-ingress-nodes if MARKING_MODE=SINGLE_MARKING

   o  Using the MARKING_MODE parameter to guide which marking is used to
      measure the CLE (but the measurement functionality is unchanged)



Charny, et al.            Expires May 21, 2008                 [Page 13]


Internet-Draft           PCN with Single Marking           November 2007


4.4.  Proposed Optional Renaming of the Marking and Marking Thresholds

   Previous work on example mechanisms
   [I-D.briscoe-tsvwg-cl-architecture] implementing the architecture of
   [I-D.eardley-pcn-architecture] assumed that the semantics of
   admission control marking and termination marking differ.
   Specifically, it was assumed that for termination purposes the
   semantics of the marking is related to the excess rate over the
   configured (termination) rate, or even more precisely, the amount of
   traffic that remains unmarked (sustainable rate) after the excess
   traffic is marked.  Some of the recent proposals assume yet different
   marking semantics [I-D.babiarz-pcn-3sm],
   [I-D.westberg-pcn-load-control].

   Even though specific association with marking semantics and function
   (admission vs termination) has been assumed in prior work, it is
   important to note that in the current architecture draft
   [I-D.eardley-pcn-architecture], the associations of specific marking
   semantics (virtual queue vs excess rate) with specific functions
   (admission vs termination) are actually *not* directly assumed.  In
   fact , the architecture document does not explicitly define the
   marking mechanism, but rather states the existence of two different
   marking mechanisms, and also allows implementation of either one or
   both of these mechanisms in a PCN- domain.

   We argue that this separation of the marking semantics from the
   functional use of the marking is important to make sure that devices
   supporting the same marking can interoperate in delivering the
   function which is based on specific supported marking semantics.

   To divorce the function (admission vs termination) and the semantics
   (excess rate marking, virtual queue marking), it may be beneficial to
   rename the marking to be associated with the semantics rather than
   the function to explicitly disassociate the two functions.
   Specifically, it may be beneficial to change the "admission-marking"
   and "termination-marking" currently defined in the architecture as
   "Type Q" or "virtual-queue-based" marking, and "Type R" or "excess-
   rate-based" marking.  Of course, other choices of the naming are
   possible (including keeping the ones currently used in
   [I-D.eardley-pcn-architecture]).

   With this renaming, the dual marking approach in
   [I-D.briscoe-tsvwg-cl-architecture] would require PCN-internal-nodes
   to support both Type R and Type Q marking, while SM would require
   support of Type-R marking only.

   We conclude by emphasizing that the changes proposed here amount to
   merely a renaming rather than a change to the proposed architecture,



Charny, et al.            Expires May 21, 2008                 [Page 14]


Internet-Draft           PCN with Single Marking           November 2007


   and are therefore entirely optional.

4.5.  An Optimization Using a Single Configuration Parameter for Single
      Marking

   We note finally that it is possible to use a single configuration
   constant U instead of two constants (U and MARKING_TYPE).
   Specifically, one can simply interpret the value of U=1 as the dual-
   marking approach (equivalent to MARKING_TYPE=DUAL_MARKING) and use
   U>1 to indicate SM.  This is discussed in detail in Section 9.


5.  Incremental Deployment Considerations

   As most of today's routers already implement a token bucket,
   implementing token-bucket based excess-rate marking at PCN-ingress
   nodes is a relatively small incremental step for most of today's
   implementations.  Implementing an additional metering and marking
   scheme in the datapath required by the dual-marking approach without
   encountering performance degradation is a larger step.  The SM
   approach may be used as an intermediate step towards the deployment
   of a dual-marking approach in the sense that routers implementing
   single-marking functionality only may be deployed first and then
   incrementally upgraded to CL.

   The deployment steps might be as follows:

   o  Initially all PCN-ingress-nodes might implement Excess-rate (Type
      R) type marking and metering only

   o  All PCN-boundary nodes implement the full functionality as
      described in this document (including the configuration parameters
      MARKING_TYPE and U) from the start.  Since the PCN-boundary-node
      behavior is enabled by simply changing the values of the
      configuration parameters, all boundary nodes become immediately
      compatible with both dual-marking (CL) and single-marking.

   o  Initially all boundary nodes are configured parameter settings
      indicating SM option.

   o  When a PCN-internal node with dual-marking functionality replaces
      a subset of PCN-internal-nodes, the virtual-queue-based (Type Q)
      marking is simply ignored by the boundary nodes until all PCN-
      internal-nodes in the PCN-domain implement the dual-marking
      metering and marking.  At that time the value of the configuration
      parameters may be reset to at all boundary nodes to indicate the
      Dual Marking configuration.




Charny, et al.            Expires May 21, 2008                 [Page 15]


Internet-Draft           PCN with Single Marking           November 2007


   o  Note that if a subset of PCN-boundary-nodes communicates only with
      each other, and all PCN-internal-nodes their traffic traverses
      have been upgraded, this subset of nodes can be upgraded to two
      dual-marking behavior while the rest of the PCN-domain can still
      run the SM case.  This would entail configuring two thresholds at
      the PCN-internal-nodes, and setting the value of the configuration
      parameters appropriately in this subset.

   o  Finally note that if the configuration parameter U is configured
      per ingress-egress-pair rather than per boundary node, then each
      ingress-egress pair can be upgraded to the dual marking
      simultaneously.  While we do not recommend that U is defined on a
      per-ingress-egress pair, such possibility should be noted and
      considered.


6.  Tradeoffs, Issues and Limitations of Single Marking Approach

6.1.  Global Configuration Requirements

   An obvious restriction necessary for the single-marking approach is
   that the ratio of (implicit) termination and admission thresholds
   remains the same on all links in the PCN region.  While clearly a
   limitation, this does not appear to be particularly crippling, and
   does not appear to outweigh the benefits of reducing the overhead in
   the router implementation and savings in codepoints in the case of a
   single PCN domain, or in the case of multiple concatenated PCN
   regions.  The case when this limitation becomes more inconvenient is
   when an operator wants to merge two previously separate PCN regions
   (which may have different admission-to-termination ratios) into a
   single PCN region.  In this case it becomes necessary to do a
   network-wide reconfiguration to align the settings.

   The fixed ratio between the implicit termination rate and the
   configured-admissible-rate also has an implications on traffic
   engineering considerations.  Those are discussed in section 7.7
   below.

   SM also requires that all PCN-boundary-nodes use the same setting of
   the global parameters U and MARKING_MODE.

6.2.  Assumptions on Loss

   Just as in the case of [I-D.briscoe-tsvwg-cl-architecture], the
   approach presented in this draft assumes that the configured-
   admissible-rate is configured at each link below the service rate of
   the traffic using PCN.  This assumption is significant because the
   algorithm relies on the fact that if admission threshold is exceeded,



Charny, et al.            Expires May 21, 2008                 [Page 16]


Internet-Draft           PCN with Single Marking           November 2007


   enough marked traffic reaches the pcn-egress-node to reach the
   configured CLE level.  If this condition does not hold, then traffic
   may get dropped without ever triggering admission decision.

6.3.  Effect of Reaction Timescale of Admission Mechanism

   As mentioned earlier in this draft, there is a potential concern that
   slower reaction time of admissions mechanism presented in this draft
   compared to [I-D.briscoe-tsvwg-cl-architecture] may result in
   overshoot when the load grows rapidly, and undershoot when the load
   drops rapidly.  While this is a valid concern theoretically, it
   should be noted that at least for the traffic and parameters used in
   the simulation study reported here, there was no indication that this
   was a problem.

6.4.  Performance Implications and Tradeoffs

   Replacement of a relatively well-studied queue-based measurement-
   based admission control approach by a cruder excess-rate measurement
   technique raises a number of algorithmic and performance concerns
   that need to be carefully evaluated.  For example, a token-bucket
   excess rate measurement is expected to be substantially more
   sensitive to traffic burstiness and parameter setting, which may have
   a significant effect in the case of lower levels of traffic
   aggregation, especially for variable-rate traffic such as video.  In
   addition, the appropriate timescale of rate measurement needs to be
   carefully evaluated, and in general it depends on the degree of
   expected traffic variability which is frequently unknown.

   In view of that, an initial performance comparison of the use token-
   bucket based excess-rate metering is presented in the following
   section.  Within the constraints of this study, the performance
   tradeoffs observed between the queue-based technique for admission
   control suggested in [I-D.briscoe-tsvwg-cl-architecture] and a
   simpler token-bucket-based excess rate measurement for admission
   control do not appear to be a cause of substantial concern for cases
   when traffic aggregation is reasonably high at the bottleneck links
   as well as on a per ingress-egress pair basis.  Details of the
   simulation study, as well as additional discussion of its
   implications are presented in section 7.

   Also, one mitigating consideration in favor of the simpler mechanism
   is that in a typical DiffServ environment, the real-time traffic is
   expected to be served at a higher priority and/or the target
   admission rate is expected to be substantially below the speed at
   which the real-time queue is actually served.  If these assumptions
   hold, then there is some margin of safety for an admission control
   algorithm, making the requirements for admission control more



Charny, et al.            Expires May 21, 2008                 [Page 17]


Internet-Draft           PCN with Single Marking           November 2007


   forgiving to bounded errors - see additional discussion in section 7.

   Flow Termination mechanisms of Single Marking and CL are both based
   on excess-rate metering and marking, as so it may be inferred that
   their performance is similar.  However, there is a subtle difference
   between the two mechanisms stemming from the fact that in SM, packets
   continue to be marked when traffic has reduced between the (implicit)
   termination threshold and the (explicit) admission threshold.  This
   "extra" marking may result in over-termination compared to CL,
   especially in multi-bottleneck topologies.  We quantify this over-
   termination in Sections 7 and 8.  While we believe that the extent of
   this over-termination is tolerable for practical purposes, it needs
   to be taken into account when considering performance tradeoffs of
   the two mechanisms.

6.5.  Effect on Proposed Anti-Cheating Mechanisms

   Replacement of the queue-based admission control mechanism of
   [I-D.briscoe-tsvwg-cl-architecture] by an excess-rate based admission
   marking changing the semantics of the pre-congestion marking, and
   consequently interferes with mechanisms for cheating detection
   discussed in [I-D.briscoe-tsvwg-re-ecn-border-cheat].  Implications
   of excess-rate based marking on the anti-cheating mechanisms need to
   be considered.

6.6.  ECMP Handling

   An issue not directly addressed by neither the dual-marking approach
   described in [I-D.briscoe-tsvwg-cl-architecture] nor the single-
   marking approach described in this draft is that if ECMP is enabled
   in the PCN-domain, then the PCN-edge nodes do not have a way of
   knowing whether specific flows in the ingress-egress aggregate (IEA)
   followed the same path or not.  If multiple paths are followed, then
   some of those paths may be experiencing pre-congestion marking, and
   some are not.  Hence, for example, an ingress node may choose to
   terminate a flow which takes an entirely un-congested path.  This
   will not only unnecessarily terminate some flows, but also will not
   eliminate congestion on the actually congested path.  While
   eventually, after several iterations, the correct number of flows
   might be terminated on the congestion path, this is clearly
   suboptimal, as the termination takes longer, and many flows are
   potentially terminated unnecessarily.

   Two approaches for solving this problem were proposed in
   draft-babiarz-pcn-explicit-marking and
   draft-westberg-pcn-load-control.  The former handles ECMP by
   terminating those flows that are termination-marked as soon as the
   termination marking is seen.  The latter uses an additional DiffServ



Charny, et al.            Expires May 21, 2008                 [Page 18]


Internet-Draft           PCN with Single Marking           November 2007


   marking/codepoint to mark all packets of the flows passing through a
   congestion point, with the PCN-boundary-nodes terminating only those
   flows which are marked with this additional marks.  Both of these
   approaches also differ in the termination-marking semantics, but we
   omit the discussion of these differences as they can be considered
   largely independent of the ECMP issue.

   It should be noted that although not proposed in this draft, either
   of these ideas can be used with dual- and single- marking approaches
   discussed here.  Specifically, in CL, when a PCN-ingress-node decides
   which flows to terminate, it can choose for termination only those
   flows that are termination-marked.  Likewise, at the cost of an
   additional (DiffServ) codepoint, a PCN-internal-node can mark all
   packets of all flows using this additional marking, and then the PCN-
   boundary-nodes can use this additional marking to guide their flow
   termination decisions.  In SM, since only one codepoint is used, this
   approach will result in choosing only those flows for termination
   which traverse at least one link where the traffic level is above the
   admission threshold.  This may result in termination of the some
   flows erroneously.

   Either of these approaches appears to imply changes to the PCN
   architecture as proposed in draft-eardley-pcn-architecture-00.  Such
   changes have not been considered in this draft at this point.

6.7.  Traffic Engineering Considerations

   Dual-marking PCN can be viewed as a replacement for Resilient Network
   Provisioning (RNP).  It is reasonable to expect that an operator
   currently using DiffServ provisioning for real-time traffic might
   consider a move to PCN.  For such a move it is necessary to
   understand how to set the PCN rate thresholds to make sure that the
   move to PCN does not detrimentally affect the guarantees currently
   offered to the operator.

   The key question addressed in this section is how to set PCN
   admission and termination thresholds in the dual marking approach or
   the single admission threshold and the scaling factor U reflecting
   the implicit termination threshold in the single-marking approach so
   that the result is "not worse" than provisioning in the amount of
   traffic that can be admitted.  Even more specifically we will address
   what if any are the tradeoffs between the dual-marking and the
   single-approach arise when answering this question.  This question
   was first raised in [Menth] and is further addressed below.

   Typically, RNP would size the network (in this specific case traffic
   that is expected to use PCN) by making sure that capacity available
   for this (PCN) type of traffic is sufficient for PCN traffic under



Charny, et al.            Expires May 21, 2008                 [Page 19]


Internet-Draft           PCN with Single Marking           November 2007


   "normal" circumstances ( that is, under no failure condition, for a
   given traffic matrix), and under a specific set of single failure
   scenarios (e.g. failure of each individual single link).  Some of the
   obvious limitations of such provisioning is that

   o  the traffic matrix is often not known well, and at times,
      especially during flash-crowds, the actual traffic matrix can
      differ substantially from the one assumed by provisioning

   o  unpredicted, non-planned failures can occur (e.g. multiple links,
      nodes, etc), causing overload.

   It is specifically such unplanned cases that serve as the motivation
   for PCN.  Yet, one may want to make sure that for cases that RNP can
   (and does today) plan for, PCN does no worse when an operator makes
   the decision to implement PCN on a currently provisioned network.
   This question directly relates to the choice of the PCN configured
   admission and termination thresholds.

   For the dual-marking approach, where the termination and admission
   thresholds are set independently on any link, one can address this
   issue as follows [Menth].  If a provisioning tool is available, for a
   given traffic matrix, one can determine the utilization of any link
   used by traffic expected to use PCN under the no-failure condition,
   and simply set the configured-admissible-rate to that "no-failure
   utilization".  Then a network using PCN will be able to admit as much
   traffic as the RNP, and will reject any traffic that exceeds the
   expected traffic matrix.  To address resiliency against a set of
   planned failures, one can use RNP to find the worst-case utilization
   of any link under the set of all provisioned failures, and then set
   the configured-termination-rate to that worst case utilization.

   Clearly, such setting of PCN thresholds with the dual-marking
   approach will achieve the following goals:

   o  PCN will admit the same traffic matrix as used by RNP and will
      protect it against all planned failures without terminating any
      traffic

   o  When traffic deviates from the planned traffic matrix, PCN will
      admit such traffic as long as the total usage of any link (without
      failure) does not exceed the configured-admission threshold, and
      all admitted traffic will be protected against all planned
      failures

   o  Additional traffic will not be admitted under the no-failure
      conditions, and traffic exceeding configure-termination threshold
      during non-planned failures will be terminated.



Charny, et al.            Expires May 21, 2008                 [Page 20]


Internet-Draft           PCN with Single Marking           November 2007


   o  Under non-planned failures, some of the planned traffic matrix may
      be terminated, but the remaining traffic will be able to receive
      its QoS treatment.

   The above argues that an operator moving from a purely provisioned
   network to a PCN network can find the settings of the PCN threshold
   with dual marking in such a way that all admitted traffic is
   protected against all planned failures.

   It is easy to see that with the single-marking scheme, the above
   approach does not work directly [Menth].  Indeed, the ratio between
   the configured-termination thresholds and the configured-admissible-
   rate may not be constant on all links.  Since the single-marking
   approach requires the (implicit) termination rate to be within a
   fixed factor of the configured admission rate, it can be argued (as
   was argued in [Menth].) that one needs to set the system-wide ratio U
   between the (implicit) termination threshold and the configured
   admission threshold to correspond to the largest ratio between the
   worst case resilient utilization and the no-failure utilization of
   RNP, and set the admission threshold on each link to the worst case
   resilient utilization divided by that system wide ratio.  Such
   approach would result in lower admission thresholds on some links
   than that of the dual-marking setting of the admission threshold
   proposed above.  It can therefore be argued that PCN with SM will be
   able to admit *less* traffic that can be fully protected under the
   planned set of failures than both RNP and the dual-marking approach.

   However, the settings of the single-marking threshold proposed above
   are not the only one possible, and in fact we propose here that the
   settings are chosen differently.  Such different settings (described
   below) will result in the following properties of the PCN network:

   o  PCN will admit the same traffic matrix as used by RNP *or more*

   o  The traffic matrix assumed by RNP will be fully protected against
      all planned failures without terminating any admitted traffic

   o  When traffic deviates from the planned traffic matrix, PCN will
      admit such traffic as long as the total usage of any link (without
      failure) does not exceed the configured-admission threshold,
      However, not all admitted traffic will be protected against all
      planned failures (i.e. even under planned failures, traffic
      exceeding the planned traffic matrix may be preempted)

   o  Under non-planned failures, some of the planned traffic matrix may
      be terminated, but the remaining traffic will be able to receive
      its QoS treatment.




Charny, et al.            Expires May 21, 2008                 [Page 21]


Internet-Draft           PCN with Single Marking           November 2007


   It is easy to see that all of these properties can be achieved if
   instead of using the largest ratio between worst case resilient
   utilization to the no-failure utilization of RNP across all links for
   setting the system wide constant U in the single-marking approach as
   proposed in [Menth], one uses the *smallest* ratio, and set the
   configured-admissible-rate to the worst case resilient utilization
   divided by that ratio.  With such setting, the configured-admissions
   threshold on each link is at least as large as the non-failure RNP
   utilization (and hence the planned traffic matrix is always
   admitted), and the implicit termination threshold is at the worst
   case planned resilient utilization of RNP on each link (and hence the
   planned traffic matrix will be fully protected against the planned
   failures).  Therefore, with such settings, the single-marking draft
   does as well as RNP or dual-marking with respect to the planned
   matrix and planned failures.  In fact, unlike the dual marking
   approach, it can admit more traffic on some links than the planned
   traffic matrix would allow, but it is only guaranteed to protect up
   to the planned traffic matrix under planned failures.

   In summary, we have argued that both the single-marking approach and
   the dual-marking approach can be configured to ensure that PCN "does
   no worse" than RNP for the planned matrix and the planned failure
   conditions, (and both can do better than RNP under non-planned
   conditions).  The tradeoff between the two is that although the
   planned traffic matrix can be admitted with protection guarantees
   against planned failures with both approaches, the nature of the
   guarantee for the admitted traffic is different.  Dual marking (with
   the settings proposed) would protect all admitted traffic but would
   not admit more than planned), while SM (with the settings proposed)
   will admit more traffic than planned, but will not guarantee
   protection against planned failures for traffic exceeding planned
   utilization.


7.  Performance Evaluation Comparison

7.1.  Relationship to other drafts

   Initial simulation results of admission and termination mechanisms of
   [I-D.briscoe-tsvwg-cl-architecture] were reported in
   [I-D.briscoe-tsvwg-cl-phb].  A follow-up study of these mechanisms is
   presented in a companion draft
   draft-zhang-cl-performance-evaluation-02.txt.  The previous versions
   of this draft concentrated on a performance comparison of the
   virtual-queue-based admission control mechanism of
   [I-D.briscoe-tsvwg-cl-phb] and the token-bucket-based admission
   control described in section 2 of this draft.  In this version, we
   added performance evaluation of the Flow Termination function of SM.



Charny, et al.            Expires May 21, 2008                 [Page 22]


Internet-Draft           PCN with Single Marking           November 2007


   The Flow Termination results are discussed in section 7.3

7.2.  Admission Control: High Level Conclusions

   The results of this study indicate that there is a potential that a
   reasonable complexity/performance tradeoff may be viable for the
   choice of admission control algorithm.  In turn, this suggests that
   using a single codepoint and metering technique for admission and
   termination may be a viable option.

   The key high-level conclusions of the simulation study comparing the
   performance of queue-based and token-based admission control
   algorithms are summarized below:

   1.  At reasonable level of aggregation at the bottleneck and per
       ingress-egress pair traffic, both algorithms perform reasonably
       well for the range of traffic models considered.

   2.  Both schemes are stressed for small levels of ingress-egress pair
       aggregation levels of bursty traffic (e.g. a single video-like
       bursty SVD flow per ingress-egress pair).  However, while the
       queue-based scheme results in tolerable performance even at low
       levels of per ingress-egress aggregation, the token-bucket-based
       scheme is substantially more sensitive to parameter setting than
       the queue-based scheme, and its performance for the high rate
       bursty SVD traffic with low levels of ingress-egress aggregation
       is quite poor unless parameters are chosen carefully to curb the
       error.  It should be noted that the SVD traffic model used in
       this study is expected to be substantially more challenging for
       both admission and termination mechanisms that the actual video
       traffic, as the latter is expected to be much smoother than the
       bursty on-off model with high peak-to-mean ratio we used.  This
       expectation is confirmed by the fact that simulations with actual
       video traces reported in this version of the draft reveal that
       the performance of the video traces is much closer to that of VBR
       voice than of our crude SVD on-off model.

   3.  Even for small per ingress-egress pair aggregation, reasonable
       performance across a range of traffic models can be obtained for
       both algorithms (with a narrower range of parameter setting for
       the token-bucket based approach) .  However, at very low ingress-
       egress aggregation, the token bucket scheme is substantially more
       sensitive to parameter variations than the virtual-queue scheme.
       In general, the token-bucket scheme performance is quite brittle
       at very low aggregations, and displays substantial performance
       degradation with BATCH traffic, as well synchronization effects
       resulting in substantial over-admission (see section 8.4.2)




Charny, et al.            Expires May 21, 2008                 [Page 23]


Internet-Draft           PCN with Single Marking           November 2007


   4.  The absolute value of round-trip time (RTT) or the RTT difference
       between different ingress-egress pair within the range of
       continental propagation delays does not appear to have a visible
       effect on the performance of both algorithms.

   5.  There is no substantial effect on the bottleneck utilization of
       multi-bottleneck topologies for both schemes.  Both schemes
       suffer substantial unfairness (and possibly complete starvation)
       of the long-haul aggregates traversing multiple bottlenecks
       compared to short-haul flows (a property shared by other MBAC
       algorithms as well).  Token-bucket scheme displayed somewhat
       larger unfairness than the virtual-queue scheme.

7.3.  Flow Termination Results

   A consequence of using just a single metering and marking and a
   single marking encoding in SM is that when the traffic level is
   between admission and (implicit) termination threshold, traffic
   continues to be marked in SM (because it exceeds the admission
   threshold at which the metering occurs).  This is in contrast to CL
   when termination marking stops as soon as the traffic falls below the
   termination threshold.  This subtle difference results in a visible
   performance impact on the Termination algorithm of SM, as discussed
   in the next subsections.  Specifically:

   o  SM requires more ingress-egress aggregation than CL (and the
      amount of aggregation needed for the termination function is
      higher than that of admission - see sections 7.3.1 and 8.3).

   o  In the multiple bottleneck scenario, where PCN traffic exceeds the
      configured (admission) rate on multiple links, additional over-
      termination may occur over that already reported for CL (see
      sections 7.3.2 and 8.3 for more detail).

7.3.1.  Sensitivity to Low Ingress-Egress aggregation levels

   In SM, the sustainable termination rate is inferred to from the
   Sustainable (Admission) Rate, by multiplying it by a system-wide
   constant U. In the case of a single bottleneck, a fluid model in
   which marking is uniformly distributed among the contending IEAs, the
   Termination Function of CL and SM would be identical.  However, in
   reality, as shown in draft-zhang-performance-evaluation, excess-rate
   marking does not get distributed among contending IEAs completely
   uniformly, and at low ingress-egress aggregations, some IEAs get
   marked more than others.  As a result, when traffic is close below
   the (implicit) termination threshold at the bottleneck , some IEAs
   get excessively marked, while some get less than their "fair" share
   of marking.  This causes a false termination event at the PCN-



Charny, et al.            Expires May 21, 2008                 [Page 24]


Internet-Draft           PCN with Single Marking           November 2007


   ingress-nodes corresponding to those IEAs which get excessively
   marked, even though the bottleneck load did not exceed the (implicit)
   termination threshold.  This effect is especially pronounced for low
   and medium aggregates of highly bursty traffic.

   We investigated how much aggregation is needed to remove this effect
   completely, and found that the number of flows in the IEAs necessary
   to reduce this error to within 4-10% ranged from about 50 to 150 for
   different traffic types we tested (see section 8.3 in the Appendix
   for more detailed results).

   We also found that for lower aggregation levels, the results could be
   improved to be comparable with CL with respect to over-termination if
   the ingresses used EWMA smoothing to the ratio of marked and unmarked
   traffic when triggering the termination event.  Such smoothing,
   however, would add latency to the termination decision.  The exact
   magnitude of this additional latency depends on the value of the
   global parameter U, the extent of the overload (i.e. excess over
   (implicit) termination threshold), and the exponential weight in the
   EWMA smoothing.  In the range of parameters we investigated that seem
   practically reasonable, the additional latency is bounded by 1-2 sec
   (see detailed results in section 8.3).  Such smoothing is not
   necessary at larger levels of ingress-egress aggregation.

   In conclusion, to avoid over-termination on a single bottleneck due
   to non-uniformity of packet marking distribution among contending
   IEAs, SM needs substantially more ingress-egress aggregation than CL,
   if no additional mechanism are used to smooth the termination
   trigger.

7.3.2.  Over-termination in the Multi-bottleneck Scenarios

   As we showed in draft-zhang-performance-evaluation, when long-haul
   flows traverse more than one bottleneck, each addition bottleneck
   incurs additional termination-marking, which causes long-haul
   terminates more than its fair-share, and the unfairness might in turn
   cause over-termination on the upstream bottleneck.

   SM has a similar issue.  However, in addition, this issue is further
   amplified, as discussed below.  This amplification is due to the fact
   that in SM, metering is done at the lower (admission) threshold, and
   so the quantity of the additional marking received at subsequent
   bottleneck is amplified by factor U (ratio of termination/admission
   threshold).  It in turn reduces the sustainable rate (i.e. the rate
   of unmarked packets), as seen by the PCN-egress-node.  It can be
   shown that this additional marking generally results in SM
   terminating more traffic than CL under the same circumstances, when
   multiple bottlenecks are traversed.  The degree of over-termination



Charny, et al.            Expires May 21, 2008                 [Page 25]


Internet-Draft           PCN with Single Marking           November 2007


   strongly depends on the number of bottlenecks in the topology, and on
   the degree of bottleneck overload above the (implicit) termination
   threshold.

   To understand the significance of this over-termination in practice,
   we randomly generated ~50,000 random traffic matrices on the 5-BTN
   topology (see section 8 for detail), choosing the settings of the
   admission threshold randomly on each link.  We chose U randomly in
   the interval 1.0<U<3.0 for each experiment.  In these experiments,
   the overload on the "tightest" bottleneck ranged from 1-10X, and in
   different experiments the actual number of links where traffic
   exceeded (implicit) termination threshold ranges from 0 to 5.  Table
   7.1 below shows the distribution of over-termination for the subset
   of 19689 experiments with 1.2<U<2.0.  We chose this subset because we
   believe this is a reasonable range for the choice of U in practice.
   We report the full distribution for the entire range of U we
   experimented with in section 8.3.

   (preamble)
    --------------------------------------------------------
   | Alg. |   Distribution of Over-Termination Percentage   |
   |      | 0.0-0.1 | 0.1-0.2 | 0.2-0.3 | 0.3-0.4 | 0.4-0.5 |
   |------|---------|---------|---------|---------|---------|
   |  CL  |  0.985  |  0.013  |  0.000  |  0.000  |  0.000  |
   |------|---------|---------|---------|---------|---------|
   |  SM  |  0.659  |  0.294  |  0.043  |  0.001  |  0.000  |
    --------------------------------------------------------
   (Table 7.1.  Distribution of over-termination percentage in 19689
   experiments with 1.2 <U<2.0. )

   We refer the reader to Section 8.3 of the Appendix for a more
   detailed discussion on this issue.

7.4.  Future work

   This study is but the first step in performance evaluation of the SM
   algorithm.  Further evaluation should include a range of
   investigation, including the following

   o  effect of signaling delays/probing

   o  effect of loss of marked packets


8.  Appendix A:  Simulation Details






Charny, et al.            Expires May 21, 2008                 [Page 26]


Internet-Draft           PCN with Single Marking           November 2007


8.1.  Simulation Setup and Environment

8.1.1.  Network and Signaling Models

   Network topologies used in this study are shown in the Figures below.
   The network is modeled as either Single Link (Fig. A.1), Multi Link
   Network with a single bottleneck (termed "RTT", Fig. A.2), or a range
   of multi-bottleneck topologies shown in Fig. A.3 (termed "Parking
   Lot").


                          A --- B


   Figure A.1: Simulated Single Link Network.


                             A

                                \

                             B  -  D - F

                                /
                             C
   Figure A.2: Simulated Multi Link Network.


         A--B--C     A--B--C--D      A--B--C--D--E--F
         |  |  |     |  |  |  |      |  |  |  |  |  |
         |  |  |     |  |  |  |      |  |  |  |  |  |
         D  E  F     E  F  G  H      G  H  I  J  K  L

           (a)          (b)                (c)
   Figure A.3: Simulated Multiple-bottleneck (Parking Lot )Topologies.

   Figure A.1 shows a single link between an ingress and an egress node,
   all flows enter at node A and depart at node B. This topology is used
   for the basic verification of the behavior of the algorithms with
   respect to a single IEA in isolation.

   In Figure A.2, A set of ingresses (A,B,C) are connected to an
   interior node in the network (D).  This topology is used to study the
   behavior of the algorithm where many IEAs share a single bottleneck
   link.  The number of ingresses varied in different simulation
   experiments in the range of 2-100.  All links have generally
   different propagation delays, in the range 1ms - 100 ms (although in
   some experiments all propagation delays are set the same.  This node



Charny, et al.            Expires May 21, 2008                 [Page 27]


Internet-Draft           PCN with Single Marking           November 2007


   D in turn is connected to the egress (F).  In this topology,
   different sets of flows between each ingress and the egress converge
   on the single link D-F, where pre-congestion notification algorithm
   is enabled.  The capacities of the ingress links are not limiting,
   and hence no PCN is enable on those.  The bottleneck link D-F is
   modeled with a 10ms propagation delay in all simulations.  Therefore
   the range of round-trip delays in the experiments is from 22ms to
   220ms.

   Another type of network of interest is multi-bottleneck (or Parking
   Lot, PLT for short) topology.  The simplest PLT with 2 bottlenecks is
   illustrated in Fig A.3(a).  An example traffic matrix with this
   network on this topology is as follows:

   o  an aggregate of "2-hop" flows entering the network at A and
      leaving at C (via the two links A-B-C)

   o  an aggregate of "1-hop" flows entering the network at D and
      leaving at E (via A-B)

   o  an aggregate of "1-hop" flows entering the network at E and
      leaving at F (via B-C)

   In the 2-hop PLT shown in Fig. A.3(a) the points of congestion are
   links A--B and B--C.  Capacity of all other links is not limiting.
   We also experiment with larger PLT topologies with 3 bottlenecks(see
   Fig A.3(b)) and 5 bottlenecks ( Fig A.3 (c)).  In all cases, we
   simulated one ingress-egress pair that carries the aggregate of
   "long" flows traversing all the N bottlenecks (where N is the number
   of bottleneck links in the PLT topology), and N ingress-egress pairs
   that carry flows traversing a single bottleneck link and exiting at
   the next "hop".  In all cases, only the "horizontal" links in Fig.
   A.3 were the bottlenecks, with capacities of all "vertical" links
   non-limiting.  Propagation delays for all links in all PLT topologies
   are set to 1ms.

   Due to time limitations, other possible traffic matrices (e.g. some
   of the flows traversing a subset of several bottleneck links) have
   not yet been considered and remain the area for future investigation.

   Our simulations concentrated primarily on the range of capacities of
   'bottleneck' links with sufficient aggregation - above 10 Mbps for
   voice and 622 Mbps for SVD, up to 2.4 Gbps.  But we also investigated
   slower 'bottleneck' links down to 512 Kbps in some experiments.
   Higher rate bottleneck speeds wee not considered due to the
   simulation time limitations.  It should generally be expected that
   the higher link speeds will result in higher levels of aggregation,
   and hence generally better performance of the measurement-based



Charny, et al.            Expires May 21, 2008                 [Page 28]


Internet-Draft           PCN with Single Marking           November 2007


   algorithms.  Therefore is seems reasonable to believe that the link
   speeds studied do provide meaningful evaluation targets.

   In the simulation model, a call requests arrives at the ingress and
   immediately sends a message to the egress.  The message arrives at
   the egress after the propagation time plus link processing time (but
   no queuing delay).  When the egress receives this message, it
   immediately responds to the ingress with the current Congestion-
   Level-Estimate.  If the Congestion-Level-Estimate is below the
   specified CLE-threshold, the call is admitted, otherwise it is
   rejected.  An admitted call sends packets according to one of the
   chosen traffic models for the duration of the call (see next
   section).  Propagation delay from source to the ingress and from
   destination to the egress is assumed negligible and is not modeled.

   In the simulation model of admission control, a call request arrives
   at the ingress and immediately sends a message to the egress.  The
   message arrives at the egress after the propagation time plus link
   processing time (but no queuing delay).  When the egress receives
   this message, it immediately responds to the ingress with the current
   Congestion Level Estimate.  If the Congestion Level Estimate is below
   the specified CLE- threshold, the call is admitted, otherwise it is
   rejected.  For Flow Termination, once the ingress node of a PCN-
   domain decides to terminate a flow, that flow is preempted
   immediately and sends no more packets from that time on.  The life of
   a flow outside the domain described above is not modeled.
   Propagation delay from source to the ingress and from destination to
   the egress is assumed negligible and is not modeled.

8.1.2.  Traffic Models

   Four types of traffic were simulated (CBR voice, on-off traffic
   approximating voice with silence compression, and on-off traffic with
   higher peak and mean rates (we termed the latter "Synthetic Video"
   (SVD) as the chosen peak and mean rate was similar to that of an MPEG
   video stream. (but for SVD no attempt was made to match any other
   parameters of this traffic to those of a video stream), and finally
   real video traces from
   http://www.tkn.tu-berlin.de/research/trace/trace.html (courtesy
   Telecommunication Networks Group of Technical University of Berlin).

   The distribution of flow duration was chosen to be exponentially
   distributed with mean 1min, regardless of the traffic type.  In most
   of the experiments flows arrived according to a Poisson distribution
   with mean arrival rate chosen to achieve a desired amount of overload
   over the configured-admissible-rate in each experiment.  Overloads in
   the range 1x to 5x and underload with 0.95x have been investigated.
   Note that the rationale for looking at the load 1 and below is to see



Charny, et al.            Expires May 21, 2008                 [Page 29]


Internet-Draft           PCN with Single Marking           November 2007


   if any significant amount of "false rejects" would be seen (i.e. one
   would assume that all traffic should be accepted if the total demand
   is below the admission threshold).  For on-off traffic, on and off
   periods were exponentially distributed with the specified mean.
   Traffic parameters for each type are summarized below:

8.1.2.1.  Voice Traffic Models

   Table A.1 below describes all voice codecs we modeled in our
   simulation results.

   The first two rows correspond to our two basic models corresponding
   to the older G.711 encoding with and without silence compression.
   These two models are referred simply as "CBR" and "VBR" in the
   reported simulation results.

   We also simulated several "mixes" of the different codecs reported in
   the table below.  The primary mix consists of equal proportion of all
   voice codecs listed below.  We have also simulated various other mix
   consist different proportion of the subset of all codecs.  Though
   these result are not reported in this draft due to their similarities
   to the primary mix result.

 --------------------------------------------------------------------------
| Name/Codecs | Packet Size | Inter-Arrival | On/Off Period | Average Rate |
|             |   (Bytes)   |   Time (ms)   |      Ratio    |    (kbps)    |
 --------------------------------------------------------------------------
|  "CBR"      |     160     |      20       |      1        |      64      |
 --------------------------------------------------------------------------
|  "VBR"      |     160     |      20       |     0.34      |     21.75    |
 --------------------------------------------------------------------------
|  G.711 CBR  |     200     |      20       |      1        |      80      |
 --------------------------------------------------------------------------
|  G.711 VBR  |     200     |      20       |     0.4       |      32      |
 --------------------------------------------------------------------------
|  G.711 CBR  |     120     |      10       |      1        |      96      |
 --------------------------------------------------------------------------
|  G.711 VBR  |     120     |      10       |     0.4       |     38.4     |
 --------------------------------------------------------------------------
|  G.729 CBR  |     60      |      20       |      1        |      24      |
 --------------------------------------------------------------------------
|  G.729 VBR  |     60      |      20       |     0.4       |      9.6     |
 --------------------------------------------------------------------------
   Table A.1 Simulated Voice Codices.







Charny, et al.            Expires May 21, 2008                 [Page 30]


Internet-Draft           PCN with Single Marking           November 2007


8.1.2.2.  "Synthetic Video":  High Rate ON-OFF traffic with Video-like
          Mean and Peak Rates ("SVD")

   This model is on-off traffic with video-like mean-to-peak ratio and
   mean rate approximating that of an MPEG-2 video stream.  No attempt
   is made to simulate any other aspects of a real video stream, and
   this model is merely that of on-off traffic.  Although there is no
   claim that this model represents the performance of video traffic
   under the algorithms in question adequately, intuitively, this model
   should be more challenging for a measurement-based algorithm than the
   actual MPEG video, and as a result, 'good' or "reasonable"
   performance on this traffic model indicates that MPEG traffic should
   perform at least as well.  We term this type of traffic SVD for
   "Synthetic Video".

   o  Long term average rate 4 Mbps

   o  On Period mean duration 340ms; during the on-period the packets
      are sent at 12 Mbps (1500 byte packets, packet inter-arrival: 1ms)

   o  Off Period mean duration 660m

8.1.2.3.  Real Video Traces (VTR)

   We used a publicly available library of frame size traces of long
   MPEG-4 and H.263 encoded video obtained from
   http://www.tkn.tu-berlin.de/research/trace/trace.html.  Each trace in
   that repository s roughly 60 minutes in length, consisting of a list
   of records in the format of <FrameArrivalTime, FrameSize>.  Among the
   160 available traces, we picked the two with the highest average rate
   (averaged over the trace length, in this case, 60 minutes.  In
   addition, the two also have a similar average rate).  The trace file
   used in the simulation is the concatenation of the two.

   Since the duration of the flow in our simulation is much smaller than
   the length of the trace, we checked whether the expected rate of flow
   corresponds to the trace's long term average.  To do so, we simulated
   a number of flows starting from random locations in the trace with
   duration chosen to be exponentially distributed with the mean of
   1min.  The results show that the expected rate of flow is roughly the
   same as the trace's average.

   In summary, our simulations use a set of segments of the 120 min
   trace chosen at random offset from the beginning and with mean
   duration of 1 min.

   Since the traces provide only the frame size, we also simulated
   packetization of the frame as a CBR segment with packet size and



Charny, et al.            Expires May 21, 2008                 [Page 31]


Internet-Draft           PCN with Single Marking           November 2007


   inter-arrival time corresponding to those of our SVD model.  Since
   the frame size is not always a multiple of the chosen packet size,
   the last packet in a frame may be shorter than 1500 bytes chosen for
   the SVD encoding.

   Traffic characteristics for our VTR models are summarized below:

   o  Average rate 769 Kbps

   o  Each frame is sent with packet length 1500 bytes and packet inter-
      arrival time 1ms

   o  No traffic is sent between frames.

8.1.2.4.  Randomization of Base Traffic Models

   To emulate some degree of network-introduced jitter, in some
   experiments we implemented limited randomization of the base models
   by randomly moving the packet by a small amount of time around its
   transmission time in the corresponding base traffic model.  More
   specifically, for each packet we chose a random number R, which is
   picked from uniform distribution in a "randomization-interval", and
   delayed the packet by R compared to its ideal departure time.  We
   choose randomization-interval to be a fraction of packet-
   interarrival-time of the CBR portion of the corresponding base model.
   To simulate a range of queuing delays, we varied this fraction from
   0.0001 to 0.1.  While we do not claim this to be an adequate model
   for network-introduced jitter, we chose it for the simplicity of
   implementation as a means to gain insight on any simulation artifacts
   of strictly CBR traffic generation.  We implemented randomized
   versions of all 5 traffic streams (CBR, VBR, MIX, SVD and VTR) by
   randomizing the CBR portion of each model

8.1.3.  Performance Metrics

   In all our experiments we use as performance metric the percent
   deviation of the mean rate achieved in the experiment from the
   expected load level.  We term these "over-admission" and "over-
   termination" percentages, depending on the type of the experiment.

   More specifically, our experiments measure the actual achieved
   throughput at 50 ms intervals, and then compute the average of these
   50ms rate samples over the duration of the experiment (where
   relevant, excluding warmup/startup conditions).  We then compare this
   experiment average to the desired traffic load.

   Initially in our experiments we also computed the variance of the
   traffic around the mean, and found that in the vast majority of the



Charny, et al.            Expires May 21, 2008                 [Page 32]


Internet-Draft           PCN with Single Marking           November 2007


   experiments it was quite small.  Therefore, in this draft we omit the
   variance and limit the reporting to the over-admission and over-
   termination percentages only.

8.2.  Admission Control

8.2.1.  Parameter Settings

8.2.1.1.  Queue-based settings

   All the queue-based simulations were run with the following Virtual
   Queue thresholds:

   o  virtual-queue-rate: configured-admissible-rate, 1/2 link speed

   o  min-marking-threshold: 5ms at virtual-queue-rate

   o  max-marking-threshold: 15ms at virtual-queue-rate

   o  virtual-queue-upper-limit: 20ms at virtual-queue-rate

   At the egress, the CLE is computed as an exponential weighted moving
   average (EWMA) on an interval basis, with 100ms measurement interval
   chosen in all simulations.  We simulated the EWMA weight ranging 0.1
   to 0.9.  The CLE threshold is chosen to be 0.05, 0.15, 0.25, and 0.5.

8.2.1.2.  Token Bucket Settings

   The token bucket rate is set to the configured-admissible-rate, which
   is half of the link speed in all experiments.  Token bucket depth
   ranges from 64 to 512 packets.  Our simulation results indicate that
   depth of token bucket has no significant impact on the performance of
   the algorithms and hence, in the rest of the section, we only present
   the result with 256 packets bucket depth.

   The CLE is calculated using EWMA just as in the case of virtual-queue
   settings, with weights from 0.1 to 0.9.  The CLE thresholds are
   chosen to be 0.0001, 0.001, 0.01, 0.05 in this case.  Note that the
   since meaning of the CLE is different for the Token bucket and queue-
   based algorithms, so there is no direct correspondence between the
   choice of the CLE thresholds in the two cases.

8.2.2.  Sensitivity to EWMA weight and CLE

   Table A.2 summarized the comparison result of over-admission-
   percentage values from 15 experiments with different [weight, CLE
   threshold] settings for each type of traffic and each topology.  The
   Ratio of the demand on the bottleneck link to the configured



Charny, et al.            Expires May 21, 2008                 [Page 33]


Internet-Draft           PCN with Single Marking           November 2007


   admission threshold is set to 5x.  (In the results for 0.95x can be
   found in previous draft).  For parking lot topologies we report the
   worst case result across all bottlenecks.  We present here only the
   extreme value over the range of resulting over-admission-percentage
   values.

   We found that the virtual-queue admission control algorithm works
   reliably with the range of parameters we simulated, for all five
   types of traffic.  In addition, except for SVD, the performance is
   insensitive to the parameters change under all tested topologies.
   For SVD, the algorithms does show certain sensitivity to the tested
   parameters.  The high level conclusion that can be drawn is that
   (predictably) high peak-to-mean ratio SVD traffic is substantially
   more stressful to the queue-based admission control algorithm, but a
   set of parameters exists that keeps the over-admission within about
   -4% - +7% of the expected load even for the bursty SVD traffic.

   The token bucket-based admission control algorithm shows higher
   sensitivity to the parameter settings compared to the virtual queue
   based algorithm.  It is important to note here that for the token
   bucket-based admission control no traffic will be marked until the
   rate of traffic exceeds the configured admission rate by the chosen
   CLE.  As a consequence, even with the ideal performance of the
   algorithms, the over-admission-percentage will not be 0, rather it is
   expected to equal to CLE threshold if the algorithm performs as
   expected.  Therefore, a more meaningful metric for the token-based
   results is actually the over-admission-percentage (listed below)
   minus the corresponding (CLE threshold * 100).  For example, for CLE
   = 0.01, one would expect that 1% over-admission is inherently
   embedded in the algorithm.  When comparing the performance of token
   bucket (with the adjusted over-admission-percentage) to its
   corresponding virtual queue result, we found that token bucket
   performs only slightly worse for voice-like CBR VBR, and MIX traffic.

   The results for SVD traffic require some additional commentary.  Note
   from the results in Table A.2. in the Single Link topology the
   performance of the token-based solution is comparable to the
   performance of the queue-based scheme.  However, for the RTT
   topology, the worse case performance for SVD traffic becomes very
   bad, with up to 23% over-admission in a high overload.  We
   investigated two potential causes of this drastic degradation of
   performance by concentrating on two key differences between the
   Single Link and the RTT topologies: the difference in the round-trip
   times and the degree of aggregation in a per ingress-egress pair
   aggregate.

   To investigate the effect of the difference in round-trip times, we
   also conducted a subset of the experiments described above using the



Charny, et al.            Expires May 21, 2008                 [Page 34]


Internet-Draft           PCN with Single Marking           November 2007


   RTT topology that has the same RTT across all ingress-egress pairs
   rather than the range of RTTs in one experiment.  We found out that
   neither the absolute nor the relative difference in RTT between
   different ingress-egress pairs appear to have any visible effect on
   the over-load performance or the fairness of both algorithms (we do
   not present these results here as their are essentially identical to
   those in Table A.2).  In view of that and noting that in the RTT
   topology we used for these experiments for the SVD traffic, there is
   only 1 highly bursty flow per ingress, we believe that the severe
   degradation of performance in this topology is directly attributable
   to the lack of traffic aggregation on the ingress-egress pair basis.

   (preamble)
    -------------------------------------------------
   | Type |  Topo  |    Over Admission Perc Stats    |
   |      |        |  Queue-based   |  Bucket-Based  |
   |      |        |  Min     Max   |  Min     Max   |
   |------|--------|---------------------------------|
   |      | S.Link | 0.224   1.105  | -0.99   1.373  |
   | CBR  |   RTT  | 0.200   1.192  | 6.495   9.403  |
   |      |   PLT  | -0.93   0.990  | -2.24   2.215  |
   |-------------------------------------------------|
   |      | S.Link | -0.07   1.646  | -2.94   2.760  |
   | VBR  |   RTT  | -0.11   1.830  | -1.92   6.384  |
   |      |   PLT  | -1.48   1.644  | -4.34   3.707  |
   |-------------------------------------------------|
   |      | S.Link | -0.14   1.961  | -2.85   2.153  |
   | MIX  |   RTT  | -0.46   1.803  | -3.18   2.445  |
   |      |   PLT  | -1.62   1.031  | -3.69   2.955  |
   |-------------------------------------------------|
   |      | S.Link | -0.05   1.581  | -2.36   2.247  |
   | VTR  |   RTT  | -0.57   1.313  | -1.44   4.947  |
   |      |   PLT  | -1.24   1.071  | -3.05   2.828  |
   |-------------------------------------------------|
   |      | S.Link | -2.73   6.525  | -11.25  6.227  |
   | SVD  |   RTT  | -2.98   5.357  | -4.30   23.48  |
   |      |   PLT  | -4.84   4.294  | -11.40  6.126  |
    -------------------------------------------------
   Table A.2 Parameter sensitivity: Queue-based v.s.  Token Bucket-
   based.  For the single bottleneck topologies (S. Link and RTT) the
   overload column represents the ratio of the mean demand on the
   bottleneck link to the configured admission threshold.  For parking
   lot topologies we report the worst case result across all
   bottlenecks.  We present here only the worst case value over the
   range of resulting over-admission-percentage values.






Charny, et al.            Expires May 21, 2008                 [Page 35]


Internet-Draft           PCN with Single Marking           November 2007


8.2.3.  Effect of Ingress-Egress Aggregation

   To investigate the effect of Ingress-Egress Aggregation, we fix a
   particular EWMA weight and CLE setting (in this case, weight=0.3, for
   virtual queue scheme CLE=0.05, and for the token bucket scheme
   CLE=0.0001), vary the level of ingress-egress aggregation by using
   RTT topologies with different number of ingresses.

   Table A.3 shows the change of over-admission-percentage with respect
   to the increase in the number of ingress for both virtual queue and
   token bucket.  For all traffic, the leftmost column in the represents
   the case with the largest aggregation (only two ingresses), while the
   right most column represents the lowest level of aggregation
   (expected number calls per ingress is just 1 in this case).  In all
   experiments the aggregate load on the bottleneck is the same across
   each traffic type (with the aggregate load being evenly divided
   between all ingresses).

   As seen from Table A.3. the virtual queue based approach is
   relatively insensitive to the level of ingress-egress aggregation.
   On the other hand, the Token Bucket based approach is performing
   significantly worse at lower levels of ingress-egress aggregation.
   For example for CBR (with expect 1-call per ingress), the over-
   admission-percentage can be as bad as 45%.



























Charny, et al.            Expires May 21, 2008                 [Page 36]


Internet-Draft           PCN with Single Marking           November 2007


   (preamble)
   --------------------------------------------------------------------
  |       | Type |                  Number of Ingresses                |
  |       |------|---------------------------------------------------- |
  |       |      |   2    |   10   |   70   |  300   |  600   |  1000  |
  |       | CBR  | 1.003  | 1.024  | 0.976  | 0.354  | -1.45  | 0.396  |
  |       |------------------------------------------------------------|
  |       |      |   2    |   10   |   70   |  300   |  600   |  1800  |
  |       | VBR  | 1.021  | 1.117  | 1.006  | 0.979  | 0.721  | -0.85  |
  |       |------------------------------------------------------------|
  |Virtual|      |   2    |   10   |   70   |  300   |  600   |  1000  |
  | Queue | MIX  | 1.080  | 1.163  | 1.105  | 1.042  | 1.132  | 1.098  |
  | Based |------------------------------------------------------------|
  |       |      |   2    |   10   |   70   |  140   |  300   |  600   |
  |       | VTR  | 1.109  | 1.053  | 0.842  | 0.859  | 0.856  | 0.862  |
  |       |------------------------------------------------------------|
  |       |      |   2    |   10   |   35   |   70   |  140   |  300   |
  |       | SVD  | -0.08  | 0.009  | -0.11  | -0.286 | -1.56  | 0.914  |
   --------------------------------------------------------------------
   --------------------------------------------------------------------
  |       | Type |                  Number of Ingresses                |
  |       |------|---------------------------------------------------- |
  |       |      |   2    |   10   |  100   |  300   |  600   |  1000  |
  |       | CBR  | 0.725  | 0.753  | 7.666  | 21.16  | 33.69  | 44.58  |
  |       |------------------------------------------------------------|
  |       |      |   2    |   10   |  100   |  300   |  600   |  1800  |
  |       | VBR  | 0.532  | 0.477  | 1.409  | 3.044  | 5.812  | 14.80  |
  |Token  |------------------------------------------------------------|
  |Bucket |      |   2    |   10   |  100   |  300   |  600   |  1800  |
  |Based  | MIX  | 0.736  | 0.649  | 1.960  | 4.652  | 10.31  | 27.69  |
  |       |------------------------------------------------------------|
  |       |      |   2    |   10   |   70   |  140   |  300   |  600   |
  |       | VTR  | 0.758  | 0.889  | 1.335  | 1.694  | 4.128  | 13.28  |
  |       |------------------------------------------------------------|
  |       |      |   2    |   10   |   35   |  100   |  140   |  300   |
  |       | SVD  | -1.64  | -0.93  | 0.237  | 4.732  | 7.103  | 8.799  |
   --------------------------------------------------------------------
   (Table A.3 Synchronization effect with low Ingress-Egress
   Aggregation: Queue-based v.s.  Token bucket-based)

   Our investigation reveals that the cause of the poor performance of
   the token bucket scheme in our experiments is attributed directly to
   the same "synchronization" effect as was earlier described in the
   Termination (preemption) results in
   draft-zhang-pcn-performance-evaluation, and to which we refer the
   reader for a more detailed description of this effect.  In short
   however, for CBR traffic, a periodic pattern arises where packets of
   a given flow see roughly the same state of the token bucket at the



Charny, et al.            Expires May 21, 2008                 [Page 37]


Internet-Draft           PCN with Single Marking           November 2007


   bottleneck, and hence either all get marked, or all do not get
   marked.  As a result, at low levels of aggregation a subset of
   ingresses always get their packets marked, while some other ingresses
   do not.

   As reported in draft-zhang-pcn-performance-evaluation, in the case of
   Termination this synchronization effect is beneficial to the
   algorithm.  In contrast, for Admission, this synchronization is
   detrimental to the algorithm performance at low aggregations.  This
   can be easily explained by noting that ingresses which packets do not
   get marked continue admitting new traffic even if the aggregate
   bottleneck load has been reached or exceeded.  Since most of the
   other traffic patterns contain large CBR segments, this effect is
   seen with other traffic types as well, although to a different
   extent.

   A natural initial reaction can be to write-off this effect as purely
   a simulation artifact.  In fact, one can expect that if some jitter
   is introduced into the strict CBR traffic pattern so that the packet
   transmission is longer strictly periodic, then the "synchronization"
   effect might be easily broken.

   To verify whether this is indeed the case, we ran the experiment with
   same topologies and parameter settings, but with randomized version
   of the base traffic types.  The results are summarized in Table A.4.
   Note, the column label with f (e.g. 0.0001) correspond to randomized
   traffic with a randomization-interval of f x packet-interarrival-
   time.  It also means that on average, the packets are delayed by f x
   packet-interarrival-time / 2.  In addition, the column of "No-Rand"
   actually correspond to the token bucket results in Table A.3).  It
   turns out that indeed introducing enough jitter does break the
   synchronization effect and the performance of the algorithm much
   improves.  However, it takes sufficient amount of the randomization
   before it is noticed.  For instance, in the CBR graph, the only
   column that shows no aggregation effect is the one labeled with
   "0.05", which translates to expected packet deviation from its ideal
   CBR transmit time of 0.5ms.  While 0.5ms per-hop deviation is not
   unreasonable to expect, in well provisioned networks with a
   relatively small amount of voice traffic in the priority queue one
   might find lower levels of network-induced jitter.  In any case,
   these results indicates the "synchronization" effect can not be
   completely written off as a simulation artifact.  The good news,
   however, that this effect is visible only at very low ingress-egress
   aggregation levels, and as the ingress-egress aggregation increases,
   the effect quickly disappears.

   We observed the synchronization effect consistently across all types
   of traffic we tested with the exception of VTR.  VTR also exhibits



Charny, et al.            Expires May 21, 2008                 [Page 38]


Internet-Draft           PCN with Single Marking           November 2007


   some aggregation effect - however randomization of its CBR portion
   has almost have no effect on performance.  We suspect this is because
   the randomization we perform is at packet level, while the
   synchronization that seems to be causing the performance degradation
   at low ingress-egress aggregation for VTR traffic occurs at frame-
   level.  Although our investigation of this issue is not completed
   yet, our preliminary results show that if we calculating random
   deviation for our artificially induced jitter using frame inter-
   arrival time instead of packet-interarrival-time, we can reduce the
   over-admission percentage for VTR to roughly 3%.  It is unclear
   however, whether such randomization at the frame level meaningfully
   reflects network-introduced jitter.







































Charny, et al.            Expires May 21, 2008                 [Page 39]


Internet-Draft           PCN with Single Marking           November 2007


    ----------------------------------------------------------------
   |     |  No.  |         Randomization Interval                   |
   |     | Ingr  | No-Rand | 0.0001 | 0.001 | 0.005 | 0.01  | 0.05  |
   |----------------------------------------------------------------|
   |     |    2  |  0.725  | 0.683  | 0.784 | 0.725 | 0.772 | 0.787 |
   |     |   10  |  0.753  | 0.725  | 0.543 | 0.645 | 0.733 | 0.854 |
   |     |  100  |  7.666  | 5.593  | 2.706 | 1.454 | 1.226 | 0.692 |
   | CBR |  300  |  21.16  | 15.52  | 6.699 | 3.105 | 2.478 | 1.624 |
   |     |  600  |  33.69  | 25.51  | 11.41 | 6.021 | 4.676 | 2.916 |
   |     | 1000  |  44.58  | 36.20  | 17.03 | 7.094 | 5.371 | 3.076 |
   |----------------------------------------------------------------|
   |     |    2  |  0.532  | 0.645  | 0.670 | 0.555 | 0.237 | 0.740 |
   |     |   10  |  0.477  | 0.596  | 0.703 | 0.494 | 0.662 | 0.533 |
   |     |  100  |  1.409  | 1.236  | 1.043 | 0.810 | 1.202 | 1.016 |
   | VBR |  300  |  3.044  | 2.652  | 2.093 | 1.588 | 1.755 | 1.671 |
   |     |  600  |  5.812  | 4.913  | 3.539 | 2.963 | 2.803 | 2.277 |
   |     | 1800  |  14.80  | 12.59  | 8.039 | 6.587 | 5.694 | 4.733 |
   |----------------------------------------------------------------|
   |     |    2  |  0.736  | 0.753  | 0.627 | 0.751 | 0.850 | 0.820 |
   |     |   10  |  0.649  | 0.737  | 0.780 | 0.824 | 0.867 | 0.787 |
   |     |  100  |  1.960  | 1.705  | 1.428 | 1.160 | 1.149 | 1.034 |
   | MIX |  300  |  4.652  | 4.724  | 3.760 | 2.692 | 2.449 | 2.027 |
   |     |  600  |  10.31  | 9.629  | 7.289 | 5.520 | 4.958 | 3.710 |
   |     | 1000  |  17.21  | 15.96  | 11.05 | 8.700 | 7.382 | 5.061 |
   |     | 1800  |  27.69  | 23.46  | 16.53 | 12.04 | 10.84 | 8.563 |
   |----------------------------------------------------------------|
   |     |    2  |  0.758  | 0.756  | 0.872 | 0.894 | 0.825 | 0.849 |
   |     |   10  |  0.889  | 0.939  | 0.785 | 0.704 | 0.843 | 0.574 |
   |     |   70  |  1.335  | 1.101  | 1.066 | 1.181 | 0.978 | 0.946 |
   | VTR |  140  |  1.694  | 1.162  | 1.979 | 1.791 | 1.684 | 1.573 |
   |     |  300  |  4.128  | 4.191  | 3.545 | 3.307 | 3.964 | 3.465 |
   |     |  600  |  13.28  | 13.76  | 13.81 | 13.18 | 12.97 | 12.35 |
   |----------------------------------------------------------------|
   |     |    2  |  -1.64  | -2.30  | -2.14 | -1.61 | -1.01 | -0.89 |
   |     |   10  |  -0.93  | -1.65  | -2.41 | -2.98 | -2.58 | -2.27 |
   |     |   35  |  0.237  | -0.31  | -0.35 | -1.02 | -0.96 | -2.16 |
   | SVD |  100  |  4.732  | 4.640  | 4.152 | 2.287 | 1.887 | -0.03 |
   |     |  140  |  7.103  | 6.002  | 5.560 | 4.974 | 3.619 | 0.091 |
   |     |  300  |  8.799  | 10.72  | 9.840 | 7.530 | 6.281 | 4.270 |
    ----------------------------------------------------------------

   (Table A.4 Ingress-Egress Aggregation: Token-based results for
   Randomized traffic))

   Finally, we investigated the impact of call arrival assumptions at
   different levels of ingress-egress aggregation by comparing the
   results with Poisson and BATCH arrivals.  We reported in
   draft-zhang-pcn-performance-evaluation that virtual queue -based



Charny, et al.            Expires May 21, 2008                 [Page 40]


Internet-Draft           PCN with Single Marking           November 2007


   admission is relatively insensitive to the BATCH vs Poisson arrivals,
   even at lower aggregation levels.  In contrast, the call arrival
   assumption does affect the performance of token bucket-based
   algorithm, and causes substantial degradation of performance at low
   ingress-egress aggregation level.  An example result with CBR traffic
   is presented in table A.5.  Here we use batch arrival with mean = 5.
   The results show that with the lowest aggregation, the batch arrival
   gives worse result than the normal Poisson arrival, however, as the
   level of aggregation become sufficient (e.g. 100 ingress, 10 call/
   ingress), the difference becomes insignificant.  This behavior is
   consistent across all types of traffic.

   (preamble)
    ----------------------------------------------------------------
   |     |  No.  |          Deviation Interval                      |
   |     | Ingr | No-Rand | 0.0001 | 0.001 | 0.005 | 0.01  | 0.05  |
   |----------------------------------------------------------------|
   |     |    2  |  0.918  | 1.007  | 0.836 | 0.933 | 1.014 | 0.971 |
   |     |   10  |  1.221  | 0.936  | 0.767 | 0.906 | 0.920 | 0.857 |
   |     |  100  |  8.857  | 7.092  | 3.265 | 1.821 | 1.463 | 1.036 |
   | CBR |  300  |  29.39  | 22.59  | 8.596 | 4.979 | 4.550 | 2.165 |
   |     |  600  |  43.36  | 37.12  | 17.37 | 10.02 | 8.005 | 4.223 |
   |     | 1000  |  63.60  | 50.36  | 25.48 | 12.82 | 9.339 | 6.219 |
   |----------------------------------------------------------------|
   (Table A.5 In/Egress Aggregation with batch traffic: Token-based
   results )

8.2.4.  Effect of Multiple Bottlenecks

   The results in Table A.2 (Section 9.5.1, parameter sensitivity study)
   implied that from the bottleneck point of view, the performance on
   the multiple-bottleneck topology, for all types of traffic, is
   comparable to the ones on the SingleLink, for both queue-based and
   token bucket-based algorithms.  However, the results in Table A.2
   only show the worst case values over all bottleneck links.  In this
   section we consider two other aspects of the Multiple Bottleneck
   effects: relative performance at individual bottlenecks and fairness
   of bandwidth usage between the short- and the long- haul IEAs.

8.2.4.1.  Relative performance of different bottlenecks

   In Table A.5, we show a snapshot of the behavior with 5 bottleneck
   topology, with the goal of studying the performance of different
   bottlenecks more closely.  Here, the over-admission-percentage
   displayed is an average across all 15 experiments with different
   [weight, CLE] setting.  (We do observe the same behavior in each of
   the individual experiment, hence providing a summarized statistics is
   meaningful).



Charny, et al.            Expires May 21, 2008                 [Page 41]


Internet-Draft           PCN with Single Marking           November 2007


   One differences in token-bucket case vs the queue-based admissions in
   the PLT topology case revealed in Table A.6 is that there appears to
   be a consistent relationship between the position of the bottleneck
   link (how far downstream it is) and its over-admission-percentage.
   The data shows the further downstream the bottleneck is, the more it
   tends to over-admit, regardless the type of the traffic.  The exact
   cause of this phenomenon is yet to be explained, but the effect of it
   seems to be insignificant in magnitude, at least in the experiments
   we ran.

   (preamble)
    ---------------------------------------------------------
   |       | Traffic |            Bottleneck LinkId          |
   |       |   Type  |   1   |   2   |   3   |   4   |   5   |
   |       |-------------------------------------------------|
   |       |   CBR   | 0.288 | 0.286 | 0.238 | 0.332 | 0.306 |
   |       |-------------------------------------------------|
   |       |   VBR   | 0.319 | 0.420 | 0.257 | 0.341 | 0.254 |
   | Queue |-------------------------------------------------|
   | Based |   MIX   | 0.363 | 0.394 | 0.312 | 0.268 | 0.205 |
   |       |-------------------------------------------------|
   |       |   VTR   | 0.466 | 0.309 | 0.223 | 0.363 | 0.317 |
   |       |-------------------------------------------------|
   |       |   SVD   | 0.319 | 0.420 | 0.257 | 0.341 | 0.254 |
   |---------------------------------------------------------
   |       | Traffic |            Bottleneck LinkId          |
   |       |   Type  |   1   |   2   |   3   |   4   |   5   |
   |       |-------------------------------------------------|
   |       |   CBR   | 0.121 | 0.300 | 0.413 | 0.515 | 0.700 |
   |       |-------------------------------------------------|
   | Token |   VBR   | -0.07 | 0.251 | 0.496 | 0.698 | 1.044 |
   |Bucket |-------------------------------------------------|
   | Based |   MIX   | 0.042 | 0.350 | 0.468 | 0.716 | 0.924 |
   |       |-------------------------------------------------|
   |       |   VTR   | 0.277 | 0.488 | 0.642 | 0.907 | 1.117 |
   |       |-------------------------------------------------|
   |       |   SVD   | -2.64 | -2.50 | -1.72 | -1.57 | -1.19 |
    ---------------------------------------------------------

   Table A.6 Bottleneck Performance: queue-based v.s. token bucket-based

8.2.4.2.  (Un)Fairness Between Different Ingress-Egress pairs

   It was reported in draft-zhang-pcn-performance-evaluation that
   virtual-queue-based admission control favors significantly short-haul
   connection over long-haul connections.  As was discussed there, this
   property is in fact common for measurement-based admission control
   algorithms (see for example [Jamin] for a discussion).  It is common



Charny, et al.            Expires May 21, 2008                 [Page 42]


Internet-Draft           PCN with Single Marking           November 2007


   knowledge that in the limit of large demands, long-haul connections
   can be completely starved.  We show in
   draft-zhang-performance-evaluation that in fact starvation of long-
   haul connections can occur even with relatively small (but constant)
   overloads.  We identify there that the primary reason for it is a de-
   synchronization of the "congestion periods" at different bottlenecks,
   resulting in the long-haul connections almost always seeing at least
   one bottleneck and hence almost never being allowed to admit new
   flows.  We refer the reader to that draft for more detail.

   Here we investigate the comparative behavior of the token-bucket
   based scheme and virtual queue based scheme with respect to fairness.

   The fairness is illustrated using the ratio between bandwidth of the
   long-haul aggregates and the short-haul aggregates.  As is
   intuitively expected, (and also confirmed experimentally), the
   unfairness is the larger the higher the demand, and the more
   bottlenecks traversed by the long-haul aggregate Therefore, we report
   here the "worst case" results across our experiments corresponding to
   the 5x demand overload and the 5-PLT topology.

   Table A.7 summaries, at 5x overload, with CLE=0.05 (for virtual
   queue), 0.0001(for token bucket), the fairness results to different
   weight and topology.  We display the ratio as function of time, in 10
   sec increments, (the reported ratios are averaged over the
   corresponding 10 simulation-second interval).  The result presented
   in this section uses the aggregates that traverse the first
   bottleneck.  The results on all other bottlenecks are extremely
   similar.






















Charny, et al.            Expires May 21, 2008                 [Page 43]


Internet-Draft           PCN with Single Marking           November 2007


   (preamble)
 ---------------------------------------------------------------------------
|       |Topo|Weight|               Simulation  Time (s)                    |
|       |    |      |  10  |  20  |  30  |  40  |  50  |  60  |  70  |  80  |
|       |-------------------------------------------------------------------|
|       |    |  0.1 | 0.99 | 1.04 | 1.14 | 1.14 | 1.23 | 1.23 | 1.35 | 1.46 |
|       |PLT5|  0.5 | 1.00 | 1.17 | 1.24 | 1.41 | 1.81 | 2.13 | 2.88 | 3.05 |
|       |    |  0.9 | 1.03 | 1.42 | 1.74 | 2.14 | 2.44 | 2.91 | 3.83 | 4.20 |
|       |-------------------------------------------------------------------|
|Virtual|    |  0.1 | 1.02 | 1.08 | 1.15 | 1.29 | 1.33 | 1.38 | 1.37 | 1.42 |
|Queue  |PLT3|  0.5 | 1.02 | 1.04 | 1.07 | 1.19 | 1.24 | 1.30 | 1.34 | 1.33 |
|Based  |    |  0.9 | 1.02 | 1.09 | 1.23 | 1.41 | 1.65 | 2.10 | 2.63 | 3.18 |
|       |-------------------------------------------------------------------|
|       |    |  0.1 | 1.02 | 0.98 | 1.03 | 1.11 | 1.22 | 1.21 | 1.25 | 1.31 |
|       |PLT2|  0.5 | 1.02 | 1.06 | 1.14 | 1.17 | 1.15 | 1.31 | 1.41 | 1.41 |
|       |    |  0.9 | 1.02 | 1.04 | 1.11 | 1.30 | 1.56 | 1.61 | 1.62 | 1.67 |
 ---------------------------------------------------------------------------
 ----------------------------------------------------------------------------
|       |Topo|Weight|               Simulation  Time (s)                    |
|       |    |      |  10  |  20  |  30  |  40  |  50  |  60  |  70  |  80  |
|       |-------------------------------------------------------------------|
|       |    |  0.1 | 1.03 | 1.48 | 1.83 | 2.34 | 2.95 | 3.33 | 4.32 | 4.65 |
|       |PLT5|  0.5 | 1.08 | 1.53 | 1.90 | 2.44 | 3.04 | 3.42 | 4.47 | 4.83 |
|       |    |  0.9 | 1.08 | 1.48 | 1.80 | 2.26 | 2.82 | 3.19 | 4.23 | 4.16 |
|       |-------------------------------------------------------------------|
|Token  |    |  0.1 | 1.02 | 1.26 | 1.45 | 1.57 | 1.69 | 1.76 | 1.92 | 1.94 |
|Bucket |PLT3|  0.5 | 1.07 | 1.41 | 1.89 | 2.36 | 2.89 | 3.63 | 3.70 | 3.82 |
|Based  |    |  0.9 | 1.07 | 1.33 | 1.59 | 1.94 | 2.41 | 2.80 | 2.75 | 2.90 |
|       |-------------------------------------------------------------------|
|       |    |  0.1 | 1.03 | 1.10 | 1.43 | 2.06 | 2.28 | 2.85 | 3.09 | 2.90 |
|       |PLT2|  0.5 | 1.07 | 1.32 | 1.47 | 1.72 | 1.71 | 1.81 | 1.89 | 1.94 |
|       |    |  0.9 | 1.09 | 1.27 | 1.51 | 1.86 | 1.82 | 1.88 | 1.88 | 2.06 |
 -------------------------------------------------------------------
   Table A.7 Fairness performance: Virtual Queue v.s.  Token Bucket.
   The numbers in the cells represent the ratio between the bandwidth of
   the long- and short-haul aggregates.  Each row represents the time
   series of these results in 10 simulation second increments.

   To summarize, we observed consistent beatdown effect across all
   experiments for both virtual-queue and token-bucket admission
   algorithms, although the exact extent of the unfairness depends on
   the demand overload, topology and parameters settings.  To further
   quantify the effect of these factors remains an area of future work.
   We also note that the cause of the beatdown effect appears to be
   largely independent of the specific algorithm, and is likely to be
   relevant to other PCN proposals as well.





Charny, et al.            Expires May 21, 2008                 [Page 44]


Internet-Draft           PCN with Single Marking           November 2007


8.3.  Termination Control

8.3.1.  Ingress-Egress Aggregation Experiments

   In this section, we investigate sensitivity of the Flow Termination
   Function of SM.  From our admission control experiments it is clear
   that SM is extremely sensitive to very low IE-aggregation (on the
   order of 1-10 flows), limiting applicability of SM at these
   aggregation levels.  We show here that the Termination Function of CL
   requires even more IE aggregation, as we quantify in this section.

   The table below shows comparative accuracy of CL and SM at different
   aggregation levels in a single bottleneck topology with multiple IEAs
   sharing the bottleneck.  As can be seen from this table, the actual
   degree of IE aggregation necessary to achieve an over-termination
   within 10% ranges from ~50 to about ~150 for different traffic types
   (note that extremely bursty high-rate SVD traffic the maximum number
   of flows in an IEA we ran was 69, which was not sufficient to reach a
   10% over-termination error bound we targeted.  We did not run higher
   number of SVD flows per IEA due to time limitations).































Charny, et al.            Expires May 21, 2008                 [Page 45]


Internet-Draft           PCN with Single Marking           November 2007


    --------------------------------------------
   |     |  No.  | Flow per | Over-Term. Perc.  |
   |     | Ingre |  Ingre   |   CL    |   SM    |
   |--------------------------------------------|
   |     |    2  |   285    | -0.106  |  4.112  |
   | CBR |   10  |    57    |  0.388  |  6.710  |
   |     |   35  |    16    |  1.035  |  14.64  |
   |     |   70  |     8    |  0.727  |  16.39  |
   |--------------------------------------------|
   |     |    2  |   849    |  0.912  |  2.808  |
   | VBR |   10  |   169    |  4.032  |  10.47  |
   |     |   35  |    48    |  2.757  |  22.26  |
   |     |  100  |    16    |  3.966  |  22.52  |
   |--------------------------------------------|
   |     |    2  |   662    |  1.297  |  3.672  |
   | MIX |   10  |   132    |  2.698  |  7.809  |
   |     |   35  |    37    |  1.978  |  14.83  |
   |     |  100  |    13    |  4.265  |  17.29  |
   |--------------------------------------------|
   |     |    2  |   158    |  3.513  |  3.718  |
   | VTR |   10  |    31    |  4.532  |  14.82  |
   |     |   35  |     9    |  6.842  |  22.95  |
   |     |   70  |     4    |  8.458  |  22.31  |
   |--------------------------------------------|
   |     |    2  |    69    |  7.811  |  20.90  |
   | SVD |   10  |    13    |  10.69  |  27.38  |
   |     |   35  |     4    |  8.322  |  20.78  |
    --------------------------------------------
   Table A.8 Over-termination comparison between CL and SM at medium/
   high IE aggregation

   It turns out that the reason for this higher sensitivity to low
   ingress-egress aggregation lies in the non-uniformity in the marking
   distribution across different IEAs.  As a result of this non-
   uniformity , when traffic is close below the (implicit) termination
   threshold at the bottleneck , some IEAs get excessively marked,
   causing a false termination event at the corresponding PCN-ingress-
   nodes, in turn causing extra over-termination.













Charny, et al.            Expires May 21, 2008                 [Page 46]


Internet-Draft           PCN with Single Marking           November 2007


    ----------------------------------------------
   |     |  No.  | Flow per | Over-Term. Perc.    |
   |     | Ingre |  Ingre   |   SM    |   SM-SM   |
   |----------------------------------------------|
   |     |    2  |    285   |  4.112  |   2.243   |
   | CBR |   10  |     57   |  6.710  |   3.142   |
   |     |   35  |     16   |  14.64  |   6.549   |
   |     |   70  |      8   |  16.39  |   8.496   |
   |----------------------------------------------|
   |     |    2  |    849   |  2.808  |   0.951   |
   | VBR |   10  |    169   |  10.47  |   4.096   |
   |     |   35  |     48   |  22.26  |   6.987   |
   |     |  100  |     16   |  22.52  |   8.567   |
   |----------------------------------------------|
   |     |    2  |    662   |  3.672  |   2.574   |
   | MIX |   10  |    132   |  7.809  |   3.822   |
   |     |   35  |     37   |  14.83  |   4.936   |
   |     |  100  |     13   |  17.29  |   6.956   |
   |----------------------------------------------|
   |     |    2  |    158   |  3.718  |   3.866   |
   | VTR |   10  |     31   |  14.82  |   7.507   |
   |     |   35  |      9   |  22.95  |   10.29   |
   |     |   70  |      4   |  22.31  |   8.528   |
   |----------------------------------------------|
   |     |    2  |     69   |  20.90  |   9.272   |
   | SVD |   10  |     13   |  27.38  |   12.46   |
   |     |   35  |      4   |  20.78  |   10.14   |
    ----------------------------------------------
   Table A.9 Over-termination comparison between SM and SM with smoothed
   trigger.  Here EWMA weight = 0.9 (heavy history)

   We investigated whether this effect can be removed by smoothing
   (using EWMA) the ratio between marked and unmarked traffic that we
   use at the ingress node to trigger the termination event.  Table A.9
   above presents the results for the EWMA weight of 0.9 corresponding
   to a long history.  It can be seen that such smoothing does in fact
   help reduce over-termination.  However, it also increases the
   reaction time of flow termination.  This increased latency grows for
   larger U and decreases with the increase in the excess load over the
   (implicit) termination threshold.

   Table A.10 quantifies this extra delay (Note: these results are for
   100 ms measurement intervals at the ingress, and for negligible
   round-trip time.  The actual extra latency is obtained by adding the
   RTT to the results of table 8.3.






Charny, et al.            Expires May 21, 2008                 [Page 47]


Internet-Draft           PCN with Single Marking           November 2007


   (preamble)
   --------------------------------
   |U \ R | 0.2 | 0.3 | 0.4 | 0.5 |
   --------------------------------
   | 1.1  | 0.5 | 0.4 | 0.3 | 0.3 |
   --------------------------------
   | 1.3  | 1.0 | 0.8 | 0.7 | 0.7 |
   --------------------------------
   | 1.5  | 1.4 | 1.1 | 1.0 | 0.9 |
   --------------------------------
   | 1.7  | 1.6 | 1.4 | 1.2 | 1.1 |
   --------------------------------
   | 1.9  | 1.8 | 1.6 | 1.4 | 1.3 |
   --------------------------------
   | 2.0  | 1.9 | 1.6 | 1.5 | 1.4 |
   --------------------------------

   Table A.10.  Additional latency due to smoothing of termination
   signal and the PCN-ingress-node (in sec; W=0.9)

   We note that this smoothing is only necessary at the lower range of
   the IE aggregation levels we considered, and is not necessary as soon
   as the aggregation level reaches 50-150 flows (for different traffic
   types) in our experiments.  For the lower aggregation level, the
   smoothing may be useful, at the expense of the additional latency.

8.3.2.  Multiple Bottlenecks Experiments

   As discussed in Section 7.3, the fact that SM marks traffic when the
   bottleneck load is below (implicit) termination threshold but above
   the configured admission threshold, causes additional "beat-down"
   effect of flows traversing multiple bottlenecks, compared to the
   beat-down effect already observed for CL in
   draft-zhang-performance-evaluation.

   We start with the setup with 2- and 5-PLT topology similar to that of
   draft-zhang-performance-evaluation.  That is, at failure event time,
   all bottleneck links have a load of roughly 3/4 of its link size.  In
   addition, the long IEA constitutes 2/3 of this load, while the short
   one is 1/3.  Table below shows the comparative over-termination on
   the bottlenecks (2 and 5 PLT topology) for both CL and SM.  The
   bottleneck rows are ordered based on the flow traversal order (from
   upstream to downstream).

   As in the results we presented in draft-zhang-performance-evaluation,
   we report over-termination compared to the "reference" over-
   termination which we compute as follows for the multi-bottleneck
   topology.  We take each link in the topology separately and compute



Charny, et al.            Expires May 21, 2008                 [Page 48]


Internet-Draft           PCN with Single Marking           November 2007


   the "rate-proportionally fair" rates that each IEA sharing this
   bottleneck will need to be reduced to (in proportion to their
   demands), so that the load on that bottleneck independently becomes
   equal to the termination threshold (this threshold being implicit for
   SM, explicit for CL), assuming the initial sum of rates exceeds this
   threshold.  After this is done independently for each bottleneck, we
   assign each IEA the smallest of its scaled down rates across all
   bottlenecks.  We then compute the "reference" utilization on each
   link by summing up the scaled down rates of each IEA sharing this
   link.  Our over-termination is then reported in reference to this
   "reference" utilization.  We note that this reference utilization may
   frequently be already below the termination threshold of a given
   link.  This can happen easily in the case when a large number of
   flows sharing a given link is "bottlenecked" elsewhere.

    -------------------------------------------------------------------
   |  Topo.   |     CBR     |     VBR     |     VTR     |     SVD      |
   | 2/5 PLT  |  CL     SM  |  CL     SM  |  CL     SM  |  CL      SM  |
    -------------------------------------------------------------------
   |2   BN1   | 5.93  20.93 | 6.49  21.31 | 9.07  21.88 | 9.28   23.18 |
   |    BN2   | 0.56   9.89 | 2.21   9.89 | 3.61   8.74 | 7.99   12.92 |
   |-------------------------------------------------------------------|
   |    BN1   | 9.63  35.04 | 10.9  34.06 | 11.41 36.30 | 14.23  39.37 |
   |    BN2   | 4.54  23.51 | 6.19  22.83 | 5.66  23.53 | 9.67   28.45 |
   |5   BN3   | 2.05  23.36 | 2.46  23.18 | 3.47  24.64 | 5.73   27.01 |
   |    BN4   | 0.90  23.78 | 1.40  23.46 | 3.13  24.02 | 3.98   27.59 |
   |    BN5   | 0.00  24.08 | 0.30  23.11 | 2.81  23.83 | 5.54   28.45 |
    -------------------------------------------------------------------
   Table A.11 Over-termination comparison of CL and SM for 2 and 5 PLT
   topology

   We note that in these experiments SM does significantly worse than CL
   across all traffic.  The most upstream bottleneck suffers the most
   over-termination due to the fact that the long-haul IA gets severely
   beaten down, while the short-haul flows terminate their fair share.
   (In this experiment almost 90% of the long-haul IA is terminated).

   In our PLT setup each IEA is heavily aggregated, so we do not expect
   smoothing of the termination trigger to have a significant effect.
   Table A.12 Summarizes the performance of the same setup with
   smoothing.










Charny, et al.            Expires May 21, 2008                 [Page 49]


Internet-Draft           PCN with Single Marking           November 2007


    ------------------------------------------------------------------
   |  Topo.   |     CBR     |     VBR     |     VTR     |     SVD     |
   | 2/5 PLT  |  SM   SM-SM |  SM   SM-SM |  SM   SM-SM |  SM   SM-SM |
    ------------------------------------------------------------------|
   |2   BN1   | 20.93 14.29 | 21.31 13.18 | 21.88 15.14 | 23.18 16.53 |
   |    BN2   | 9.89  14.37 | 9.89  13.32 | 8.74  14.22 | 12.92 16.89 |
   |------------------------------------------------------------------|
   |    BN1   | 35.04 24.27 | 34.06 23.17 | 36.30 23.98 | 39.37 30.83 |
   |    BN2   | 23.51 24.15 | 22.83 23.87 | 23.53 24.21 | 28.45 31.60 |
   |5   BN3   | 23.36 23.94 | 23.18 23.67 | 24.64 25.23 | 27.01 29.65 |
   |    BN4   | 23.78 24.56 | 23.46 23.86 | 24.02 25.04 | 27.59 29.25 |
   |    BN5   | 24.08 24.24 | 23.11 24.08 | 23.83 24.95 | 28.45 29.94 |
    -------------------------------------------------------------------
   Table A.12.  Over-termination comparison of SM and smoothed SM for 2
   and 5 PLT topology

   Smaller over-termination on the upstream bottlenecks, especially 1)
   is due to the fact that with smoothing, the short IEA on the
   bottleneck 1 did not terminate at all, which makes it the bottleneck
   1 less over-terminated than in the case of SM.  The reason for this
   is that in the smooth-SM, the additional markings received (due to
   multi-bottleneck effect) by the long IEA make its smoothing process
   much faster than the short IEA.  Again in this particular setup,
   there is enough flows in the long IEA to be terminated and bring the
   bottleneck load way below the termination threshold, while short IEA
   never gets to react.

   Our next task was to investigate whether particularly bad performance
   of SM in this case is a common occurrence.  To do so, we took the
   5-PLT topology and generated on the order of ~50,000 random traffic
   matrices, and random settings of the admission threshold and the
   parameter U, resulting in creation of anywhere from 0 to 5
   bottlenecks on this topology in each experiment.  We limited the
   range of U from 1.0 to 3.0, and the maximum overload on any link was
   at most ~10x of the (implicit) termination threshold.  To enable us
   to run these many experiments in a reasonable time, we implemented a
   fluid model, and later compared its accuracy with the packet
   simulations on a subset of topologies to confirm reliability of the
   fluid model simulation results (see below).

   Table A.13 gives a summary of the experimental frequency of the
   setups with a particular range of the termination error on the most
   loaded bottleneck.  In addition, we checked whether the termination
   error in those setups is so big as to bring the load on the most
   loaded bottleneck below its admission threshold.  This data is shown
   by summarizing the experimental frequency of experiments where the
   resulting load after termination (End-Load) is above the admission
   threshold .  Since the frequency of these cases depends on the number



Charny, et al.            Expires May 21, 2008                 [Page 50]


Internet-Draft           PCN with Single Marking           November 2007


   of bottlenecks in the experiment, we report this by the number of
   bottlenecks.

   We split the results in Table A.13 into those with small U (where the
   (implicit) termination threshold is very close to the admission
   threshold), medium U (where the implicit termination threshold is
   between 1.2 - and 2 times the admission threshold, and large U
   (greater than 2 times admission threshold).

   Given the accuracy results from our packet experiments, it seems that
   the reasonable setting of U must be at least 120% of the admission
   threshold to reduce the probability that the termination error will
   bring the load below admission threshold.  Therefore, we present the
   results for small U for completeness only.  We also believe that in
   practice setups with large U>2.0 should be rare, and hence we report
   the large U results separately as well.

   At a high level the results of table A.13 imply that for small U,
   over-termination is small, but it is enough to frequently drive the
   bottleneck load below admission threshold, especially with larger
   number of bottlenecks.  For large U, the over-termination is larger,
   but the bottleneck load almost never falls below admission threshold.
   Finally, for medium U, which, in our opinion is the case of practical
   importance, the over-termination for SM is below 10% in ~65% of the
   experiments, is within 20% for about 30% of the experiments, and
   between 30 an d 40% for the remaining 5%.  In contrast, CL remains
   within 10% over-termination most of the time.  For this medium U,
   this over-termination almost never causes the bottleneck load to drop
   below admission threshold for up to 3 bottlenecks, while ~10% of the
   4 bottleneck cases and ~20% of the 5-bottleneck cases do drop below
   the admission threshold.  Note that CL also occasionally drives the
   load below admission threshold, albeit not as often as SM - e.g. in
   ~3% of the simulations for the 4 bottlenecks and about 7% for the 5
   bottlenecks, for medium U.

   We note that the cause of the after-termination event load falling
   below admission threshold, as well as a partial cause for the over-
   termination reported below is partially due to the fact that some of
   the flows going through the most overloaded bottleneck are
   nevertheless passing another bottleneck elsewhere.  Even if the
   overall overload on that other bottleneck may not be as high, that
   (other) bottleneck nevertheless may be driving some of the IEAs down
   to a smaller rate than the bottleneck with the largest overload we
   are considering.  This is confirmed by the fact that the Reference
   termination occasionally also falls below the admission threshold as
   well - see REF rows of the "End-load Above Threshold" tables in Table
   A.13.




Charny, et al.            Expires May 21, 2008                 [Page 51]


Internet-Draft           PCN with Single Marking           November 2007


   (preamble)
   Small U    Total Expr: 5185   1.0 < U <= 1.2

    ------------------------------------------------------
   | Alg. | Distribution of Over-Termination Percentage  |
   |      | 0-10% | 10-20% | 20-30%  | 30-40%  | 40-50%  |
   |------|-------|--------|---------|---------|---------|
   |  CL  | 0.986 |  0.013 |  0.000  |  0.000  |  0.000  |
   |------|-------|--------|---------|---------|---------|
   |  SM  | 0.968 |  0.031 |  0.000  |  0.000  |  0.000  |
    -----------------------------------------------------|

    ------------------------------------------------------
   | Alg. |   Fract. End-load Above Admission Threshold    |
   |      | 0 BN  | 1 BN  | 2 BN  | 3 BN  | 4 BN  | 5 BN  |
   |------|-------|-------|-------|-------|-------|-------|
   |  CL  |   1   | 0.914 | 0.851 | 0.732 | 0.490 | 0.357 |
   |------|-------|-------|-------|-------|-------|-------|
   |  SM  |   1   | 0.934 | 0.869 | 0.725 | 0.488 | 0.374 |
   |------------------------------------------------------|
   |  REF |   1   | 0.980 | 0.982 | 0.977 | 0.978 | 0.968 |
   |------------------------------------------------------|

   Medium U    Total Expr: 19689   1.2 < U < 2.0

    --------------------------------------------------------
   | Alg. |   Distribution of Over-Termination Percentage   |
   |      |  0-10%  | 10-20%  |  20-30% | 30-40%  | 40-50%  |
   |------|---------|---------|---------|---------|---------|
   |  CL  |  0.985  |  0.013  |  0.000  |  0.000  |  0.000  |
   |------|---------|---------|---------|---------|---------|
   |  SM  |  0.659  |  0.294  |  0.043  |  0.001  |  0.000  |
    --------------------------------------------------------

    ------------------------------------------------------
   | Alg. |   Fract. End-load Above Admission Threshold    |
   |      | 0 BN  | 1 BN  | 2 BN  | 3 BN  | 4 BN  | 5 BN  |
   |------|-------|-------|-------|-------|-------|-------|
   |  CL  |   1   | 0.991 | 0.993 | 0.991 | 0.969 | 0.928 |
   |------|-------|-------|-------|-------|-------|-------|
   |  SM  |   1   | 0.990 | 0.991 | 0.977 | 0.909 | 0.831 |
   |------------------------------------------------------|
   | REF  |   1   | 0.991 | 0.994 | 0.994 | 0.996 | 0.993 |
   -------------------------------------------------------|

   Large U    Total Expr: 25129   2.0 <= U <= 3.0

    -------------------------------------------------------



Charny, et al.            Expires May 21, 2008                 [Page 52]


Internet-Draft           PCN with Single Marking           November 2007


   | Alg. |   Distribution of Over-Termination Percentage  |
   |      |  0-10%  | 10-20%  | 20-30%  | 30-40%  | 40-50% |
   |------|---------|---------|---------|---------|--------|
   |  CL  |  0.984  |  0.014  |  0.000  |  0.000  |  0.000 |
   |------|---------|---------|---------|---------|--------|
   |  SM  |  0.254  |  0.384  |  0.275  |  0.075  |  0.008 |
    --------------------------------------------------------

    ------------------------------------------------------
   | Alg. |   Fract. End-load Above Admission Threshold    |
   |      | 0 BN  | 1 BN  | 2 BN  | 3 BN  | 4 BN  | 5 BN  |
   |------|-------|-------|-------|-------|-------|-------|
   |  CL  |   1   | 0.998 | 0.998 | 0.997 | 0.997 | 0.996 |
   |------|-------|-------|-------|-------|-------|-------|
   |  SM  |   1   | 0.997 | 0.995 | 0.992 | 0.969 | 0.945 |
    ------------------------------------------------------
   |  REF |   1   | 0.998 | 0.998 | 0.997 | 0.998 | 0.999 |
   --------------------------------------------------------


   Table A.13.  Distribution of over-termination percentage and
   frequency of the load after termination event (denoted "End-Load")
   remaining above admission threshold.  The REF rows in the "End-load
   Above Admission Threshold" tables correspond to the Reference
   termination against which the over-termination percentage is computed

   To investigate how the fluid simulation results relate to the packet
   simulation, we also ran a subset of approximately 2000 experiments
   through a packet simulator with the same parameter settings and
   traffic loads as the corresponding fluid simulations, and compared
   the results.  We found that the error in most experiments is
   relatively small, allowing us to conjecture that statistics of the
   fluid experiments adequately approximate the expected packet-level
   results in these experiments.

   The distribution of the error between the fluid and packet simulation
   results for these experiments is shown in the following table:














Charny, et al.            Expires May 21, 2008                 [Page 53]


Internet-Draft           PCN with Single Marking           November 2007


   (preamble)
    --------------------------------------------------------
   | Alg. | Error Dist. in  Over-Term. Perc. (Fluid-Packet) |
   |      |-0.2~-0.1|-0.1~0.0 | 0.0~0.1 | 0.1~0.2 | 0.2-0.4 |
   |------|---------|---------|---------|---------|---------|
   |  CL  |  0.000  |  0.032  |  0.963  |  0.000  |  0.000  |
   |------|---------|---------|---------|---------|---------|
   |  SM  |  0.003  |  0.334  |  0.640  |  0.013  |  0.000  |
    --------------------------------------------------------

   Table A.14 Error Between Fluid and Packet Simulations in about 2000
   experiments

   As can bee seen, the error is relatively contained.

   Finally, we ran a similar set of fluid experiments with the smoothed
   trigger signal (see section 8.3.1), and found that there is no
   visible difference in the statistical performance of the smoothed
   version and non-smoothed version of the algorithm.  In some cases
   smoothing performed better than the non-smoothed version, as in the
   example reported in Tables A11 and A12 , and in other cases non-
   smoothed version outperformed the smoothed version, with the overall
   distribution of over-termination errors remaining extremely similar
   for smoothed and non-smoothed versions.  We therefore conclude that
   smoothing is necessary only to deal with low levels of ingress-egress
   aggregations, and have no effect on the over-termination in the
   multi-bottleneck scenario as long as the IEAs are sufficiently
   aggregated.


9.  Appendix B. Controlling The Single Marking Configuration with a
    Single Parameter

9.1.  Assumption

   This section assumes that TM-marking is used for SM marking encoding.

9.2.  Details of the Proposed Enhancements to PCN Architecture

9.2.1.  PCN-Internal-Node

   No substantive change is required for the PCN framework (as defined
   in [I-D.eardley-pcn-architecture]) to enable SM Operation in the PCN
   Internal Node.  The architecture already allows the implementation of
   only one marking and metering algorithm at the PCN-internal-node.

   However, we propose to rename the terms "configured-admissible-rate"
   and "configured-termination-rate" to "Type Q threshold" and "Type R"



Charny, et al.            Expires May 21, 2008                 [Page 54]


Internet-Draft           PCN with Single Marking           November 2007


   threshold.  The architecture should allow configuring either one of
   these thresholds or both at the PCN-ingress node.  The type of the
   threshold determines the type of the marking semantics/algorithm
   associated with the threshold.

9.2.2.  PCN-Egress-Node

   The only proposed change at the PCN-egress-node is the addition of a
   single (globally defined) configuration constant U. The setting of
   this constant defines the type of marking CLE is measured against.
   If U=1, the system defaults to the dual-marking behavior and the CLE
   is measured against Type Q marked packets.  If U>1, the CLE is
   measured against Type R marked traffic.  No other change is required.

   In more detail,

   o  If U=1, a PCN-egress-node expects to receive either Type Q marking
      only (the network implements virtual-queue-based admission only),
      or Type R marking only (the system implements excess-rate-based
      flow termination only), or both (the system implements dual-
      marking admission and termination).

   o  If U>1, a PCN egress node expects to receive only type-R marking
      (the network implements single-marking approach).

   o  If U=1 and Type-Q marking is received (as indicated by the
      encoding in the PCN packets), then the PCN-egress-node always
      measures the CLE (fraction of traffic carrying Type-Q marks) on a
      per-ingress basis against Type Q marking.  This represents no
      change (other than renaming "admission-marked-packets" to "type
      Q-marked" packets) compared to the current architecture.  The PCN-
      egress-node then signals the (type Q-based) CLE to the PCN-
      ingress-node - again as already enabled by the current PCN
      architecture.

   o  If U=1 and a PCN-egress-node receives "Type R" marking (as
      indicated in the encoding of the PCN packets), it measures
      sustainable rate with respect to Type-R marked traffic, (i.e. it
      measures the amount of traffic without the "Type-R" marks).  This
      also is just a renaming change (with termination-marking renamed
      to "Type R" marking) and is fully compatible with the current PCN
      architecture.

   o  If U > 1, the PCN-egress node computes both the CLE and the
      Sustainable rate with respect to Type-R marking.

   o  Once computed, the CLE and/or the Sustainable rate are
      communicated to the PCN-ingress-node as described in



Charny, et al.            Expires May 21, 2008                 [Page 55]


Internet-Draft           PCN with Single Marking           November 2007


      [I-D.eardley-pcn-architecture].

9.2.3.  PCN-Ingress-Node

   The only proposed change at the PCN-ingress-node is the addition of a
   single (globally defined) configuration constant U (in fact, this is
   the same constant as defined for the PCN-egress-node, so U in fact is
   a per PCN-boundary-node constant; its value however is assumed to be
   global for all PCN-boundary nodes in the PCN-domain (or at least a
   subset of nodes communicating with each other only)).  The value of
   this constant is used to multiply the sustainable rate received from
   a given PCN-egress-node to compute the rate threshold used for flow
   termination decisions.  The value U=1 corresponds to the dual-marking
   approach, and results in using the sustainable rate received from the
   PCN-egress-node directly.  The value U>1 corresponds to the SM
   approach and its (globally defined) value signifies the desired
   system-wide implicit ratio between flow termination and flow
   admission thresholds as described in Section 2.

   Note that constant U is assumed to be defined per PCN-boundary node
   (i.e. the ingress and the egress functions of the PCN-boundary-node
   use the same configuration constant to guide their behavior.

   In more detail:

   o  A PCN-ingress-node receives CLE and/or Sustainable Rate from each
      PCN-egress-node it has traffic to.  This is fully compatible with
      PCN architecture as described in [I-D.eardley-pcn-architecture].

   o  A PCN-ingress-node bases its admission decisions on the value of
      CLE.  Specifically, once the value of CLE exceeds a configured
      threshold, the PCN-ingress-node stops admitting new flows.  It
      restarts admitting when the CLE value goes down below the
      specified threshold.  This is fully compatible with PCN
      architecture as described in draft-earley-pcn-architecture-00.

   o  A PCN-ingress node receiving a Sustainable Rate from a particular
      PCN-egress node measures its traffic to that egress node.  This
      again is fully compatible with PCN architecture as described in
      draft-earley-pcn-architecture-00.

   o  The PCN-ingress-node computes the desired Termination Rate to a
      particular PCN-egress-node by multiplying the sustainable rate
      from a given PCN-egress-node by the value of the configuration
      parameter U. This computation step represents a proposed change to
      the current version of [I-D.eardley-pcn-architecture].





Charny, et al.            Expires May 21, 2008                 [Page 56]


Internet-Draft           PCN with Single Marking           November 2007


   o  Once the Termination Rate is computed, it is used for the flow
      termination decision in a manner fully compatible with
      [I-D.eardley-pcn-architecture].  Namely the PCN-ingress-node
      compares the measured traffic rate destined to the given PCN-
      egress-node with the computed Termination rate for that egress
      node, and terminates a set of traffic flows to reduce the rate
      exceeding that Termination rate.  This is fully compatible with
      [I-D.eardley-pcn-architecture].


10.  Security Considerations

   TBD


11.  References

11.1.  Normative References

   [RFC2119]  Bradner, S., "Key words for use in RFCs to Indicate
              Requirement Levels", BCP 14, RFC 2119, March 1997.

11.2.  Informative References

   [I-D.babiarz-pcn-3sm]
              Babiarz, J., "Three State PCN Marking",
              draft-babiarz-pcn-3sm-00 (work in progress), July 2007.

   [I-D.briscoe-tsvwg-cl-architecture]
              Briscoe, B., "An edge-to-edge Deployment Model for Pre-
              Congestion Notification: Admission  Control over a
              DiffServ Region", draft-briscoe-tsvwg-cl-architecture-04
              (work in progress), October 2006.

   [I-D.briscoe-tsvwg-cl-phb]
              Briscoe, B., "Pre-Congestion Notification marking",
              draft-briscoe-tsvwg-cl-phb-03 (work in progress),
              October 2006.

   [I-D.briscoe-tsvwg-re-ecn-border-cheat]
              Briscoe, B., "Emulating Border Flow Policing using Re-ECN
              on Bulk Data", draft-briscoe-tsvwg-re-ecn-border-cheat-01
              (work in progress), June 2006.

   [I-D.briscoe-tsvwg-re-ecn-tcp]
              Briscoe, B., "Re-ECN: Adding Accountability for Causing
              Congestion to TCP/IP", draft-briscoe-tsvwg-re-ecn-tcp-04
              (work in progress), July 2007.



Charny, et al.            Expires May 21, 2008                 [Page 57]


Internet-Draft           PCN with Single Marking           November 2007


   [I-D.davie-ecn-mpls]
              Davie, B., "Explicit Congestion Marking in MPLS",
              draft-davie-ecn-mpls-01 (work in progress), October 2006.

   [I-D.eardley-pcn-architecture]
              Eardley, P., "Pre-Congestion Notification Architecture",
              draft-eardley-pcn-architecture-00 (work in progress),
              June 2007.

   [I-D.lefaucheur-emergency-rsvp]
              Faucheur, F., "RSVP Extensions for Emergency Services",
              draft-lefaucheur-emergency-rsvp-02 (work in progress),
              June 2006.

   [I-D.westberg-pcn-load-control]
              Westberg, L., "LC-PCN: The Load Control PCN Solution",
              draft-westberg-pcn-load-control-02 (work in progress),
              November 2007.

   [I-D.zhang-pcn-performance-evaluation]
              Zhang, X., "Performance Evaluation of CL-PHB Admission and
              Termination Algorithms",
              draft-zhang-pcn-performance-evaluation-02 (work in
              progress), July 2007.

11.3.  References

   [Jamin]    "A Measurement-based Admission Control Algorithm for
              Integrated Services Packet Networks", 1997.

   [Menth]    "PCN-Based Resilient Network Admission Control: The Impact
              of a Single Bit", 2007.


Authors' Addresses

   Anna Charny
   Cisco Systems, Inc.
   1414 Mass. Ave.
   Boxborough, MA  01719
   USA

   Email: acharny@cisco.com








Charny, et al.            Expires May 21, 2008                 [Page 58]


Internet-Draft           PCN with Single Marking           November 2007


   Xinyang (Joy) Zhang
   Cisco Systems, Inc. and Cornell University
   1414 Mass. Ave.
   Boxborough, MA  01719
   USA

   Email: joyzhang@cisco.com


   Francois Le Faucheur
   Cisco Systems, Inc.
   Village d'Entreprise Green Side - Batiment T3 ,
   400 Avenue de Roumanille, 06410 Biot Sophia-Antipolis,
   France

   Email: flefauch@cisco.com


   Vassilis Liatsos
   Cisco Systems, Inc.
   1414 Mass. Ave.
   Boxborough, MA  01719
   USA

   Email: vliatsos@cisco.com


























Charny, et al.            Expires May 21, 2008                 [Page 59]


Internet-Draft           PCN with Single Marking           November 2007


Full Copyright Statement

   Copyright (C) The IETF Trust (2007).

   This document is subject to the rights, licenses and restrictions
   contained in BCP 78, and except as set forth therein, the authors
   retain all their rights.

   This document and the information contained herein are provided on an
   "AS IS" basis and THE CONTRIBUTOR, THE ORGANIZATION HE/SHE REPRESENTS
   OR IS SPONSORED BY (IF ANY), THE INTERNET SOCIETY, THE IETF TRUST AND
   THE INTERNET ENGINEERING TASK FORCE DISCLAIM ALL WARRANTIES, EXPRESS
   OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF
   THE INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED
   WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE.


Intellectual Property

   The IETF takes no position regarding the validity or scope of any
   Intellectual Property Rights or other rights that might be claimed to
   pertain to the implementation or use of the technology described in
   this document or the extent to which any license under such rights
   might or might not be available; nor does it represent that it has
   made any independent effort to identify any such rights.  Information
   on the procedures with respect to rights in RFC documents can be
   found in BCP 78 and BCP 79.

   Copies of IPR disclosures made to the IETF Secretariat and any
   assurances of licenses to be made available, or the result of an
   attempt made to obtain a general license or permission for the use of
   such proprietary rights by implementers or users of this
   specification can be obtained from the IETF on-line IPR repository at
   http://www.ietf.org/ipr.

   The IETF invites any interested party to bring to its attention any
   copyrights, patents or patent applications, or other proprietary
   rights that may cover technology that may be required to implement
   this standard.  Please address the information to the IETF at
   ietf-ipr@ietf.org.


Acknowledgment

   Funding for the RFC Editor function is provided by the IETF
   Administrative Support Activity (IASA).





Charny, et al.            Expires May 21, 2008                 [Page 60]


Html markup produced by rfcmarkup 1.129c, available from https://tools.ietf.org/tools/rfcmarkup/