draft-ietf-forces-framework-13.txt   rfc3746.txt 
Internet Draft L. Yang Network Working Group L. Yang
Expiration: July 2004 Intel Corp. Request for Comments: 3746 Intel Corp.
File: draft-ietf-forces-framework-13.txt R. Dantu Category: Informational R. Dantu
Working Group: ForCES Univ. of North Texas Univ. of North Texas
T. Anderson T. Anderson
Intel Corp. Intel Corp.
R. Gopal R. Gopal
Nokia Nokia
April 2004
January 2004
Forwarding and Control Element Separation (ForCES) Framework Forwarding and Control Element Separation (ForCES) Framework
draft-ietf-forces-framework-13.txt
Status of this Memo Status of this Memo
This document is an Internet-Draft and is in full conformance with This memo provides information for the Internet community. It does
all provisions of Section 10 of RFC2026. Internet-Drafts are not specify an Internet standard of any kind. Distribution of this
working documents of the Internet Engineering Task Force (IETF), memo is unlimited.
its areas, and its working groups. Note that other groups may also
distribute working documents as Internet-Drafts.
Internet-Drafts are draft documents valid for a maximum of six
months and may be updated, replaced, or obsoleted by other
documents at any time. It is inappropriate to use Internet-Drafts
as reference material or to cite them other than as ``work in
progress.''
The list of current Internet-Drafts can be accessed at
http://www.ietf.org/ietf/1id-abstracts.txt.
The list of Internet-Draft Shadow Directories can be accessed at
http://www.ietf.org/shadow.html.
Copyright Notice Copyright Notice
Copyright (C) The Internet Society (2003). All Rights Reserved. Copyright (C) The Internet Society (2004). All Rights Reserved.
Abstract Abstract
This document defines the architectural framework for the ForCES This document defines the architectural framework for the ForCES
(Forwarding and Control Element Separation) network elements, and (Forwarding and Control Element Separation) network elements, and
identifies the associated entities and the interactions among them. identifies the associated entities and their interactions.
Table of Contents Table of Contents
1. Definitions...................................................3
1.1. Conventions used in this document........................3 1. Definitions. . . . . . . . . . . . . . . . . . . . . . . . . . 2
1.2. Terminologies............................................3 1.1. Conventions used in this document . . . . . . . . . . . . 2
1.2. Terminologies . . . . . . . . . . . . . . . . . . . . . . 3
2. Introduction to Forwarding and Control Element Separation 2. Introduction to Forwarding and Control Element Separation
(ForCES).........................................................5 (ForCES) . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
3. Architecture..................................................9 3. Architecture . . . . . . . . . . . . . . . . . . . . . . . . . 8
3.1. Control Elements and Fr Reference Point.................10 3.1. Control Elements and Fr Reference Point . . . . . . . . . 10
3.2. Forwarding Elements and Fi reference point..............11 3.2. Forwarding Elements and Fi reference point. . . . . . . . 11
3.3. CE Managers.............................................15 3.3. CE Managers . . . . . . . . . . . . . . . . . . . . . . . 14
3.4. FE Managers.............................................15 3.4. FE Managers . . . . . . . . . . . . . . . . . . . . . . . 14
4. Operational Phases...........................................15 4. Operational Phases . . . . . . . . . . . . . . . . . . . . . . 15
4.1. Pre-association Phase...................................15 4.1. Pre-association Phase . . . . . . . . . . . . . . . . . . 15
4.1.1. Fl Reference Point.................................15 4.1.1. Fl Reference Point . . . . . . . . . . . . . . . . 15
4.1.2. Ff Reference Point.................................16 4.1.2. Ff Reference Point . . . . . . . . . . . . . . . . 16
4.1.3. Fc Reference Point.................................17 4.1.3. Fc Reference Point . . . . . . . . . . . . . . . . 17
4.2. Post-association Phase and Fp reference point...........17 4.2. Post-association Phase and Fp reference point . . . . . . 17
4.2.1. Proximity and Interconnect between CEs and FEs.....18 4.2.1. Proximity and Interconnect between CEs and FEs . . 18
4.2.2. Association Establishment..........................18 4.2.2. Association Establishment. . . . . . . . . . . . . 18
4.2.3. Steady-state Communication.........................20 4.2.3. Steady-state Communication . . . . . . . . . . . . 19
4.2.4. Data Packets across Fp reference point.............20 4.2.4. Data Packets across Fp reference point . . . . . . 21
4.2.5. Proxy FE...........................................22 4.2.5. Proxy FE . . . . . . . . . . . . . . . . . . . . . 22
4.3. Association Re-establishment............................22 4.3. Association Re-establishment. . . . . . . . . . . . . . . 22
4.3.1. CE graceful restart................................22 4.3.1. CE graceful restart. . . . . . . . . . . . . . . . 23
4.3.2. FE restart.........................................24 4.3.2. FE restart . . . . . . . . . . . . . . . . . . . . 24
5. Applicability to RFC1812.....................................25 5. Applicability to RFC 1812. . . . . . . . . . . . . . . . . . . 25
5.1. General Router Requirements.............................25 5.1. General Router Requirements . . . . . . . . . . . . . . . 25
5.2. Link Layer..............................................26 5.2. Link Layer. . . . . . . . . . . . . . . . . . . . . . . . 26
5.3. Internet Layer Protocols................................27 5.3. Internet Layer Protocols. . . . . . . . . . . . . . . . . 27
5.4. Internet Layer Forwarding...............................28 5.4. Internet Layer Forwarding . . . . . . . . . . . . . . . . 27
5.5. Transport Layer.........................................29 5.5. Transport Layer . . . . . . . . . . . . . . . . . . . . . 28
5.6. Application Layer -- Routing Protocols..................29 5.6. Application Layer -- Routing Protocols. . . . . . . . . . 29
5.7. Application Layer -- Network Management Protocol........29 5.7. Application Layer -- Network Management Protocol. . . . . 29
6. Summary......................................................30 6. Summary. . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
7. Acknowledgements.............................................30 7. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . 30
8. Security Considerations......................................30 8. Security Considerations. . . . . . . . . . . . . . . . . . . . 30
8.1. Analysis of Potential Threats Introduced by ForCES......31 8.1. Analysis of Potential Threats Introduced by ForCES. . . . 31
8.1.1. "Join" or "Remove" Message Flooding on CEs.........31 8.1.1. "Join" or "Remove" Message Flooding on CEs . . . . 31
8.1.2. Impersonation Attack...............................32 8.1.2. Impersonation Attack . . . . . . . . . . . . . . . 31
8.1.3. Replay Attack......................................32 8.1.3. Replay Attack. . . . . . . . . . . . . . . . . . . 31
8.1.4. Attack during Fail Over............................32 8.1.4. Attack during Fail Over. . . . . . . . . . . . . . 32
8.1.5. Data Integrity.....................................32 8.1.5. Data Integrity . . . . . . . . . . . . . . . . . . 32
8.1.6. Data Confidentiality...............................33 8.1.6. Data Confidentiality . . . . . . . . . . . . . . . 32
8.1.7. Sharing security parameters........................33 8.1.7. Sharing security parameters. . . . . . . . . . . . 33
8.1.8. Denial of Service Attack via External Interface....33 8.1.8. Denial of Service Attack via External Interface. . 33
8.2. Security Recommendations for ForCES.....................34 8.2. Security Recommendations for ForCES . . . . . . . . . . . 33
8.2.1. Using TLS with ForCES..............................34 8.2.1. Using TLS with ForCES. . . . . . . . . . . . . . . 34
8.2.2. Using IPsec with ForCES............................35 8.2.2. Using IPsec with ForCES. . . . . . . . . . . . . . 35
9. Normative References.........................................37 9. References . . . . . . . . . . . . . . . . . . . . . . . . . . 37
10. Informative References......................................37 9.1. Normative References. . . . . . . . . . . . . . . . . . . 37
11. Authors' Addresses..........................................38 9.2. Informative References. . . . . . . . . . . . . . . . . . 37
12. Intellectual Property Right.................................39 10. Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . 39
13. Full Copyright Statement....................................39 11. Full Copyright Statement . . . . . . . . . . . . . . . . . . . 40
1. Definitions 1. Definitions
1.1. Conventions used in this document 1.1. Conventions used in this document
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
"SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
this document are to be interpreted as described in RFC 2119. document are to be interpreted as described in BCP 14, RFC 2119 [1].
1.2. Terminologies 1.2. Terminologies
A set of terminology associated with the ForCES requirements is A set of terminology associated with the ForCES requirements is
defined in [3] and we only include the definitions that are most defined in [4] and we only include the definitions that are most
relevant to this document here. relevant to this document here.
Addressable Entity (AE) - An entity that is directly addressable Addressable Entity (AE) - An entity that is directly addressable
given some interconnect technology. For example, on IP networks, given some interconnect technology. For example, on IP networks, it
it is a device to which we can communicate using an IP address; on is a device to which we can communicate using an IP address; on a
a switch fabric, it is a device to which we can communicate using a switch fabric, it is a device to which we can communicate using a
switch fabric port number. switch fabric port number.
Physical Forwarding Element (PFE) - An AE that includes hardware Physical Forwarding Element (PFE) - An AE that includes hardware used
used to provide per-packet processing and handling. This hardware to provide per-packet processing and handling. This hardware may
may consist of (but is not limited to) network processors, ASICs consist of (but is not limited to) network processors, ASICs
(Application-Specific Integrated Circuits), or general purpose (Application-Specific Integrated Circuits), or general purpose
processors, installed on line cards, daughter boards, mezzanine processors, installed on line cards, daughter boards, mezzanine
cards, or in stand-alone boxes. cards, or in stand-alone boxes.
PFE Partition - A logical partition of a PFE consisting of some PFE Partition - A logical partition of a PFE consisting of some
subset of each of the resources (e.g., ports, memory, forwarding subset of each of the resources (e.g., ports, memory, forwarding
table entries) available on the PFE. This concept is analogous to table entries) available on the PFE. This concept is analogous to
that of the resources assigned to a virtual switching element as that of the resources assigned to a virtual switching element as
described in [8]. described in [9].
Physical Control Element (PCE) - An AE that includes hardware used Physical Control Element (PCE) - An AE that includes hardware used to
to provide control functionality. This hardware typically includes provide control functionality. This hardware typically includes a
a general purpose processor. general purpose processor.
PCE Partition - A logical partition of a PCE consisting of some PCE Partition - A logical partition of a PCE consisting of some
subset of each of the resources available on the PCE. subset of each of the resources available on the PCE.
Forwarding Element (FE) - A logical entity that implements the Forwarding Element (FE) - A logical entity that implements the ForCES
ForCES Protocol. FEs use the underlying hardware to provide per- Protocol. FEs use the underlying hardware to provide per-packet
packet processing and handling as directed by a CE via the ForCES processing and handling as directed by a CE via the ForCES Protocol.
Protocol. FEs may happen to be a single blade (or PFE), a FEs may happen to be a single blade (or PFE), a partition of a PFE,
partition of a PFE or multiple PFEs. or multiple PFEs.
Control Element (CE) - A logical entity that implements the ForCES Control Element (CE) - A logical entity that implements the ForCES
Protocol and uses it to instruct one or more FEs how to process Protocol and uses it to instruct one or more FEs on how to process
packets. CEs handle functionality such as the execution of control packets. CEs handle functionality such as the execution of control
and signaling protocols. CEs may consist of PCE partitions or and signaling protocols. CEs may consist of PCE partitions or whole
whole PCEs. PCEs.
ForCES Network Element (NE) - An entity composed of one or more CEs ForCES Network Element (NE) - An entity composed of one or more CEs
and one or more FEs. To entities outside an NE, the NE represents and one or more FEs. An NE usually hides its internal organization
a single point of management. Similarly, an NE usually hides its from external entities and represents a single point of management to
internal organization from external entities. entities outside the NE.
Pre-association Phase - The period of time during which an FE Pre-association Phase - The period of time during which an FE Manager
Manager (see below) and a CE Manager (see below) are determining (see below) and a CE Manager (see below) are determining whether an
whether an FE and a CE should be part of the same network element. FE and a CE should be part of the same network element. It is
It is possible for some elements of the NE to be in pre-association possible for some elements of the NE to be in pre-association phase
phase while other elements are in the post-association phase. while other elements are in the post-association phase.
Post-association Phase - The period of time during which an FE does Post-association Phase - The period of time during which an FE knows
know which CE is to control it and vice versa, including the time which CE is to control it and vice versa, including the time during
during which the CE and FE are establishing communication with one which the CE and FE are establishing communication with one another.
another.
ForCES Protocol - While there may be multiple protocols used within ForCES Protocol - While there may be multiple protocols used within
the overall ForCES architecture, the term "ForCES Protocol" refers the overall ForCES architecture, the term "ForCES Protocol" refers
only to the ForCES post-association phase protocol (see below). only to the ForCES post-association phase protocol (see below).
ForCES Post-Association Phase Protocol - The protocol used for ForCES Post-Association Phase Protocol - The protocol used for post-
post-association phase communication between CEs and FEs. This association phase communication between CEs and FEs. This protocol
protocol does not apply to CE-to-CE communication, FE-to-FE does not apply to CE-to-CE communication, FE-to-FE communication, or
communication, nor to communication between FE and CE managers. to communication between FE and CE managers. The ForCES Protocol is
The ForCES Protocol is a master-slave protocol in which FEs are a master-slave protocol in which FEs are slaves and CEs are masters.
slaves and CEs are masters. This protocol includes both the This protocol includes both the management of the communication
management of the communication channel (e.g., connection channel (e.g., connection establishment, heartbeats) and the control
establishment, heartbeats) and the control messages themselves. messages themselves. This protocol could be a single protocol or
This protocol could be a single protocol or could consist of could consist of multiple protocols working together, and may be
multiple protocols working together, and may be unicast based or unicast or multicast based. A separate protocol document will
multicast based. A separate protocol document will specify this specify this information.
information.
FE Manager - A logical entity that operates in the pre-association FE Manager - A logical entity that operates in the pre-association
phase and is responsible for determining to which CE(s) an FE phase and is responsible for determining to which CE(s) an FE should
should communicate. This process is called CE discovery and may communicate. This process is called CE discovery and may involve the
involve the FE manager learning the capabilities of available CEs. FE manager learning the capabilities of available CEs. An FE manager
An FE manager may use anything from a static configuration to a may use anything from a static configuration to a pre-association
pre-association phase protocol (see below) to determine which CE(s) phase protocol (see below) to determine which CE(s) to use; however,
to use; however, this is currently out of scope. Being a logical this is currently out of scope. Being a logical entity, an FE
entity, an FE manager might be physically combined with any of the manager might be physically combined with any of the other logical
other logical entities mentioned in this section. entities mentioned in this section.
CE Manager - A logical entity that operates in the pre-association CE Manager - A logical entity that operates in the pre-association
phase and is responsible for determining to which FE(s) a CE should phase and is responsible for determining to which FE(s) a CE should
communicate. This process is called FE discovery and may involve communicate. This process is called FE discovery and may involve the
the CE manager learning the capabilities of available FEs. A CE CE manager learning the capabilities of available FEs. A CE manager
manager may use anything from a static configuration to a pre- may use anything from a static configuration to a pre-association
association phase protocol (see below) to determine which FE to phase protocol (see below) to determine which FE to use; however,
use, however this is currently out of scope. Being a logical this is currently out of scope. Being a logical entity, a CE manager
entity, a CE manager might be physically combined with any of the might be physically combined with any of the other logical entities
other logical entities mentioned in this section. mentioned in this section.
Pre-association Phase Protocol - A protocol between FE managers and Pre-association Phase Protocol - A protocol between FE managers and
CE managers that is used to determine which CEs or FEs to use. A CE managers that is used to determine which CEs or FEs to use. A
pre-association phase protocol may include a CE and/or FE pre-association phase protocol may include a CE and/or FE capability
capability discovery mechanism. Note that this capability discovery mechanism. Note that this capability discovery process is
discovery process is wholly separate from (and does not replace) wholly separate from (and does not replace) that used within the
that used within the ForCES Protocol. However, the two capability ForCES Protocol. However, the two capability discovery mechanisms
discovery mechanisms may utilize the same FE model. may utilize the same FE model.
FE Model - A model that describes the logical processing functions FE Model - A model that describes the logical processing functions of
of an FE. an FE.
ForCES Protocol Element - An FE or CE. ForCES Protocol Element - An FE or CE.
Intra-FE topology - Representation of how a single FE is realized Intra-FE topology - Representation of how a single FE is realized by
by combining possibly multiple logical functional blocks along combining possibly multiple logical functional blocks along multiple
multiple data path. This is defined by the FE model. data paths. This is defined by the FE model.
FE Topology - Representation of how the multiple FEs in a single NE FE Topology - Representation of how the multiple FEs in a single NE
are interconnected. Sometimes it is called inter-FE topology, to are interconnected. Sometimes it is called inter-FE topology, to be
be distinguished from intra-FE topology used by the FE model. distinguished from intra-FE topology used by the FE model.
Inter-FE topology - see FE Topology. Inter-FE topology - See FE Topology.
2. Introduction to Forwarding and Control Element Separation (ForCES) 2. Introduction to Forwarding and Control Element Separation (ForCES)
An IP network element (NE) appears to external entities as a An IP network element (NE) appears to external entities as a
monolithic piece of network equipment, e.g., a router, NAT, monolithic piece of network equipment, e.g., a router, NAT, firewall,
firewall, or load balancer. Internally, however, an IP network or load balancer. Internally, however, an IP network element (NE)
element (NE) (such as a router) is composed of numerous logically (such as a router) is composed of numerous logically separated
separated entities that cooperate to provide a given functionality entities that cooperate to provide a given functionality (such as
(such as routing). Two types of network element components exist: routing). Two types of network element components exist: control
control element (CE) in control plane and forwarding element (FE) element (CE) in control plane and forwarding element (FE) in
in forwarding plane (or data plane). Forwarding elements typically forwarding plane (or data plane). Forwarding elements are typically
are ASIC, network-processor, or general-purpose processor-based ASIC, network-processor, or general-purpose processor-based devices
devices that handle data path operations for each packet. Control that handle data path operations for each packet. Control elements
elements are typically based on general-purpose processors that are typically based on general-purpose processors that provide
provide control functionality like routing and signaling protocols. control functionality, like routing and signaling protocols.
ForCES aims to define a framework and associated protocol(s) to ForCES aims to define a framework and associated protocol(s) to
standardize information exchange between the control and forwarding standardize information exchange between the control and forwarding
plane. Having standard mechanisms allows CEs and FEs to become plane. Having standard mechanisms allows CEs and FEs to become
physically separated standard components. This physical separation physically separated standard components. This physical separation
accrues several benefits to the ForCES architecture. Separate accrues several benefits to the ForCES architecture. Separate
components would allow component vendors to specialize in one components would allow component vendors to specialize in one
component without having to become experts in all components. component without having to become experts in all components.
Standard protocol also allows the CEs and FEs from different Standard protocol also allows the CEs and FEs from different
component vendors to interoperate with each other and hence it component vendors to interoperate with each other and hence it
becomes possible for system vendors to integrate together the CEs becomes possible for system vendors to integrate together the CEs and
and FEs from different component suppliers. This interoperability FEs from different component suppliers. This interoperability
translates into more design choices and flexibility for the system translates into increased design choices and flexibility for the
vendors. Overall, ForCES will enable rapid innovation in both the system vendors. Overall, ForCES will enable rapid innovation in both
control and forwarding planes while maintaining interoperability. the control and forwarding planes while maintaining interoperability.
Scalability is also easily provided by this architecture in that Scalability is also easily provided by this architecture in that
additional forwarding or control capacity can be added to existing additional forwarding or control capacity can be added to existing
network elements without the need for forklift upgrades. network elements without the need for forklift upgrades.
------------------------- ------------------------- ------------------------- -------------------------
| Control Blade A | | Control Blade B | | Control Blade A | | Control Blade B |
| (CE) | | (CE) | | (CE) | | (CE) |
------------------------- ------------------------- ------------------------- -------------------------
^ | ^ | ^ | ^ |
| | | | | | | |
skipping to change at page 7, line 9 skipping to change at page 6, line 36
|Router | |Router | |Router | |Router | |Router | |Router |
|Blade #1 | |Blade #2 | |Blade #N | |Blade #1 | |Blade #2 | |Blade #N |
| (FE) | | (FE) | | (FE) | | (FE) | | (FE) | | (FE) |
------------ ------------ ------------ ------------ ------------ ------------
^ | ^ | ^ | ^ | ^ | ^ |
| | | | . . . | | | | | | . . . | |
| V | V | V | V | V | V
Figure 1. A router configuration example with separate blades. Figure 1. A router configuration example with separate blades.
One example of such physical separation is at the blade level. One example of such physical separation is at the blade level. Figure
Figure 1 shows such an example configuration of a router, with two 1 shows such an example configuration of a router, with two control
control blades and multiple forwarding blades, all interconnected blades and multiple forwarding blades, all interconnected into a
into a switch fabric backplane. In such a chassis configuration, switch fabric backplane. In such a chassis configuration, the
the control blades are the CEs while the router blades are FEs, and control blades are the CEs while the router blades are the FEs, and
the switch fabric backplane provides the physical interconnect for the switch fabric backplane provides the physical interconnect for
all the blades. Control blade A may be the primary CE while all the blades. Control blade A may be the primary CE while control
control blade B may be the backup CE providing redundancy. It is blade B may be the backup CE providing redundancy. It is also
also possible to have a redundant switch fabric for high possible to have a redundant switch fabric for high availability
availability support. Routers today with this kind of support. Routers today with this kind of configuration use
configuration use proprietary interfaces for messaging between CEs proprietary interfaces for messaging between CEs and FEs. The goal
and FEs. The goal of ForCES is to replace such proprietary of ForCES is to replace such proprietary interfaces with a standard
interfaces with a standard protocol. With a standard protocol like protocol. With a standard protocol like ForCES implemented on all
ForCES implemented on all blades, it becomes possible for control blades, it becomes possible for control blades from vendor X and
blades from vendor X and forwarding blades from vendor Y to work forwarding blades from vendor Y to work seamlessly together in one
seamlessly together in one chassis. chassis.
------- ------- ------- -------
| CE1 | | CE2 | | CE1 | | CE2 |
------- ------- ------- -------
^ ^ ^ ^
| | | |
V V V V
============================================ Ethernet ============================================ Ethernet
^ ^ . . . ^ ^ ^ . . . ^
| | | | | |
skipping to change at page 7, line 46 skipping to change at page 7, line 25
------- ------- -------- ------- ------- --------
| FE#1| | FE#2| | FE#n | | FE#1| | FE#2| | FE#n |
------- ------- -------- ------- ------- --------
^ | ^ | ^ | ^ | ^ | ^ |
| | | | | | | | | | | |
| V | V | V | V | V | V
Figure 2. A router configuration example with separate boxes. Figure 2. A router configuration example with separate boxes.
Another level of physical separation between the CEs and FEs can be Another level of physical separation between the CEs and FEs can be
at the box level. In such configuration, all the CEs and FEs are at the box level. In such a configuration, all the CEs and FEs are
physically separated boxes, interconnected with some kind of high physically separated boxes, interconnected with some kind of high
speed LAN connection (like Gigabit Ethernet). These separated CEs speed LAN connection (like Gigabit Ethernet). These separated CEs
and FEs are only one hop away from each other within a local area and FEs are only one hop away from each other within a local area
network. The CEs and FEs communicate to each other by running network. The CEs and FEs communicate to each other by running
ForCES, and the collection of these CEs and FEs together become one ForCES, and the collection of these CEs and FEs together become one
routing unit to the external world. Figure 2 shows such an example. routing unit to the external world. Figure 2 shows such an example.
In both examples shown here, the same physical interconnect is used In both examples shown here, the same physical interconnect is used
for both CE-to-FE and FE-to-FE communication. However, that does for both CE-to-FE and FE-to-FE communication. However, that does not
not have to be the case. One reason to use different interconnects have to be the case. One reason to use different interconnects is
is that CE-to-FE interconnect does not have to be as fast as the that the CE-to-FE interconnect does not have to be as fast as the
FE-to-FE interconnect, so the more expensive fast connections can FE-to-FE interconnect, so the more faster and more expensive
be saved for FE-to-FE. The separate interconnects may also provide connections can be saved for FE-to-FE. The separate interconnects
reliability and redundancy benefits for the NE. may also provide reliability and redundancy benefits for the NE.
Some examples of control functions that can be implemented in the Some examples of control functions that can be implemented in the CE
CE include routing protocols like RIP, OSPF and BGP, control and include routing protocols like RIP, OSPF, and BGP, control and
signaling protocols like RSVP (Resource Reservation Protocol), LDP signaling protocols like RSVP (Resource Reservation Protocol), LDP
(Label Distribution Protocol) for MPLS, etc. Examples of (Label Distribution Protocol) for MPLS, etc. Examples of forwarding
forwarding functions in the FE include LPM (longest prefix match) functions in the FE include LPM (longest prefix match) forwarder,
forwarder, classifiers, traffic shaper, meter, NAT (Network Address classifiers, traffic shaper, meter, NAT (Network Address
Translators), etc. Figure 3 provides example functions in both CE Translators), etc. Figure 3 provides example functions in both CE
and FE. Any given NE may contain one or many of these CE and FE and FE. Any given NE may contain one or many of these CE and FE
functions in it. The diagram also shows that ForCES Protocol is functions in it. The diagram also shows that the ForCES Protocol is
used to transport both the control messages for ForCES itself and used to transport both the control messages for ForCES itself and the
the data packets that are originated/destined from/to the control data packets that are originated/destined from/to the control
functions in CE (e.g., routing packets). Section 4.2.4 provides functions in the CE (e.g., routing packets). Section 4.2.4 provides
more detail on this. more detail on this.
------------------------------------------------- -------------------------------------------------
| | | | | | | | | | | | | |
|OSPF |RIP |BGP |RSVP |LDP |. . . | |OSPF |RIP |BGP |RSVP |LDP |. . . |
| | | | | | | | | | | | | |
------------------------------------------------- -------------------------------------------------
| ForCES Interface | | ForCES Interface |
------------------------------------------------- -------------------------------------------------
^ ^ ^ ^
skipping to change at page 9, line 4 skipping to change at page 8, line 29
v v v v
------------------------------------------------- -------------------------------------------------
| ForCES Interface | | ForCES Interface |
------------------------------------------------- -------------------------------------------------
| | | | | | | | | | | | | |
|LPM Fwd|Meter |Shaper |NAT |Classi-|. . . | |LPM Fwd|Meter |Shaper |NAT |Classi-|. . . |
| | | | |fier | | | | | | |fier | |
------------------------------------------------- -------------------------------------------------
| FE resources | | FE resources |
------------------------------------------------- -------------------------------------------------
Figure 3. Examples of CE and FE functions
Figure 3. Examples of CE and FE functions.
A set of requirements for control and forwarding separation is A set of requirements for control and forwarding separation is
identified in [3]. This document describes a ForCES architecture identified in [4]. This document describes a ForCES architecture
that satisfies the architectural requirements of that document and that satisfies the architectural requirements of [4] and defines a
defines a framework for ForCES network elements and the associated framework for ForCES network elements and the associated entities to
entities to facilitate protocol definition. Whenever necessary, facilitate protocol definition. Whenever necessary, this document
this document uses many examples to illustrate the issues and/or uses many examples to illustrate the issues and/or possible solutions
possible solutions in ForCES. These examples are intended to be in ForCES. These examples are intended to be just examples, and
just examples, and should not be taken as the only or definite ways should not be taken as the only or definite ways of doing certain
of doing certain things. It is expected that separate document things. It is expected that a separate document will be produced by
will be produced by the ForCES working group to specify the ForCES the ForCES working group to specify the ForCES Protocol.
Protocol.
3. Architecture 3. Architecture
This section defines the ForCES architectural framework and the This section defines the ForCES architectural framework and the
associated logical components. This ForCES framework defines associated logical components. This ForCES framework defines
components of ForCES NEs including several ancillary components. components of ForCES NEs, including several ancillary components.
These components may be connected in different kinds of topologies These components may be connected in different kinds of topologies
for flexible packet processing. for flexible packet processing.
--------------------------------------- ---------------------------------------
| ForCES Network Element | | ForCES Network Element |
-------------- Fc | -------------- -------------- | -------------- Fc | -------------- -------------- |
| CE Manager |---------+-| CE 1 |------| CE 2 | | | CE Manager |---------+-| CE 1 |------| CE 2 | |
-------------- | | | Fr | | | -------------- | | | Fr | | |
| | -------------- -------------- | | | -------------- -------------- |
| Fl | | | Fp / | | Fl | | | Fp / |
skipping to change at page 10, line 15 skipping to change at page 9, line 39
Fr: CE-CE interface Fr: CE-CE interface
Fc: Interface between the CE Manager and a CE Fc: Interface between the CE Manager and a CE
Ff: Interface between the FE Manager and an FE Ff: Interface between the FE Manager and an FE
Fl: Interface between the CE Manager and the FE Manager Fl: Interface between the CE Manager and the FE Manager
Fi/f: FE external interface Fi/f: FE external interface
Figure 4. ForCES Architectural Diagram Figure 4. ForCES Architectural Diagram
The diagram in Figure 4 shows the logical components of the ForCES The diagram in Figure 4 shows the logical components of the ForCES
architecture and their relationships. There are two kinds of architecture and their relationships. There are two kinds of
components inside a ForCES network element: control element (CE) components inside a ForCES network element: control element (CE) and
and forwarding element (FE). The framework allows multiple forwarding element (FE). The framework allows multiple instances of
instances of CE and FE inside one NE. Each FE contains one or more CE and FE inside one NE. Each FE contains one or more physical media
physical media interfaces for receiving and transmitting packets interfaces for receiving and transmitting packets from/to the
from/to the external world. The aggregation of these FE interfaces external world. The aggregation of these FE interfaces becomes the
becomes the NE's external interfaces. In addition to the external NE's external interfaces. In addition to the external interfaces,
interfaces, there must also exist some kind of interconnect within there must also exist some kind of interconnect within the NE so that
the NE so that the CE and FE can communicate with each other, and the CE and FE can communicate with each other, and one FE can forward
one FE can forward packets to another FE. The diagram also shows packets to another FE. The diagram also shows two entities outside
two entities outside of the ForCES NE: CE Manager and FE Manager. of the ForCES NE: CE Manager and FE Manager. These two ancillary
These two ancillary entities provide configuration to the entities provide configuration to the corresponding CE or FE in the
corresponding CE or FE in the pre-association phase (see Section pre-association phase (see Section 4.1).
4.1).
For convenience, the logical interactions between these components For convenience, the logical interactions between these components
are labeled by reference points Fp, Fc, Ff, Fr, Fl, and Fi, as are labeled by reference points Fp, Fc, Ff, Fr, Fl, and Fi, as shown
shown in Figure 4. The FE external interfaces are labeled as Fi/f. in Figure 4. The FE external interfaces are labeled as Fi/f. More
More detail is provided in Section 3 and 4 for each of these detail is provided in Section 3 and 4 for each of these reference
reference points. All these reference points are important in points. All these reference points are important in understanding
understanding the ForCES architecture, however, the ForCES Protocol the ForCES architecture, however, the ForCES Protocol is only defined
is only defined over one reference point -- Fp. over one reference point -- Fp.
The interface between two ForCES NEs is identical to the interface The interface between two ForCES NEs is identical to the interface
between two conventional routers and these two NEs exchange the between two conventional routers and these two NEs exchange the
protocol packets through the external interfaces at Fi/f. ForCES protocol packets through the external interfaces at Fi/f. ForCES NEs
NEs connect to existing routers transparently. connect to existing routers transparently.
3.1. Control Elements and Fr Reference Point 3.1. Control Elements and Fr Reference Point
It is not necessary to define any protocols across the Fr reference It is not necessary to define any protocols across the Fr reference
point to enable control and forwarding separation for simple point to enable control and forwarding separation for simple
configurations like single CE and multiple FEs. However, this configurations like single CE and multiple FEs. However, this
architecture permits multiple CEs to be present in a network architecture permits multiple CEs to be present in a network element.
element. In cases where an implementation uses multiple CEs, the In cases where an implementation uses multiple CEs, the invariant
invariant that the CEs and FEs together appear as a single NE must that the CEs and FEs together appear as a single NE must be
be maintained. maintained.
Multiple CEs may be used for redundancy, load sharing, distributed Multiple CEs may be used for redundancy, load sharing, distributed
control, or other purposes. Redundancy is the case where one or control, or other purposes. Redundancy is the case where one or more
more CEs are prepared to take over should an active CE fail. Load CEs are prepared to take over should an active CE fail. Load sharing
sharing is the case where two or more CEs are concurrently active is the case where two or more CEs are concurrently active and any
and any request that can be serviced by one of the CEs can also be request that can be serviced by one of the CEs can also be serviced
serviced by any of the other CEs. For both redundancy and load by any of the other CEs. For both redundancy and load sharing, the
sharing, the CEs involved are equivalently capable. The only CEs involved are equivalently capable. The only difference between
difference between these two cases is in terms of how many active these two cases is in terms of how many active CEs there are
CEs there are. Distributed control is the case where two or more simultaneously. Distributed control is the case where two or more
CEs are concurrently active but certain requests can only be CEs are concurrently active but certain requests can only be serviced
serviced by certain CEs. by certain CEs.
When multiple CEs are employed in a ForCES NE, their internal When multiple CEs are employed in a ForCES NE, their internal
organization is considered an implementation issue that is beyond organization is considered an implementation issue that is beyond the
the scope of ForCES. CEs are wholly responsible for coordinating scope of ForCES. CEs are wholly responsible for coordinating amongst
amongst themselves via the Fr reference point to provide themselves via the Fr reference point to provide consistency and
consistency and synchronization. However, ForCES does not define synchronization. However, ForCES does not define the implementation
the implementation or protocols used between CEs, nor does it or protocols used between CEs, nor does it define how to distribute
define how to distribute functionality among CEs. Nevertheless, functionality among CEs. Nevertheless, ForCES will support
ForCES will support mechanisms for CE redundancy or fail over, and mechanisms for CE redundancy or fail over, and it is expected that
it is expected that vendors will provide redundancy or fail over vendors will provide redundancy or fail over solutions within this
solutions within this framework. framework.
3.2. Forwarding Elements and Fi reference point 3.2. Forwarding Elements and Fi reference point
An FE is a logical entity that implements the ForCES Protocol and An FE is a logical entity that implements the ForCES Protocol and
uses the underlying hardware to provide per-packet processing and uses the underlying hardware to provide per-packet processing and
handling as directed by a CE. It is possible to partition one handling as directed by a CE. It is possible to partition one
physical FE into multiple logical FEs. It is also possible for one physical FE into multiple logical FEs. It is also possible for one
FE to use multiple physical FEs. The mapping between physical FE to use multiple physical FEs. The mapping between physical FE(s)
FE(s) and the logical FE(s) is beyond the scope of ForCES. For and logical FE(s) is beyond the scope of ForCES. For example, a
example, a logical partition of a physical FE can be created by logical partition of a physical FE can be created by assigning some
assigning some portion of each of the resources (e.g., ports, portion of each of the resources (e.g., ports, memory, forwarding
memory, forwarding table entries) available on the ForCES physical table entries) available on the ForCES physical FE to each of the
FE to each of the logical FEs. Such concept of FE virtualization logical FEs. Such a concept of FE virtualization is analogous to a
is analogous to a virtual switching element as described in [8]. virtual switching element as described in [9]. If FE virtualization
If FE virtualization occurs only in the pre-association phase, it occurs only in the pre-association phase, it has no impact on ForCES.
has no impact on ForCES. However, if FE virtualization results in However, if FE virtualization results in a resource change taken from
resource change taken from an existing FE (already participating in an existing FE (already participating in ForCES post-association
ForCES post-association phase), the ForCES Protocol needs to be phase), the ForCES Protocol needs to be able to inform the CE of such
able to inform the CE of such change via asynchronous messages (see a change via asynchronous messages (see [4], Section 5, requirement
[3], Section 5, requirement #6). #6).
FEs perform all packet processing functions as directed by CEs.
FEs have no initiative of their own. Instead, FEs are slaves and FEs perform all packet processing functions as directed by CEs. FEs
only do as they are told. FEs may communicate with one or more CEs have no initiative of their own. Instead, FEs are slaves and only do
as they are told. FEs may communicate with one or more CEs
concurrently across reference point Fp. FEs have no notion of CE concurrently across reference point Fp. FEs have no notion of CE
redundancy, load sharing, or distributed control. Instead, FEs redundancy, load sharing, or distributed control. Instead, FEs
accept commands from any CE authorized to control them, and it is accept commands from any CE authorized to control them, and it is up
up to the CEs to coordinate among themselves to achieve redundancy, to the CEs to coordinate among themselves to achieve redundancy, load
load sharing or distributed control. The idea is to keep FEs as sharing, or distributed control. The idea is to keep FEs as simple
simple and dumb as possible so that FEs can focus their resource on and dumb as possible so that FEs can focus their resources on the
the packet processing functions. Unless otherwise configured or packet processing functions. Unless otherwise configured or
determined by a ForCEs Protocol exchange, each FE will process determined by a ForCEs Protocol exchange, each FE will process
authorized incoming commands directed at it as it receives them on authorized incoming commands directed at it as it receives them on a
a first come first serve basis. first come first serve basis.
For example, in Figure 5, FE1 and FE2 can be configured to accept For example, in Figure 5, FE1 and FE2 can be configured to accept
commands from both the primary CE (CE1) and the backup CE (CE2). commands from both the primary CE (CE1) and the backup CE (CE2).
Upon detection of CE1 failure, perhaps across the Fr or Fp Upon detection of CE1 failure, perhaps across the Fr or Fp reference
reference point, CE2 is configured to take over activities of CE1. point, CE2 is configured to take over activities of CE1. This is
This is beyond the scope of ForCES and is not discussed further. beyond the scope of ForCES and is not discussed further.
Distributed control can be achieved in a similar fashion, without Distributed control can be achieved in a similar fashion, without
much intelligence on the part of FEs. For example, FEs can be much intelligence on the part of FEs. For example, FEs can be
configured to detect RSVP and BGP protocol packets, and forward configured to detect RSVP and BGP protocol packets, and forward RSVP
RSVP packets to one CE and BGP packets to another CE. Hence, FEs packets to one CE and BGP packets to another CE. Hence, FEs may need
may need to do packet filtering for forwarding packets to specific to do packet filtering for forwarding packets to specific CEs.
CEs.
------- Fr ------- ------- Fr -------
| CE1 | ------| CE2 | | CE1 | ------| CE2 |
------- ------- ------- -------
| \ / | | \ / |
| \ / | | \ / |
| \ / | | \ / |
| \/Fp | | \/Fp |
| /\ | | /\ |
| / \ | | / \ |
| / \ | | / \ |
------- Fi ------- ------- Fi -------
| FE1 |<----->| FE2 | | FE1 |<----->| FE2 |
------- ------- ------- -------
Figure 5. CE redundancy example. Figure 5. CE redundancy example.
This architecture permits multiple FEs to be present in an NE. [3] This architecture permits multiple FEs to be present in an NE. [4]
dictates that the ForCES Protocol must be able to scale to at least dictates that the ForCES Protocol must be able to scale to at least
hundreds of FEs (see [3] Section 5, requirement #11). Each of hundreds of FEs (see [4] Section 5, requirement #11). Each of these
these FEs may potentially have a different set of packet processing FEs may potentially have a different set of packet processing
functions, with different media interfaces. FEs are responsible functions, with different media interfaces. FEs are responsible for
for basic maintenance of layer-2 connectivity with other FEs and basic maintenance of layer-2 connectivity with other FEs and with
with external entities. Many layer-2 media include sophisticated external entities. Many layer-2 media include sophisticated control
control protocols. The FORCES Protocol (over the Fp reference protocols. The FORCES Protocol (over the Fp reference point) will be
point) will be able to carry messages for such protocols so that, able to carry messages for such protocols so that, in keeping with
in keeping with the dumb FE model, the CE can provide appropriate the dumb FE model, the CE can provide appropriate intelligence and
intelligence and control over these media. control over these media.
When multiple FEs are present, ForCES requires that packets must be When multiple FEs are present, ForCES requires that packets must be
able to arrive at the NE by one FE and leave the NE via a different able to arrive at the NE by one FE and leave the NE via a different
FE (See [3], Section 5, Requirement #3). Packets that enter the NE FE (See [4], Section 5, Requirement #3). Packets that enter the NE
via one FE and leave the NE via a different FE are transferred via one FE and leave the NE via a different FE are transferred
between FEs across the Fi reference point. Fi reference point between FEs across the Fi reference point. The Fi reference point
could be used by FEs to discovery their (inter-FE) topology, could be used by FEs to discover their (inter-FE) topology, perhaps
perhaps during pre-association phase. The Fi reference point is a during the pre-association phase. The Fi reference point is a
separate protocol from the Fp reference point and is not currently separate protocol from the Fp reference point and is not currently
defined by the ForCES architecture. defined by the ForCES Protocol.
FEs could be connected in different kinds of topologies and packet FEs could be connected in different kinds of topologies and packet
processing may spread across several FEs in the topology. Hence, processing may spread across several FEs in the topology. Hence,
logical packet flow may be different from physical FE topology. logical packet flow may be different from physical FE topology.
Figure 6 provides some topology examples. When it is necessary to Figure 6 provides some topology examples. When it is necessary to
forward packets between FEs, the CE needs to understand the FE forward packets between FEs, the CE needs to understand the FE
topology. The FE topology may be queried from the FEs by the CEs topology. The FE topology may be queried from the FEs by the CEs via
via ForCES Protocol, but the FEs are not required to provide that the ForCES Protocol, but the FEs are not required to provide that
information to the CEs. So, the FE topology information may also be information to the CEs. So, the FE topology information may also be
gathered by other means outside of the ForCES Protocol (like inter- gathered by other means outside of the ForCES Protocol (like inter-FE
FE topology discovery protocol). topology discovery protocol).
----------------- -----------------
| CE | | CE |
----------------- -----------------
^ ^ ^ ^ ^ ^
/ | \ / | \
/ v \ / v \
/ ------- \ / ------- \
/ +->| FE3 |<-+ \ / +->| FE3 |<-+ \
/ | | | | \ / | | | | \
v | ------- | v v | ------- | v
------- | | ------- ------- | | -------
| FE1 |<-+ +->| FE2 | | FE1 |<-+ +->| FE2 |
| |<--------------->| | | |<--------------->| |
------- ------- ------- -------
^ | ^ | ^ | ^ |
| | | | | | | |
| v | v | v | v
(a) Full mesh among FE1, FE2 and FE3. (a) Full mesh among FE1, FE2, and FE3
----------- -----------
| CE | | CE |
----------- -----------
^ ^ ^ ^ ^ ^ ^ ^
/ | | \ / | | \
/------ | | ------\ /------ | | ------\
v v v v v v v v
------- ------- ------- ------- ------- ------- ------- -------
| FE1 |<->| FE2 |<->| FE3 |<->| FE4 | | FE1 |<->| FE2 |<->| FE3 |<->| FE4 |
skipping to change at page 14, line 47 skipping to change at page 14, line 30
| ----------- | | | ----------- | |
| | FE4 |<----------------------| | | | FE4 |<----------------------| |
| ----------- | | ----------- |
| | ^ | | | ^ |
| v | | | v | |
| | | |
|----------------------------------------| |----------------------------------------|
(c) Multiple FEs connected by a ring (c) Multiple FEs connected by a ring
Figure 6. Some examples of FE topology. Figure 6. Some examples of FE topology
3.3.CE Managers 3.3.CE Managers
CE managers are responsible for determining which FEs a CE should CE managers are responsible for determining which FEs a CE should
control. It is legitimate for CE managers to be hard-coded with control. It is legitimate for CE managers to be hard-coded with the
the knowledge of with which FEs its CEs should communicate with. A knowledge of with which FEs its CEs should communicate with. A CE
CE manager may also be physically embedded into a CE and be manager may also be physically embedded into a CE and be implemented
implemented as a simple keypad or other direct configuration as a simple keypad or other direct configuration mechanism on the CE.
mechanism on the CE. Finally, CE managers may be physically and Finally, CE managers may be physically and logically separate
logically separate entities that configure the CE with FE entities that configure the CE with FE information via such
information via such mechanisms as COPS-PR [6] or SNMP [4]. mechanisms as COPS-PR [7] or SNMP [5].
3.4. FE Managers 3.4. FE Managers
FE managers are responsible for determining with which CE any FE managers are responsible for determining with which CE any
particular FE should initially communicate. Like CE managers, no particular FE should initially communicate. Like CE managers, no
restrictions are placed on how an FE manager decides with which CE restrictions are placed on how an FE manager decides with which CE
its FEs should communicate, nor are restrictions placed on how FE its FEs should communicate, nor are restrictions placed on how FE
managers are implemented. Each FE should have one and only one FE managers are implemented. Each FE should have one and only one FE
manager, while different FEs may have the same or different FE manager, while different FEs may have the same or different FE
manager(s). Each manager can choose to exist and operate manager(s). Each manager can choose to exist and operate
independently of other manager. independently of other manager.
4. Operational Phases 4. Operational Phases
Both FEs and CEs require some configuration in place before they Both FEs and CEs require some configuration to be in place before
can start information exchange and function as a coherent network they can start information exchange and function as a coherent
element. Two operational phases are identified in this framework: network element. Two operational phases are identified in this
pre-association and post-association. framework: pre-association and post-association.
4.1.Pre-association Phase 4.1.Pre-association Phase
Pre-association phase is the period of time during which an FE The Pre-association phase is the period of time during which an FE
Manager and a CE Manager are determining whether an FE and a CE Manager and a CE Manager are determining whether an FE and a CE
should be part of the same network element. The protocols used should be part of the same network element. The protocols used
during this phase may include all or some of the message exchange during this phase may include all or some of the message exchange
over Fl, Ff and Fc reference points. However, all these may be over Fl, Ff, and Fc reference points. However, all these may be
optional and none of this is within the scope of ForCES Protocol. optional and none of this is within the scope of the ForCES Protocol.
4.1.1. Fl Reference Point 4.1.1. Fl Reference Point
CE managers and FE managers may communicate across the Fl reference CE managers and FE managers may communicate across the Fl reference
point in the pre-association phase in order to determine whether an point in the pre-association phase in order to determine whether an
individual CE and FE, or a set of CEs and FEs should be associated. individual CE and FE, or a set of CEs and FEs should be associated.
Communication across the Fl reference point is optional in this Communication across the Fl reference point is optional in this
architecture. No requirements are placed on this reference point. architecture. No requirements are placed on this reference point.
CE managers and FE managers may be operated by different entities. CE managers and FE managers may be operated by different entities.
The operator of the CE manager may not want to divulge, except to The operator of the CE manager may not want to divulge, except to
specified FE managers, any characteristics of the CEs it manages. specified FE managers, any characteristics of the CEs it manages.
Similarly, the operator of the FE manager may not want to divulge Similarly, the operator of the FE manager may not want to divulge FE
FE characteristics, except to authorized entities. As such, CE characteristics, except to authorized entities. As such, CE managers
managers and FE managers may need to authenticate one another. and FE managers may need to authenticate one another. Subsequent
Subsequent communication between CE managers and FE managers may communication between CE managers and FE managers may require other
require other security functions such as privacy, non-repudiation, security functions such as privacy, non-repudiation, freshness, and
freshness, and integrity. integrity.
FE Manager FE CE Manager CE FE Manager FE CE Manager CE
| | | | | | | |
| | | | | | | |
|(security exchange) | | |(security exchange) | |
1|<------------------------------>| | 1|<------------------------------>| |
| | | | | | | |
|(a list of CEs and their attributes) | |(a list of CEs and their attributes) |
2|<-------------------------------| | 2|<-------------------------------| |
| | | | | | | |
|(a list of FEs and their attributes) | |(a list of FEs and their attributes) |
3|------------------------------->| | 3|------------------------------->| |
| | | | | | | |
| | | | | | | |
|<----------------Fl------------>| | |<----------------Fl------------>| |
Figure 7. An example of message exchange over Fl reference point Figure 7. An example of a message exchange over the Fl reference
point
Once the necessary security functions have been performed, the CE Once the necessary security functions have been performed, the CE and
and FE managers communicate to determine which CEs and FEs should FE managers communicate to determine which CEs and FEs should
communicate with each other. At the very minimum, the CE and FE communicate with each other. At the very minimum, the CE and FE
managers need to learn of the existence of available FEs and CEs managers need to learn of the existence of available FEs and CEs
respectively. This discovery process may entail one or both respectively. This discovery process may entail one or both managers
managers learning the capabilities of the discovered ForCES learning the capabilities of the discovered ForCES protocol elements.
protocol elements. Figure 7 shows an example of possible message Figure 7 shows an example of a possible message exchange between the
exchange between CE manager and FE manager over Fl reference point. CE manager and FE manager over the Fl reference point.
4.1.2. Ff Reference Point 4.1.2. Ff Reference Point
The Ff reference point is used to inform forwarding elements of the The Ff reference point is used to inform forwarding elements of the
association decisions made by the FE manager in pre-association association decisions made by the FE manager in the pre-association
phase. Only authorized entities may instruct an FE with respect to phase. Only authorized entities may instruct an FE with respect to
which CE should control it. Therefore, privacy, integrity, which CE should control it. Therefore, privacy, integrity,
freshness, and authentication are necessary between the FE manager freshness, and authentication are necessary between the FE manager
and FEs when the FE manager is remote to the FE. Once the and FEs when the FE manager is remote to the FE. Once the
appropriate security has been established, the FE manager instructs appropriate security has been established, the FE manager instructs
the FEs across this reference point to join a new NE or to the FEs across this reference point to join a new NE or to disconnect
disconnect from an existing NE. The FE Manager could also assign from an existing NE. The FE Manager could also assign unique FE
unique FE identifiers to the FEs using this reference point. The identifiers to the FEs using this reference point. The FE
FE identifiers are useful in post association phase to express FE identifiers are useful in the post association phase to express FE
topology. Figure 8 shows example of message exchange over Ff topology. Figure 8 shows example of a message exchange over the Ff
reference point. reference point.
FE Manager FE CE Manager CE FE Manager FE CE Manager CE
| | | | | | | |
| | | | | | | |
|(security exchange) |(security exchange) |(security exchange) |(security exchange)
1|<------------>|authentication 1|<----------->|authentication 1|<------------>|authentication 1|<----------->|authentication
| | | | | | | |
|(FE ID, attributes) |(CE ID, attributes) |(FE ID, attributes) |(CE ID, attributes)
2|<-------------|request 2|<----------->|request 2|<-------------|request 2|<------------|request
| | | | | | | |
3|------------->|response 3|------------>|response 3|------------->|response 3|------------>|response
|(corresponding CE ID) |(corresponding FE ID) |(corresponding CE ID) |(corresponding FE ID)
| | | | | | | |
| | | | | | | |
|<-----Ff----->| |<-----Fc---->| |<-----Ff----->| |<-----Fc---->|
Figure 8. Examples of message exchange Figure 8. Examples of a message exchange
over Ff and Fc reference points. over the Ff and Fc reference points
Note that the FE manager function may be co-located with the FE Note that the FE manager function may be co-located with the FE (such
(such as by manual keypad entry of the CE IP address), in which as by manual keypad entry of the CE IP address), in which case this
case this reference point is reduced to a built-in function. reference point is reduced to a built-in function.
4.1.3. Fc Reference Point 4.1.3. Fc Reference Point
The Fc reference point is used to inform control elements of the The Fc reference point is used to inform control elements of the
association decisions made by CE managers in pre-association phase. association decisions made by CE managers in the pre-association
When the CE manager is remote, only authorized entities may phase. When the CE manager is remote, only authorized entities may
instruct a CE to control certain FEs. Privacy, integrity, instruct a CE to control certain FEs. Privacy, integrity, freshness,
freshness and authentication are also required across this and authentication are also required across this reference point in
reference point in such a configuration. Once appropriate security such a configuration. Once appropriate security has been
has been established, the CE manager instructs CEs as to which FEs established, the CE manager instructs the CEs as to which FEs they
they should control and how they should control them. Figure 8 should control and how they should control them. Figure 8 shows
shows example of message exchange over Fc reference point. example of a message exchange over the Fc reference point.
As with the FE manager and FEs, configurations are possible where As with the FE manager and FEs, configurations are possible where the
the CE manager and CE are co-located and no protocol is used for CE manager and CE are co-located and no protocol is used for this
this function. function.
4.2. Post-association Phase and Fp reference point 4.2. Post-association Phase and Fp reference point
Post-association phase is the period of time during which an FE and
CE have been configured with information necessary to contact each The Post-association phase is the period of time during which an FE
other and includes both association establishment and steady-state and CE have been configured with information necessary to contact
communication. The communication between CE and FE is performed each other and includes both association establishment and steady-
across the Fp ("p" meaning protocol) reference point. ForCES state communication. The communication between CE and FE is
Protocol is exclusively used for all communication across the Fp performed across the Fp ("p" meaning protocol) reference point.
reference point. ForCES Protocol is exclusively used for all communication across the
Fp reference point.
4.2.1. Proximity and Interconnect between CEs and FEs 4.2.1. Proximity and Interconnect between CEs and FEs
The ForCES Working Group has made a conscious decision that the The ForCES Working Group has made a conscious decision that the first
first version of ForCES will be focused on "very close" CE/FE version of ForCES will be focused on "very close" CE/FE localities in
localities in IP networks. Very Close localities consist of IP networks. Very Close localities consist of control and forwarding
control and forwarding elements that either are components in the elements that are either components in the same physical box, or
same physical box, or are separated at most by one local network separated at most by one local network hop ([8]). CEs and FEs can be
hop ([7]). CEs and FEs can be connected by a variety of connected by a variety of interconnect technologies, including
interconnect technologies, including Ethernet connections, Ethernet connections, backplanes, ATM (cell) fabrics, etc. ForCES
backplanes, ATM (cell) fabrics, etc. ForCES should be able to should be able to support each of these interconnects (see [4]
support each of these interconnects (see [3] Section 5, requirement Section 5, requirement #1). When the CEs and FEs are separated
#1). When the CEs and FEs are separated beyond a single L3 routing beyond a single L3 routing hop, the ForCES Protocol will make use of
hop, the ForCES Protocol will make use of an existing RFC2914 an existing RFC 2914 [3] compliant L4 protocol with adequate
compliant L4 protocol with adequate reliability, security and reliability, security, and congestion control (e.g., TCP, SCTP) for
congestion control (e.g. TCP, SCTP) for transport purposes. transport purposes.
4.2.2. Association Establishment 4.2.2. Association Establishment
FE CE FE CE
| | | |
|(Security exchange.) | |(Security exchange.) |
1|<--------------------->| 1|<--------------------->|
| | | |
|(Let me join the NE please.) |(Let me join the NE please.)
2|---------------------->| 2|---------------------->|
skipping to change at page 19, line 9 skipping to change at page 18, line 48
|(Initial config for FE -- optional) |(Initial config for FE -- optional)
5|<----------------------| 5|<----------------------|
| | | |
|(I am ready to go. Shall I?) |(I am ready to go. Shall I?)
6|---------------------->| 6|---------------------->|
| | | |
|(Go ahead!) | |(Go ahead!) |
7|<----------------------| 7|<----------------------|
| | | |
Figure 9. Example of message exchange between CE and FE Figure 9. Example of a message exchange between CE and FE
over Fp to establish NE association over Fp to establish an NE association
As an example, figure 9 shows some of the message exchange that may As an example, figure 9 shows some of the message exchange that may
happen before the association between the CE and FE is fully happen before the association between the CE and FE is fully
established. Either the CE or FE can initiate the connection. established. Either the CE or FE can initiate the connection.
Security handshake is necessary to authenticate the two Security handshake is necessary to authenticate the two communication
communication endpoints to each other before any further message endpoints to each other before any further message exchange can
exchange can happen. The security handshake should include mutual happen. The security handshake should include mutual authentication
authentication and authorization between the CE and FE, but the and authorization between the CE and FE, but the exact details depend
exact details depend on the security solution chosen by ForCES on the security solution chosen by the ForCES Protocol.
Protocol. Authorization can be as simple as checking against the Authorization can be as simple as checking against the list of
list of authorized end points provided by the FE or CE manager authorized end points provided by the FE or CE manager during the
during the pre-association phase. Both authentication and pre-association phase. Both authentication and authorization must be
authorization must be successful before the association can be successful before the association can be established. If either
established. If either authentication or authorization fails, the authentication or authorization fails, the end point must not be
end point must not be allowed to join the NE. After the successful allowed to join the NE. After the successful security handshake,
security handshake, message authentication and confidentiality are message authentication and confidentiality are still necessary for
still necessary for the on-going information exchange between the the on-going information exchange between the CE and FE, unless some
CE and FE, unless some form of physical security exists. Whenever form of physical security exists. Whenever a packet fails
a packet fails authentication, it must be dropped and a authentication, it must be dropped and a notification may be sent to
notification may be sent to alert the sender of the potential alert the sender of the potential attack. Section 8 provides more
attack. Section 8 provides more details on the security details on the security considerations for ForCES.
considerations for ForCES.
After the successful security handshake, the FE needs to inform the After the successful security handshake, the FE needs to inform the
CE of its own capability and optionally its topology in relation to CE of its own capability and optionally its topology in relation to
other FEs. The capability of the FE is represented by the FE model, other FEs. The capability of the FE shall be represented by the FE
described in a separate document. The model would allow an FE to model, as required in [4] (Section 6, requirement #1). The model
describe what kind of packet processing functions it contains, in would allow an FE to describe what kind of packet processing
what order the processing happens, what kinds of configurable functions it contains, in what order the processing happens, what
parameters it allows, what statistics it collects and what events kinds of configurable parameters it allows, what statistics it
it might throw, etc. Once such information is available to the CE, collects, and what events it might throw, etc. Once such information
the CE may choose to send some initial or default configuration to is available to the CE, the CE may choose to send some initial or
the FE so that the FE can start receiving and processing packets default configuration to the FE so that the FE can start receiving
correctly. Such initialization may not be necessary if the FE and processing packets correctly. Such initialization may not be
already obtains the information from its own bootstrap process. necessary if the FE already obtains the information from its own
Once the necessary initial information is exchanged, the process of bootstrap process. Once the necessary initial information is
association is completed. Packet processing and forwarding at the exchanged, the process of association is completed. Packet
FE cannot begin until association is established. After the processing and forwarding at the FE cannot begin until association is
association is established, the CE and FE enter steady-state established. After the association is established, the CE and FE
communication. enter steady-state communication.
4.2.3. Steady-state Communication 4.2.3. Steady-state Communication
Once an association is established between the CE and FE, the Once an association is established between the CE and FE, the ForCES
ForCES Protocol is used by the CE and FE over Fp reference point to Protocol is used by the CE and FE over the Fp reference point to
exchange information to facilitate packet processing. exchange information to facilitate packet processing.
FE CE FE CE
| | | |
|(Add these new routes.)| |(Add these new routes.)|
1|<----------------------| 1|<----------------------|
| | | |
|(Successful.) | |(Successful.) |
2|---------------------->| 2|---------------------->|
| | | |
skipping to change at page 20, line 37 skipping to change at page 20, line 27
|(Reply with stats collected.) |(Reply with stats collected.)
2|---------------------->| 2|---------------------->|
| | | |
| | | |
|(My port is down, with port #.) |(My port is down, with port #.)
1|---------------------->| 1|---------------------->|
| | | |
|(Here is a new forwarding table) |(Here is a new forwarding table)
2|<----------------------| 2|<----------------------|
| | | |
Figure 10. Examples of message exchange between CE and FE
Figure 10. Examples of a message exchange between CE and FE
over Fp during steady-state communication over Fp during steady-state communication
Based on the information acquired through CEs' control processing, Based on the information acquired through CEs' control processing,
CEs will frequently need to manipulate the packet-forwarding CEs will frequently need to manipulate the packet-forwarding
behaviors of their FE(s) by sending instructions to FEs. For behaviors of their FE(s) by sending instructions to FEs. For
example, Figure 10 shows message exchange examples in which the CE example, Figure 10 shows message exchange examples in which the CE
sends new routes to the FE so that the FE can add them to its sends new routes to the FE so that the FE can add them to its
forwarding table. The CE may query the FE for statistics collected forwarding table. The CE may query the FE for statistics collected
by the FE and the FE may notify the CE of important events such as by the FE and the FE may notify the CE of important events such as
port failure. port failure.
skipping to change at page 21, line 26 skipping to change at page 21, line 29
| Router | | | Router | | | Router | | | Router | |
| A | | | B | | | A | | | B | |
---------+----------- -----------+---------- ---------+----------- -----------+----------
v ^ v ^
| | | |
| | | |
------------------->--------------- ------------------->---------------
Figure 11. Example to show data packet flow between two NEs. Figure 11. Example to show data packet flow between two NEs.
Control plane protocol packets (such as RIP, OSPF messages) Control plane protocol packets (such as RIP, OSPF messages) addressed
addressed to any of NE's interfaces are typically redirected by the to any of NE's interfaces are typically redirected by the receiving
receiving FE to its CE, and CE may originate packets and have its FE to its CE, and CE may originate packets and have its FE deliver
FE deliver them to other NEs. Therefore, ForCES Protocol over Fp them to other NEs. Therefore, the ForCES Protocol over Fp not only
not only transports the ForCES Protocol messages between CEs and transports the ForCES Protocol messages between CEs and FEs, but also
FEs, but also encapsulates the data packets from control plane encapsulates the data packets from control plane protocols.
protocols. Moreover, one FE may be controlled by multiple CEs for Moreover, one FE may be controlled by multiple CEs for distributed
distributed control. In this configuration, the control protocols control. In this configuration, the control protocols supported by
supported by the FORCES NEs may spread across multiple CEs. For the FORCES NEs may spread across multiple CEs. For example, one CE
example, one CE may support routing protocols like OSPF and BGP, may support routing protocols like OSPF and BGP, while a signaling
while a signaling and admission control protocol like RSVP is and admission control protocol like RSVP is supported in another CE.
supported in another CE. FEs are configured to recognize and FEs are configured to recognize and filter these protocol packets and
filter these protocol packets and forward them to the corresponding forward them to the corresponding CE.
CE.
Figure 11 shows one example of how the BGP packets originated by Figure 11 shows one example of how the BGP packets originated by
router A are passed to router B. In this example, the ForCES router A are passed to router B. In this example, the ForCES
Protocol is used to transport the packets from the CE to the FE Protocol is used to transport the packets from the CE to the FE
inside router A, and then from the FE to the CE inside router B. inside router A, and then from the FE to the CE inside router B. In
In light of the fact that the ForCES Protocol is responsible for light of the fact that the ForCES Protocol is responsible for
transporting both the control messages and the data packets between transporting both the control messages and the data packets between
the CE and FE over Fp reference point, it is possible to use either the CE and FE over the Fp reference point, it is possible to use
a single protocol or multiple protocols to achieve this. either a single protocol or multiple protocols.
4.2.5. Proxy FE 4.2.5. Proxy FE
In the case where a physical FE cannot implement (e.g., due to the In the case where a physical FE cannot implement (e.g., due to the
lack of a general purpose CPU) the ForCES Protocol directly, a lack of a general purpose CPU) the ForCES Protocol directly, a proxy
proxy FE can be used to terminate the Fp reference point instead of FE can be used to terminate the Fp reference point instead of the
the physical FE. This allows the CE communicate to the physical FE physical FE. This allows the CE to communicate to the physical FE
via the proxy by using ForCES, while the proxy manipulates the via the proxy by using ForCES, while the proxy manipulates the
physical FE using some intermediary form of communication (e.g., a physical FE using some intermediary form of communication (e.g., a
non-ForCES protocol or DMA). In such an implementation, the non-ForCES protocol or DMA). In such an implementation, the
combination of the proxy and the physical FE becomes one logical FE combination of the proxy and the physical FE becomes one logical FE
entity. It is also possible that one proxy act on behalf of entity. It is also possible for one proxy to act on behalf of
multiple physical FEs. multiple physical FEs.
One needs to be aware of the security implication introduced by the One needs to be aware of the security implication introduced by the
proxy FE. Since the physical FE is not capable of implementing proxy FE. Since the physical FE is not capable of implementing
ForCES itself, the security mechanism of ForCES can only secure the ForCES itself, the security mechanism of ForCES can only secure the
communication channel between the CE and the proxy FE, but not all communication channel between the CE and the proxy FE, but not all
the way to the physical FE. It is recommended that other security the way to the physical FE. It is recommended that other security
mechanisms (including physical security property) be employed to mechanisms (including physical security property) be employed to
ensure the security between the CE and the physical FE. ensure the security between the CE and the physical FE.
4.3. Association Re-establishment 4.3. Association Re-establishment
FEs and CEs may join and leave NEs dynamically (see [3] Section 5, FEs and CEs may join and leave NEs dynamically (see [4] Section 5,
requirements #12). When an FE or CE leaves the NE, the association requirements #12). When an FE or CE leaves the NE, the association
with the NE is broken. If the leaving party rejoins an NE later, with the NE is broken. If the leaving party rejoins an NE later, to
to re-establish the association, it may need to re-enter the pre- re-establish the association, it may need to re-enter the pre-
association phase. Loss of association can also happen association phase. Loss of association can also happen unexpectedly
unexpectedly due to loss of connection between the CE and the FE. due to a loss of connection between the CE and the FE. Therefore,
Therefore, the framework allows the bi-directional transition the framework allows the bi-directional transition between these two
between these two phases, but the ForCES Protocol is only phases, but the ForCES Protocol is only applicable for the post-
applicable for the post-association phase. However, the protocol association phase. However, the protocol should provide mechanisms
should provide mechanisms to support association re-establishment. to support association re-establishment. This includes the ability
This includes the ability for CEs and FEs to determine when there for CEs and FEs to determine when there is a loss of association
is a loss of association between them, ability to restore between them, and to restore association and efficient state
association and efficient state (re)synchronization mechanisms (see (re)synchronization mechanisms (see [4] Section 5, requirement #7).
[3] Section 5, requirement #7). Note that security association and Note that security association and state must also be re-established
state must be also re-established to guarantee the same level of to guarantee the same level of security (including both
security (including both authentication and authorization) exists authentication and authorization) exists before and after the
before and after the association re-establishment. association re-establishment.
When an FE leaves or joins an existing NE that is already in When an FE leaves or joins an existing NE that is already in
operation, the CE needs to be aware of the impact on FE topology operation, the CE needs to be aware of the impact on FE topology and
and deals with the change accordingly. deal with the change accordingly.
4.3.1. CE graceful restart 4.3.1. CE graceful restart
The failure and restart of the CE in a router can potentially cause The failure and restart of the CE in a router can potentially cause
much stress and disruption on the control plane throughout a much stress and disruption on the control plane throughout a network
network. Because when a CE has to restart for any reason, the because in restarting a CE for any reason, the router loses routing
router loses routing adjacencies or sessions with its routing adjacencies or sessions with its routing neighbors. Neighbors who
neighbors. Neighbors who detect the lost adjacency normally re- detect the lost adjacency normally re-compute new routes and then
compute new routes and then send routing updates to their own send routing updates to their own neighbors to communicate the lost
neighbors to communicate the lost adjacency. Their neighbors do adjacency. Their neighbors do the same thing to propagate throughout
the same thing to propagate throughout the network. In the the network. In the meantime, the restarting router cannot receive
meantime, the restarting router cannot receive traffic from other traffic from other routers because the neighbors have stopped using
routers because the neighbors have stopped using the router's the router's previously advertised routes. When the restarting
previously advertised routes. When the restarting router restores router restores adjacencies, neighbors must once again re-compute new
adjacencies, neighbors must once again re-compute new routes and routes and send out additional routing updates. The restarting
send out additional routing updates. The restarting router is router is unable to forward packets until it has re-established
unable to forward packets until it has re-established routing routing adjacencies with neighbors, received route updates through
adjacencies with neighbors, received route updates through these these adjacencies, and computed new routes. Until convergence takes
adjacencies, and computed new routes. Until convergence takes place throughout the network, packets may be lost in transient black
place throughout the network, packets may be lost in transient holes or forwarding loops.
black holes or forwarding loops.
A high availability mechanism known as the "graceful restart" has A high availability mechanism known as the "graceful restart" has
been used by the IP routing protocols (OSPF [10], BGP [11]) and been used by the IP routing protocols (OSPF [11], BGP [12], IS-IS
MPLS label distribution protocol (LDP [9]) to help minimize the [13]) and MPLS label distribution protocol (LDP [10]) to help
negative effects on routing throughout an entire network caused by minimize the negative effects on routing throughout an entire network
a restarting router. Route flap on neighboring routers is avoided, caused by a restarting router. Route flap on neighboring routers is
and a restarting router can continue to forward packets that would avoided, and a restarting router can continue to forward packets that
otherwise be dropped. would otherwise be dropped.
While the details differ from protocol to protocol, the general While the details differ from protocol to protocol, the general idea
idea behind the graceful restart mechanism remains the same. With behind the graceful restart mechanism remains the same. With the
the graceful restart, a restarting router can inform its neighbors graceful restart, a restarting router can inform its neighbors when
when it restarts. The neighbors may detect the lost adjacency but it restarts. The neighbors may detect the lost adjacency but do not
do not recompute new routes or send routing updates to their recompute new routes or send routing updates to their neighbors. The
neighbors. The neighbors also hold on to the routes received from neighbors also hold on to the routes received from the restarting
the restarting router before restart and assume they are still router before restart and assume they are still valid for a limited
valid for a limited time. By doing so, the restarting router's FEs time. By doing so, the restarting router's FEs can also continue to
can also continue to receive and forward traffic from other receive and forward traffic from other neighbors for a limited time
neighbors for a limited time by using the routes they already have. by using the routes they already have. The restarting router then
The restarting router then re-establishes routing adjacencies, re-establishes routing adjacencies, downloads updated routes from all
downloads updated routes from all its neighbors, recomputes new its neighbors, recomputes new routes, and uses them to replace the
routes and uses them to replace the older routes it was using. It older routes it was using. It then sends these updated routes to its
then sends these updated routes to its neighbors and signals the neighbors and signals the completion of the graceful restart process.
completion of the graceful restart process.
Non-stop forwarding is a requirement for graceful restart. It is Non-stop forwarding is a requirement for graceful restart. It is
necessary so a router can continue to forward packets while it is necessary so a router can continue to forward packets while it is
downloading routing information and recomputing new routes. This downloading routing information and recomputing new routes. This
ensures that packets will not be dropped. As one can see, one of ensures that packets will not be dropped. As one can see, one of the
the benefits afforded by the separation of CE and FE is exactly the benefits afforded by the separation of CE and FE is exactly the
ability of non-stop forwarding in the face of the CE failure and ability of non-stop forwarding in the face of the CE failure and
restart. The support of dynamic changes to CE/FE association in restart. The support of dynamic changes to CE/FE association in
ForCES also makes it compatible with high availability mechanisms ForCES also makes it compatible with high availability mechanisms,
such as graceful restart. such as graceful restart.
ForCES should be able to support CE graceful restart easily. When ForCES should be able to support a CE graceful restart easily. When
the association is established the first time, the CE must inform the association is established the first time, the CE must inform the
the FEs what to do in the case of CE failure. If graceful restart FEs what to do in the case of a CE failure. If graceful restart is
is not supported, the FEs may be told to stop packet processing all not supported, the FEs may be told to stop packet processing all
together if its CE fails. If graceful restart is supported, the together if its CE fails. If graceful restart is supported, the FEs
FEs should be told to cache and hold on to its FE state including should be told to cache and hold on to its FE state, including the
the forwarding tables across the restarts. A timer must be forwarding tables across the restarts. A timer must be included so
included so that the timeout causes such cached state to expire that the timeout causes such a cached state to eventually expire.
eventually. Those timers should be settable by the CE. Those timers should be settable by the CE.
4.3.2. FE restart 4.3.2. FE restart
In the same example in Figure 5, assuming CE1 is the working CE for In the same example in Figure 5, assuming CE1 is the working CE for
the moment, what would happen if one of the FEs, say FE1, leaves the moment, what would happen if one of the FEs, say FE1, leaves the
the NE temporarily? FE1 may voluntarily decide to leave the NE temporarily? FE1 may voluntarily decide to leave the association.
association. Alternatively, FE1 may stop functioning simply due to Alternatively, FE1 may stop functioning simply due to unexpected
unexpected failure. In the former case, CE1 receives a "leave- failure. In the former case, CE1 receives a "leave-association
association request" from FE1. In the latter, CE1 detects the request" from FE1. In the latter, CE1 detects the failure of FE1 by
failure of FE1 by some other means. In both cases, CE1 must inform some other means. In both cases, CE1 must inform the routing
the routing protocols of such an event, most likely prompting a protocols of such an event, most likely prompting a reachability and
reachability and SPF (Shortest Path First) recalculation and SPF (Shortest Path First) recalculation and associated downloading of
associated downloading of new FIBs from CE1 to the other remaining new FIBs from CE1 to the other remaining FEs (only FE2 in this
FEs (only FE2 in this example). Such recalculation and FIB update example). Such recalculation and FIB updates will also be propagated
will also be propagated from CE1 to the NE's neighbors that are from CE1 to the NE's neighbors that are affected by the connectivity
affected by the connectivity of FE1. of FE1.
When FE1 decides to rejoin again, or when it restarts again from When FE1 decides to rejoin again, or when it restarts again after the
the failure, FE1 needs to re-discover its master (CE). This can be failure, FE1 needs to re-discover its master (CE). This can be
achieved by several means. It may re-enter the pre-association achieved by several means. It may re-enter the pre-association phase
phase and get that information from its FE manager. It may and get that information from its FE manager. It may retrieve the
retrieve the previous CE information from its cache, if it can previous CE information from its cache, if it can validate the
validate the information freshness. Once it discovers its CE, it information freshness. Once it discovers its CE, it starts message
starts message exchange with the CE to re-establish the association exchange with the CE to re-establish the association, as outlined in
just as outlined in Figure 9, with the possible exception that it Figure 9, with the possible exception that it might be able to bypass
might be able to bypass the transport of the complete initial the transport of the complete initial configuration. Suppose that
configuration. Suppose that FE1 still has its routing table and FE1 still has its routing table and other state information from the
other state information from the last association. Instead of last association. Instead of re-sending all the information, it may
sending all the information again from scratch, it may be able to be able to use a more efficient mechanism to re-sync the state with
use more efficient mechanism to re-sync up the state with its CE if its CE, if such a mechanism is supported by the ForCES Protocol. For
such mechanism is supported by the ForCES Protocol. For example, example, CRC-32 of the state might give a quick indication of whether
CRC-32 of the state might give a quick indication of whether or not or not the state is in-sync with its CE. By comparing its state with
the state is in-sync with its CE. By comparing its state with the the CE first, it sends an information update
CE first, it sends an information update only if it is needed. only if it is needed. The ForCES Protocol may choose to implement
ForCES Protocol may choose to implement similar optimization similar optimization mechanisms, but it may also choose not to, as
mechanisms, but it may also choose not to, as this is not a this is not a requirement.
requirement.
5. Applicability to RFC1812 5. Applicability to RFC1812
[3] Section 5, requirement #9 dictates "Any proposed ForCES [4] Section 5, requirement #9 dictates "Any proposed ForCES
architecture must explain how that architecture supports all of the architecture must explain how that architecture supports all of the
router functions as defined in RFC1812." RFC1812 discusses many router functions as defined in RFC 1812." RFC 1812 [2] discusses
important requirements for IPv4 routers from the link layer to the many important requirements for IPv4 routers from the link layer to
application layer. This section addresses the relevant the application layer. This section addresses the relevant
requirements in RFC1812 for implementing IPv4 routers based on requirements in RFC1812 for implementing IPv4 routers based on
ForCES architecture and explains how ForCES satisfies these ForCES architecture and explains how ForCES satisfies these
requirements by providing guidelines on how to separate the requirements by providing guidelines on how to separate the
functionalities required into forwarding plane and control plane. functionalities required into the forwarding plane and control plane.
In general, the forwarding plane carries out the bulk of the per- In general, the forwarding plane carries out the bulk of the per-
packet processing that is required at line speed, while the control packet processing that is required at line speed, while the control
plane carries most of the computationally complex operations that plane carries most of the computationally complex operations that are
are typical of the control and signaling protocols. However, it is typical of the control and signaling protocols. However, it is
impossible to draw a rigid line to divide the processing into CEs impossible to draw a rigid line to divide the processing into CEs and
and FEs cleanly. Nor should the ForCES architecture limit the FEs cleanly and the ForCES architecture should not limit the
innovative approaches in control and forwarding plane separation. innovative approaches in control and forwarding plane separation. As
As more and more processing power is available in the FEs, some of more and more processing power is available in the FEs, some of the
the control functions that traditionally are performed by CEs may control functions that traditionally are performed by CEs may now be
now be moved to FEs for better performance and scalability. Such moved to FEs for better performance and scalability. Such offloaded
offloaded functions may include part of ICMP or TCP processing, or functions may include part of ICMP or TCP processing, or part of
part of routing protocols. Once off-loaded onto the forwarding routing protocols. Once off-loaded onto the forwarding plane, such
plane, such CE functions, even though logically belonging to the CE functions, even though logically belonging to the control plane,
control plane, now become part of the FE functions. Just like the now become part of the FE functions. Just like the other logical
other logical functions performed by FEs, such off-loaded functions functions performed by FEs, such off-loaded functions must be
must be expressed as part of the FE model so that the CEs can expressed as part of the FE model so that the CEs can decide how to
decide how to best take advantage of these off-loaded functions best take advantage of these off-loaded functions when present on the
when present on the FEs. FEs.
5.1. General Router Requirements 5.1. General Router Requirements
Routers have at least two or more logical interfaces. When CEs and Routers have at least two or more logical interfaces. When CEs and
FEs are separated by ForCES within a single NE, some additional FEs are separated by ForCES within a single NE, some additional
interfaces are needed for intra-NE communications. Figure 12 shows interfaces are needed for intra-NE communications, as illustrated in
an example to illustrate that. This NE contains one CE and two figure 12. This NE contains one CE and two FEs. Each FE has four
FEs. Each FE has four interfaces; two of them are used for interfaces; two of them are used for receiving and transmitting
receiving and transmitting packets to the external world, while the packets to the external world, while the other two are for intra-NE
other two are for intra-NE connections. CE has two logical connections. CE has two logical interfaces #9 and #10, connected to
interfaces #9 and #10, connected to interfaces #3 and #6 from FE1 interfaces #3 and #6 from FE1 and FE2, respectively. Interface #4
and FE2, respectively. Interface #4 and #5 are connected for FE1- and #5 are connected for FE1-FE2 communication. Therefore, this
FE2 communication. Therefore, this router NE provides four router NE provides four external interfaces (#1, 2, 7, and 8).
external interfaces (#1, 2, 7 and 8).
--------------------------------- ---------------------------------
| router NE | | router NE |
| ----------- ----------- | | ----------- ----------- |
| | FE1 | | FE2 | | | | FE1 | | FE2 | |
| ----------- ----------- | | ----------- ----------- |
| 1| 2| 3| 4| 5| 6| 7| 8| | | 1| 2| 3| 4| 5| 6| 7| 8| |
| | | | | | | | | | | | | | | | | | | |
| | | | +----+ | | | | | | | | +----+ | | | |
| | | | | | | | | | | | | | | |
skipping to change at page 26, line 31 skipping to change at page 26, line 27
| | | -------------- | | | | | | -------------- | | |
| | | | | | | | | | | |
-----+--+----------------+--+---- -----+--+----------------+--+----
| | | | | | | |
| | | | | | | |
Figure 12. A router NE example with four interfaces. Figure 12. A router NE example with four interfaces.
IPv4 routers must implement IP to support its packet forwarding IPv4 routers must implement IP to support its packet forwarding
function, which is driven by its FIB (Forwarding Information Base). function, which is driven by its FIB (Forwarding Information Base).
This Internet layer forwarding (see RFC1812 [1] Section 5) This Internet layer forwarding (see RFC 1812 [2] Section 5)
functionality naturally belongs to FEs in the ForCES architecture. functionality naturally belongs to FEs in the ForCES architecture.
A router may implement transport layer protocols (like TCP and UDP) A router may implement transport layer protocols (like TCP and UDP)
that are required to support application layer protocols (see that are required to support application layer protocols (see RFC
RFC1812 [1] Section 6). One important class of application 1812 [2] Section 6). One important class of application protocols is
protocols is routing protocols (see RFC1812 [1] Section 7). In routing protocols (see RFC 1812 [2] Section 7). In the ForCES
ForCES architecture, routing protocols are naturally implemented by architecture, routing protocols are naturally implemented by CEs.
CEs. Routing protocols require routers communicate with each Routing protocols require that routers communicate with each other.
other. This communication between CEs in different routers is This communication between CEs in different routers is supported in
supported in ForCES by FEs' ability to redirect data packets ForCES by FEs' ability to redirect data packets addressed to routers
addressed to routers (i.e., NEs) and CEs' ability to originate (i.e., NEs), and the CEs' ability to originate packets and have them
packets and have them delivered by their FEs. This communication delivered by their FEs. This communication occurs across the Fp
occurs across Fp reference point inside each router and between reference point inside each router and between neighboring routers'
neighboring routers' external interfaces, as illustrated in Figure external interfaces, as illustrated in Figure 11.
11.
5.2.Link Layer 5.2.Link Layer
Since FEs own all the external interfaces for the router, FEs need
to conform to the link layer requirements in RFC1812. Arguably, Since FEs own all the external interfaces for the router, FEs need to
ARP support may be implemented in either CEs or FEs. As we will conform to the link layer requirements in RFC 1812 [2]. Arguably,
see later, a number of behaviors that RFC1812 mandates fall into ARP support may be implemented in either CEs or FEs. As we will see
this category -- they may be performed by the FE and may be later, a number of behaviors that RFC 1812 mandates fall into this
performed by the CE. A general guideline is needed to ensure category -- they may be performed by the FE and may be performed by
interoperability between separated control and forwarding planes. the CE. A general guideline is needed to ensure interoperability
The guideline we offer here is that CEs MUST be capable of these between separated control and forwarding planes. The guideline we
kind of operations while FEs MAY choose to implement them. FE offer here is that CEs MUST be capable of these kinds of operations
model should indicate its capabilities in this regard so that CEs while FEs MAY choose to implement them. The FE model should indicate
can decide where these functions are implemented. its capabilities in this regard so that CEs can decide where these
functions are implemented.
Interface parameters, including MTU, IP address, etc., must be Interface parameters, including MTU, IP address, etc., must be
configurable by CEs via ForCES. CEs must be able to determine configurable by CEs via ForCES. CEs must be able to determine
whether a physical interface in an FE is available to send packets whether a physical interface in an FE is available to send packets or
or not. FEs must also inform CEs the status change of the not. FEs must also inform CEs of the status change of the interfaces
interfaces (like link up/down) via ForCES. (like link up/down) via ForCES.
5.3.Internet Layer Protocols 5.3.Internet Layer Protocols
Both FEs and CEs must implement IP protocol and all mandatory Both FEs and CEs must implement the IP protocol and all mandatory
extensions as RFC1812 specified. CEs should implement IP options extensions as RFC1812 specified. CEs should implement IP options
like source route and record route while FEs may choose to like source route and record route while FEs may choose to implement
implement those as well. The timestamp option should be those as well. The timestamp option should be implemented by FEs to
implemented by FEs to insert the timestamp most accurately. The FE insert the timestamp most accurately. The FE must interpret the IP
must interpret the IP options that it understands and preserve the options that it understands and preserve the rest unchanged for use
rest unchanged for use by CEs. Both FEs and CEs might choose to by CEs. Both FEs and CEs might choose to silently discard packets
silently discard packets without sending ICMP errors, but such without sending ICMP errors, but such events should be logged and
events should be logged and counted. FEs may report statistics for counted. FEs may report statistics for such events to CEs via
such events to CEs via ForCES. ForCES.
When multiple FEs are involved to process packets, the appearance When multiple FEs are involved to process packets, the appearance of
of single NE must be strictly maintained. For example, Time-To- a single NE must be strictly maintained. For example, Time-To-Live
Live (TTL) must be decremented only once within a single NE. For (TTL) must be decremented only once within a single NE. For example,
example, it can be always decremented by the last FE with egress it can be always decremented by the last FE with egress function.
function.
FEs must receive and process normally any packets with a broadcast FEs must receive and process normally any packets with a broadcast
destination address or a multicast destination address that the destination address or a multicast destination address that the
router has asked to receive. When IP multicast is supported in router has asked to receive. When IP multicast is supported in
routers, IGMP is implemented in CEs. CEs are also required of ICMP routers, IGMP is implemented in CEs. CEs are also required of ICMP
support, while it is optional for FEs to support ICMP. Such an support, while it is optional for FEs to support ICMP. Such an
option can be communicated to CEs as part of the FE model. option can be communicated to CEs as part of the FE model. Therefore,
Therefore, FEs can always rely upon CEs to send out ICMP error FEs can always rely upon CEs to send out ICMP error messages, but FEs
messages, but FEs also have the option to generate ICMP error also have the option of generating ICMP error messages themselves.
messages themselves.
5.4.Internet Layer Forwarding 5.4.Internet Layer Forwarding
IP forwarding is implemented by FEs. When the routing table is IP forwarding is implemented by FEs. When the routing table is
updated at CEs, ForCES is used to send the new route entries from updated at the CEs, ForCES is used to send the new route entries from
CEs to FEs. Each FE has its own forwarding table and uses this the CEs to FEs. Each FE has its own forwarding table and uses this
table to direct packets to the next hop interface. table to direct packets to the next hop interface.
Upon receiving IP packets, the FE verifies the IP header and Upon receiving IP packets, the FE verifies the IP header and
processes most of the IP options. Some options cannot be processed processes most of the IP options. Some options cannot be processed
until the routing decision has been made. The routing decision is until the routing decision has been made. The routing decision is
made after examining the destination IP address. If the made after examining the destination IP address. If the destination
destination address belongs to the router itself, the packets are address belongs to the router itself, the packets are filtered and
filtered and either processed locally or forwarded to CE, depending either processed locally or forwarded to the CE, depending upon the
upon the instructions set-up by CE. Otherwise, the FE determines instructions set-up by the CE. Otherwise, the FE determines the next
the next hop IP address by looking up in its forwarding table. The hop IP address by looking in its forwarding table. The FE also
FE also determines the network interface it uses to send the determines the network interface it uses to send the packets.
packets. Sometimes an FE may need to forward the packets to Sometimes an FE may need to forward the packets to another FE before
another FE before packets can be forwarded out to the next hop. packets can be forwarded out to the next hop. Right before packets
Right before packets are forwarded out to the next hop, the FE are forwarded out to the next hop, the FE decrements TTL by 1 and
decrements TTL by 1 and processes any IP options that cannot be processes any IP options that could not be processed before. The FE
processed before. The FE performs any IP fragmentation if performs IP fragmentation if necessary, determines the link layer
necessary, determines link layer address (e.g., by ARP), and address (e.g., by ARP), and encapsulates the IP datagram (or each of
encapsulates the IP datagram (or each of the fragments thereof) in the fragments thereof) in an appropriate link layer frame and queues
an appropriate link layer frame and queues it for output on the it for output on the interface selected.
interface selected.
Other options mentioned in RFC1812 for IP forwarding may also be Other options mentioned in RFC 1812 [2] for IP forwarding may also be
implemented at FEs, for example, packet filtering. implemented at FEs, for example, packet filtering.
FEs typically forward packets destined locally to CEs. FEs may FEs typically forward packets destined locally to CEs. FEs may also
also forward exceptional packets (packets that FEs do not know how forward exceptional packets (packets that FEs do not know how to
to handle) to CEs. CEs are required to handle packets forwarded by handle) to CEs. CEs are required to handle packets forwarded by FEs
FEs for whatever different reasons. It might be necessary for for whatever reason. It might be necessary for ForCES to attach some
ForCES to attach some meta-data with the packets to indicate the meta-data with the packets to indicate the reasons of forwarding from
reasons of forwarding from FEs to CEs. Upon receiving packets with FEs to CEs. Upon receiving packets with meta-data from FEs, CEs can
meta-data from FEs, CEs can decide to either process the packets decide to either process the packets themselves, or pass the packets
themselves, or pass the packets to the upper layer protocols to the upper layer protocols including routing and management
including routing and management protocols. If CEs are to process protocols. If CEs are to process the packets by themselves, CEs may
the packets by themselves, CEs may choose to discard the packets, choose to discard the packets, or modify and re-send the packets.
or modify and re-send the packets. CEs may also originate new CEs may also originate new packets and deliver them to FEs for
packets and deliver them to FEs for further forwarding. further forwarding.
Any state change during router operation must also be handled Any state change during router operation must also be handled
correctly according to RFC1812. For example, when an FE ceases correctly according to RFC1812. For example, when an FE ceases
forwarding, the entire NE may continue forwarding packets, but it forwarding, the entire NE may continue forwarding packets, but it
needs to stop advertising routes that are affected by the failed needs to stop advertising routes that are affected by the failed FE.
FE.
5.5. Transport Layer 5.5. Transport Layer
Transport layer is typically implemented at CEs to support higher The Transport layer is typically implemented at CEs to support higher
layer application protocols like routing protocols. In practice, layer application protocols like routing protocols. In practice,
this means that most CEs implement both the Transmission Control this means that most CEs implement both the Transmission Control
Protocol (TCP) and the User Datagram Protocol (UDP). Protocol (TCP) and the User Datagram Protocol (UDP).
Both CEs and FEs need to implement ForCES Protocol. If some layer- Both CEs and FEs need to implement the ForCES Protocol. If some
4 transport is used to support ForCES, then both CEs and FEs need layer-4 transport is used to support ForCES, then both CEs and FEs
to implement the L4 transport and ForCES Protocols. need to implement the L4 transport and ForCES Protocols.
5.6. Application Layer -- Routing Protocols 5.6. Application Layer -- Routing Protocols
Interior and exterior routing protocols are implemented on CEs. Interior and exterior routing protocols are implemented on CEs. The
The routing packets originated by CEs are forwarded to FEs for routing packets originated by CEs are forwarded to FEs for delivery.
delivery. The results of such protocols (like forwarding table The results of such protocols (like forwarding table updates) are
updates) are communicated to FEs via ForCES. communicated to FEs via ForCES.
For performance or scalability reasons, portions of the control For performance or scalability reasons, portions of the control plane
plane functions that need faster response may be moved from the CEs functions that need faster response may be moved from the CEs and
and off-loaded onto the FEs. For example in OSPF, the Hello off-loaded onto the FEs. For example, in OSPF, the Hello protocol
protocol packets are generated and processed periodically. When packets are generated and processed periodically. When done at the
done at CEs, the inbound Hello packets have to traverse from the CEs, the inbound Hello packets have to traverse from the external
external interfaces at the FEs to the CEs via the internal CE-FE interfaces at the FEs to the CEs via the internal CE-FE channel.
channel. Similarly, the outbound Hello packets have to go from the Similarly, the outbound Hello packets have to go from the CEs to the
CEs to the FEs and to the external interfaces. Frequent Hello FEs and to the external interfaces. Frequent Hello updates place
updates place heavy processing overhead on the CEs and can heavy processing overhead on the CEs and can overwhelm the CE-FE
overwhelm the CE-FE channel as well. Since typically there are far channel as well. Since typically there are far more FEs than CEs in
more FEs than CEs in a router, the off-loaded Hello packets are a router, the off-loaded Hello packets are processed in a much more
processed in a much more distributed and scalable fashion. By distributed and scalable fashion. By expressing such off-loaded
expressing such off-loaded functions in the FE model, we can ensure functions in the FE model, we can ensure interoperability. However,
interoperability. However, the exact description of the off-loaded the exact description of the off-loaded functionality corresponding
functionality corresponding to the off-loaded functions expressed to the off-loaded functions expressed in the FE model are not part of
in the FE model are not part of the model itself and will need to the model itself and will need to be worked out as a separate
be worked out as a separate specification. specification.
5.7. Application Layer -- Network Management Protocol 5.7. Application Layer -- Network Management Protocol
RFC1812 also dictates "Routers MUST be manageable by SNMP." In RFC 1812 [2] also dictates that "Routers MUST be manageable by SNMP".
general, for post-association phase, most external management tasks In general, for the post-association phase, most external management
(including SNMP) should be done through interaction with the CE in tasks (including SNMP) should be done through interaction with the CE
order to support the appearance of a single functional device. in order to support the appearance of a single functional device.
Therefore, it is recommended that SNMP agent be implemented by CEs Therefore, it is recommended that an SNMP agent be implemented by CEs
and the SNMP messages received by FEs be redirected to their CEs. and that the SNMP messages received by FEs be redirected to their
AgentX framework defined in RFC2741 ([5]) may be applied here such CEs. AgentX framework defined in RFC 2741 ([6]) may be applied here
that CEs act in the role of master agent to process SNMP protocol such that CEs act in the role of master agent to process SNMP
messages while FEs act in the role of subagent to provide access to protocol messages while FEs act in the role of subagent to provide
the MIB objects residing on FEs. AgentX protocol messages between access to the MIB objects residing on FEs. AgentX protocol messages
the master agent (CE) and the subagent (FE) are encapsulated and between the master agent (CE) and the subagent (FE) are encapsulated
transported via ForCES, just like data packets from any other and transported via ForCES, just like data packets from any other
application layer protocols. application layer protocols.
6. Summary 6. Summary
This document defines an architectural framework for ForCES. It This document defines an architectural framework for ForCES. It
identifies the relevant components for a ForCES network element, identifies the relevant components for a ForCES network element,
including (one or more) FEs, (one or more) CEs, one optional FE including (one or more) FEs, (one or more) CEs, one optional FE
manager, and one optional CE manager. It also identifies the manager, and one optional CE manager. It also identifies the
interaction among these components and discusses all the major interaction among these components and discusses all the major
reference points. It is important to point out that, among all the reference points. It is important to point out that, among all the
reference points, only the Fp interface between CEs and FEs is reference points, only the Fp interface between CEs and FEs is within
within the scope of ForCES. ForCES alone may not be enough to the scope of ForCES. ForCES alone may not be enough to support all
support all desirable NE configurations. However, we believe that desirable NE configurations. However, we believe that ForCES over an
ForCES over Fp interface is the most important element in realizing Fp interface is the most important element in realizing physical
physical separation and interoperability of CEs and FEs, and hence separation and interoperability of CEs and FEs, and hence the first
the first interface that ought to be standardized. Simple and interface that ought to be standardized. Simple and useful
useful configurations can still be implemented with only CE-FE configurations can still be implemented with only CE-FE interface
interface being standardized, e.g., single CE with full-meshed FEs. being standardized, e.g., single CE with full-meshed FEs.
7. Acknowledgements 7. Acknowledgements
Joel M. Halpern gave us many insightful comments and suggestions Joel M. Halpern gave us many insightful comments and suggestions and
and pointed out several major issues. T. Sridhar suggested that pointed out several major issues. T. Sridhar suggested that the
the AgentX protocol could be used with SNMP to manage the ForCES AgentX protocol could be used with SNMP to manage the ForCES network
network elements. Susan Hares pointed out the issue of graceful elements. Susan Hares pointed out the issue of graceful restart with
restart with ForCES. Russ Housley, Avri Doria, Jamal Hadi Salim ForCES. Russ Housley, Avri Doria, Jamal Hadi Salim, and many others
and many others in the ForCES mailing list also provided valuable in the ForCES mailing list also provided valuable feedback.
feedback.
8. Security Considerations 8. Security Considerations
The NE administrator has the freedom to determine the exact The NE administrator has the freedom to determine the exact security
security configuration that is needed for the specific deployment. configuration that is needed for the specific deployment. For
For example, ForCES may be deployed between CEs and FEs connected example, ForCES may be deployed between CEs and FEs connected to each
to each other inside a box over a backplane. In such scenario, other inside a box over a backplane. In such a scenario, physical
physical security of the box ensures that most of the attacks such security of the box ensures that most of the attacks, such as man-
as man-in-the-middle, snooping, and impersonation are not possible, in-the-middle, snooping, and impersonation, are not possible, and
and hence ForCES architecture may rely on the physical security of hence the ForCES architecture may rely on the physical security of
the box to defend against these attacks and protocol mechanisms may the box to defend against these attacks and protocol mechanisms may
be turned off. However, it is also shown that denial of service be turned off. However, it is also shown that denial of service
attack via external interface as described below in Section 8.1.8 attacks via external interfaces as described below in Section 8.1.8
is still a potential threat even for such "all-in-one-box" is still a potential threat, even for such an "all-in-one-box"
deployment scenario and hence the rate limiting mechanism is still deployment scenario and hence the rate limiting mechanism is still
necessary. This is just one example to show that it is important necessary. This is just one example to show that it is important to
to assess the security needs of the ForCES-enabled network elements assess the security needs of the ForCES-enabled network elements
under different deployment scenarios. It should be possible for under different deployment scenarios. It should be possible for the
the administrator to configure the level of security needed for the administrator to configure the level of security needed for the
ForCES Protocol. ForCES Protocol.
In general, the physical separation of two entities usually results In general, the physical separation of two entities usually results
in a potentially insecure link between the two entities and hence in a potentially insecure link between the two entities and hence
much stricter security measurements are required. For example, we much stricter security measurements are required. For example, we
pointed out in Section 4.1 that authentication becomes necessary pointed out in Section 4.1 that authentication becomes necessary
between CE manager and FE manager, between CE and CE manager, between the CE manager and FE manager, between the CE and CE manager,
between FE and FE manager in some configurations. The physical and between the FE and FE manager in some configurations. The
separation of CE and FE also imposes serious security requirement physical separation of the CE and FE also imposes serious security
for ForCES Protocol over Fp interface. This section first attempts requirements for the ForCES Protocol over the Fp interface. This
to describe the security threats that may be introduced by the section first attempts to describe the security threats that may be
physical separation of the FEs and the CEs, and then it provides introduced by the physical separation of the FEs and CEs, and then it
recommendation and guidelines for secure operation and management provides recommendations and guidelines for the secure operation and
of ForCES Protocol over Fp interface based on existing standard management of the ForCES Protocol over the Fp interface based on
security solutions. existing standard security solutions.
8.1. Analysis of Potential Threats Introduced by ForCES 8.1. Analysis of Potential Threats Introduced by ForCES
This section provides the threat analysis for ForCES, with a focus This section provides the threat analysis for ForCES, with a focus on
on Fp interface. Each threat is described in details with the the Fp interface. Each threat is described in detail with the
effects on the ForCES Protocol entities or/and the NE as a whole, effects on the ForCES Protocol entities or/and the NE as a whole, and
and the required functionalities that need to be in place to defend the required functionalities that need to be in place to defend the
the threat. threat.
8.1.1. "Join" or "Remove" Message Flooding on CEs 8.1.1. "Join" or "Remove" Message Flooding on CEs
Threats: A malicious node could send a stream of false "join NE" Threats: A malicious node could send a stream of false "join NE" or
or "remove from NE" requests on behalf of non-existent or "remove from NE" requests on behalf of a non-existent or unauthorized
unauthorized FE to legitimate CEs at a very rapid rate and thereby FE to legitimate CEs at a very rapid rate, and thereby creating
create unnecessary state in the CEs. unnecessary state in the CEs.
Effects: If by maintaining state for non-existent or unauthorized Effects: If maintaining state for non-existent or unauthorized FEs, a
FEs, a CE may become unavailable for other processing and hence CE may become unavailable for other processing and hence suffer from
suffer from denial of service (DoS) attack similar to the TCP SYN a denial of service (DoS) attack similar to the TCP SYN DoS. If
DoS. If multiple CEs are used, the unnecessary state information multiple CEs are used, the unnecessary state information may also be
may also be conveyed to multiple CEs via Fr interface (e.g., from conveyed to multiple CEs via the Fr interface (e.g., from the active
the active CE to the stand-by CE) and hence subject multiple CEs to CE to the stand-by CE) and hence subject multiple CEs to a DoS
DoS attack. attack.
Requirement: A CE that receives a "join" or "remove" request Requirement: A CE that receives a "join" or "remove" request should
should not create any state information until it has authenticated not create any state information until it has authenticated the FE
the FE endpoint. endpoint.
8.1.2. Impersonation Attack 8.1.2. Impersonation Attack
Threats: A malicious node can impersonate a CE or FE and send out Threats: A malicious node can impersonate a CE or FE and send out
false messages. false messages.
Effects: The whole NE could be compromised. Effects: The whole NE could be compromised.
Requirement: The CE or FE must authenticate the message as having Requirement: The CE or FE must authenticate the message as having
come from an FE or CE on the list of the authorized ForCES elements come from an FE or CE on the list of the authorized ForCES elements
(provided by the CE or FE Manager in the pre-association phase) (provided by the CE or FE Manager in the pre-association phase)
before accepting and processing it. before accepting and processing it.
8.1.3. Replay Attack 8.1.3. Replay Attack
Threat: A malicious node could replay the entire message previously Threat: A malicious node could replay the entire message previously
sent by an FE or CE entity to get around authentication. sent by an FE or CE entity to get around authentication.
Effect: The NE could be compromised. Effect: The NE could be compromised.
Requirement: Replay protection mechanism needs to be part of the Requirement: A replay protection mechanism needs to be part of the
security solution to defend against this attack. security solution to defend against this attack.
8.1.4. Attack during Fail Over 8.1.4. Attack during Fail Over
Threat: A malicious node may exploit the CE fail-over mechanism to Threat: A malicious node may exploit the CE fail-over mechanism to
take over the control of NE. For example, suppose two CEs, say CE-A take over the control of NE. For example, suppose two CEs, say CE-A
and CE-B, are controlling several FEs. CE-A is active and CE-B is and CE-B, are controlling several FEs. CE-A is active and CE-B is
stand-by. When CE-A fails, CE-B is taking over the active CE stand-by. When CE-A fails, CE-B is taking over the active CE
position. The FEs already had a trusted relationship with CE-A, position. The FEs already had a trusted relationship with CE-A, but
but the FEs may not have the same trusted relationship established the FEs may not have the same trusted relationship established with
with CE-B prior to the fail-over. A malicious node can take over CE-B prior to the fail-over. A malicious node can take over as CE-B
as CE-B if such trusted relationship has not been established prior if such a trusted relationship has not been established prior to or
or during the fail-over. during the fail-over.
Effect: The NE may be compromised after such insecure fail-over. Effect: The NE may be compromised after such insecure fail-over.
Requirement: The level of trust relationship between the stand-by Requirement: The level of trust between the stand-by CE and the FEs
CE and the FEs must be as strong as the one between the active CE must be as strong as the one between the active CE and the FEs. The
and the FEs. The security association between the FEs and the security association between the FEs and the stand-by CE may be
stand-by CE may be established prior to fail-over. If not already established prior to fail-over. If not already in place, such
in place, such security association must be re-established before security association must be re-established before the stand-by CE
the stand-by CE takes over. takes over.
8.1.5. Data Integrity 8.1.5. Data Integrity
Threats: A malicious node may inject false messages to legitimate
Threats: A malicious node may inject false messages to a legitimate
CE or FE. CE or FE.
Effect: An FE or CE receives the fabricated packet and performs Effect: An FE or CE receives the fabricated packet and performs an
incorrect or catastrophic operation. incorrect or catastrophic operation.
Requirement: Protocol messages require integrity protection. Requirement: Protocol messages require integrity protection.
8.1.6. Data Confidentiality 8.1.6. Data Confidentiality
Threat: When FE and CE are physically separated, a malicious node Threat: When FE and CE are physically separated, a malicious node may
may eavesdrop the messages in transit. Some of the messages are eavesdrop the messages in transit. Some of the messages are critical
critical to the functioning of the whole network, while others may to the functioning of the whole network, while others may contain
contain confidential business data. Leaking of such information confidential business data. Leaking of such information may result
may result in compromise even beyond the immediate CE or FE. in compromise even beyond the immediate CE or FE.
Effect: Sensitive information might be exposed between CE and FE. Effect: Sensitive information might be exposed between the CE and FE.
Requirement: Data confidentiality between FE and CE must be Requirement: Data confidentiality between the FE and CE must be
available for sensitive information. available for sensitive information.
8.1.7. Sharing security parameters 8.1.7. Sharing security parameters
Threat: Consider a scenario where several FEs communicating to the Threat: Consider a scenario where several FEs are communicating to
same CE share the same authentication keys for the Fp interface. the same CE and sharing the same authentication keys for the Fp
If any FE or the CE is compromised, all other entities are interface. If any FE or CE is compromised, all other entities are
compromised. compromised.
Effect: The whole NE is compromised. Effect: The whole NE is compromised.
Recommendation: To avoid this side effect, it's better to configure Recommendation: To avoid this side effect, it's better to configure
different security parameters for each FE-CE communication over Fp different security parameters for each FE-CE communication over the
interface. Fp interface.
8.1.8. Denial of Service Attack via External Interface 8.1.8. Denial of Service Attack via External Interface
Threat: When an FE receives a packet that is destined for its CE, Threat: When an FE receives a packet that is destined for its CE, the
the FE forwards the packet over the Fp interface. Malicious node FE forwards the packet over the Fp interface. A malicious node can
can generate huge message storm like routing protocol packets etc. generate a huge message storm like routing protocol packets etc.
through the external Fi/f interface so that the FE has to process through the external Fi/f interface so that the FE has to process and
and forward all packets to CE through Fp interface. forward all packets to the CE through the Fp interface.
Effect: CE encounters resource exhaustion and bandwidth starvation Effect: The CE encounters resource exhaustion and bandwidth
on Fp interface due to an overwhelming number of packets from FEs. starvation on Fp interface due to an overwhelming number of packets
from FEs.
Requirement: Some sort of rate limiting mechanism MUST to be in Requirement: Some sort of rate limiting mechanism MUST be in place at
place at both the FE and CE. Rate Limiter SHOULD be configured at both the FE and CE. The Rate Limiter SHOULD be configured at the FE
the FE for each message type that are being received through Fi/F for each message type being received through the Fi/f interface.
interface.
8.2. Security Recommendations for ForCES 8.2. Security Recommendations for ForCES
The requirements document [3] suggested that ForCES Protocol should The requirements document [4] suggested that the ForCES Protocol
support reliability over Fp interface, but no particular transport should support reliability over the Fp interface, but no particular
protocol is yet specified for ForCES. This framework document does transport protocol is yet specified for ForCES. This framework
not intend to specify the particular transport either, and so we document does not intend to specify the particular transport either,
only provide recommendations and guidelines based on the existing and so we only provide recommendations and guidelines based on the
standard security protocols [18] that can work with the common existing standard security protocols [18] that can work with the
transport candidates suitable for ForCES. common transport candidates suitable for ForCES.
We review two existing security protocol solutions, namely IPsec We review two existing security protocol solutions, namely IPsec (IP
(IP Security) [14] or TLS (Transport Layer Security) [13]. TLS Security) [15] and TLS (Transport Layer Security) [14]. TLS works
works with reliable transports such as TCP or SCTP for unicast, with reliable transports such as TCP or SCTP for unicast, while IPsec
while IPsec can be used with any transport (UDP, TCP, SCTP) and can be used with any transport (UDP, TCP, SCTP) and supports both
supports both unicast and multicast. Both TLS and IPsec can be unicast and multicast. Both TLS and IPsec can be used potentially to
used potentially to satisfy all of the security requirements for satisfy all of the security requirements for the ForCES Protocol. In
ForCES Protocol. In addition, other approaches may be used as well addition, other approaches that satisfy the requirements can be used
but are not documented here, including using L2 security mechanisms as well, but are not documented here, including the use of L2
for a given L2 interconnect technology, as long as the requirements security mechanisms for a given L2 interconnect technology.
can be satisfied.
When ForCES is deployed between CEs and FEs inside a box or a When ForCES is deployed between CEs and FEs inside a box or a
physically secured room, authentication, confidentiality and physically secured room, authentication, confidentiality, and
integrity may be provided by the physical security of the box and integrity may be provided by the physical security of the box. Thus,
so the security mechanisms may be turned off, depending on the the security mechanisms may be turned off, depending on the
networking topology and its administration policy. However, it is networking topology and its administration policy. However, it is
important to realize that even if the NE is in a single-box, the important to realize that even if the NE is in a single-box, the DoS
DoS attacks as described in Section 8.1.8 can still be launched attacks as described in Section 8.1.8 can still be launched through
through Fi/f interfaces. Therefore, it is important to have the the Fi/f interfaces. Therefore, it is important to have the
corresponding counter-measurement in place even for single-box corresponding counter-measurement in place, even for single-box
deployment. deployment.
8.2.1. Using TLS with ForCES 8.2.1. Using TLS with ForCES
TLS [13] can be used if a reliable unicast transport such as TCP or TLS [14] can be used if a reliable unicast transport such as TCP or
SCTP is used for ForCES over the Fp interface. The TLS handshake SCTP is used for ForCES over the Fp interface. The TLS handshake
protocol is used during association establishment or re- protocol is used during the association establishment or re-
establishment phase to negotiate a TLS session between the CE and establishment phase to negotiate a TLS session between the CE and FE.
FE. Once the session is in place, the TLS record protocol is used Once the session is in place, the TLS record protocol is used to
to secure ForCES communication messages between the CE and FE. secure ForCES communication messages between the CE and FE.
A basic outline of how TLS can be used with ForCES is described A basic outline of how TLS can be used with ForCES is described
below. Steps 1) till 7) complete the security handshake as below. Steps 1) through 7) complete the security handshake as
illustrated in Figure 9 while step 8) is for all the further illustrated in Figure 9, while step 8) is for all further
communication between the CE and FE, including the rest of messages communication between the CE and FE, including the rest of the
after the security handshake shown in Figure 9 and the steady-state messages after the security handshake shown in Figure 9 and the
communication shown in Figure 10. steady-state communication shown in Figure 10.
1) During the Pre-association phase, all FEs are configured with the
CEs (including both the active CE and the standby CE).
2) The FE establishes a TLS connection with the CE (master) and
negotiates a cipher suite.
3) The FE (slave) gets the CE certificate, validates the signature,
checks the expiration date, and checks whether the certificate has
been revoked.
4) The CE (master) gets the FE certificate and performs the same
validation as the FE in step 3).
5) If any of the checks fail in step 3) or step 4), the endpoint must
generate an error message and abort.
1) During Pre-association phase all FEs are configured with
the CEs (including both the active CE and the standby CE).
2) The FE establishes a TLS connection with the CE (master)
and negotiates a cipher suite.
3) The FE (slave) gets the CE certificate, validates the
signature, checks the expiration date, checks if the
certificate has been revoked.
4) The CE (master) gets the FE certificate and performs the
same validation as the FE in step 3).
5) If any of the check fails in step 3) or step 4), endpoint
must generate an error message and abort.
6) After successful mutual authentication, a TLS session is 6) After successful mutual authentication, a TLS session is
established between CE and FE. established between the CE and FE.
7) The FE sends a "join NE" message to the CE. 7) The FE sends a "join NE" message to the CE.
8) The FE and CE use TLS session for further communication.
8) The FE and CE use the TLS session for further communication.
Note that there are different ways for the CE and FE to validate a Note that there are different ways for the CE and FE to validate a
received certificate. One way is to configure the FE Manager or CE received certificate. One way is to configure the FE Manager or CE
Manager or other central component as CA, so that the CE or FE can Manager or other central component as CA, so that the CE or FE can
query this pre-configured CA to validate that the certificate has query this pre-configured CA to validate that the certificate has not
not been revoked. Another way is to have the CE and the FE been revoked. Another way is to have the CE and FE directly
configured directly a list of valid certificates in the pre- configure a list of valid certificates in the pre-association phase.
association phase.
In the case of fail-over, it is the responsibility of the active CE In the case of fail-over, it is the responsibility of the active CE
and the standby CE to synchronize ForCES states including the TLS and the standby CE to synchronize ForCES states, including the TLS
states to minimize the state reestablishment during fail-over. states to minimize the state re-establishment during fail-over. Care
Care must be taken to ensure that the standby CE is also must be taken to ensure that the standby CE is also authenticated in
authenticated in the same way as the active CE, either before or the same way as the active CE, either before or during the fail-over.
during the fail-over.
8.2.2. Using IPsec with ForCES 8.2.2. Using IPsec with ForCES
IPsec [14] can be used with any transport protocol, such as UDP, IPsec [15] can be used with any transport protocol, such as UDP,
SCTP and TCP over Fp interface for ForCES. When using IPsec, we SCTP, and TCP, over the Fp interface for ForCES. When using IPsec,
recommend using ESP in transport mode for ForCES because message we recommend using ESP in the transport mode for ForCES because
confidentiality is required for ForCES. message confidentiality is required for ForCES.
IPsec can be used with both manual and automated SA and IPsec can be used with both manual and automated SA and cryptographic
cryptographic key management. But IPsec's replay protection key management. But IPsec's replay protection mechanisms are not
mechanisms are not available if manual key management is used. available if manual key management is used. Hence, automatic key
Hence, automatic key management is recommended if replay protection management is recommended if replay protection is deemed important.
is deemed important. Otherwise, manual key management might be Otherwise, manual key management might be sufficient for some
sufficient for some deployment scenarios, esp. when the number of deployment scenarios, especially when the number of CEs and FEs is
CEs and FEs is relatively small. It is recommended that the keys relatively small. It is recommended that the keys be changed
be changed periodically even for manual key management. periodically, even for manual key management.
IPsec can support both unicast and multicast transport. At the time IPsec can support both unicast and multicast transport. At the time
this document was published, MSEC working group is actively working this document was published, the MSEC working group was actively
on standardizing protocols to provide multicast security [17]. working on standardizing protocols to provide multicast security
Multicast-based solutions relying on IPsec should specify how to [17]. Multicast-based solutions relying on IPsec should specify how
meet the security requirements in [3]. to meet the security requirements in [4].
Unlike TLS, IPsec provides security services between the CE and FE Unlike TLS, IPsec provides security services between the CE and FE at
at IP level, and so the security handshake as illustrated in Figure IP level, so the security handshake, as illustrated in Figure 9
9 amounts to a "no-op" when manual key management is used. The amounts to a "no-op" when manual key management is used. The
following outline the steps taken for ForCES in such a case. following outlines the steps taken for ForCES in such a case.
1) During Pre-association phase all FEs are configured with 1) During the Pre-association phase, all the FEs are configured with
the CEs (including active CE and standby CE) and SA parameters CEs (including the active CE and standby CE) and SA parameters
manually. manually.
2) The FE sends a "join NE" message to the CE. This message
and all others that follow are afforded security service
according to the manually configured IPsec SA parameters, but
replay protection is not available.
It is up to the administrator to decide whether to share the same 2) The FE sends a "join NE" message to the CE. This message and all
key across multiple FE-CE communication, but it is recommended that others that follow are afforded security service according to the
different keys be used. Similarly, it is recommended that manually configured IPsec SA parameters, but replay protection is
different keys be used for inbound and outbound traffic. not available.
If automatic key management is needed, IKE [15] can be used for It is up to the administrator to decide whether to share the same key
that purpose. Other automatic key distribution techniques such as across multiple FE-CE communication, but it is recommended that
Kerberos may be used as well. The key exchange process different keys be used. Similarly, it is recommended that different
constitutes the security handshake as illustrated in Figure 9. The keys be used for inbound and outbound traffic.
following shows the steps involved in using IKE with IPsec for
ForCES. Steps 1) to 6) constitute the security handshake in Figure
9.
1) During Pre-association phase all FEs are configured with If automatic key management is needed, IKE [16] can be used for that
the CEs (including active CE and standby CE), IPsec policy purpose. Other automatic key distribution techniques, such as
etc. Kerberos, may be used as well. The key exchange process constitutes
2) The FE kicks off IKE process and tries to establish an the security handshake as illustrated in Figure 9. The following
IPsec SA with the CE (master). The FE (Slave) gets the CE shows the steps involved in using IKE with IPsec for ForCES. Steps
certificate as part of the IKE negotiation. The FE validates 1) to 6) constitute the security handshake in Figure 9.
signature, checks the expiration date, checks if the
certificate has been revoked.
3) The CE (master) gets the FE certificate and performs the
same check as the FE in step 2).
4) If any of the check fails in step 2) or step 3), the 1) During the Pre-association phase, all FEs are configured with the
endpoint must generate an error message and abort. CEs (including active CE and standby CE), IPsec policy etc.
5) After successful mutual authentication, IPsec session is
2) The FE kicks off the IKE process and tries to establish an IPsec
SA with the CE (master). The FE (Slave) gets the CE certificate
as part of the IKE negotiation. The FE validates the signature,
checks the expiration date, and checks whether the certificate has
been revoked.
3) The CE (master) gets the FE certificate and performs the same
check as the FE in step 2).
4) If any of the checks fail in step 2) or step 3), the endpoint must
generate an error message and abort.
5) After successful mutual authentication, the IPsec session is
established between the CE and FE. established between the CE and FE.
6) The FE sends a "join NE" message to CE. No SADB entry is
6) The FE sends a "join NE" message to the CE. No SADB entry is
created in FE yet. created in FE yet.
7) The FE and CE use the IPsec session for further
communication.
FE Manager or CE Manager or other central component can be used as 7) The FE and CE use the IPsec session for further communication.
CA for validating CE and FE certificates during the IKE process.
Alternatively, during the pre-association phase, the CE and FE can The FE Manager, CE Manager, or other central component can be used as
be configured directly with the required information such as a CA for validating CE and FE certificates during the IKE process.
certificates or passwords etc depending upon the type of Alternatively, during the pre-association phase, the CE and FE can be
configured directly with the required information, such as
certificates or passwords etc., depending upon the type of
authentication that administrator wants to configure. authentication that administrator wants to configure.
In the case of fail-over, it is the responsibility of active CE and In the case of fail-over, it is the responsibility of the active CE
standby CE to synchronize ForCES states and IPsec states to and standby CE to synchronize ForCES states and IPsec states to
minimize the state reestablishment during fail-over. minimize the state re-establishment during fail-over. Alternatively,
Alternatively, the FE needs to establish different IPsec SA during the FE needs to establish a different IPsec SA during the startup
the startup operation itself with each CE. This will minimize the operation itself with each CE. This will minimize the periodic state
periodic state transfer across IPsec layer though Fr (CE-CE) transfer across the IPsec layer though the Fr (CE-CE) Interface.
Interface.
9. Normative References 9. References
[1] Baker, F., "Requirements for IP Version 4 Routers", RFC 1812, 9.1. Normative References
June 1995.
[2] Floyd, S., "Congestion Control Principles", RFC 2914, September [1] Bradner, S., "Key words for use in RFCs to Indicate Requirement
2000. Levels", BCP 14, RFC 2119, March 1997.
[3] Khosravi, H. and Anderson, T., "Requirements for Separation of [2] Baker, F., Ed., "Requirements for IP Version 4 Routers", RFC
IP Control and Forwarding", RFC 3654, November 2003. 1812, June 1995.
10. Informative References [3] Floyd, S., "Congestion Control Principles", BCP 41, RFC 2914,
September 2000.
[4] Case, J., et al., "Introduction and Applicability Statements [4] Khosravi, H. and Anderson, T., Eds., "Requirements for
for Internet Standard Management Framework", RFC 3410, December Separation of IP Control and Forwarding", RFC 3654, November
2002. 2003.
[5] Daniele, M. et al., "Agent Extensibility (AgentX) Protocol 9.2. Informative References
Version 1", RFC 2741, January 2000.
[6] Chan, K. et al., "COPS Usage for Policy Provisioning (COPS- [5] Case, J., Mundy, R., Partain, D. and B. Stewart, "Introduction
PR)", RFC 3084, March 2001. and Applicability Statements for Internet Standard Management
Framework", RFC 3410, December 2002.
[7] Crouch, A. et al., "ForCES Applicability Statement", work in [6] Daniele, M., Wijnen, B., Ellison, M. and D. Francisco, "Agent
progress, June 2003, <draft-ietf-forces-applicability-02.txt>. Extensibility (AgentX) Protocol Version 1", RFC 2741, January
2000.
[8] Anderson, T. and J. Buerkle, "Requirements for the Dynamic [7] Chan, K., Seligson, J., Durham, D., Gai, S., McCloghrie, K.,
Partitioning of Switching Elements", RFC 3532, May 2003. Herzog, S., Reichmeyer, F., Yavatkar, R. and A. Smith, "COPS
Usage for Policy Provisioning (COPS-PR)", RFC 3084, March 2001.
[9] Leelanivas, M. et al., "Graceful Restart Mechanism for Label [8] Crouch, A. et al., "ForCES Applicability Statement", Work in
Distribution Protocol", RFC 3478, February 2003. Progress.
[10] Moy, J. et al., "Graceful OSPF Restart", RFC 3623, November [9] Anderson, T. and J. Buerkle, "Requirements for the Dynamic
Partitioning of Switching Elements", RFC 3532, May 2003.
[10] Leelanivas, M., Rekhter, Y. and R. Aggarwal, "Graceful Restart
Mechanism for Label Distribution Protocol", RFC 3478, February
2003. 2003.
[11] Sangli, S. et al., "Graceful Restart Mechanism for BGP", work [11] Moy, J., Pillay-Esnault, P. and A. Lindem, "Graceful OSPF
in progress, September 2003, < draft-ietf-idr-restart-08.txt>. Restart", RFC 3623, November 2003.
[12] Shand, M. and L. Ginsberg, "Restart Signaling for IS-IS", work [12] Sangli, S. et al., "Graceful Restart Mechanism for BGP", Work in
in progress, July 2003, <draft-ietf-isis-restart-04.txt>. Progress.
[13] Dierks, T. and C. Allen, "The TLS Protocol, version 1.0", RFC [13] Shand, M. and L. Ginsberg, "Restart Signaling for IS-IS", Work
in Progress.
[14] Dierks, T. and C. Allen, "The TLS Protocol Version 1.0", RFC
2246, January 1999. 2246, January 1999.
[14] Kent, S. and R. Atkinson, "Security Architecture for the [15] Kent, S. and R. Atkinson, "Security Architecture for the
Internet Protocol", RFC 2401, November 1998. Internet Protocol", RFC 2401, November 1998.
[15] Harkins, D. and D. Carrel, "The Internet Key Exchange (IKE) ", [16] Harkins, D. and D. Carrel, "The Internet Key Exchange (IKE)",
RFC 2409, November 1998. RFC 2409, November 1998.
[16] Bellovin, S., "Guidelines for Mandating the Use of Ipsec", [17] Hardjono, T. and Weis, B. "The Multicast Group Security
work in progress, October 2003, <draft-bellovin-useipsec-02.txt>. Architecture", RFC 3740, March 2004.
[17] Hardjono, T. and Weis, B. "The Multicast Security
Architecture", work in progress, November 2003, <draft-ietf-msec-
arch-04.txt>.
[18] S. Bellovin, J. Schiller, and C. Kaufman, "Security Mechanisms [18] Bellovin, S., Schiller, J. and C. Kaufman, Eds., "Security
for the Internet", RFC 3631, December, 2003. Mechanisms for the Internet", RFC 3631, December 2003.
11. Authors' Addresses 10. Authors' Addresses
L. Lily Yang L. Lily Yang
Intel Corp., MS JF3-206, Intel Corp., MS JF3-206,
2111 NE 25th Avenue 2111 NE 25th Avenue
Hillsboro, OR 97124, USA Hillsboro, OR 97124, USA
Phone: +1 503 264 8813 Phone: +1 503 264 8813
Email: lily.l.yang@intel.com EMail: lily.l.yang@intel.com
Ram Dantu Ram Dantu
Department of Computer Science, Department of Computer Science,
University of North Texas, University of North Texas,
Denton, TX 76203, USA Denton, TX 76203, USA
Phone: +1 940 565 2822 Phone: +1 940 565 2822
Email: rdantu@unt.edu EMail: rdantu@unt.edu
Todd A. Anderson Todd A. Anderson
Intel Corp. Intel Corp.
2111 NE 25th Avenue 2111 NE 25th Avenue
Hillsboro, OR 97124, USA Hillsboro, OR 97124, USA
Phone: +1 503 712 1760 Phone: +1 503 712 1760
Email: todd.a.anderson@intel.com EMail: todd.a.anderson@intel.com
Ram Gopal Ram Gopal
Nokia Research Center Nokia Research Center
5, Wayside Road, 5, Wayside Road,
Burlington, MA 01803, USA Burlington, MA 01803, USA
Phone: +1 781 993 3685 Phone: +1 781 993 3685
Email: ram.gopal@nokia.com EMail: ram.gopal@nokia.com
12. Intellectual Property Right 11. Full Copyright Statement
Copyright (C) The Internet Society (2004). This document is subject
to the rights, licenses and restrictions contained in BCP 78, and
except as set forth therein, the authors retain all their rights.
This document and the information contained herein are provided on an
"AS IS" basis and THE CONTRIBUTOR, THE ORGANIZATION HE/SHE
REPRESENTS OR IS SPONSORED BY (IF ANY), THE INTERNET SOCIETY AND THE
INTERNET ENGINEERING TASK FORCE DISCLAIM ALL WARRANTIES, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF
THE INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED
WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE.
Intellectual Property
The IETF takes no position regarding the validity or scope of any The IETF takes no position regarding the validity or scope of any
intellectual property or other rights that might be claimed to Intellectual Property Rights or other rights that might be claimed
pertain to the implementation or use of the technology described in to pertain to the implementation or use of the technology
this document or the extent to which any license under such rights described in this document or the extent to which any license
might or might not be available; neither does it represent that it under such rights might or might not be available; nor does it
has made any effort to identify any such rights. Information on represent that it has made any independent effort to identify any
the IETF's procedures with respect to rights in standards-track and such rights. Information on the procedures with respect to
standards-related documentation can be found in RFC 2026. Copies rights in RFC documents can be found in BCP 78 and BCP 79.
of claims of rights made available for publication and any
Copies of IPR disclosures made to the IETF Secretariat and any
assurances of licenses to be made available, or the result of an assurances of licenses to be made available, or the result of an
attempt made to obtain a general license or permission for the use attempt made to obtain a general license or permission for the use
of such proprietary rights by implementors or users of this of such proprietary rights by implementers or users of this
specification can be obtained from the IETF Secretariat. specification can be obtained from the IETF on-line IPR repository
at http://www.ietf.org/ipr.
The IETF invites any interested party to bring to its attention any
copyrights, patents or patent applications, or other proprietary
rights which may cover technology that may be required to practice
this standard. Please address the information to the IETF
Executive Director.
13. Full Copyright Statement
Copyright (C) The Internet Society (2003). All Rights Reserved. The IETF invites any interested party to bring to its attention
any copyrights, patents or patent applications, or other
proprietary rights that may cover technology that may be required
to implement this standard. Please address the information to the
IETF at ietf-ipr@ietf.org.
This document and translations of it may be copied and furnished to Acknowledgement
others, and derivative works that comment on or otherwise explain
it or assist in its implementation may be prepared, copied,
published and distributed, in whole or in part, without restriction
of any kind, provided that the above copyright notice and this
paragraph are included on all such copies and derivative works.
However, this document itself may not be modified in any way, such
as by removing the copyright notice or references to the Internet
Society or other Internet organizations, except as needed for the
purpose of developing Internet standards in which case the
procedures for copyrights defined in the Internet Standards process
must be followed, or as required to translate it into languages
other than English.
The limited permissions granted above are perpetual and will not be Funding for the RFC Editor function is currently provided by the
revoked by the Internet Society or its successors or assigns. Internet Society.
This document and the information contained herein is provided on
an "AS IS" basis and THE INTERNET SOCIETY AND THE INTERNET
ENGINEERING TASK FORCE DISCLAIMS ALL WARRANTIES, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF
THE INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED
WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE."
 End of changes. 

This html diff was produced by rfcdiff 1.25, available from http://www.levkowetz.com/ietf/tools/rfcdiff/