draft-ietf-forces-framework-00.txt   draft-ietf-forces-framework-01.txt 
Internet Draft L. Yang Internet Draft L. Yang
Expiration: December 2002 Intel Labs Expiration: March 2003 Intel Labs
File: draft-ietf-forces-framework-00.txt R. Dantu File: draft-ietf-forces-framework-01.txt R. Dantu
Working Group: ForCES Netrake Corp. Working Group: ForCES Netrake Corp.
T. Anderson T. Anderson
Intel Labs Intel Labs
June 2002 Sept 2002
ForCES Architectural Framework ForCES Architectural Framework
draft-ietf-forces-framework-00.txt draft-ietf-forces-framework-01.txt
Status of this Memo Status of this Memo
This document is an Internet-Draft and is in full conformance with This document is an Internet-Draft and is in full conformance with
all provisions of Section 10 of RFC2026. Internet-Drafts are all provisions of Section 10 of RFC2026. Internet-Drafts are
working documents of the Internet Engineering Task Force (IETF), its working documents of the Internet Engineering Task Force (IETF), its
areas, and its working groups. Note that other groups may also areas, and its working groups. Note that other groups may also
distribute working documents as Internet-Drafts. distribute working documents as Internet-Drafts.
Internet-Drafts are draft documents valid for a maximum of six Internet-Drafts are draft documents valid for a maximum of six
skipping to change at line 100 skipping to change at line 100
could consist of multiple protocols working together. could consist of multiple protocols working together.
FE Manager - A logical entity that operates in the pre-association FE Manager - A logical entity that operates in the pre-association
phase and is responsible for determining to which CE(s) a FE should phase and is responsible for determining to which CE(s) a FE should
communicate. This process is called CE discovery and may involve communicate. This process is called CE discovery and may involve
the FE manager learning the capabilities of available CEs. A FE the FE manager learning the capabilities of available CEs. A FE
manager may use anything from a static configuration to a pre- manager may use anything from a static configuration to a pre-
association phase protocol (see below) to determine which CE(s) to association phase protocol (see below) to determine which CE(s) to
use. Being a logical entity, a FE manager might be physically use. Being a logical entity, a FE manager might be physically
Yang, et. al. Expires December 2002 [Page 2] Yang, et. al. Expires March 2003 [Page 2]
combined with any of the other logical entities mentioned in this combined with any of the other logical entities mentioned in this
section. section.
CE Manager - A logical entity that operates in the pre-association CE Manager - A logical entity that operates in the pre-association
phase and is responsible for determining to which FE(s) a CE should phase and is responsible for determining to which FE(s) a CE should
communicate. This process is called FE discovery and may involve communicate. This process is called FE discovery and may involve
the CE manager learning the capabilities of available FEs. A CE the CE manager learning the capabilities of available FEs. A CE
manager may use anything from a static configuration to a pre- manager may use anything from a static configuration to a pre-
association phase protocol (see below) to determine which FE to use. association phase protocol (see below) to determine which FE to use.
Being a logical entity, a CE manager might be physically combined Being a logical entity, a CE manager might be physically combined
skipping to change at line 155 skipping to change at line 155
and FEs allow these components to be physically separated. This and FEs allow these components to be physically separated. This
physical separation accrues several benefits to the ForCES physical separation accrues several benefits to the ForCES
architecture. Separate components would allow component vendors to architecture. Separate components would allow component vendors to
specialize in one component without having to become experts in all specialize in one component without having to become experts in all
components. It also allows CEs and FEs from different component components. It also allows CEs and FEs from different component
vendors to interoperate with each other and hence it becomes vendors to interoperate with each other and hence it becomes
possible for system vendors to integrate together CEs and FEs from possible for system vendors to integrate together CEs and FEs from
different component vendors. This translates into a lot more design different component vendors. This translates into a lot more design
choices and flexibility to the system vendors. Overall, ForCES will choices and flexibility to the system vendors. Overall, ForCES will
Yang, et. al. Expires December 2002 [Page 3] Yang, et. al. Expires March 2003 [Page 3]
enable rapid innovation in both the control and forwarding planes enable rapid innovation in both the control and forwarding planes
while maintaining interoperability. Scalability is also easily while maintaining interoperability. Scalability is also easily
provided by this architecture in that additional forwarding or provided by this architecture in that additional forwarding or
control capacity can be added to existing network elements without control capacity can be added to existing network elements without
the need for forklift upgrades. the need for forklift upgrades.
One example of such physical separation is at the blade level. One example of such physical separation is at the blade level.
Figure 1 shows an example configuration of a router, with two Figure 1 shows an example configuration of a router, with two
control blades and multiple router (forwarding) blades, all control blades and multiple router (forwarding) blades, all
interconnected into a switch fabric backplane. In such chassis interconnected into a switch fabric backplane. In such chassis
skipping to change at line 209 skipping to change at line 209
Another level of physical separation between the CEs and FEs can be Another level of physical separation between the CEs and FEs can be
at the box level. In such configuration, all the CEs and FEs are at the box level. In such configuration, all the CEs and FEs are
physically separated boxes, interconnected with some kind of high physically separated boxes, interconnected with some kind of high
speed LAN connection (like Gigabit Ethernet). These separated CEs speed LAN connection (like Gigabit Ethernet). These separated CEs
and FEs are only one hop away from each other within a local area and FEs are only one hop away from each other within a local area
network. The CEs and FEs communicate to each other by running network. The CEs and FEs communicate to each other by running
ForCES, and the collection of these CEs and FEs together become one ForCES, and the collection of these CEs and FEs together become one
routing unit to the external world. Figure 2 shows such an example. routing unit to the external world. Figure 2 shows such an example.
Yang, et. al. Expires December 2002 [Page 4] Yang, et. al. Expires March 2003 [Page 4]
In this example, the same physical interconnect (Ethernet) is shared In this example, the same physical interconnect (Ethernet) is shared
for both CE-to-FE and FE-to-FE communication. However, that does not for both CE-to-FE and FE-to-FE communication. However, that does not
have to be the case. One reason to use different interconnect might have to be the case. One reason to use different interconnect might
be that CE-to-FE interconnect does not have to be as fast as the FE- be that CE-to-FE interconnect does not have to be as fast as the FE-
to-FE interconnect, so the more expensive fast ports can be saved to-FE interconnect, so the more expensive fast ports can be saved
for FE-to-FE. There may also be reliability and redundancy benefits for FE-to-FE. The separate interconnects may also provide
for the NE. reliability and redundancy benefits for the NE.
------- ------- ------- -------
| CE1 | | CE2 | | CE1 | | CE2 |
------- ------- ------- -------
^ ^ ^ ^
| | | |
V V V V
============================================ Ethernet ============================================ Ethernet
^ ^ ^ ^ ^ ^
| | | | | |
skipping to change at line 261 skipping to change at line 261
------------------------------------------------- -------------------------------------------------
| | | | | | | | | | | | | |
|LPM Fwd|Meter |Shaper |NAT |Classi-| | |LPM Fwd|Meter |Shaper |NAT |Classi-| |
| | | | |fier | | | | | | |fier | |
------------------------------------------------- -------------------------------------------------
| FE resources | | FE resources |
------------------------------------------------- -------------------------------------------------
Figure 3. Examples of CE and FE functions Figure 3. Examples of CE and FE functions
Yang, et. al. Expires December 2002 [Page 5] Yang, et. al. Expires March 2003 [Page 5]
Some examples of control functions that can be implemented in the CE Some examples of control functions that can be implemented in the CE
include routing protocols like RIP, OSPF and BGP, control and include routing protocols like RIP, OSPF and BGP, control and
signaling protocols like CAC (Call Admission Control), LDP (Label signaling protocols like CAC (Call Admission Control), LDP (Label
Distribution Protocol) for MPLS, etc. Examples of forwarding Distribution Protocol) for MPLS, etc. Examples of forwarding
functions in FE include LPM (longest prefix match) forwarder, functions in FE include LPM (longest prefix match) forwarder,
classifiers, traffic shaper, meter, NAT, etc. Figure 3 shows a classifiers, traffic shaper, meter, NAT, etc. Figure 3 shows a
diagram with examples in both CE and FE. Any given NE may contain diagram with examples in both CE and FE. Any given NE may contain
one or many of these CE and FE functions in it. The diagram also one or many of these CE and FE functions in it. The diagram also
shows that ForCES protocol is used to transport both the control shows that ForCES protocol is used to transport both the control
messages for ForCES itself and the data packets that are messages for ForCES itself and the data packets that are
originated/destined from/to the control functions in CE (e.g., originated/destined from/to the control functions in CE (e.g.,
routing packets). Section 5.2 provides more detail on this. routing packets). Section 5.2.4 provides more detail on this.
A set of requirements for control and forwarding separation is A set of requirements for control and forwarding separation is
identified in [FORCES-REQ]. This document describes a ForCES identified in [FORCES-REQ]. This document describes a ForCES
architecture that satisfies the architectural requirements of that architecture that satisfies the architectural requirements of that
document and defines a framework for ForCES network elements and document and defines a framework for ForCES network elements and
associated entities to facilitate protocol definition. associated entities to facilitate protocol definition.
4. Architecture 4. Architecture
This section defines the ForCES architectural framework and This section defines the ForCES architectural framework and the
associated logical components. This ForCES framework defines associated logical components. This ForCES framework defines
components of ForCES NEs and also includes several ancillary components of ForCES NEs including several ancillary components.
components. These components can be connected in different kinds of These components may be connected in different kinds of topologies
topologies for flexible packet processing. for flexible packet processing.
--------------------------------------- ---------------------------------------
| ForCES Network Element | | ForCES Network Element |
-------------- Fc | -------------- -------------- | -------------- Fc | -------------- -------------- |
| CE Manager |---------+-| CE 1 |------| CE 2 | | | CE Manager |---------+-| CE 1 |------| CE 2 | |
-------------- | | | Fr | | | -------------- | | | Fr | | |
| | -------------- -------------- | | | -------------- -------------- |
| Fl | | | Fp / | | Fl | | | Fp / |
| | Fp| |----------| / | | | Fp| |----------| / |
| | | |/ | | | | |/ |
skipping to change at line 316 skipping to change at line 316
----+--+--+--+----------+--+--+--+----- ----+--+--+--+----------+--+--+--+-----
| | | | | | | | | | | | | | | |
| | | | | | | | | | | | | | | |
Fi/f Fi/f Fi/f Fi/f
Figure 4. ForCES Architectural Diagram Figure 4. ForCES Architectural Diagram
The diagram in Figure 4 shows the logical components of the ForCES The diagram in Figure 4 shows the logical components of the ForCES
architecture and their relationships. There are two kinds of architecture and their relationships. There are two kinds of
Yang, et. al. Expires December 2002 [Page 6] Yang, et. al. Expires March 2003 [Page 6]
components inside a ForCES network element: control element (CE) and components inside a ForCES network element: control element (CE) and
forwarding element (FE). The framework allows multiple instances of forwarding element (FE). The framework allows multiple instances of
CE and FE inside one NE. Each FE contains one or more physical media CE and FE inside one NE. Each FE contains one or more physical media
interfaces for receiving and transmitting packets from/to the interfaces for receiving and transmitting packets from/to the
external world. The aggregation of these FE interfaces becomes the external world. The aggregation of these FE interfaces becomes the
NEs external interfaces. In addition to the external interfaces, NEs external interfaces. In addition to the external interfaces,
there must also exist some kind of interconnect within the NE so there must also exist some kind of interconnect within the NE so
that the CE and FE can communicate with each other, and one FE can that the CE and FE can communicate with each other, and one FE can
forward packets to another FE. The diagram also shows two entities forward packets to another FE. The diagram also shows two entities
outside of the ForCES NE: CE Manager and FE Manager. These two outside of the ForCES NE: CE Manager and FE Manager. These two
entities provide configuration to the corresponding CE or FE in the entities provide configuration to the corresponding CE or FE in the
pre-association phase (see Section 5.1). There is no defined role pre-association phase (see Section 5.1). There is no defined role
for FE Manager and CE Manager in post-association phase, thus these for FE Manager and CE Manager in post-association phase, thus these
logical components are not considered part of the ForCES NE. logical components are not considered part of the ForCES NE.
For convenience, the logical interactions between these components For convenience, the logical interactions between these components
are labeled by reference points Fp, Fc, Ff, Fr, Fl, and Fi. The FE are labeled by reference points Fp, Fc, Ff, Fr, Fl, and Fi, as shown
interface is labeled as Fi/f. More detail is provided in Section 4 in Figure 4. The FE external interfaces are labeled as Fi/f. More
and 5 for each of these reference points. It is important to detail is provided in Section 4 and 5 for each of these reference
understand all these reference points from the architecture point of points. All these reference points are important in understanding
view, however, the ForCES protocol is only defined for one reference the ForCES architecture, however, the ForCES protocol is only
point (Fp). defined over one reference point -- Fp.
The interface between two ForCES NEs is identical to the interface The interface between two ForCES NEs is identical to the interface
between two conventional routers and these two NEs exchange the between two conventional routers and these two NEs exchange the
protocol packets through the interfaces at Fi/f. ForCES NEs connect protocol packets through the external interfaces at Fi/f. ForCES NEs
to existing routers transparently. connect to existing routers transparently.
4.1. Control Elements and Fr Reference Point 4.1. Control Elements and Fr Reference Point
It is not necessary to define any protocols across the Fr reference It is not necessary to define any protocols across the Fr reference
point to enable control and forwarding separation for simple point to enable control and forwarding separation for simple
configurations like single CE and multiple FEs. However, this configurations like single CE and multiple FEs. However, this
architecture permits multiple CEs to be present in a network architecture permits multiple CEs to be present in a network
element. In cases where an implementation uses multiple CEs, it is element. In cases where an implementation uses multiple CEs, it is
expected the invariant that the CEs and FEs together appear as a expected the invariant that the CEs and FEs together appear as a
single NE MUST be maintained. single NE MUST be maintained.
skipping to change at line 369 skipping to change at line 369
and where any request that can be serviced by one of the CEs can and where any request that can be serviced by one of the CEs can
also be serviced by any of the other CEs. In both redundancy and also be serviced by any of the other CEs. In both redundancy and
load sharing, the CEs involved are equivalently capable. The only load sharing, the CEs involved are equivalently capable. The only
difference between these two cases is in terms of how many active difference between these two cases is in terms of how many active
CEs there are. Distributed control is the case where two or more CEs there are. Distributed control is the case where two or more
CEs are concurrently active but where certain requests can only be CEs are concurrently active but where certain requests can only be
serviced by certain CEs. serviced by certain CEs.
When multiple CEs are employed in a ForCES NE, their internal When multiple CEs are employed in a ForCES NE, their internal
organization is considered an implementation issue that is beyond organization is considered an implementation issue that is beyond
the scope of the ForCES effort. CEs are wholly responsible for the scope of ForCES. CEs are wholly responsible for coordinating
Yang, et. al. Expires December 2002 [Page 7] Yang, et. al. Expires March 2003 [Page 7]
coordinating amongst themselves via the Fr reference point to amongst themselves via the Fr reference point to provide consistency
provide consistency and synchronization. However, ForCES does not and synchronization. However, ForCES does not define the
define the implementation or protocols used between CEs, nor does it implementation or protocols used between CEs, nor does it define how
define how to distribute functionality among CEs. Nevertheless, to distribute functionality among CEs. Nevertheless, ForCES will
ForCES will support mechanisms for CE redundancy or fail over, and support mechanisms for CE redundancy or fail over, and it is
it is expected that vendors will provide redundancy or fail over expected that vendors will provide redundancy or fail over solutions
solutions within this framework. within this framework.
4.2. Forwarding Elements and Fi reference point 4.2. Forwarding Elements and Fi reference point
FEs perform per-packet processing and handling as directed by CEs. FEs perform per-packet processing and handling as directed by CEs.
FEs have no initiative of their own. Instead, FEs are slaves and FEs have no initiative of their own. Instead, FEs are slaves and
only do as they are told. FEs may communicate with one or more CEs only do as they are told. FEs may communicate with one or more CEs
concurrently across reference point Fp. FEs have no notion of CE concurrently across reference point Fp. FEs have no notion of CE
redundancy, load sharing, or distributed control. Instead, FEs redundancy, load sharing, or distributed control. Instead, FEs
accept commands from any CE authorized to control them, and it is up accept commands from any CE authorized to control them, and it is up
to the CEs to coordinate among themselves to achieve redundancy, to the CEs to coordinate among themselves to achieve redundancy,
skipping to change at line 424 skipping to change at line 424
issuing commands to the FEs, it might need to recheck the FEs' state issuing commands to the FEs, it might need to recheck the FEs' state
and instruct FEs whether or not it is ok to preserve their current and instruct FEs whether or not it is ok to preserve their current
state. state.
Distributed control can be achieved in the similar fashion, without Distributed control can be achieved in the similar fashion, without
much intelligence on the part of FEs. For example, FEs can be much intelligence on the part of FEs. For example, FEs can be
configured to detect RSVP and BGP protocol packets, and forward RSVP configured to detect RSVP and BGP protocol packets, and forward RSVP
packets to one CE and BGP packets to another CE. Hence, FEs may need packets to one CE and BGP packets to another CE. Hence, FEs may need
to do packet filtering for forwarding packets to specific CEs. to do packet filtering for forwarding packets to specific CEs.
Yang, et. al. Expires December 2002 [Page 8] Yang, et. al. Expires March 2003 [Page 8]
This architecture permits multiple FEs to be present in a NE. This architecture permits multiple FEs to be present in a NE.
[FORCES-REQ] dictates that the ForCES protocol MUST be able to scale [FORCES-REQ] dictates that the ForCES protocol MUST be able to scale
to at least hundreds of FEs (see [FORCES-REQ] Section 5, requirement to at least hundreds of FEs (see [FORCES-REQ] Section 5, requirement
#11). Each of these FEs may potentially have a different set of #11). Each of these FEs may potentially have a different set of
packet processing functions, with different media interfaces. FEs packet processing functions, with different media interfaces. FEs
are responsible for basic maintenance of layer-2 connectivity with are responsible for basic maintenance of layer-2 connectivity with
other FEs and with external entities. Many layer-2 media include other FEs and with external entities. Many layer-2 media include
sophisticated control protocols. The FORCES protocol (over the Fp sophisticated control protocols. The FORCES protocol (over the Fp
reference point) will be able to carry messages for such protools so reference point) will be able to carry messages for such protools so
that, in keeping with the "dumb FE model" the CE can provide that, in keeping with the "dumb FE model" the CE can provide
skipping to change at line 477 skipping to change at line 477
^ | ^ | ^ | ^ |
| | | | | | | |
| v | v | v | v
(a) Full mesh among FE1, FE2 and FE3. (a) Full mesh among FE1, FE2 and FE3.
----------- -----------
| CE | | CE |
----------- -----------
Yang, et. al. Expires December 2002 [Page 9] Yang, et. al. Expires March 2003 [Page 9]
^ ^ ^ ^ ^ ^ ^ ^
/ | | \ / | | \
/------ | | ------\ /------ | | ------\
v v v v v v v v
------- ------- ------- ------- ------- ------- ------- -------
| FE1 |<->| FE2 |<->| FE3 |<->| FE4 | | FE1 |<->| FE2 |<->| FE3 |<->| FE4 |
------- ------- ------- ------- ------- ------- ------- -------
^ | ^ | ^ | ^ | ^ | ^ | ^ | ^ |
| | | | | | | | | | | | | | | |
| v | v | v | v | v | v | v | v
skipping to change at line 530 skipping to change at line 530
CE managers are responsible for determining which FEs a CE should CE managers are responsible for determining which FEs a CE should
control. It is legitimate for CE managers to be hard-coded with the control. It is legitimate for CE managers to be hard-coded with the
knowledge of with which FEs its CEs should communicate. A CE manager knowledge of with which FEs its CEs should communicate. A CE manager
may also be physically embedded into a CE and be implemented as a may also be physically embedded into a CE and be implemented as a
simple keypad or other direct configuration mechanism on the CE. simple keypad or other direct configuration mechanism on the CE.
Finally, CE managers may be physically and logically separate Finally, CE managers may be physically and logically separate
entities that configure the CE with FE information via such entities that configure the CE with FE information via such
mechanisms as COPS-PR [RFC3084] or SNMP [RFC1157]. mechanisms as COPS-PR [RFC3084] or SNMP [RFC1157].
Yang, et. al. Expires December 2002 [Page 10] Yang, et. al. Expires March 2003 [Page 10]
4.4. FE Managers 4.4. FE Managers
FE managers are responsible for determining to which CE any FE managers are responsible for determining to which CE any
particular FE should initially communicate. Like CE managers, no particular FE should initially communicate. Like CE managers, no
restrictions are placed on how a FE manager decides to which CEs its restrictions are placed on how a FE manager decides to which CEs its
FEs should communicate, nor are restrictions placed on how FE FEs should communicate, nor are restrictions placed on how FE
managers are implemented. managers are implemented.
5. Operational Phases 5. Operational Phases
skipping to change at line 584 skipping to change at line 584
CE managers and FE managers may communicate across the Fl reference CE managers and FE managers may communicate across the Fl reference
point in the pre-association phase in order to determine which CEs point in the pre-association phase in order to determine which CEs
and FEs should communicate with each other. Communication across and FEs should communicate with each other. Communication across
the Fl reference point is optional in this architecture. No the Fl reference point is optional in this architecture. No
requirements are placed on this reference point. requirements are placed on this reference point.
CE managers and FE managers may be operated by different entities. CE managers and FE managers may be operated by different entities.
The operator of the CE manager may not want to divulge, except to The operator of the CE manager may not want to divulge, except to
specified FE managers, any characteristics of the CEs it manages. specified FE managers, any characteristics of the CEs it manages.
Yang, et. al. Expires December 2002 [Page 11] Yang, et. al. Expires March 2003 [Page 11]
Similarly, the operator of the FE manager may not want to divulge FE Similarly, the operator of the FE manager may not want to divulge FE
characteristics, except to authorized entities. As such, CE characteristics, except to authorized entities. As such, CE
managers and FE managers may need to authenticate one another. managers and FE managers may need to authenticate one another.
Subsequent communication between CE managers and FE managers may Subsequent communication between CE managers and FE managers may
require other security functions such as privacy, non-repudiation, require other security functions such as privacy, non-repudiation,
freshness, and integrity. freshness, and integrity.
Once the necessary security functions have been performed, the CE Once the necessary security functions have been performed, the CE
and FE managers communicate to determine which CEs and FEs should and FE managers communicate to determine which CEs and FEs should
communicate with each other. At the very minimum, the CE and FE communicate with each other. At the very minimum, the CE and FE
skipping to change at line 637 skipping to change at line 637
established, FE manager instructs FEs across this reference point to established, FE manager instructs FEs across this reference point to
join a new NE or to disconnect from an existing NE. Figure 8 shows join a new NE or to disconnect from an existing NE. Figure 8 shows
example of message exchange over Ff reference point. example of message exchange over Ff reference point.
Note that when the FE manager function may be co-located with the FE Note that when the FE manager function may be co-located with the FE
(such as by manual keypad entry of the CE IP address), in which case (such as by manual keypad entry of the CE IP address), in which case
this reference point is reduced to a built-in function. this reference point is reduced to a built-in function.
5.1.3. Fc Reference Point 5.1.3. Fc Reference Point
Yang, et. al. Expires December 2002 [Page 12] Yang, et. al. Expires March 2003 [Page 12]
The Fc reference point is used to inform control elements of the The Fc reference point is used to inform control elements of the
association decisions made by CE managers in pre-association phase. association decisions made by CE managers in pre-association phase.
When the CE manager is remote, only authorized entities may instruct When the CE manager is remote, only authorized entities may instruct
a CE to control certain FEs. Privacy, integrity, freshness and a CE to control certain FEs. Privacy, integrity, freshness and
authentication are also required across this reference point in such authentication are also required across this reference point in such
a configuration. Once appropriate security has been established, a configuration. Once appropriate security has been established,
the CE manager instructs CEs as to which FEs they should control and the CE manager instructs CEs as to which FEs they should control and
how they should control them. Figure 7 shows example of message how they should control them. Figure 7 shows example of message
exchange over Fc reference point. exchange over Fc reference point.
skipping to change at line 692 skipping to change at line 692
As an example, figure 9 shows some of the message exchange that need As an example, figure 9 shows some of the message exchange that need
to happen before the association between CE and FE is fully to happen before the association between CE and FE is fully
established. Typically, FE would need to inform the CE of its own established. Typically, FE would need to inform the CE of its own
capability and its topology in relation to other FEs. The capability capability and its topology in relation to other FEs. The capability
of FE is represented by FE model, described in another separate of FE is represented by FE model, described in another separate
document. The model would allow FE to describe what kind of packet document. The model would allow FE to describe what kind of packet
processing functions it contains, in what order these processing processing functions it contains, in what order these processing
happen, what kind of configurable parameters it allows, what happen, what kind of configurable parameters it allows, what
Yang, et. al. Expires December 2002 [Page 13] Yang, et. al. Expires March 2003 [Page 13]
statistics it collects and what events it might throw, etc. Once statistics it collects and what events it might throw, etc. Once
such information is available to CE, CE sends all the necessary such information is available to CE, CE sends all the necessary
configuration to FE so that FE can start receiving and processing configuration to FE so that FE can start receiving and processing
packets. For example, CE might need to send a snapshot of the packets. For example, CE might need to send a snapshot of the
current routing table to FE so that FE can start routing packets current routing table to FE so that FE can start routing packets
correctly. Once FE starts accepting packets for processing, we say correctly. Once FE starts accepting packets for processing, we say
the association of this FE with its CE is now established. From then the association of this FE with its CE is now established. From then
on, CE and FE enter steady-state communication as described in on, CE and FE enter steady-state communication as described in
5.2.2. 5.2.2.
skipping to change at line 741 skipping to change at line 741
| | | |
|(Go ahead!) | |(Go ahead!) |
10|<----------------------| 10|<----------------------|
| | | |
Figure 9. Example of message exchange between CE and FE Figure 9. Example of message exchange between CE and FE
over Fp to establish NE association over Fp to establish NE association
5.2.3. Steady-state Communication 5.2.3. Steady-state Communication
Yang, et. al. Expires December 2002 [Page 14] Yang, et. al. Expires March 2003 [Page 14]
Once an association is established between the CE and the FE, the Once an association is established between the CE and the FE, the
ForCES protocol is used by the CE and the FE over Fp reference point ForCES protocol is used by the CE and the FE over Fp reference point
to exchange information to facilitate packet processing. to exchange information to facilitate packet processing.
FE CE FE CE
| | | |
|(Add these new routes.)| |(Add these new routes.)|
1|<----------------------| 1|<----------------------|
| | | |
|(Successful.) | |(Successful.) |
skipping to change at line 796 skipping to change at line 796
Control packets (such as RIP, OSPF messages) addressed to any of Control packets (such as RIP, OSPF messages) addressed to any of
NE's interfaces are typically redirected by the receiving FE to its NE's interfaces are typically redirected by the receiving FE to its
CE, and CE may originate packets and have its FE deliver them to CE, and CE may originate packets and have its FE deliver them to
other NEs. Therefore, the communication across the Fp reference other NEs. Therefore, the communication across the Fp reference
point includes not only the control messages from CEs to FEs and the point includes not only the control messages from CEs to FEs and the
status or statistics report from FEs to CEs, but also the data status or statistics report from FEs to CEs, but also the data
packets that are redirected between them. Moreover, one FE may be packets that are redirected between them. Moreover, one FE may be
controlled by multiple CEs. In this configuration, the control controlled by multiple CEs. In this configuration, the control
protocols supported by the FORCES NEs may spread across multiple protocols supported by the FORCES NEs may spread across multiple
Yang, et. al. Expires December 2002 [Page 15] Yang, et. al. Expires March 2003 [Page 15]
CEs. For example, one CE may support OSPF and another supports BGP. CEs. For example, one CE may support OSPF and another supports BGP.
FEs are configured to recognize these protocol packets and forward FEs are configured to recognize these protocol packets and forward
them to the corresponding CE. them to the corresponding CE.
Figure 11 shows one example of how the OSPF packets originated by Figure 11 shows one example of how the OSPF packets originated by
router A are passed to router B. In this example, ForCES protocol is router A are passed to router B. In this example, ForCES protocol is
used to transport the packets from CE to FE inside router A, and used to transport the packets from CE to FE inside router A, and
then from FE to CE inside router B. In light of the fact that ForCES then from FE to CE inside router B. In light of the fact that ForCES
protocol is responsible to transport both the control messages and protocol is responsible to transport both the control messages and
the data packets between CE and FE over Fp reference point, it is the data packets between CE and FE over Fp reference point, it is
skipping to change at line 849 skipping to change at line 849
form of communication (e.g., a non-ForCES protocol or DMA). In such form of communication (e.g., a non-ForCES protocol or DMA). In such
an implementation, the combination of the proxy and the physical FE an implementation, the combination of the proxy and the physical FE
becomes one logical FE entity. becomes one logical FE entity.
5.3. Association Re-establishment 5.3. Association Re-establishment
FEs and CEs may join and leave NEs dynamically (see [FORCES-REQ] FEs and CEs may join and leave NEs dynamically (see [FORCES-REQ]
Section 5, requirements #12 and #13). When a FE or CE leaves the NE, Section 5, requirements #12 and #13). When a FE or CE leaves the NE,
the association with the NE is broken. If the leaving party rejoins the association with the NE is broken. If the leaving party rejoins
Yang, et. al. Expires December 2002 [Page 16] Yang, et. al. Expires March 2003 [Page 16]
a NE later, to re-establish the association, it may or may not need a NE later, to re-establish the association, it may or may not need
to re-enter the pre-association phase. Loss of association can also to re-enter the pre-association phase. Loss of association can also
happen unexpectedly due to loss of connection between the CE and the happen unexpectedly due to loss of connection between the CE and the
FE. Therefore, the framework allows the bi-directional transition FE. Therefore, the framework allows the bi-directional transition
between these two phases, but the ForCES protocol is only applicable between these two phases, but the ForCES protocol is only applicable
for the post-association phase. However, the protocol should provide for the post-association phase. However, the protocol should provide
mechanisms to support association re-establishment (see [FORCES-REQ] mechanisms to support association re-establishment (see [FORCES-REQ]
Section 5, requirement #7). Section 5, requirement #7).
Let's use the example in Figure 5 to see what happens when the Let's use the example in Figure 5 to see what happens when the
skipping to change at line 903 skipping to change at line 903
if it is needed. if it is needed.
6. Applicability to RFC1812 6. Applicability to RFC1812
[FORCES-REQ] Section 5, requirement #9 dictates that "All proposed [FORCES-REQ] Section 5, requirement #9 dictates that "All proposed
ForCES architecture MUST explain how that architecture may be ForCES architecture MUST explain how that architecture may be
applied to support all of a router's functions as defined in applied to support all of a router's functions as defined in
[RFC1812]." RFC1812 discusses many important requirements for IPv4 [RFC1812]." RFC1812 discusses many important requirements for IPv4
routers from the link layer to the application layer. This section routers from the link layer to the application layer. This section
Yang, et. al. Expires December 2002 [Page 17] Yang, et. al. Expires March 2003 [Page 17]
addresses the relevant requirements for implementing IPv4 routers addresses the relevant requirements for implementing IPv4 routers
based on ForCES architecture and how ForCES satisfies these based on ForCES architecture and how ForCES satisfies these
requirements. requirements.
6.1. General Router Requirements 6.1. General Router Requirements
Routers have at least two or more logical interfaces. When CEs and Routers have at least two or more logical interfaces. When CEs and
FEs are separated by ForCES within a single NE, some additional FEs are separated by ForCES within a single NE, some additional
interfaces are needed for intra-NE communications. Figure 12 shows interfaces are needed for intra-NE communications. Figure 12 shows
an example to illustrate that. This NE contains one CE and two FEs. an example to illustrate that. This NE contains one CE and two FEs.
skipping to change at line 957 skipping to change at line 957
that are required to support application layer protocols (see that are required to support application layer protocols (see
[RFC1812] Section 6). One important class of application protocols [RFC1812] Section 6). One important class of application protocols
is routing protocols (see [RFC1812] Section 7). In ForCES is routing protocols (see [RFC1812] Section 7). In ForCES
architecture, routing protocols are naturally implemented by CEs. architecture, routing protocols are naturally implemented by CEs.
Routing protocols require routers communicate with each other. This Routing protocols require routers communicate with each other. This
communication between CEs in different routers is supported in communication between CEs in different routers is supported in
ForCES by the FEs' ability to redirect data packets ForCES by the FEs' ability to redirect data packets
addressed to routers (i.e., NEs) and CEs' ability to originate addressed to routers (i.e., NEs) and CEs' ability to originate
packets and have them delivered by their FEs. This communication packets and have them delivered by their FEs. This communication
Yang, et. al. Expires December 2002 [Page 18] Yang, et. al. Expires March 2003 [Page 18]
occurs across Fp reference point inside each router and between occurs across Fp reference point inside each router and between
neighboring routers' interfaces, as illustrated in Figure 11. neighboring routers' interfaces, as illustrated in Figure 11.
6.2.Link Layer 6.2.Link Layer
Since FEs own all the external interfaces for the router, FEs need Since FEs own all the external interfaces for the router, FEs need
to conform to all the link layer requirements in RFC1812, including to conform to the link layer requirements in RFC1812. Theoretically,
ARP support. Interface parameters, including MTU, IP address, etc., ARP support may be implemented in either CEs or FEs. As we will see
must be configurable by CEs via ForCES. FEs must also inform CEs the later, a number of behaviors that RFC1812 mandates fall into this
status change of the interfaces (like link up/down) via ForCES. category -- they may be performed by the FE and may be performed by
the CE. A general guideline is needed to ensure interoperability
between separated control and forwarding planes. The guideline we
offer here is that CEs are required to be capable of these
operations while FEs may or may not choose to implement them. FE
model should indicate its capabilities in this regard.
Interface parameters, including MTU, IP address, etc., must be
configurable by CEs via ForCES. CEs must be able to determine
whether a physical interface in an FE is available to send packets
or not. FEs must also inform CEs the status change of the interfaces
(like link up/down) via ForCES.
6.3.Internet Layer Protocols 6.3.Internet Layer Protocols
Both FEs and CEs must implement IP protocol and all mandatory Both FEs and CEs must implement IP protocol and all mandatory
options as RFC1812 specified. If an FE receives a packet containing extensions as RFC1812 specified. CEs should implement IP options
a completed source route, the packet has reached its final like source route and record route while FEs may choose to implement
destination. The option as received (the recorded route) MUST be those as well. Timestamp option should be implemented by FEs to
passed up to the transport layer (or to ICMP message processing) at insert the timestamp most accurately. FE must interpret the IP
CEs. Both FEs and CEs might choose to silently discard packets options that it understands and preserve the rest unchanged for use
without sending ICMP errors, and such events should be logged. FEs by CEs. Both FEs and CEs might choose to silently discard packets
can report statistics for such events to CEs via ForCES. without sending ICMP errors, but such events should be logged and
counted. FEs can report statistics for such events to CEs via
ForCES.
When multiple FEs are involved to process packets, the appearance of When multiple FEs are involved to process packets, the appearance of
single NE must be strictly maintained. For example, Time-To-Live single NE must be strictly maintained. For example, Time-To-Live
(TTL) must be decremented only once within a single NE. For example, (TTL) must be decremented only once within a single NE. For example,
it can be always decremented by the last FE with egress function. it can be always decremented by the last FE with egress function.
Similarly, when source
When IP multicast is supported in routers, IGMP is implemented in FEs must receive and process normally any packets with a broadcast
CEs. CEs are also required of ICMP support, while it is optional for destination address or a multicast destination address that the
FEs to support ICMP. Such an option can be communicated to CEs as router has asked to receive. When IP multicast is supported in
part of the FE model. Therefore, FEs can always rely upon CEs to routers, IGMP is implemented in CEs. CEs are also required of ICMP
send out ICMP error messages, but FEs also have the option to support, while it is optional for FEs to support ICMP. Such an
generate ICMP error messages themselves. option can be communicated to CEs as part of the FE model.
Therefore, FEs can always rely upon CEs to send out ICMP error
messages, but FEs also have the option to generate ICMP error
messages themselves.
6.4.Internet Layer Forwarding 6.4.Internet Layer Forwarding
Yang, et. al. Expires March 2003 [Page 19]
IP forwarding is implemented by FEs. After routing protocol update IP forwarding is implemented by FEs. After routing protocol update
its routing tables at CEs, ForCES is used to send the new routing its routing tables at CEs, ForCES is used to send the new routing
table entries from CEs to FEs. Each FE has its own routing table and table entries from CEs to FEs. Each FE has its own routing table and
uses this table to direct packets to the next hop interface. Note uses this table to direct packets to the next hop interface.
that the routing table could be different from FE to FE because each
FE has a different set of next hop interfaces.
Upon receiving IP packets, FEs verify the IP header and process most
of the IP options. Some options can't be processed until the routing
decision has been made. Routing decision is made after examining the
destination IP address. If the destination address belongs to the
router itself, the packets are forwarded to CEs. Otherwise, FEs
determine the next hop IP address by looking up in its routing
table. FEs also determine the network interface it uses to send the
packets. Sometimes an FE may need to forward the packets to another
FE before packets can be forwarded out to the next hop. Right before
Yang, et. al. Expires December 2002 [Page 19] Upon receiving IP packets, FE verifies the IP header and process
packets are forwarded out to the next hop, FEs decrement TTL by 1 most of the IP options. Some options can't be processed until the
and process any IP options that can not be processed before. FEs routing decision has been made. Routing decision is made after
perform any IP fragmentation if necessary, determine link layer examining the destination IP address. If the destination address
address (e.g., by ARP), and encapsulates the IP datagram (or each of belongs to the router itself, the packets are forwarded to CE.
the fragments thereof) in an appropriate link layer frame and queues Otherwise, FE determines the next hop IP address by looking up in
it for output on the interface selected. its routing table. FE also determines the network interface it uses
to send the packets. Sometimes FE may need to forward the packets to
another FE before packets can be forwarded out to the next hop.
Right before packets are forwarded out to the next hop, FE
decrements TTL by 1 and processes any IP options that cannot be
processed before. FE performs any IP fragmentation if necessary,
determines link layer address (e.g., by ARP), and encapsulates the
IP datagram (or each of the fragments thereof) in an appropriate
link layer frame and queues it for output on the interface selected.
Other options mentioned in RFC1812 for IP forwarding may also be Other options mentioned in RFC1812 for IP forwarding may also be
implemented at FEs, for example, packet filtering. implemented at FEs, for example, packet filtering.
FEs typically forward packets destined locally to CEs. FEs may also FEs typically forward packets destined locally to CEs. FEs may also
forward exceptional packets (packets that FEs don't know how to forward exceptional packets (packets that FEs don't know how to
handle) to CEs. CEs are required to handle packets forwarded by FEs handle) to CEs. CEs are required to handle packets forwarded by FEs
for whatever different reasons. It might be necessary for ForCES to for whatever different reasons. It might be necessary for ForCES to
attach some meta-data with the packets to indicate the reasons of attach some meta-data with the packets to indicate the reasons of
forwarding from FEs to CEs. Upon receiving packets with meta-data forwarding from FEs to CEs. Upon receiving packets with meta-data
skipping to change at line 1048 skipping to change at line 1060
forwarding, the entire NE may continue forwarding packets, but it forwarding, the entire NE may continue forwarding packets, but it
needs to stop advertising routes that are affected by the failed FE. needs to stop advertising routes that are affected by the failed FE.
6.5.Transport Layer 6.5.Transport Layer
Transport layer is typically implemented at CEs to support higher Transport layer is typically implemented at CEs to support higher
layer application protocols like routing protocols. In practice, layer application protocols like routing protocols. In practice,
this means that most CEs implement both the Transmission Control this means that most CEs implement both the Transmission Control
Protocol (TCP) and the User Datagram Protocol (UDP). Protocol (TCP) and the User Datagram Protocol (UDP).
Both CEs and FEs need to implement ForCES protocol. If any layer-4 Both CEs and FEs need to implement ForCES protocol. If some layer-4
transport is used to support ForCES, then both CEs and FEs need to transport is used to support ForCES, then both CEs and FEs need to
implement the L4 transport and ForCES protocols. It is possible that implement the L4 transport and ForCES protocols. It is possible that
all FEs inside an NE implements only one such protocol entity. all FEs inside an NE implements only one such protocol entity.
Yang, et. al. Expires March 2003 [Page 20]
6.6. Application Layer -- Routing Protocols 6.6. Application Layer -- Routing Protocols
Both interior routing protocols and exterior routing protocols are Both interior routing protocols and exterior routing protocols are
implemented on CEs. The routing packets originated by CEs are implemented on CEs. The routing packets originated by CEs are
forwarded to FEs for delivery. The results of such protocols (like forwarded to FEs for delivery. The results of such protocols (like
routing table update) are communicated to FEs via ForCES. routing table update) are communicated to FEs via ForCES.
6.7. Application Layer -- Network Management Protocol 6.7. Application Layer -- Network Management Protocol
RFC1812 also dictates "Routers MUST be manageable by SNMP." (see RFC1812 also dictates "Routers MUST be manageable by SNMP." (see
[RFC1157] Section 8) In general, for post-association phase, most [RFC1157] Section 8) In general, for post-association phase, most
external management tasks (including SNMP) SHOULD be done through external management tasks (including SNMP) SHOULD be done through
interaction with the CE in order to support the appearance of a interaction with the CE in order to support the appearance of a
Yang, et. al. Expires December 2002 [Page 20]
single functional device. Therefore, it is recommended that SNMP single functional device. Therefore, it is recommended that SNMP
management agent be implemented by CEs and the SNMP messages sent to management agent be implemented by CEs and the SNMP messages sent to
FEs be redirected to their CEs. This also requires that ForCES In FEs be redirected to their CEs. This also requires that ForCES In
certain conditions (e.g. CE/FE disconnection), it may be useful to certain conditions (e.g. CE/FE disconnection), it may be useful to
allow SNMP to be used to diagnose and repair problems. However, care allow SNMP to be used to diagnose and repair problems. However, care
should be taken when exercising such mechanisms and guidelines are should be taken when exercising such mechanisms and guidelines are
provided in [FORCES-REQ], Section 5, requirement #4. provided in [FORCES-REQ], Section 5, requirement #4.
7. Summary 7. Summary
skipping to change at line 1090 skipping to change at line 1101
including (one or more) FEs, (one or more) CEs, FE manager including (one or more) FEs, (one or more) CEs, FE manager
(optional), and CE manager (optional). It also identifies the (optional), and CE manager (optional). It also identifies the
interaction among these components and discusses all the major interaction among these components and discusses all the major
reference points. It is important to point out that, among all the reference points. It is important to point out that, among all the
reference points, only the interface between CEs and FEs is within reference points, only the interface between CEs and FEs is within
the scope of ForCES. ForCES alone may not be enough to support all the scope of ForCES. ForCES alone may not be enough to support all
different NE configurations. However, we believe ForCES is the most different NE configurations. However, we believe ForCES is the most
important element in realizing the physical separation and important element in realizing the physical separation and
interoperability of CEs and FEs, and hence the first interface that interoperability of CEs and FEs, and hence the first interface that
ought to be standardized. Simple and useful configurations can still ought to be standardized. Simple and useful configurations can still
be implemented with only CE-FE interface standardized, e.g., single be implemented with only CE-FE interface being standardized, e.g.,
CE with full-meshed FEs and static configuration without the need single CE with full-meshed FEs and static configuration without the
for CE/FE managers. need for CE/FE managers.
8. Security Considerations 8. Security Considerations
The security necessary across each reference point except Fp is The security necessary across each reference point except Fp is
discussed throughout the document. In general, the physical discussed throughout the document. In general, the physical
separation of two entities usually requires much stricter security separation of two entities usually requires much stricter security
measurement in place. For example, we pointed out in Section 5.1 measurement in place. For example, we pointed out in Section 5.1
that authentication becomes necessary between CE manager and FE that authentication becomes necessary between CE manager and FE
manager, between CE and CE manager, between FE and FE manager in manager, between CE and CE manager, between FE and FE manager in
some configuration. The physical separation of CE and FE also some configuration. The physical separation of CE and FE also
imposes serious security requirement for ForCES protocol. The imposes serious security requirement for ForCES protocol. The
security requirements for reference point Fp (i.e., ForCES protocol) security requirements for reference point Fp (i.e., ForCES protocol)
are discussed in detail in [FORCES-REQ] Section 8. are discussed in detail in [FORCES-REQ] Section 8.
Yang, et. al. Expires March 2003 [Page 21]
9. Intellectual Property Right 9. Intellectual Property Right
The authors are not aware of any intellectual property right issues The authors are not aware of any intellectual property right issues
pertaining to this document. pertaining to this document.
10. Normative References 10. Normative References
[RFC2914] S. Floyd, "Congestion Control Principles", RFC2914, [RFC2914] S. Floyd, "Congestion Control Principles", RFC2914,
September 2000. September 2000.
[RFC1157] J. Case, et. al., "A Simple Network Management Protocol [RFC1157] J. Case, et. al., "A Simple Network Management Protocol
(SNMP)", RFC1157, May 1990. (SNMP)", RFC1157, May 1990.
Yang, et. al. Expires December 2002 [Page 21]
[RFC3084] K. Chan, et. al., "COPS Usage for Policy Provisioning [RFC3084] K. Chan, et. al., "COPS Usage for Policy Provisioning
(COPS-PR)", RFC3084, March 2001. (COPS-PR)", RFC3084, March 2001.
[RFC1812] F. Baker, "Requirements for IP Version 4 Routers", [RFC1812] F. Baker, "Requirements for IP Version 4 Routers",
RFC1812, June 1995. RFC1812, June 1995.
11. Informative References 11. Informative References
[FORCES-REQ] T. Anderson, et. al., "Requirements for Separation of [FORCES-REQ] T. Anderson, et. al., "Requirements for Separation of
IP Control and Forwarding", work in progress, February 2002, <draft- IP Control and Forwarding", work in progress, February 2002, <draft-
ietf-forces-requirements-02.txt>. ietf-forces-requirements-02.txt>.
[FORCES-APP] A. Crouch, et. al., "ForCES Applicability Statement", [FORCES-APP] A. Crouch, et. al., "ForCES Applicability Statement",
work in progress, February 2002, <draft-ietf-forces-applicability- work in progress, February 2002, <draft-ietf-forces-applicability-
00.txt>. 00.txt>.
12. Acknowledgments 12. Acknowledgments
Joel M. Halpern gave us many insightful comments and suggestions and Joel M. Halpern gave us many insightful comments and suggestions and
pointed out several major issues. Many of our colleagues and people pointed out several major issues. Many of our colleagues and people
in the ForCES mailing list also provided valuable feedback, in the ForCES mailing list also provided valuable feedback.
including Alan Crouch, Patrick Droz, David Putzolu, Hormuzd M.
Khosravi, and Ryszard Dyrga.
13. Authors' Addresses 13. Authors' Addresses
Lily L. Yang Lily L. Yang
Intel Labs Intel Labs
2111 NE 25th Avenue 2111 NE 25th Avenue
Hillsboro, OR 97124 USA Hillsboro, OR 97124 USA
Phone: +1 503 264 8813 Phone: +1 503 264 8813
Email: lily.l.yang@intel.com Email: lily.l.yang@intel.com
Ram Dantu Ram Dantu
Netrake Corporation Netrake Corporation
3000 Technology Drive 3000 Technology Drive
Plano, Texas 75074 Plano, Texas 75074
Phone: +1 214 291 1111 Phone: +1 214 291 1111
Email: ramd@netrake.com Email: ramd@netrake.com
Todd A. Anderson Todd A. Anderson
Yang, et. al. Expires March 2003 [Page 22]
Intel Labs Intel Labs
2111 NE 25th Avenue 2111 NE 25th Avenue
Hillsboro, OR 97124 USA Hillsboro, OR 97124 USA
Phone: +1 503 712 1760 Phone: +1 503 712 1760
Email: todd.a.anderson@intel.com Email: todd.a.anderson@intel.com
1. Abstract........................................................1 1. Abstract........................................................1
2. Definitions.....................................................1 2. Definitions.....................................................1
Yang, et. al. Expires December 2002 [Page 22]
3. Introduction to Forwarding and Control Element Separation 3. Introduction to Forwarding and Control Element Separation
(ForCES)...........................................................3 (ForCES)...........................................................3
4. Architecture....................................................6 4. Architecture....................................................6
4.1. Control Elements and Fr Reference Point....................7 4.1. Control Elements and Fr Reference Point....................7
4.2. Forwarding Elements and Fi reference point.................8 4.2. Forwarding Elements and Fi reference point.................8
4.3. CE Managers...............................................10 4.3. CE Managers...............................................10
4.4. FE Managers...............................................11 4.4. FE Managers...............................................11
5. Operational Phases.............................................11 5. Operational Phases.............................................11
5.1. Pre-association Phase.....................................11 5.1. Pre-association Phase.....................................11
5.1.1. Fl Reference Point...................................11 5.1.1. Fl Reference Point...................................11
skipping to change at line 1197 skipping to change at line 1206
5.2.3. Steady-state Communication...........................14 5.2.3. Steady-state Communication...........................14
5.2.4. Data Packets across Fp reference point...............15 5.2.4. Data Packets across Fp reference point...............15
5.2.5. Proxy FE.............................................16 5.2.5. Proxy FE.............................................16
5.3. Association Re-establishment..............................16 5.3. Association Re-establishment..............................16
6. Applicability to RFC1812.......................................17 6. Applicability to RFC1812.......................................17
6.1. General Router Requirements...............................18 6.1. General Router Requirements...............................18
6.2. Link Layer................................................19 6.2. Link Layer................................................19
6.3. Internet Layer Protocols..................................19 6.3. Internet Layer Protocols..................................19
6.4. Internet Layer Forwarding.................................19 6.4. Internet Layer Forwarding.................................19
6.5. Transport Layer...........................................20 6.5. Transport Layer...........................................20
6.6. Application Layer -- Routing Protocols....................20 6.6. Application Layer -- Routing Protocols....................21
6.7. Application Layer -- Network Management Protocol..........20 6.7. Application Layer -- Network Management Protocol..........21
7. Summary........................................................21 7. Summary........................................................21
8. Security Considerations........................................21 8. Security Considerations........................................21
9. Intellectual Property Right....................................21 9. Intellectual Property Right....................................22
10. Normative References..........................................21 10. Normative References..........................................22
11. Informative References........................................22 11. Informative References........................................22
12. Acknowledgments...............................................22 12. Acknowledgments...............................................22
13. Authors' Addresses............................................22 13. Authors' Addresses............................................22
Yang, et. al. Expires December 2002 [Page 23] Yang, et. al. Expires March 2003 [Page 23]
 End of changes. 

This html diff was produced by rfcdiff 1.23, available from http://www.levkowetz.com/ietf/tools/rfcdiff/