draft-ietf-forces-ceha-07.txt   draft-ietf-forces-ceha-08.txt 
Network Working Group K. Ogawa Network Working Group K. Ogawa
Internet-Draft NTT Corporation Internet-Draft NTT Corporation
Intended status: Standards Track W. M. Wang Updates: 5810 (if approved) W. M. Wang
Expires: November 09, 2013 Zhejiang Gongshang University Intended status: Standards Track Zhejiang Gongshang University
E. Haleplidis Expires: April 17, 2014 E. Haleplidis
University of Patras University of Patras
J. Hadi Salim J. Hadi Salim
Mojatatu Networks Mojatatu Networks
May 08, 2013 October 14, 2013
ForCES Intra-NE High Availability ForCES Intra-NE High Availability
draft-ietf-forces-ceha-07 draft-ietf-forces-ceha-08
Abstract Abstract
This document discusses Control Element High Availability within a This document discusses Control Element High Availability within a
ForCES Network Element. ForCES Network Element. Additionally this document updates [RFC5810]
by providing new normative text for the Cold-Standby High
availability mechanism.
Status of This Memo Status of This Memo
This Internet-Draft is submitted in full conformance with the This Internet-Draft is submitted in full conformance with the
provisions of BCP 78 and BCP 79. provisions of BCP 78 and BCP 79.
Internet-Drafts are working documents of the Internet Engineering Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF). Note that other groups may also distribute Task Force (IETF). Note that other groups may also distribute
working documents as Internet-Drafts. The list of current Internet- working documents as Internet-Drafts. The list of current Internet-
Drafts is at http://datatracker.ietf.org/drafts/current/. Drafts is at http://datatracker.ietf.org/drafts/current/.
Internet-Drafts are draft documents valid for a maximum of six months Internet-Drafts are draft documents valid for a maximum of six months
and may be updated, replaced, or obsoleted by other documents at any and may be updated, replaced, or obsoleted by other documents at any
time. It is inappropriate to use Internet-Drafts as reference time. It is inappropriate to use Internet-Drafts as reference
material or to cite them other than as "work in progress." material or to cite them other than as "work in progress."
This Internet-Draft will expire on November 09, 2013. This Internet-Draft will expire on April 17, 2014.
Copyright Notice Copyright Notice
Copyright (c) 2013 IETF Trust and the persons identified as the Copyright (c) 2013 IETF Trust and the persons identified as the
document authors. All rights reserved. document authors. All rights reserved.
This document is subject to BCP 78 and the IETF Trust's Legal This document is subject to BCP 78 and the IETF Trust's Legal
Provisions Relating to IETF Documents Provisions Relating to IETF Documents
(http://trustee.ietf.org/license-info) in effect on the date of (http://trustee.ietf.org/license-info) in effect on the date of
publication of this document. Please review these documents publication of this document. Please review these documents
skipping to change at page 2, line 21 skipping to change at page 2, line 23
2.1. Document Scope . . . . . . . . . . . . . . . . . . . . . 5 2.1. Document Scope . . . . . . . . . . . . . . . . . . . . . 5
2.2. Quantifying Problem Scope . . . . . . . . . . . . . . . . 5 2.2. Quantifying Problem Scope . . . . . . . . . . . . . . . . 5
3. RFC5810 CE HA Framework . . . . . . . . . . . . . . . . . . . 6 3. RFC5810 CE HA Framework . . . . . . . . . . . . . . . . . . . 6
3.1. RFC 5810 CE HA Support . . . . . . . . . . . . . . . . . 6 3.1. RFC 5810 CE HA Support . . . . . . . . . . . . . . . . . 6
3.1.1. Cold Standby Interaction with ForCES Protocol . . . . 7 3.1.1. Cold Standby Interaction with ForCES Protocol . . . . 7
3.1.2. Responsibilities for HA . . . . . . . . . . . . . . . 9 3.1.2. Responsibilities for HA . . . . . . . . . . . . . . . 9
4. CE HA Hot Standby . . . . . . . . . . . . . . . . . . . . . . 10 4. CE HA Hot Standby . . . . . . . . . . . . . . . . . . . . . . 10
4.1. Changes to the FEPO model . . . . . . . . . . . . . . . . 10 4.1. Changes to the FEPO model . . . . . . . . . . . . . . . . 10
4.2. FEPO processing . . . . . . . . . . . . . . . . . . . . . 12 4.2. FEPO processing . . . . . . . . . . . . . . . . . . . . . 12
5. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 15 5. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 15
6. Security Considerations . . . . . . . . . . . . . . . . . . . 15 6. Security Considerations . . . . . . . . . . . . . . . . . . . 16
7. References . . . . . . . . . . . . . . . . . . . . . . . . . 16 7. References . . . . . . . . . . . . . . . . . . . . . . . . . 17
7.1. Normative References . . . . . . . . . . . . . . . . . . 16 7.1. Normative References . . . . . . . . . . . . . . . . . . 17
7.2. Informative References . . . . . . . . . . . . . . . . . 16 7.2. Informative References . . . . . . . . . . . . . . . . . 17
Appendix A. New FEPO version . . . . . . . . . . . . . . . . . . 16 Appendix A. New FEPO version . . . . . . . . . . . . . . . . . . 17
Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 25 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 27
1. Definitions 1. Definitions
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
"SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
document are to be interpreted as described in RFC 2119. document are to be interpreted as described in [RFC2119].
The following definitions are taken from [RFC3654]and [RFC3746]: The following definitions are taken from [RFC3654], [RFC3746] and
[RFC5810]. They are repeated here for convenience as needed, but the
normative definitions are found in the referenced RFCs:
o Logical Functional Block (LFB) -- A template that represents a o Logical Functional Block (LFB) -- A template that represents a
fine-grained, logically separate aspects of FE processing. fine-grained, logically separate aspects of FE processing.
o ForCES Protocol -- The protocol used at the Fp reference point in o ForCES Protocol -- The protocol used for communication
the ForCES Framework in [RFC3746]. communication between CEs and FEs. This protocol does not apply
to CE-to-CE communication, FE-to-FE communication, or to
communication between FE and CE managers. The ForCES protocol is
a master-slave protocol in which FEs are slaves and CEs are
masters. This protocol includes both the management of the
communication channel (e.g., connection establishment, heartbeats)
and the control messages themselves.
o ForCES Protocol Layer (ForCES PL) -- A layer in the ForCES o ForCES Protocol Layer (ForCES PL) -- A layer in the ForCES
architecture that embodies the ForCES protocol and the state protocol architecture that defines the ForCES protocol messages,
transfer mechanisms as defined in [RFC5810]. the protocol state transfer scheme, and the ForCES protocol
architecture itself (including requirements of ForCES TML as shown
below). Specifications of ForCES PL are defined in [RFC5810]
o ForCES Protocol Transport Mapping Layer (ForCES TML) -- A layer in o ForCES Protocol Transport Mapping Layer (ForCES TML) -- A layer in
ForCES protocol architecture that specifically addresses the ForCES protocol architecture that specifically addresses the
protocol message transportation issues, such as how the protocol protocol message transportation issues, such as how the protocol
messages are mapped to different transport media (like SCTP, IP, messages are mapped to different transport media (like SCTP, IP,
TCP, UDP, ATM, Ethernet, etc), and how to achieve and implement TCP, UDP, ATM, Ethernet, etc), and how to achieve and implement
reliability, security, etc. reliability, security, etc.
o Forwarding Element (FE) - A logical entity that implements the o Forwarding Element (FE) - A logical entity that implements the
ForCES Protocol. FEs use the underlying hardware to provide per- ForCES Protocol. FEs use the underlying hardware to provide per-
skipping to change at page 4, line 8 skipping to change at page 4, line 8
involve the CE manager learning the capabilities of available FEs. involve the CE manager learning the capabilities of available FEs.
2. Introduction 2. Introduction
Figure 1 illustrates a ForCES NE controlled by a set of redundant CEs Figure 1 illustrates a ForCES NE controlled by a set of redundant CEs
with CE1 being active and CE2 and CEN being a backup. with CE1 being active and CE2 and CEN being a backup.
----------------------------------------- -----------------------------------------
| ForCES Network Element | | ForCES Network Element |
| +-----------+ | | +-----------+ |
| | CEN | | | | CEn | |
| | (Backup) | | | | (Backup) | |
-------------- Fc | +------------+ +------------+ | | -------------- Fc | +------------+ +------------+ | |
| CE Manager |--------+-| CE1 |------| CE2 |-+ | | CE Manager |--------+-| CE1 |------| CE2 |-+ |
-------------- | | (Active) | Fr | (Backup) | | -------------- | | (Active) | Fr | (Backup) | |
| | +-------+--+-+ +---+---+----+ | | | +-------+--+-+ +---+---+----+ |
| Fl | | | Fp / | | | Fl | | | Fp / | |
| | | +---------+ / | | | | | +---------+ / | |
| | Fp| |/ |Fp | | | Fp| |/ |Fp |
| | | | | | | | | | | |
| | | Fp /+--+ | | | | | Fp /+--+ | |
skipping to change at page 4, line 44 skipping to change at page 4, line 44
Fc: Interface between the CE Manager and a CE Fc: Interface between the CE Manager and a CE
Ff: Interface between the FE Manager and an FE Ff: Interface between the FE Manager and an FE
Fl: Interface between the CE Manager and the FE Manager Fl: Interface between the CE Manager and the FE Manager
Fi/f: FE external interface Fi/f: FE external interface
Figure 1: ForCES Architecture Figure 1: ForCES Architecture
The ForCES architecture allows FEs to be aware of multiple CEs but The ForCES architecture allows FEs to be aware of multiple CEs but
enforces that only one CE be the master controller. This is known in enforces that only one CE be the master controller. This is known in
the industry as 1+N redundancy. The master CE controls the FEs via the industry as 1+N redundancy. The master CE controls the FEs via
the ForCES protocol operating in the Fp interface. If the master CE the ForCES protocol operating on the Fp interface. If the master CE
becomes faulty, a backup CE takes over and NE operation continues. becomes faulty, a backup CE takes over and NE operation continues.
By definition, the current documented setup is known as cold-standby. By definition, the current documented setup is known as cold-standby.
The set of CEs controlling an FE is static and is passed to the FE by The set of CEs controlling an FE is static and is passed to the FE by
the FE Manager (FEM) via the Ff interface and to each CE by the CE the FE Manager (FEM) via the Ff interface and to each CE by the CE
Manager (CEM) in the Fc interface during the pre-association phase. Manager (CEM) in the Fc interface during the pre-association phase.
From an FE perspective, the knobs of control for a CE set are defined From an FE perspective, the knobs of control for a CE set are defined
by the FEPO LFB in [RFC5810], Appendix B. In Section 3.1 of this by the FEPO LFB in [RFC5810], Appendix B. In Section 3.1 of this
document we discuss further details of these knobs. document we discuss further details of these knobs.
skipping to change at page 5, line 24 skipping to change at page 5, line 24
discussion in Section 4. discussion in Section 4.
At the time this document is being written, the Fr interface is out At the time this document is being written, the Fr interface is out
of scope for the ForCES architecture. However, it is expected that of scope for the ForCES architecture. However, it is expected that
organizations implementing a set of CEs will need to have the CEs organizations implementing a set of CEs will need to have the CEs
communicate to each other via the Fr interface in order to achieve communicate to each other via the Fr interface in order to achieve
the synchronization necessary for controlling the FEs. the synchronization necessary for controlling the FEs.
The problem scope addressed by this document falls into 2 areas: The problem scope addressed by this document falls into 2 areas:
1. To describe with more clarity (than [RFC5810]) how current cold- 1. To update the description of [RFC5810] with more clarity on how
standby approach operates within the NE cluster. current cold-standby approach operates within the NE cluster.
2. To describe how to evolve the [RFC5810] cold-standby setup to a 2. To describe how to evolve the [RFC5810] cold-standby setup to a
hot-standby redundancy setup to improve the failover time and NE hot-standby redundancy setup to improve the failover time and NE
availability. availability.
2.2. Quantifying Problem Scope 2.2. Quantifying Problem Scope
The NE recovery and availability is dependent on several time- The NE recovery and availability is dependent on several time-
sensitive metrics: sensitive metrics:
skipping to change at page 7, line 5 skipping to change at page 6, line 50
As mentioned earlier, although there can be multiple redundant CEs, As mentioned earlier, although there can be multiple redundant CEs,
only one CE actively controls FEs in a ForCES NE. In practice there only one CE actively controls FEs in a ForCES NE. In practice there
may be only one backup CE. At any moment in time, only one master CE may be only one backup CE. At any moment in time, only one master CE
can control an FE. In addition, the FE connects and associates to can control an FE. In addition, the FE connects and associates to
only the master CE. The FE and the CE are aware of the primary and only the master CE. The FE and the CE are aware of the primary and
one or more secondary CEs. This information (primary, secondary CEs) one or more secondary CEs. This information (primary, secondary CEs)
is configured on the FE and the CE during pre-association by the FEM is configured on the FE and the CE during pre-association by the FEM
and the CEM respectively. and the CEM respectively.
This section includes a new normative description that updates
[RFC5810] for the Cold-Standby High Availability mechanism.
Figure 2 below illustrates the Forces message sequences that the FE Figure 2 below illustrates the Forces message sequences that the FE
uses to recover the connection in current defined cold-standby uses to recover the connection in current defined cold-standby
scheme. scheme.
FE CE Primary CE Secondary FE CE Primary CE Secondary
| | | | | |
| Asso Estb,Caps exchg | | | Association Establishment | |
1 |<--------------------->| | | Capabilities Exchange | |
| | | 1 |<------------------------->| |
| state update | | | | |
2 |<--------------------->| | | State Update | |
| | | 2 |<------------------------->| |
| | | | | |
| FAILURE | | | |
| | | FAILURE |
| Asso Estb,Caps exchange | | |
3 |<------------------------------------------>| | Association Estbalishment,Capabilities Exchange |
| | 3 |<----------------------------------------------->|
| Event Report (pri CE down) | | |
4 |------------------------------------------->| | Event Report (primary CE down) |
| | 4 |------------------------------------------------>|
| state update | | |
5 |<------------------------------------------>| | State Update |
5 |<----------------------------------------------->|
Figure 2: CE Failover for Cold Standby Figure 2: CE Failover for Cold Standby
3.1.1. Cold Standby Interaction with ForCES Protocol 3.1.1. Cold Standby Interaction with ForCES Protocol
HA parameterization in an FE is driven by configuring the FE Protocol HA parametrization in an FE is driven by configuring the FE Protocol
Object (FEPO) LFB. Object (FEPO) LFB.
The FEPO CEID component identifies the current master CE and the The FEPO CEID component identifies the current master CE and the
component table BackupCEs identifies the configured backup CEs. The component table BackupCEs identifies the configured backup CEs. The
FEPO FE Heartbeat Interval, CE Heartbeat Dead Interval, and CE FEPO FE Heartbeat Interval, CE Heartbeat Dead Interval, and CE
Heartbeat policy help in detecting connectivity problems between an Heartbeat policy help in detecting connectivity problems between an
FE and CE. The CE Failover policy defines how the FE should react on FE and CE. The CE Failover policy defines how the FE should react on
a detected failure. The FEObject FEState component [RFC5812] defines a detected failure. The FEObject FEState component [RFC5812] defines
the operational forwarding status and control. The CE can turn off the operational forwarding status and control. The CE can turn off
the FE's forwarding operations by setting the FEState to AdminDisable the FE's forwarding operations by setting the FEState to AdminDisable
and can turn it on by setting it to OperEnable. Note: [RFC5812] and can turn it on by setting it to OperEnable. Note: [RFC5812]
section 5.1 has an erratta which describes the FEState as read-only section 5.1 has an errata which describes the FEState as read-only
when it should be read-write. when it should be read-write.
Figure 3 illustrates the defined state machine that facilitates the Figure 3 illustrates the defined state machine that facilitates the
recovery of connection state. recovery of connection state.
The FE connects to the CE specified on FEPO CEID component. If it The FE connects to the CE specified on FEPO CEID component. If it
fails to connect to the defined CE, it moves it to the bottom of fails to connect to the defined CE, it moves it to the bottom of
table BackupCEs and sets its CEID component to be the first CE table BackupCEs and sets its CEID component to be the first CE
retrieved from table BackupCEs. The FE then attempts to associate retrieved from table BackupCEs. The FE then attempts to associate
with the CE designated as the new primary CE. The FE continues with the CE designated as the new primary CE. The FE continues
through this procedure until it successfully connects to one of the through this procedure until it successfully connects to one of the
CEs. CEs or until the CE Failover Timeout Interval (CEFTI) expires.
FE tries to associate FE tries to associate
+-->-----+ +-->-----+
| | | |
(CE changes master || | | (CE changes master || | |
CE issues Teardown || +---+--------v----+ CE issues Teardown || +---+--------v----+
Lost association) && | Pre-Association | Lost association) && | Pre-Association |
CE failover policy = 0 | (Association | CE failover policy = 0 | (Association |
+------------>-->-->| in +<----+ +------------>-->-->| in +<----+
| | progress) | | | | progress) | |
| | | | | | | |
| +--------+--------+ | | +--------+--------+ |
| CE Association | | CEFTI | CE Association | | CEFTI
| Response V | timer | Response V | timer
| +------------------+ | expires | +------------------+ | expires
| |FE issue CEPrimaryDown ^ | |FE issue CEPrimaryDown ^
| V | | V |
+-+-----------+ +------+-----+ +-+-----------+ +------+-----+
| | (CE changes master || | Not | | | (CE changes master || | Not |
| | CE issues Teardown || | Associated | | | CE issues Teardown || | Associated |
| | Lost association) && | +->---+ | | Lost association) && | +->---+
| Associated | CE Failover Policy = 1 |(May | FE | | Associated | CE Failover Policy = 1 |(May | FE |
| | | Continue |try v | | | Continue | try v
| |-------->------->------>| Forwarding)|assn | | |-------->------->------>| Forwarding)| assn|
| | Start CEFTI timer | |-<---+ | | Start CEFTI timer | |-<---+
| | | | | | | |
+-------------+ +-------+-----+ +-------------+ +-------+-----+
^ | ^ |
| Successful V | Successful V
| Association | | Association |
| Setup | | Setup |
| (Cancel CEFTI Timer) | | (Cancel CEFTI Timer) |
+_________________________________________+ +_________________________________________+
FE issue CEPrimaryDown event FE issue CEPrimaryDown event
Figure 3: FE State Machine considering HA Figure 3: FE State Machine considering HA
There are several events that trigger mastership changes: The master There are several events that trigger mastership changes: The master
CE may issue a mastership change (by changing the CEID value), or CE may issue a mastership change (by changing the CEID component), or
teardown an existing association; and last, connectivity may be lost teardown an existing association; and last, connectivity may be lost
between the CE and FE. between the CE and FE.
When communication fails between the FE and CE (which can be caused When communication fails between the FE and CE (which can be caused
by either the CE or link failure but not FE related), either the TML by either the CE or link failure but not FE related), either the TML
on the FE will trigger the FE PL regarding this failure or it will be on the FE will trigger the FE PL regarding this failure or it will be
detected using the HB messages between FEs and CEs. The detected using the heartbeat messages between FEs and CEs. The
communication failure, regardless of how it is detected, MUST be communication failure, regardless of how it is detected, MUST be
considered as a loss of association between the CE and corresponding considered as a loss of association between the CE and corresponding
FE. FE.
If the FE's FEPO CE Failover Policy is configured to mode 0 (the If the FE's FEPO CE Failover Policy is configured to mode 0 (the
default), it will immediately transition to the pre-association default), it will immediately transition to the pre-association
phase. This means that if association is later re-established with a phase. This means that if association is later re-established with a
CE, all FE state will need to be re-created. CE, all FE state will need to be re-created.
If the FE's FEPO CE Failover Policy is configured to mode 1, it If the FE's FEPO CE Failover Policy is configured to mode 1, it
indicates that the FE will run in HA restart recovery. In such a indicates that the FE will run in HA restart recovery. In such a
case, the FE transitions to the Not Associated state and the CEFTI case, the FE transitions to the Not Associated state and the CEFTI
timer [RFC5810] is started. The FE MAY continue to forward packets timer [RFC5810] is started. The FE may continue to forward packets
during this state. The FE recycles through any configured backup CEs during this state depending upon the value of the CEFailoverPolicy
in a round-robin fashion. It first adds its primary CE to the bottom component of the FEPO LFB. The FE recycles through any configured
of table BackupCEs and sets its CEID component to be the first backup CEs in a round-robin fashion. It first adds its primary CE to
secondary retrieved from table BackupCEs. The FE then attempts to the bottom of table BackupCEs and sets its CEID component to be the
associate with the CE designated as the new primary CE. If it fails first secondary retrieved from table BackupCEs. The FE then attempts
to re-associate with any CE and the CEFTI expires, the FE then to associate with the CE designated as the new primary CE. If it
fails to re-associate with any CE and the CEFTI expires, the FE then
transitions to the pre-association state and FE will operationally transitions to the pre-association state and FE will operationally
bring down its forwarding path (and set the [RFC5812] FEObject bring down its forwarding path (and set the [RFC5812] FEObject
FEState component to OperDisable). FEState component to OperDisable).
If the FE, while in the not associated state, manages to reconnect to If the FE, while in the not associated state, manages to reconnect to
a new primary CE before CEFTI expires it transitions to the a new primary CE before CEFTI expires it transitions to the
Associated state. Once re-associated, the CE may try to synchronize Associated state. Once re-associated, the CE may try to synchronize
any state that the FE may have lost during disconnection. How the CE any state that the FE may have lost during disconnection. How the CE
re-synchronizes such state is out of scope for the current ForCES re-synchronizes such state is out of scope for the current ForCES
architecture but would typically constitute the issuing of new architecture but would typically constitute the issuing of new
skipping to change at page 10, line 11 skipping to change at page 10, line 13
1. The TML controls logical connection availability and failover. 1. The TML controls logical connection availability and failover.
2. The TML also controls peer HA management. 2. The TML also controls peer HA management.
At this level, control of all lower layers, for example transport At this level, control of all lower layers, for example transport
level (such as IP addresses, MAC addresses etc) and associated links level (such as IP addresses, MAC addresses etc) and associated links
going down are the role of the TML. going down are the role of the TML.
PL Level: PL Level:
All other functionality, including configuring the HA behavior during All other functionality, including configuring the HA behavior during
setup, the CE IDs used to identify primary and secondary CEs, setup, the Control Element IDs (CE IDs) used to identify primary and
protocol messages used to report CE failure (Event Report), Heartbeat secondary CEs, protocol messages used to report CE failure (Event
messages used to detect association failure, messages to change the Report), Heartbeat messages used to detect association failure,
primary CE (Config), and other HA related operations described in messages to change the primary CE (Config), and other HA related
Section 3.1, are the PL's responsibility. operations described in Section 3.1, are the PL's responsibility.
To put the two together, if a path to a primary CE is down, the TML To put the two together, if a path to a primary CE is down, the TML
would help recover from a failure by switching over to a backup path, would help recover from a failure by switching over to a backup path,
if one is available. If the CE is totally unreachable then the PL if one is available. If the CE is totally unreachable then the PL
would be informed and it would take the appropriate actions described would be informed and it would take the appropriate actions described
before. before.
4. CE HA Hot Standby 4. CE HA Hot Standby
In this section we describe small extensions to the existing scheme In this section we describe small extensions to the existing scheme
skipping to change at page 11, line 18 skipping to change at page 11, line 21
+ 2 (Associated) represents that the FE has successfully + 2 (Associated) represents that the FE has successfully
associated with the CE associated with the CE
+ 3 (IsMaster) represents that the FE has associated with + 3 (IsMaster) represents that the FE has associated with
the CE and is the master of the FE the CE and is the master of the FE
+ 4 (LostConnection) represents that the FE was associated + 4 (LostConnection) represents that the FE was associated
with the CE at one point but lost the connection with the CE at one point but lost the connection
+ 5 (Unreachable) represents the FE deems this CE + 5 (Unreachable) represents the FE deems this CE
unreachable. i.e the FE has tried over a period to unreachable. i.e the FE has tried over a period to connect
connect to it but has failed. to it but has failed.
2. HAModeValues an unsigned char to specify selected HA mode. 2. HAModeValues an unsigned char to specify selected HA mode.
Special values are: Special values are:
+ 0 (No HA Mode) represents that the FE is not running in HA + 0 (No HA Mode) represents that the FE is not running in HA
mode mode
+ 1 (HA Mode - Cold Standby) represents that the FE is in HA + 1 (HA Mode - Cold Standby) represents that the FE is in HA
mode cold Standby mode cold Standby
skipping to change at page 11, line 42 skipping to change at page 11, line 45
3. Statistics, a complex structure, representing the 3. Statistics, a complex structure, representing the
communication statistics between the FE and CE. The communication statistics between the FE and CE. The
components are: components are:
+ RecvPackets representing the packet count received from + RecvPackets representing the packet count received from
the CE the CE
+ RecvBytes representing the byte count received from the CE + RecvBytes representing the byte count received from the CE
+ RecvErrPackets representing the erronous packets received + RecvErrPackets representing the erroneous packets received
from the CE. This component logs badly formatted packets from the CE. This component logs badly formatted packets
as well as good packets sent to the FE by the CE to set as well as good packets sent to the FE by the CE to set
components whilst that CE is not the master. Erronous components whilst that CE is not the master. Erroneous
packets are dropped(i.e not responded to). packets are dropped(i.e not responded to).
+ RecvErrBytes representing the RecvErrPackets byte count + RecvErrBytes representing the RecvErrPackets byte count
received from the CE received from the CE
+ TxmitPackets representing the packet count transmitted to + TxmitPackets representing the packet count transmitted to
the CE the CE
+ TxmitErrPackets representing the error packet count + TxmitErrPackets representing the error packet count
transmitted to the CE. Typically these would be failures transmitted to the CE. Typically these would be failures
due to communication. due to communication.
+ TxmitBytes representing the byte count transmitted to the + TxmitBytes representing the byte count transmitted to the
CE CE
+ TxmitErrBytes representing the byte count of errors from + TxmitErrBytes representing the byte count of errors from
transmit to the CE transmit to the CE
4. AllCEType, a complex structure constituing the CE ID, 4. AllCEType, a complex structure constituting the CE IDs,
Statistics and CEStatusType to reflect connection information Statistics and CEStatusType to reflect connection information
for one CE. Used in the AllCEs component array. for one CE. Used in the AllCEs component array.
2. Appended two new components: 2. Appended two new components:
1. Read-only AllCEs to hold status for all CEs. AllCEs is an 1. Read-only AllCEs to hold status for all CEs. AllCEs is an
Array of the AllCEType. Array of the AllCEType.
2. Read-write HAMode of type HAModeValues to carry the HA mode 2. Read-write HAMode of type HAModeValues to carry the HA mode
used by the FE. used by the FE.
3. Added one additional Event, PrimaryCEChanged, reporting the new 3. Added one additional Event, PrimaryCEChanged, reporting the new
master CEID when there is a mastership change. master CE ID when there is a mastership change.
Since no component from the FEPO v1 has been changed FEPO v1.1 Since no component from the FEPO v1 has been changed FEPO v1.1
retains backwards compatibility with CEs that know only version 1.0. retains backwards compatibility with CEs that know only version 1.0.
These CEs however cannot make use of the HA options that the new FEPO These CEs however cannot make use of the HA options that the new FEPO
provides. provides.
4.2. FEPO processing 4.2. FEPO processing
The FE's FEPO LFB version 1.1 AllCEs table contains all the CEIDs The FE's FEPO LFB version 1.1 AllCEs table contains all the CE IDs
that the FE may connect and associate with. The ordering of the CE that the FE may connect and associate with. The ordering of the CE
IDs in this table defines the priority order in which an FE will IDs in this table defines the priority order in which an FE will
connect to the CEs. This table is provisioned initially from the connect to the CEs. This table is provisioned initially from the
configuration plane (FEM). In the pre-association phase, the first configuration plane (FEM). In the pre-association phase, the first
CE (lowest table index) in the AllCEs table MUST be the first CE that CE (lowest table index) in the AllCEs table MUST be the first CE that
the FE will attempt to connect and associate with. If the FE fails the FE will attempt to connect and associate with. If the FE fails
to connect and associate with the first listed CE, it will attempt to to connect and associate with the first listed CE, it will attempt to
connect to the second CE and so forth, and cycles back to the connect to the second CE and so forth, and cycles back to the
beggining of the list until there is a successful association. The beginning of the list until there is a successful association. The
FE MUST associate with at least one CE. Upon a successful FE MUST associate with at least one CE. Upon a successful
association, the FEPO's CEID component identifies the current association, a component of the FEPO LFB, specifically the CEID
associated master CE. component, identifies the current associated master CE.
While it would be much simpler to have the FE not respond to any While it would be much simpler to have the FE not respond to any
messages from a CE other than the master, in practise it has been messages from a CE other than the master, in practise it has been
found to be useful to respond to queries and hearbeats from backup found to be useful to respond to queries and heartbeats from backup
CEs. For this reason, we allow backup CEs to issues queries to the CEs. For this reason, we allow backup CEs to issues queries to the
FE. Configuration messages (SET/DEL) from backup CEs MUST be dropped FE. Configuration messages (SET/DEL) from backup CEs MUST be dropped
by the FE and logged as received errors. by the FE and logged as received errors.
Asynchronous events that the master CE has subscribed to, as well as Asynchronous events that the master CE has subscribed to, as well as
heartbeats are sent to all associated-to CEs. Packet redirects heartbeats are sent to all associated-to CEs. Packet redirects
continue to be sent only to the master CE. The Heartbeat Interval, continue to be sent only to the master CE. The Heartbeat Interval,
the CEHB Policy and the FEHB Policy are global for all CEs(and the CE Heartbeat Policy (CEHB) and the FE Heartbeat Policy (FEHB) are
changed only by the master CE). global for all CEs(and changed only by the master CE).
Figure 4 illustrates the state machine that facilitates connection Figure 4 illustrates the state machine that facilitates connection
recovery with HA enabled. recovery with HA enabled.
FE tries to associate FE tries to associate
+-->-----+ +-->-----+
| | | |
(CE changes master || | | (CE changes master || | |
CE issues Teardown || +---+--------v----+ CE issues Teardown || +---+--------v----+
Lost association) && | Pre-Association | Lost association) && | Pre-Association |
skipping to change at page 13, line 38 skipping to change at page 13, line 41
| +--------+--------+ | | +--------+--------+ |
| CE Association | | CEFTI | CE Association | | CEFTI
| Response V | timer | Response V | timer
| +------------------+ | expires | +------------------+ | expires
| |FE issue CEPrimaryDown ^ | |FE issue CEPrimaryDown ^
| |FE issue PrimaryCEChanged ^ | |FE issue PrimaryCEChanged ^
| V | | V |
+-+-----------+ +------+-----+ +-+-----------+ +------+-----+
| | (CE changes master || | Not | | | (CE changes master || | Not |
| | CE issues Teardown || | Associated | | | CE issues Teardown || | Associated |
| | Lost association) && | +->-----------+ | | Lost association) && | +->----------+
| Associated | CE Failover Policy = 1 |(May |find first | | Associated | CE Failover Policy = 1 |(May | find first |
| | | Continue |associated v | | | Continue | associated v
| |-------->------->------>| Forwarding)|CE or retry | | |-------->------->------>| Forwarding)| CE or retry|
| | Start CEFTI timer | |associating | | | Start CEFTI timer | | associating|
| | | |-<-----------+ | | | |-<----------+
| | | | | | | |
+----+--------+ +-------+----+ +----+--------+ +-------+----+
| | | |
^ Found | associated CE ^ Found | associated CE
| or newly | associated CE | or newly | associated CE
| V | V
| (Cancel CEFTI Timer) | | (Cancel CEFTI Timer) |
+_________________________________________+ +_________________________________________+
FE issue CEPrimaryDown event FE issue CEPrimaryDown event
FE issue PrimaryCEChanged event FE issue PrimaryCEChanged event
skipping to change at page 14, line 18 skipping to change at page 14, line 20
Once the FE has associated with a master CE it moves to the post- Once the FE has associated with a master CE it moves to the post-
association phase (Associated state). It is assumed that the master association phase (Associated state). It is assumed that the master
CE will communicate with other CEs within the NE for the purpose of CE will communicate with other CEs within the NE for the purpose of
synchronization via the CE-CE interface. The CE-CE interface is out synchronization via the CE-CE interface. The CE-CE interface is out
of scope for this document. An election result amongst CEs may of scope for this document. An election result amongst CEs may
result in desire to change mastership to a different associated CE; result in desire to change mastership to a different associated CE;
at which point current assumed master CE will instruct the FE to use at which point current assumed master CE will instruct the FE to use
a different master CE. a different master CE.
FE CE#1 CE#2 ... CE#N FE CE#1 CE#2 ... CE#N
| | | | | | | |
| Asso Estb,Caps exchg | | | | Association Estbalishment | | |
1 |<-------------------->| | | | Capabilities Exchange | | |
| | | | 1 |<------------------------->| | |
| state update | | | | | | |
2 |<-------------------->| | | | State Update | | |
| | | | 2 |<------------------------->| | |
| Asso Estb,Caps exchg | | | | | |
3I|<--------------------------------->| | | Association Estbalishment | |
... ... ... ... | Capabilities Exchange | |
| Asso Estb,Caps exchg | 3I|<-------------------------------------->| |
3N|<------------------------------------------>| ... ... ... ...
| | | | | Association Estbalishment,Capabilities Exchange |
4 |<-------------------->| | | 3N|<----------------------------------------------->|
. . . . | | | |
4x|<-------------------->| | | 4 |<------------------------->| | |
| FAILURE | | . . . .
| | | | 4x|<------------------------->| | |
| Event Report (LastCEID changed) | | | FAILURE | |
5 |---------------------------------->|------->| | | | |
| Event Report (CE#2 is new master) | | | Event Report (LastCEID changed) | |
6 |---------------------------------->|------->| 5 |--------------------------------------->|------->|
| | | | Event Report (CE#2 is new master) | |
7 |<--------------------------------->| | 6 |--------------------------------------->|------->|
. . . . | | |
7x|<--------------------------------->| | 7 |<-------------------------------------->| |
. . . . . . . .
7x|<-------------------------------------->| |
. . . .
Figure 5: CE Failover for Hot Standby Figure 5: CE Failover for Hot Standby
While in the post-association phase, if the CE Failover Policy is set While in the post-association phase, if the CE Failover Policy is set
to 1 and HAMode set to 2 (HotStandby) then the FE, after succesfully to 1 and HAMode set to 2 (HotStandby) then the FE, after successfully
associating with the master CE, MUST attempt to connect and associate associating with the master CE, MUST attempt to connect and associate
with all the CEs that it is aware of. Figure 5 steps #1 and #2 with all the CEs that it is aware of. Figure 5 steps #1 and #2
illustrates the FE associating with CE#1 as the master and then illustrates the FE associating with CE#1 as the master and then
proceeding to steps #3I to #3N the association with backup CEs CE#2 proceeding to steps #3I to #3N the association with backup CEs CE#2
to CE#N. If the FE fails to connect or associate with some CEs, the to CE#N. If the FE fails to connect or associate with some CEs, the
FE MAY flag them as unreachable to avoid continuous attempts to FE MAY flag them as unreachable to avoid continuous attempts to
connect. The FE MAY retry to reassociate with unreachable CEs when connect. The FE MAY retry to reassociate with unreachable CEs when
possible. possible.
When the master CE for any reason is considered to be down, then the When the master CE for any reason is considered to be down, then the
FE MUST try to find the first associated CE from the list of all CEs FE MUST try to find the first associated CE from the list of all CEs
in a round-robin fashion. in a round-robin fashion.
If the FE is unable to find an associated FE in its list of CEs, then If the FE is unable to find an associated FE in its list of CEs, then
it MUST attempt to connect and associate with the first from the list it MUST attempt to connect and associate with the first from the list
of all CEs and continue in a round-robin fashion until it connects of all CEs and continue in a round-robin fashion until it connects
and associates with a CE. and associates with a CE or the CEFTI timer expires.
Once the FE selects an associated CE to use as the new master, the FE Once the FE selects an associated CE to use as the new master, the FE
issues a PrimaryCEDown Event Notification to all associated CEs to issues a PrimaryCEDown Event Notification to all associated CEs to
notify them that the last primary CE went down (and what its identity notify them that the last primary CE went down (and what its identity
was); a second event PrimaryCEChanged identifying the new master CE was); a second event PrimaryCEChanged identifying the new master CE
is sent as well to identify which CE the reporting FE considers to be is sent as well to identify which CE the reporting FE considers to be
the new master. the new master.
In most HA architectures there exists the possibility of split-brain. In most HA architectures there exists the possibility of split-brain.
However, since in our setup the FE will never accept any However, since in our setup the FE will never accept any
skipping to change at page 15, line 44 skipping to change at page 15, line 48
scope. scope.
By virtue of having multiple CE connections, the FE switchover to a By virtue of having multiple CE connections, the FE switchover to a
new master CE will be relatively much faster. The overall effect is new master CE will be relatively much faster. The overall effect is
improving the NE recovery time in case of communication failure or improving the NE recovery time in case of communication failure or
faults of the master CE. This satisfies the requirement we set to faults of the master CE. This satisfies the requirement we set to
achieve. achieve.
5. IANA Considerations 5. IANA Considerations
XXX: This document updates an IANA registered FE Protocol object Following the policies outlined in "Guidelines for Writing an IANA
Logical Functional Block (LFB). At minimal when it becomes RFC we Considerations Section in RFCs" [RFC5226], the Logical Functional
should update https://www.iana.org/assignments/forces/forces.xml Block (LFB) Class Names and Class Identifiers namespaces is updated.
section on FEPO.
A new column, LFB version, is added to the table after the LFB Class
Name. The table now reads as follows:
+----------------+---------+-----------+---------------+------------+
| LFB Class | LFB | LFB | Description | Reference |
| Identifier | Class | Version | | |
| | Name | | | |
+----------------+---------+-----------+---------------+------------+
+----------------+---------+-----------+---------------+------------+
Logical Functional Block (LFB) Class Names and Class Identifiers
The same rules applies as defined in [RFC5812] with the addition that
entries must provide the LFB version as a string.
Upon publication of this document, all current entries are assigned a
value of 1.0.
This document introduces the FE Protocol Object version 1.1 as
follows:
+--------------+------------+---------+-----------------+-----------+
| LFB Class | LFB Class | LFB | Description | Reference |
| Identifier | Name | Version | | |
+--------------+------------+---------+-----------------+-----------+
| 2 | FE | 1.1 | Defines | This |
| | Protocol | | parameters for | document |
| | Object | | the ForCES | |
| | | | protocol | |
| | | | operation | |
+--------------+------------+---------+-----------------+-----------+
Logical Functional Block (LFB) Class Names and Class Identifiers
6. Security Considerations 6. Security Considerations
Security consideration as defined in section 9 of [RFC5810] applies. Security consideration as defined in section 9 of [RFC5810] applies
securing each CE-FE communication. Multiple CEs associated with the
same FE still require the same procedure to be followed on a per-
association basis.
While CE-CE plane is outside current scope of ForCES, we recognize
that it may be subjected to attacks which may affect the CE-FE
communication.
The following considerations should be made:
1. CEs should use secure communication channels between for
coordination and keeping of state at least to avoid connection of
malicious CEs.
2. The master CE should take into account DoS and DDoS attacks from
malicious or malfunctioning CEs.
3. CEs should take into account the split-brain issue. There are
currently two fail-safes in the FE, firstly the FE has the CEID
component that denotes which CE is the master and secondly the FE
does not allow BackupCEs to configure the FE. However backup CEs
that consider that the master CE has dropped and themselves as
master should first do a sanity check and query the FE CEID
component.
7. References 7. References
7.1. Normative References 7.1. Normative References
[RFC2119] Bradner, S., "Key words for use in RFCs to Indicate [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate
Requirement Levels", BCP 14, RFC 2119, March 1997. Requirement Levels", BCP 14, RFC 2119, March 1997.
[RFC5226] Narten, T. and H. Alvestrand, "Guidelines for Writing an
IANA Considerations Section in RFCs", BCP 26, RFC 5226,
May 2008.
[RFC5810] Doria, A., Hadi Salim, J., Haas, R., Khosravi, H., Wang, [RFC5810] Doria, A., Hadi Salim, J., Haas, R., Khosravi, H., Wang,
W., Dong, L., Gopal, R., and J. Halpern, "Forwarding and W., Dong, L., Gopal, R., and J. Halpern, "Forwarding and
Control Element Separation (ForCES) Protocol Control Element Separation (ForCES) Protocol
Specification", RFC 5810, March 2010. Specification", RFC 5810, March 2010.
[RFC5812] Halpern, J. and J. Hadi Salim, "Forwarding and Control
Element Separation (ForCES) Forwarding Element Model", RFC
5812, March 2010.
7.2. Informative References 7.2. Informative References
[RFC3654] Khosravi, H. and T. Anderson, "Requirements for Separation [RFC3654] Khosravi, H. and T. Anderson, "Requirements for Separation
of IP Control and Forwarding", RFC 3654, November 2003. of IP Control and Forwarding", RFC 3654, November 2003.
[RFC3746] Yang, L., Dantu, R., Anderson, T., and R. Gopal, [RFC3746] Yang, L., Dantu, R., Anderson, T., and R. Gopal,
"Forwarding and Control Element Separation (ForCES) "Forwarding and Control Element Separation (ForCES)
Framework", RFC 3746, April 2004. Framework", RFC 3746, April 2004.
[RFC5812] Halpern, J. and J. Hadi Salim, "Forwarding and Control
Element Separation (ForCES) Forwarding Element Model", RFC
5812, March 2010.
Appendix A. New FEPO version Appendix A. New FEPO version
The xml has been validated against the schema defined in [RFC5812]. The xml has been validated against the schema defined in [RFC5812].
<LFBLibrary xmlns="urn:ietf:params:xml:ns:forces:lfbmodel:1.0" <LFBLibrary xmlns="urn:ietf:params:xml:ns:forces:lfbmodel:1.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:noNamespaceSchemaLocation="lfb-schema.xsd" provides="FEPO"> xsi:noNamespaceSchemaLocation="lfb-schema.xsd" provides="FEPO">
<!-- XXX --> <!-- XXX -->
<dataTypeDefs> <dataTypeDefs>
<dataTypeDef> <dataTypeDef>
skipping to change at page 17, line 51 skipping to change at page 19, line 21
<name>FERestartPolicyValues</name> <name>FERestartPolicyValues</name>
<synopsis> <synopsis>
The possible values of FE restart policy The possible values of FE restart policy
</synopsis> </synopsis>
<atomic> <atomic>
<baseType>uchar</baseType> <baseType>uchar</baseType>
<specialValues> <specialValues>
<specialValue value="0"> <specialValue value="0">
<name>FERestartPolicy0</name> <name>FERestartPolicy0</name>
<synopsis> <synopsis>
The FE restart restats its state from scratch The FE restarts its state from scratch
</synopsis> </synopsis>
</specialValue> </specialValue>
</specialValues> </specialValues>
</atomic> </atomic>
</dataTypeDef> </dataTypeDef>
<dataTypeDef> <dataTypeDef>
<name>HAModeValues</name> <name>HAModeValues</name>
<synopsis> <synopsis>
The possible values of HA modes The possible values of HA modes
</synopsis> </synopsis>
<atomic> <atomic>
 End of changes. 45 change blocks. 
122 lines changed or deleted 201 lines changed or added

This html diff was produced by rfcdiff 1.41. The latest version is available from http://tools.ietf.org/tools/rfcdiff/