Internet Engineering Task Force                            E. Haleplidis
Internet-Draft                                      University of Patras
Intended status: Informational                                  K. Ogawa
Expires: December 31, 2009 February 8, 2010                                NTT Corporation
                                                                 X. Wang
                                           Huawei Technologies Co., Ltd.
                                                                   C. Li
                                           Zhejiang Gongshang University
                                                           June 29,
                                                          August 7, 2009

                     ForCES Interoperability Draft
                 draft-ietf-forces-interoperability-02
                 draft-ietf-forces-interoperability-03

Status of this Memo

   This Internet-Draft is submitted to IETF in full conformance with the
   provisions of BCP 78 and BCP 79.

   Internet-Drafts are working documents of the Internet Engineering
   Task Force (IETF), its areas, and its working groups.  Note that
   other groups may also distribute working documents as Internet-
   Drafts.

   Internet-Drafts are draft documents valid for a maximum of six months
   and may be updated, replaced, or obsoleted by other documents at any
   time.  It is inappropriate to use Internet-Drafts as reference
   material or to cite them other than as "work in progress."

   The list of current Internet-Drafts can be accessed at
   http://www.ietf.org/ietf/1id-abstracts.txt.

   The list of Internet-Draft Shadow Directories can be accessed at
   http://www.ietf.org/shadow.html.

   This Internet-Draft will expire on December 31, 2009. February 8, 2010.

Copyright Notice

   Copyright (c) 2009 IETF Trust and the persons identified as the
   document authors.  All rights reserved.

   This document is subject to BCP 78 and the IETF Trust's Legal
   Provisions Relating to IETF Documents in effect on the date of
   publication of this document (http://trustee.ietf.org/license-info).
   Please review these documents carefully, as they describe your rights
   and restrictions with respect to this document.

Abstract

   This document describes the details of the interoperability test of
   the Forward and Control Element Separation (ForCES) protocol that
   will take
   took place in the University of Patras in Rio, Greece, in the
   third week of 15 and 16 July
   2009.  This informational draft provides provided necessary information, for
   all parties who wish to participate in the interoperability test.

   This update also includes the results of the test.

Table of Contents

   1.  Terminology and Conventions  . . . . . . . . . . . . . . . . .  3  4
     1.1.  Requirements Language  . . . . . . . . . . . . . . . . . .  3  4
   2.  Introduction . . . . . . . . . . . . . . . . . . . . . . . . .  4  5
     2.1.  ForCES Protocol  . . . . . . . . . . . . . . . . . . . . .  4  5
     2.2.  ForCES Model . . . . . . . . . . . . . . . . . . . . . . .  4  5
     2.3.  Transport mapping layer  . . . . . . . . . . . . . . . . .  4  5
   3.  Definitions  . . . . . . . . . . . . . . . . . . . . . . . . .  5  6
   4.  Date, Location and Access  . . . . . . . . . . . . . . . . . .  8  9
     4.1.  Date . . . . . . . . . . . . . . . . . . . . . . . . . . .  8  9
     4.2.  Location . . . . . . . . . . . . . . . . . . . . . . . . .  8  9
     4.3.  Access . . . . . . . . . . . . . . . . . . . . . . . . . .  8  9
   5.  Testbed architecture . . . . . . . . . . . . . . . . . . . . . 10
     5.1.  Local configuration  . . . . . . . . . . . . . . . . . . . 10
     5.2.  Distributed configuration  . . . . . . . . . . . . . . . . 11
   6.  Scenarios  . . . . . . . . . . . . . . . . . . . . . . . . . . 12
     6.1.  Scenario 1 - Pre-association Setup . . . . . . . . . . . . 12
     6.2.  Scenario 2 - TML priority channels connection  . . . . . . 13
     6.3.  Scenario 3 - Association Setup - Association Complete  . . 13
     6.4.  Scenario 4 - CE query  . . . . . . . . . . . . . . . . . . 13
     6.5.  Scenario 5 - Heartbeat monitoring  . . . . . . . . . . . . 14
     6.6.  Scenario 6 - Simple Config Command . . . . . . . . . . . . 14
     6.7.  Scenario 7 - Association Teardown  . . . . . . . . . . . . 14
   7.  Acknowledgements  Tested Features  . . . . . . . . . . . . . . . . . . . . . . . 16
   8.  IANA Considerations
     7.1.  ForCES Protocol Features . . . . . . . . . . . . . . . . . 16
       7.1.1.  Protocol Messages  . . . . . . . . . . . . . . . . . . 16
       7.1.2.  MainHeader Handling  . . . . . . . . . . . . . . . . . 17
   9.  Security Considerations
       7.1.3.  TLV Handling . . . . . . . . . . . . . . . . . . . . . 17
       7.1.4.  Operation Types Supported  . . . . . . . . . . . . . . 18
   10. References
     7.2.  ForCES Model Features  . . . . . . . . . . . . . . . . . . 18
       7.2.1.  Basic Atomic Types Supported . . . . . . . . . 19
     10.1. Normative References . . . . 18
       7.2.2.  Compound Types Supported . . . . . . . . . . . . . . . 19
     10.2. Informative References 18
       7.2.3.  LFBs Supported . . . . . . . . . . . . . . . . . . . . 19
   Authors' Addresses
         7.2.3.1.  FE Protocol LFB  . . . . . . . . . . . . . . . . . 19
         7.2.3.2.  FE Object LFB  . . . . . . . . . . 21

1.  Terminology and Conventions

1.1.  Requirements Language

   The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
   "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
   document are to be interpreted as described in [RFC2119].

2.  Introduction

   Forwarding and Control Element Separation (ForCES) defines an
   architectural framework and associated protocols to standardize
   information exchange between the control plane and the forwarding
   plane in a ForCES Network Element (ForCES NE).  [RFC3654] has defined
   the ForCES requirements, and [RFC3746] has defined the ForCES
   framework.

2.1.  ForCES Protocol

   The . . . . . . . . 19
     7.3.  ForCES protocol works in a master-slave mode in which FEs are
   slaves and CEs are masters.  The protocol includes commands for
   transport of Logical Function Block (LFB) configuration information, SCTP-TML Features . . . . . . . . . . . . . . . . . 20
       7.3.1.  TML Priority Ports . . . . . . . . . . . . . . . . . . 20
       7.3.2.  Message Handling at specific priorities  . . . . . . . 20
   8.  Test details . . . . . . . . . . . . . . . . . . . . . . . . . 22
   9.  Results  . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
   10. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . 29
   11. IANA Considerations  . . . . . . . . . . . . . . . . . . . . . 30
   12. Security Considerations  . . . . . . . . . . . . . . . . . . . 31
   13. References . . . . . . . . . . . . . . . . . . . . . . . . . . 32
     13.1. Normative References . . . . . . . . . . . . . . . . . . . 32
     13.2. Informative References . . . . . . . . . . . . . . . . . . 32
   Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 34

1.  Terminology and Conventions

1.1.  Requirements Language

   The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
   "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
   document are to be interpreted as described in [RFC2119].

2.  Introduction

   Forwarding and Control Element Separation (ForCES) defines an
   architectural framework and associated protocols to standardize
   information exchange between the control plane and the forwarding
   plane in a ForCES Network Element (ForCES NE).  [RFC3654] has defined
   the ForCES requirements, and [RFC3746] has defined the ForCES
   framework.

2.1.  ForCES Protocol

   The ForCES protocol works in a master-slave mode in which FEs are
   slaves and CEs are masters.  The protocol includes commands for
   transport of Logical Function Block (LFB) configuration information,
   association setup, status, and event notifications, etc.  The reader
   is encouraged to read FE-protocol [I-D.ietf-forces-protocol] for
   further information.

2.2.  ForCES Model

   The FE-MODEL [I-D.ietf-forces-model] presents a formal way to define
   FE Logical Function Blocks (LFBs) using XML.  LFB configuration
   components, capabilities, and associated events are defined when the
   LFB is formally created.  The LFBs within the FE are accordingly
   controlled in a standardized way by the ForCES protocol.

2.3.  Transport mapping layer

   The TML transports the PL messages.  The TML is where the issues of
   how to achieve transport level reliability, congestion control,
   multicast, ordering, etc. are handled.  It is expected that more than
   one TML will be standardized.  The various possible TMLs could vary
   their implementations based on the capabilities of underlying media
   and transport.  However, since each TML is standardized,
   interoperability is guaranteed as long as both endpoints support the
   same TML.  All ForCES Protocol Layer implementations MUST be portable
   across all TMLs.  Although more than one TML may be standardized for
   the ForCES Protocol, for the purposes of the interoperability test,
   the mandated MUST IMPLEMENT SCTP TML [I-D.ietf-forces-sctptml] will
   be used.

3.  Definitions

   This document follows the terminology defined by the ForCES
   Requirements in [RFC3654] and by the ForCES framework in [RFC3746].
   The definitions below are repeated below for clarity.

   o  Control Element (CE) - A logical entity that implements the ForCES
      protocol and uses it to instruct one or more FEs on how to process
      packets.  CEs handle functionality such as the execution of
      control and signaling protocols.

   o  CE Manager (CEM) - A logical entity responsible for generic CE
      management tasks.  It is particularly used during the pre-
      association setup, status, phase to determine with which FE(s) a CE should
      communicate.  This process is called FE discovery and may involve
      the CE manager learning the capabilities of available FEs.

   o  Forwarding Element (FE) - A logical entity that implements the
      ForCES protocol.  FEs use the underlying hardware to provide per-
      packet processing and handling as directed/controlled by one or
      more CEs via the ForCES protocol.

   o  FE Manager (FEM) - A logical entity responsible for generic FE
      management tasks.  It is used during pre-association phase to
      determine with which CE(s) an FE should communicate.  This process
      is called CE discovery and may involve the FE manager learning the
      capabilities of available CEs.  An FE manager may use anything
      from a static configuration to a pre-association phase protocol
      (see below) to determine which CE(s) to use.  Being a logical
      entity, an FE manager might be physically combined with any of the
      other logical entities such as FEs.

   o  ForCES Network Element (NE) - An entity composed of one or more
      CEs and one or more FEs.  To entities outside an NE, the NE
      represents a single point of management.  Similarly, an NE usually
      hides its internal organization from external entities.

   o  LFB (Logical Function Block) - The basic building block that is
      operated on by the ForCES protocol.  The LFB is a well defined,
      logically separable functional block that resides in an FE and is
      controlled by the CE via ForCES protocol.  The LFB may reside at
      the FE's datapath and process packets or may be purely an FE
      control or configuration entity that is operated on by the CE.
      Note that the LFB is a functionally accurate abstraction of the
      FE's processing capabilities, but not a hardware-accurate
      representation of the FE implementation.

   o  FE Topology - A representation of how the multiple FEs within a
      single NE are interconnected.  Sometimes this is called inter-FE
      topology, to be distinguished from intra-FE topology (i.e., LFB
      topology).

   o  LFB Class and LFB Instance - LFBs are categorized by LFB Classes.
      An LFB Instance represents an LFB Class (or Type) existence.
      There may be multiple instances of the same LFB Class (or Type) in
      an FE.  An LFB Class is represented by an LFB Class ID, and an LFB
      Instance is represented by an LFB Instance ID.  As a result, an
      LFB Class ID associated with an LFB Instance ID uniquely specifies
      an LFB existence.

   o  LFB Metadata - Metadata is used to communicate per-packet state
      from one LFB to another, but is not sent across the network.  The
      FE model defines how such metadata is identified, produced and
      consumed by the LFBs.  It defines the functionality but not how
      metadata is encoded within an implementation.

   o  LFB Component - Operational parameters of the LFBs that must be
      visible to the CEs are conceptualized in the FE model as the LFB
      components.  The LFB components include, for example, flags,
      single parameter arguments, complex arguments, and tables that the
      CE can read and/or write via the ForCES protocol (see below).

   o  LFB Topology - Representation of how the LFB instances are
      logically interconnected and placed along the datapath within one
      FE.  Sometimes it is also called intra-FE topology, to be
      distinguished from inter-FE topology.

   o  Pre-association Phase - The period of time during which an FE
      Manager and a CE Manager are determining which FE(s) and CE(s)
      should be part of the same network element.

   o  Post-association Phase - The period of time during which an FE
      knows which CE is to control it and vice versa.  This includes the
      time during which the CE and FE are establishing communication
      with one another.

   o  ForCES Protocol - While there may be multiple protocols used
      within the overall ForCES architecture, the term "ForCES protocol"
      and "protocol" refer to the Fp reference points in the ForCES
      Framework in [RFC3746].  This protocol does not apply to CE-to-CE
      communication, FE-to-FE communication, or to communication between
      FE and CE managers.  Basically, the ForCES protocol works in a
      master- slave mode in which FEs are slaves and CEs are masters.
      This document defines the specifications for this ForCES protocol.

   o  ForCES Protocol Transport Mapping Layer (ForCES TML) - A layer in
      ForCES protocol architecture that uses the capabilities of
      existing transport protocols to specifically address protocol
      message transportation issues, such as how the protocol messages
      are mapped to different transport media (like TCP, IP, ATM,
      Ethernet, etc), and how to achieve and implement reliability,
      multicast, ordering, etc.  The ForCES TML specifications are
      detailed in separate ForCES documents, one for each TML.

4.  Date, Location and Access

4.1.  Date

   The date that the Interoperability test took place was 15-16/07/2009,
   one and a half week before IETF 75, in Stockholm.

4.2.  Location

   Patras is a major harbor of Greece connecting it with Italy.

   The University of Patras is located in Rio, 10km east out of Patras.

   The following coordinates mark the Electrical and Computer
   Engineering building in the University.

   o  North: 38o17'15.99"

   o  East: 21o47'19.28"

4.3.  Access

   The best way to come to Greece is by plane to the Athens
   International Airport.

   From there there are three ways to arrive in the University of
   Patras.

   1.  Renting a car and driving to the University.  It is a maximum
       2:30 hours drive from the aiport.

   2.  Via coach station.  Get from the airport to the coach station via
       X93 bus towards the Kifissos Coach Station.  At the Coach Station
       there are buses to Patras every 30 minutes.  The Bus to Patras
       may take about 2:30 - 3:00 hours, and the ride of the X93 bus may
       take about 30 mins - 1hour depending on the traffic, so it's
       about 3:30 - 4:30 hours away with the wait at the Coach Station.

   3.  Via Train.  It is recommended you already have booked your ticket
       beforehand as there are not many trains going to Patras, and
       mostly are booked in advanced.  It is not recommended that you
       take the train to Patras, as you have to change at least 2
       trains.  In order to reach Patras from the Athens International
       Airport you need to take the Suburban Rail to Kiato.  From Kiato
       you can catch a train to Patras.  It will take you at least 5
       hours to reach Patras.

5.  Testbed architecture

   Most FEs and CEs were located locally at the University of Patras
   premises.  One party participated connecting over the internet.

   The test took place between FEs and CEs of different implementors
   with different permutations.

   All protocol messages of each scenario were monitored using a
   protocol network analyzer that tested validity.  Two tools were used:

   o  A modified tcpdump [tcpdump].

   o  A modified Ethereal [ethereal].

   All NE's in all the scenarios were comprised of one CE and one FE
   from different implementors.

5.1.  Local configuration

   Hardware/Software (CEs and FEs) that were located within the
   University of Patras premises, were connected together using
   switches.

   The scenarios were tested with only one CE associated with one or
   multiple FEs from different implementors.  The CE and the FE(s) were
   connected in one LAN as shown in the following figure.

                                  +-----+
                                  | CE1 |
                                  |Impl1|
                                  +-----+
                                     |
                                     |
                   +------------------------------------+
                   |                LAN                 |
                   +------------------------------------+
                      |       |         |          |
                      |       |   ...   |          |
                   +-----+ +-----+   +-----+   +--------+
                   | FE1 | | FE2 |   | FEn |   |Protocol|
                   |Impl1| |Impl2|   |Impln|   |Analyzer|
                   +-----+ +-----+   +-----+   +--------+

   All scenarios were tested more than once with permutation of the CE
   from different implementors.  In the next permutation, the setup were
   as shown in the following figure.

                                  +-----+
                                  | CE2 |
                                  |Impl2|
                                  +-----+
                                     |
                                     |
                   +------------------------------------+
                   |                LAN                 |
                   +------------------------------------+
                      |       |         |          |
                      |       |   ...   |          |
                   +-----+ +-----+   +-----+   +--------+
                   | FE1 | | FE2 |   | FEn |   |Protocol|
                   |Impl1| |Impl2|   |Impln|   |Analyzer|
                   +-----+ +-----+   +-----+   +--------+

5.2.  Distributed configuration

   For parties that cannot participate, public IPs can be provided and event notifications, etc.  The reader
   associations can be achieved over the internet as seen in the
   following figure.

       +-----+   +------------+   /\/\/\/\/\   +----------+   +-----+
       |FE/CE|   |Implementor |   \Internet/   |University|   |FE/CE|
       |ImplX|---|   Router   |---/        \---|  Router  |---|ImplY|
       +-----+   +------------+   \/\/\/\/\/   +----------+   +-----+

   For interoperability issues, all CEs and FEs MUST implement no
   security even in the TML.  For security, firewalls MUST be used that
   will allow only the specific IPs and the SCTP ports defined in the
   SCTP-TML draft [I-D.ietf-forces-sctptml].

6.  Scenarios

   Since the main goal of this interoperability test is encouraged to read FE-protocol [I-D.ietf-forces-protocol] for
   further information.

2.2.  ForCES Model

   The FE-MODEL [I-D.ietf-forces-model] presents a formal way test the
   basic protocol functionality, we will limit the test parameters.
   Therefore:

   1.  In the Association Setup Message, all report messages will be
       ignored.

   2.  In the Association Setup Phase, the messages, FEO OperEnable
       Event (FE to define CE), Config FEO Adminup (CE to FE) and FEO Config-
       Resp (FE to CE) will be ignored.  The CE will assume that the FE Logical Function Blocks (LFBs) using XML.  LFB configuration
   components, capabilities,
       is enabled once the LFBSelectors has been queried.

   3.  Only FullDataTLVs are going to be used and not SparseData TLV's.

   4.  There will be no transaction operations.

   5.  Each message shall have only one LFBSelector TLV, one Operation
       TLV and associated events are defined one PathDataTLV per message when these are used.

6.1.  Scenario 1 - Pre-association Setup

   While the
   LFB Pre-association setup is formally created.  The LFBs within not in the FE are accordingly
   controlled ForCES current scope it
   is an essential step before CEs and FEs communicate.  As the first
   part in a standardized way by successfull CE-FE connection the ForCES protocol.

2.3.  Transport mapping layer participating CEs and FEs
   should be able to be configured.

   In the Pre-association Phase the following configuration items MUST
   be setup regarding the CEs:

   o  The TML transports CE ID.

   o  The FE IDs that will be connected to this CE

   o  The IP of the PL messages. FEs that will connect

   o  The TML is where priority ports.

   In the issues of
   how to achieve transport level reliability, congestion control,
   multicast, ordering, etc. are handled.  It is expected Pre-association Phase the following configuration items MUST
   be setup regarding the FEs:

   o  The FE ID.

   o  The CE ID that more than
   one TML this FE will be standardized. connecting to.

   o  The various possible TMLs could vary
   their implementations based on the capabilities IP of underlying media
   and transport.  However, since each the CE that will connect to
   o  The TML priority ports.

   Once each element is standardized,
   interoperability setup and configured, Scenario 1 is guaranteed as long as both endpoints support the
   same TML.  All ForCES Protocol Layer implementations MUST be portable
   across all TMLs.  Although more than one successful.

6.2.  Scenario 2 - TML may be standardized for
   the ForCES Protocol, for the purposes of priority channels connection

   For the current interoperability test, the mandated MUST IMPLEMENT SCTP TML [RFC3654] will be used.

3.  Definitions

   This document follows used as TML.
   The TML connection with the terminology defined by associating element is needed for the ForCES
   Requirements in [RFC3654] and by
   scenario 2 to be successful.

   The SCTP-TML draft [I-D.ietf-forces-sctptml] defines 3 priority
   channels, with specific ports:

   o  High priority - Port number: 6700

   o  Medium priority - Port number: 6701

   o  Lower priority - Port number: 6702

   Once these channels have been established with each associated
   element, will the ForCES framework Scenario 2 be successful.

6.3.  Scenario 3 - Association Setup - Association Complete

   Once the Pre-association phase has been complete in [RFC3746].
   The definitions below the previous 2
   scenarios, CEs and FEs are repeated below for clarity.

      Control Element (CE) - A logical entity that implements ready to communicate using the ForCES
      protocol
   protocol, and uses it to instruct one or more enter the Association Setup stage.  In this stage the
   FEs on how attempts to process
      packets.  CEs handle functionality such as join the execution of
      control and signaling protocols.

      CE Manager (CEM) - A logical entity responsible NE.  The following ForCES protocol messages
   will be exchanged for generic CE
      management tasks.  It is particularly used during each CE-FE pair in the pre-
      association phase specified order:

   o  Association Setup Message (from FE to determine with which FE(s) a CE)

   o  Association Setup Response Message (from CE should
      communicate.  This process is called to FE)

   o  Query Message: FEO LFBSelectors(from CE to FE)

   o  Query Response: FEO LFBSelectors response (from FE discovery and may involve to CE)

   Once the associations has been initialized scenario 3 will have been
   successful.

6.4.  Scenario 4 - CE manager learning query

   Once the capabilities of available FEs.

      Forwarding Element (FE) - A logical entity that implements Association Phase stage has been complete, the
      ForCES protocol. FEs use the underlying hardware to provide per-
      packet processing and handling as directed/controlled by one or
      more CEs via
   will enter the Established stage.  In this stage the ForCES protocol.

      FE Manager (FEM) - A logical entity responsible for generic FE
      management tasks.  It is used during pre-association phase to
      determine with which CE(s) an FE should communicate.  This process is called
   continuously updated or queried.  The CE discovery and may involve should query the FE manager learning a
   specific value from the
      capabilities of available CEs.  An FE manager may use anything Object LFB and from a static configuration to a pre-association phase protocol
      (see below) to determine which CE(s) to use.  Being a logical
      entity, an FE manager might be physically combined with any of the
      other logical entities such as FEs.

      ForCES Network Element (NE) - FE Protocol LFB.
   An entity composed of one or more
      CEs example from the FE Protocol LFB is the HeartBeat Timer (FEHI) and one or more FEs.  To entities outside an NE,
   from the NE
      represents a single point FE Object LFB is the State of management.  Similarly, an NE usually
      hides its internal organization from external entities. the LFB (Logical Function Block) (FEState)

   The following ForCES protocol messages will be exchanged:

   o  Query Message

   o  Query Response Message

6.5.  Scenario 5 - Heartbeat monitoring

   The basic building block that Heartbeat (HB) Message is
      operated on by the used for one ForCES element (FE or CE)
   to asynchronously notify one or more other ForCES protocol.  The LFB is a well defined,
      logically separable functional block that resides elements in an FE and is
      controlled by the CE via
   same ForCES protocol. NE on its liveness.  The LFB may reside at
      the FE's datapath and process packets or may be purely an FE
      control or default configuration entity that is operated on by the CE.
      Note that the LFB is a functionally accurate abstraction of the
      FE's processing capabilities, but not a hardware-accurate
      representation
   Heartbeat Policy of the FE implementation. is set to 0 which means, that the FE Topology - A representation of how
   should not generate any Heartbeat messages. the multiple FEs within a
      single NE are interconnected.  Sometimes this CE is called inter-FE
      topology, to be distinguished from intra-FE topology (i.e., LFB
      topology).

      LFB Class and LFB Instance - LFBs are categorized responsible for
   checking FE liveness by LFB Classes.
      An LFB Instance represents an LFB Class (or Type) existence.
      There may be multiple instances setting the PL header ACK flag of the same LFB Class (or Type) in
      an FE.  An LFB Class is represented by an LFB Class ID, and an LFB
      Instance is represented by an LFB Instance ID.  As message
   it sends to AlwaysACK.  In this Scenario the CE should send a result, an
      LFB Class ID associated
   Heartbeat message with an LFB Instance ID uniquely specifies
      an LFB existence.

      LFB Metadata - Metadata is used to communicate per-packet state
      from one LFB the ACK flag set to another, but AlwaysACK and the FE
   should respond.

   The following ForCES protocol messages will be exchanged:

   o  Heartbeat Message

6.6.  Scenario 6 - Simple Config Command

   A config message is not sent across the network.  The
      FE model defines how such metadata is identified, produced and
      consumed by the LFBs.  It defines CE to the functionality but not how
      metadata is encoded within an implementation. FE to configure LFB Component - Operational parameters of
   components in the LFBs that must be FE.  A simple config command easily visible and
   metered would be to change the CEs are conceptualized Heartbeat configuration.  This will be
   done in two steps:

   1.  Change the FE model as Heartbeat Policy (FEHBPolicy) to value 1, to force
       the LFB
      components.  The LFB components include, for example, flags,
      single parameter arguments, complex arguments, and tables that FE to send heartbeats.

   2.  After some heartbeats from the
      CE can read and/or write via FE, the FE Heartbeat Interval
       (FEHI) will be changed.

   The following ForCES protocol (see below).

      LFB Topology messages will be exchanged:

   o  Config Message

   o  Config Response Message

6.7.  Scenario 7 - Representation of how Association Teardown

   In the LFB instances are
      logically interconnected and placed along end, the datapath within one
      FE.  Sometimes it is also called intra-FE topology, to association must be
      distinguished from inter-FE topology.

      Pre-association Phase - The period of time during which an FE
      Manager and a CE Manager terminated.  There are determining which FE(s) and CE(s)
      should be part of the same network element.

      Post-association Phase - The period of time during which an FE
      knows which CE is to control it and vice versa.  This includes the
      time during two
   scenarios by which the CE and association maybe terminated:

   1.  Normal tear down by exchanging Association Teardown Message

   2.  Irregular tear down by stopping heartbeats from a FE are establishing communication
      with one another.

      ForCES Protocol - While there or a CE.

   3.  Irregular tear down by externally shutting down/rebooting a FE or
       a CE.

   All scenarios may be multiple protocols used
      within the overall ForCES architecture, the term "ForCES protocol"
      and "protocol" refer to the Fp reference points tested in the interoperability test.

   The following ForCES
      Framework in [RFC3746].  This protocol does messages will be exchanged:

   o  Association Teardown Message

7.  Tested Features

   The features that were tested are:

7.1.  ForCES Protocol Features

                              +------------+
                              |   Feature  |
                              +------------+
                              |  Batching  |
                              |            |
                              | HeartBeats |
                              +------------+

                         ForCES Protocol Features

   Although Batching was not apply to CE-to-CE
      communication, FE-to-FE communication, or initially designed to communication between
      FE and CE managers.  Basically, the ForCES protocol works in a
      master- slave mode in which FEs are slaves and CEs are masters.
      This document defines be tested, it was
   tested during the specifications for this interoperability test.

7.1.1.  Protocol Messages

                      +----------------------------+
                      |      Protocol Message      |
                      +----------------------------+
                      |      Association Setup     |
                      |                            |
                      | Association Setup Response |
                      |                            |
                      |    Association TearDown    |
                      |                            |
                      |        Configuration       |
                      |                            |
                      |   Configuration Response   |
                      |                            |
                      |            Query           |
                      |                            |
                      |       Query Response       |
                      |                            |
                      |          HeartBeat         |
                      +----------------------------+

                          ForCES protocol. Protocol Message

7.1.2.  MainHeader Handling

                           +------------------+
                           |   Header Field   |
                           +------------------+
                           |    Correlator    |
                           |                  |
                           | Acknowledge Flag |
                           |                  |
                           |   Priority Flag  |
                           +------------------+

                            MainHeader Handling

7.1.3.  TLV Handling

                    +---------------------------------+
                    |               TLV               |
                    +---------------------------------+
                    |   Association Setup Result TLV  |
                    |                                 |
                    | Association TearDown Reason TLV |
                    |                                 |
                    |         LFBSelector TLV         |
                    |                                 |
                    |          Operation TLV          |
                    |                                 |
                    |           PathData TLV          |
                    |                                 |
                    |           FullData TLV          |
                    |                                 |
                    |            Result TLV           |
                    +---------------------------------+

                              TLVs Supported

7.1.4.  Operation Types Supported

                             +--------------+
                             |   Operation  |
                             +--------------+
                             |      Set     |
                             |              |
                             | Set Response |
                             |              |
                             |      Get     |
                             |              |
                             | Get Response |
                             |              |
                             |    Report    |
                             +--------------+

                         Operation Type Supported

7.2.  ForCES Model Features

7.2.1.  Basic Atomic Types Supported

                              +-------------+
                              | Atomic Type |
                              +-------------+
                              |    uchar    |
                              |             |
                              |    uint32   |
                              +-------------+

                       Basic Atomic Types Supported

7.2.2.  Compound Types Supported

                             +---------------+
                             | Compound Type |
                             +---------------+
                             |    structs    |
                             |               |
                             |     arrays    |
                             +---------------+

                         Compound Types Supported

7.2.3.  LFBs Supported

7.2.3.1.  FE Protocol Transport Mapping Layer (ForCES TML) - A layer in
      ForCES protocol architecture that uses the capabilities of
      existing transport protocols to specifically address protocol
      message transportation issues, such as how the protocol messages
      are mapped to different transport media (like TCP, IP, ATM,
      Ethernet, etc), and how to achieve and implement reliability,
      multicast, ordering, etc.  The LFB

                          +--------------------+
                          | Protocol DataTypes |
                          +--------------------+
                          |     CEHBPolicy     |
                          |                    |
                          |     FEHIBPolicy    |
                          +--------------------+

                         FE Protocol LFB Datatypes

                          +---------------------+
                          | Protocol Components |
                          +---------------------+
                          |         FEID        |
                          |                     |
                          |      CEHBPolicy     |
                          |                     |
                          |        CEHDI        |
                          |                     |
                          |      FEHBPolicy     |
                          |                     |
                          |         FEHI        |
                          |                     |
                          |         CEID        |
                          +---------------------+

                        FE Protocol LFB Components

7.2.3.2.  FE Object LFB

                           +------------------+
                           | Object DataTypes |
                           +------------------+
                           |   FEStateValues  |
                           |                  |
                           |  LFBSelectorType |
                           +------------------+

                          FE Object LFB Datatypes
                           +-------------------+
                           | Object Components |
                           +-------------------+
                           |    LFBSelectors   |
                           |                   |
                           |      FEState      |
                           +-------------------+

                         FE Object LFB Components

7.3.  ForCES SCTP-TML Features

7.3.1.  TML specifications are
      detailed in separate Priority Ports

                        +------------------------+
                        |          Port          |
                        +------------------------+
                        |  High priority (6700)  |
                        |                        |
                        | Medium priority (6701) |
                        |                        |
                        |   Low priority (6702)  |
                        +------------------------+

                              Priority Ports

7.3.2.  Message Handling at specific priorities

                      +----------------------------+
                      |       ForCES documents, one for each TML.

4.  Date, Location and Access

4.1.  Date

   The date that the Interoperability draft will take place has been
   specified Message       |
                      +----------------------------+
                      |      Association Setup     |
                      |                            |
                      | Association Setup Response |
                      |                            |
                      |    Association Teardown    |
                      |                            |
                      |           Config           |
                      |                            |
                      |       Config Response      |
                      |                            |
                      |            Query           |
                      |                            |
                      |       Query Response       |
                      +----------------------------+

               Message Handling at High priority (6700) Port
                            +----------------+
                            | ForCES Message |
                            +----------------+
                            |   Heartbeats   |
                            +----------------+

               Message Handling at 15-16/07/2009, one and a half week before IETF 75, in
   Stockholm.

4.2.  Location

   Patras is a major harbor of Greece connecting it with Italy.

   The University of Patras is located in Rio, 10km east out of Patras. Low priority (6702) Port

8.  Test details

   The following coordinates mark the Electrical Engineering building in
   the University.

   o  North: 38o17'15.99"

   o  East: 21o47'19.28"

4.3.  Access

   The best way to come to Greece is by plane to the Athens
   International Airport.

   From there there are three ways to arrive in the University of
   Patras.

   1.  Renting a car and driving to the University.  It is a maximum
       2:30 hours drive from the aiport.

   2.  Via coach station.  Get from the airport to the coach station via
       X93 bus towards the Kifissos Coach Station.  At the Coach Station
       there are buses to Patras every 30 minutes.  The Bus to Patras
       may take about 2:30 - 3:00 hours, and the ride of the X93 bus may
       take about 30 mins - 1hour depending on the traffic, so it's
       about 3:30 - 4:30 hours away with the wait at the Coach Station.

   3.  Via Train.  It is recommended you already have booked your ticket
       beforehand as there are not many trains going to Patras, and
       mostly are booked in advanced.  It is not recommended that you
       take the train to Patras, as you have to change at least tests occured:

   +------+----------+----------+-----------+-----------+--------------+
   | Test |    CE    |   FE(s)  |  Teardown |   Result  |    Comment   |
   | #    |          |          |   Option  |           |              |
   +------+----------+----------+-----------+-----------+--------------+
   |   1  | Zhejiang |    NTT   |  Teardown |  Success  |              |
   |      | Gongshan |          |  from FE  |           |              |
   |      | g        |          |           |           |              |
   |      |  Univers |          |           |           |              |
   |      | it    y  |          |           |           |              |
   |      |          |          |           |           |              |
   |   2
       trains.  In order to reach Patras  | Zhejiang |    NTT   |  Teardown |  Success  |              |
   |      | Gongshan |          |  from CE  |           |              |
   |      | g        |          |           |           |              |
   |      |  Univers |          |           |           |              |
   |      | it    y  |          |           |           |              |
   |      |          |          |           |           |              |
   |   3  | Zhejiang |    NTT   |   Cable   |  Success  |  Nobody saw  |
   |      | Gongshan |          | disconnec |           |  the Athens International
       Airport you need to take the Suburban Rail to Neratziotissa.
       From there you must take ISAP to Pireaus.  There you must change
       again to Suburban Rail to reach Kiato.  From Kiato you can catch
       a train to Patras.  It will take you at least loss of |
   |      | g        |          | t         |           |    cable.    |
   |      |  Univers |          |           |           |   Everybody  |
   |      | it    y  |          |           |           |   found out  |
   |      |          |          |           |           | from loss of |
   |      |          |          |           |           | PL-heartbeat |
   |      |          |          |           |           | s            |
   |      |          |          |           |           |              |
   |   4  | Zhejiang |    NTT   |  Loss of  |  Success  |   FE didn't  |
   |      | Gongshan |          |     CE    |           |     send     |
   |      | g        |          | Heartbeat |           | Teardown and |
   |      |  Univers |          | s         |           |    closed    |
   |      | it    y  |          |           |           |  connection  |
   |      |          |          |           |           |              |
   |   5 hours to reach
       Patras.

5.  Testbed architecture

   Most FEs and CEs should be located locally at the University  | Zhejiang |    NTT   |  Loss of
   Patras premises.  But if some parties would like to participate but
   cannot attend the interoperability test locally a connection over the
   internet MAY be created.

   The actual test will take place between FEs and CEs  | Untestabl |              |
   |      | Gongshan |          |     FE    | e         |              |
   |      | g        |          | Heartbeat |           |              |
   |      |  Univers |          | s         |           |              |
   |      | it    y  |          |           |           |              |
   |      |          |          |           |           |              |
   |   6  |    NTT   | Zhejiang |  Teardown |  Initial  |  CE couldn't |
   |      |          | Gongshan |  from CE  |  Failure  | handle Query |
   |      |          | g        |           |           |  Result for  |
   |      |          |  Univers |           |           |    unknown   |
   |      |          | it    y  |           |           |  LFBSelects. |
   |      |          |          |           |           |              |
   |   7  | Zhejiang | Universi |  Teardown |  Success  |   Problems   |
   |      | Gongshan | t  y of different
   implementors  |  from FE  |           |     with different permutations.

   All protocol messages of each scenario will be monitored using a
   protocol network analyzer to test validity.  Two tools shall be used:

   o  A modified tcpdump [tcpdump].     |
   |      | g        |   Patras |           |           | retransmitti |
   |      |  Univers |          |           |           | o  A modified Ethereal [ethereal].

   All NE's in all the scenarios will be comprised      n     |
   |      | it    y  |          |           |           |              |
   |      |          |          |           |           |              |
   |   8  | Zhejiang | Universi |  Teardown |  Success  |   Problems   |
   |      | Gongshan | t  y of one CE and one FE  |  from different implementors.

5.1.  Local configuration

   Hardware/Software (CEs and FEs) that will be located within the
   University of Patras premises, will be connected together using
   switches.

   The scenarios will be tested with only one CE associated  |           |     with one or
   multiple FEs from different implementors.  The CE and the FE(s) will
   be connected in one LAN as shown in the following figure.

                                  +-----+     | CE1
   |
                                  |Impl1|
                                  +-----+      | g        |
                   +------------------------------------+   Patras |                LAN           |
                   +------------------------------------+           | retransmitti |
   |      |  Univers |          |   ...           |           |
                   +-----+ +-----+   +-----+   +--------+ o      n     | FE1
   |      | FE2 it    y  |          | FEn           |   |Protocol|
                   |Impl1| |Impl2|   |Impln|   |Analyzer|
                   +-----+ +-----+   +-----+   +--------+

   All scenarios will be tested more than once with permutation           |              |
   |      |          |          |           |           |              |
   |   9  | Zhejiang | Universi |   Cable   |  Success  |  Nobody saw  |
   |      | Gongshan | t  y of  | disconnec |           |  the
   CE loss of |
   |      | g        |   Patras | t         |           |    cable.    |
   |      |  Univers |          |           |           |   Everybody  |
   |      | it    y  |          |           |           |   found out  |
   |      |          |          |           |           | from different implementors.  In the next permutation, the setup
   will be as shown in the following figure.

                                  +-----+ loss of | CE2
   |
                                  |Impl2|
                                  +-----+      |          |
                   +------------------------------------+          |                LAN           |
                   +------------------------------------+           | PL-heartbeat |
   |      |          |          |   ...           |           |
                   +-----+ +-----+   +-----+   +--------+ s            | FE1
   |      | FE2          |          | FEn           |   |Protocol|
                   |Impl1| |Impl2|   |Impln|   |Analyzer|
                   +-----+ +-----+   +-----+   +--------+

5.2.  Distributed configuration

   For parties that cannot participate, public IPs can be provided and
   associations can be achieved over the internet as seen in the
   following figure.

       +-----+   +------------+   /\/\/\/\/\   +----------+   +-----+
       |FE/CE|   |Implementor           |   \Internet/   |University|   |FE/CE|
       |ImplX|---|   Router   |---/        \---|  Router  |---|ImplY|
       +-----+   +------------+   \/\/\/\/\/   +----------+   +-----+

   For interoperability issues, all CEs and FEs MUST implement no
   security even in the TML.  For security, firewalls MUST be used that
   will allow only the specific IPs and the SCTP ports defined in the
   SCTP-TML draft [I-D.ietf-forces-sctptml].

6.  Scenarios

   Since the main goal              |
   |  10  | Zhejiang | Universi |  Loss of  |  Success  |              |
   |      | Gongshan | t  y of this interoperability test is to test the
   basic protocol functionality, we will limit the test parameters.
   Therefore:

   1.  In the Association Setup Message, all report messages will be
       ignored.

   2.  In the Association Setup Phase, the messages, FEO OperEnable
       Event (FE to CE), Config FEO Adminup (CE to FE) and FEO Config-
       Resp (FE to CE) will be ignored.  The  |     CE will assume that the FE
       is enabled once the LFBSelectors has been queried.

   3.  Only FullDataTLVs are going to be used and not SparseData TLV's.

   4.  There will be no transaction operations.

   5.  Each message shall have only one LFBSelector TLV, one Operation
       TLV and one PathDataTLV per message when these are used.

6.1.  Scenario 1 - Pre-association Setup

   While the Pre-association setup is not in the ForCES current scope    |           |              |
   |      | g        |   Patras | Heartbeat |           |              |
   |      |  Univers |          | s         |           |              |
   |      | it
   is an essential step before CEs and FEs communicate.  As the first
   part in a successfull CE-FE connection the participating CEs and FEs
   should be able to be configured.

   In the Pre-association Phase the following configuration items MUST
   be setup regarding the CEs:

   o  The    y  |          |           |           |              |
   |      |          |          |           |           |              |
   |  11  |    NTT   | Zhejiang |  Teardown |  Success  |   Test# 6.   |
   |      |          | Gongshan |  from CE ID.

   o  The  | on Repeat |   Problems   |
   |      |          | g        |           |           |     fixed    |
   |      |          |  Univers |           |           |              |
   |      |          | it    y  |           |           |              |
   |      |          |          |           |           |              |
   |  12  |    NTT   | Zhejiang |  Teardown |  Success  |              |
   |      |          | Gongshan |  from FE IDs that will be connected to this CE

   o  The IP of the FEs that will connect

   o  The TML priority ports.

   In the Pre-association Phase the following configuration items MUST
   be setup regarding  |           |              |
   |      |          | g        |           |           |              |
   |      |          |  Univers |           |           |              |
   |      |          | it    y  |           |           |              |
   |      |          |          |           |           |              |
   |  13  |    NTT   | Zhejiang |   Cable   |  Success  |  Nobody saw  |
   |      |          | Gongshan | disconnec |           |  the FEs:

   o  The FE ID. loss of |
   |      |          | g        | t         |           |    cable.    |
   |      |          |  Univers |           |           |   Everybody  |
   |      |          | it    y  |           |           |   found out  |
   |      |          |          |           |           | from loss of |
   |      |          |          |           |           | PL-heartbeat |
   |      |          |          |           |           | s      .     |
   |      |          |          |           |           |              |
   |  14  |    NTT   | Zhejiang |  Loss of  |  Success  |   Problems   |
   |      |          | Gongshan |     CE    |           |     with     |
   |      |          | g        | Heartbeat |           | retransmitti |
   |      |          |  Univers | s         |           | o  The      n     |
   |      |          | it    y  |           |           |              |
   |      |          |          |           |           |              |
   |  15  | Universi | Zhejiang |  Teardown |  Success  |   CE ID that this FE will be connecting to.

   o  The IP didn't  |
   |      | t  y of the CE that will connect to
   o  The TML priority ports.

   Once each element is setup and configured, Scenario 1 is successful.

6.2.  Scenario 2 - TML priority channels connection

   For the current interoperability test, the SCTP will be used as TML.
   The TML connection with the associating element is needed for the
   scenario 2 to be successful.

   The SCTP-TML draft [I-D.ietf-forces-sctptml] defines 3 priority
   channels, with specific ports:

   o  High priority - Port number: 6700

   o  Medium priority - Port number: 6701

   o  Lower priority - Port number: 6702

   Once these channels have been established with each associated
   element, will the Scenario 2 be successful.

6.3.  Scenario 3 - Association Setup - Association Complete

   Once the Pre-association phase has been complete in the previous 2
   scenarios, CEs and FEs are ready to communicate using the ForCES
   protocol, and enter the Association Setup stage.  In this stage the
   FEs attempts to join the NE.  The following ForCES protocol messages
   will be exchanged for each CE-FE pair in the specified order:  | Gongshan |  from FE  |           |   terminat   |
   |      |   Patras | g        |           |           |     after    |
   |      |          |  Univers |           |           |    sending   |
   |      |          | it    y  |           |           |  Teardown.   |
   |      |          |          |           |           |    FE did    |
   |      |          |          |           |           |              |
   |  16  | Universi | Zhejiang |  Teardown |  Success  |   Problems   |
   |      | t  y of  | Gongshan |  from CE  |           |     with     |
   |      |   Patras | g        |           |           | retransmitti |
   |      |          |  Univers |           |           | o  Association Setup Message (from      n     |
   |      |          | it    y  |           |           |              |
   |      |          |          |           |           |              |
   |  17  | Universi | Zhejiang |  Loss of  |  Success  |   FE to CE)

   o  Association Setup Response Message (from didn't  |
   |      | t  y of  | Gongshan |     CE to FE)    |           |     send     |
   |      |   Patras | g        | Heartbeat |           | Teardown and |
   |      |          |  Univers | s         |           |    closed    |
   |      |          | it    y  |           |           |  connection  |
   |      |          |          |           |           |              |
   |  18  | Zhejiang |   NTT &  |  Teardown |  Success  |              |
   |      | Gongshan | Universi |  from CE  |           |              |
   |      | g        | t  y of  |           |           |              |
   |      |  Univers |  Patrasx |           |           |              |
   |      | it    y  | 2        |           |           |              |
   |      |          |          |           |           |              |
   |  19  |    NTT   | Zhejiang |  Teardown |  Success  |              |
   |      |          | Gongshan |  from CE  |           |              |
   |      |          | g        |           |           |              |
   |      |          |  Univers |           |           |              |
   |      |          | it   y & |           |           |              |
   |      |          |   Univer |           |           |              |
   |      |          | sit  y o  Query Message: FEO LFBSelectors(from |           |           |              |
   |      |          | f  Patra |           |           |              |
   |      |          | sx2      |           |           |              |
   |      |          |          |           |           |              |
   |  20  | Universi |   NTT &  |  Teardown |  Success  |              |
   |      | t  y of  | Zhejiang |  from CE to FE)  |           |              |
   |      |   Patras | Gongshan |           |           |              |
   |      |          | g        |           |           |              |
   |      |          |  Univers |           |           |              |
   |      |          | it   y & |           |           |              |
   |      |          |   Univer |           |           |              |
   |      |          | sit  y o |           |           |              |
   |      |          | f  Patra |           |           |              |
   |      |          | sx2      |           |           |              |
   |      |          |          |           |           |              |
   |  21  | Universi | Zhejiang |  Batching |  Success  |              |
   |      | t  y of  | Gongshan | Query Response: FEO LFBSelectors response (from and |           |              |
   |      |   Patras | g        |   Config  |           |              |
   |      |          |  Univers |           |           |              |
   |      |          | it    y  |           |           |              |
   |      |          |          |           |           |              |
   |  22  | Universi |    NTT   |  Teardown |  Success  |              |
   |      | t  y of  |          |  from FE to CE)

   Once the associations has been initialized scenario 3 will have been
   successful.

6.4.  Scenario 4 -  |           |              |
   |      |   Patras |          |           |           |              |
   |      |          |          |           |           |              |
   |  23  | Universi |    NTT   |  Teardown |  Success  |              |
   |      | t  y of  |          |  from CE query

   Once the Association Phase stage has been complete, the FEs and CEs
   will enter the Established stage.  In this stage the  |           |              |
   |      |   Patras |          |           |           |              |
   |      |          |          |           |           |              |
   |  24  | Universi |    NTT   |  Loss of  |  Success  |   FE is
   continuously updated or queried.  The didn't  |
   |      | t  y of  |          |     CE should query    |           |     send     |
   |      |   Patras |          | Heartbeat |           | Teardown and |
   |      |          |          | s         |           |    closed    |
   |      |          |          |           |           |  connection  |
   |      |          |          |           |           |              |
   |  25  | Universi |    NTT   |   Cable   |  Success  |  Nobody saw  |
   |      | t  y of  |          | disconnec |           |  the FE a
   specific value loss of |
   |      |   Patras |          | t         |           |    cable.    |
   |      |          |          |           |           |   Everybody  |
   |      |          |          |           |           |   found out  |
   |      |          |          |           |           | from the FE Object LFB and loss of |
   |      |          |          |           |           | PL-heartbeat |
   |      |          |          |           |           | s            |
   |      |          |          |           |           |              |
   |  26  |    NTT   | Universi |  Teardown |  Success  |              |
   |      |          | t  y of  |  from the FE Protocol LFB.
   An example  |           |              |
   |      |          |   Patras |           |           |              |
   |      |          |          |           |           |              |
   |  27  |    NTT   | Universi |  Teardown |  Success  |              |
   |      |          | t  y of  |  from the CE  |           |              |
   |      |          |   Patras |           |           |              |
   |      |          |          |           |           |              |
   |  28  |    NTT   | Universi |  Loss of  |  Success  |   FE Protocol LFB is didn't  |
   |      |          | t  y of  |     CE    |           |     send     |
   |      |          |   Patras | Heartbeat |           | Teardown and |
   |      |          |          | s         |           |    closed    |
   |      |          |          |           |           |  connection  |
   |      |          |          |           |           |              |
   |  29  |    NTT   | Universi |   Cable   |  Success  |  Nobody saw  |
   |      |          | t  y of  | disconnec |           |  the HeartBeat Timer (FEHI) and loss of |
   |      |          |   Patras | t         |           |    cable.    |
   |      |          |          |           |           |   Everybody  |
   |      |          |          |           |           |   found out  |
   |      |          |          |           |           | from the FE Object LFB is the State loss of the LFB (FEState) |
   |      |          |          |           |           | PL-heartbeat |
   |      |          |          |           |           | s            |
   +------+----------+----------+-----------+-----------+--------------+

                          Interoperability Tests

9.  Results

   All implementations were found to be interoperable with each other.

   All scenarios were tested successfully.

   The following ForCES protocol issues were found and dealt with.

   1.   Some messages will be exchanged:

   o  Query Message

   o  Query Response Message

6.5.  Scenario 5 - Heartbeat monitoring

   The Heartbeat (HB) Message is used for one ForCES element (FE or CE) were sent to asynchronously notify one or more other ForCES elements in the
   same ForCES NE wrong priority channels.  There
        was some ambiguities on its liveness.  The default configuration of the
   Heartbeat Policy of the FE is set to 0 which means, SCTP-TML draft that the FE
   should not generate any Heartbeat messages. the have been
        corrected.

   2.   At some point, a CE is responsible for
   checking FE liveness by setting the PL header ACK flag of the sent a TearDown message
   it sends to AlwaysACK.  In this Scenario the FE.  The CE should send a
   Heartbeat message with
        expected the ACK flag set FE to AlwaysACK shut down the connection, and the FE
   should respond.

   The following ForCES protocol messages will be exchanged:

   o  Heartbeat Message

6.6.  Scenario 6 - Simple Config Command

   A config message is sent by waited
        the CE to shut down the FE to configure LFB
   components connection and were caught in a
        deadlock.  This was a code bug and was fixed.

   3.   Sometimes the FE.  A simple config command easily visible association setup message, only on the remote
        connection test, although sent, was not received by the other
        end and
   metered would be to change made impossible the Heartbeat configuration. association.  This will was caused by
        network problems.

   4.   An implementation did not take into account that the padding in
        TLVs MUST NOT be
   done included in two steps:

   1.  Change the FE Heartbeat Policy (FEHBPolicy) to value 1, to force lenght of the FE TLV.  This was a
        code bug and was fixed.

   5.   EM Flag was set to send heartbeats.

   2. reserved by a CE and was not ignored by the
        FE.  This was a code bug and was fixed.

   6.   After some heartbeats from the FE, FEHBPolicy was set to 1 the FE Heartbeat Interval
       (FEHI) will be changed.

   The following ForCES protocol messages will be exchanged:

   o  Config Message

   o  Config Response Message

6.7.  Scenario 7 - Association Teardown

   In the end, didn't send any
        HeartBeats.  This was a code bug and was fixed.

   7.   Some FE's sent HeartBeats with the association must be terminated.  There are two
   scenarios by which ACK flag with a value other
        than NoACK.  The CE responded.  This was a code bug and was
        fixed.

   8.   When a cable was disconnected, the TML didn't realize that.  The
        association maybe terminated:

   1.  Normal tear down by exchanging Association Teardown Message

   2.  Irregular tear down by stopping heartbeats from was dropped due to heartbeats, this was a FE or success,
        but this is an implementation issue implementers should keep in
        mind.  This is a CE.

   3.  Irregular tear down by externally shutting down/rebooting SCTP options issue.  Nothing was needed to be
        done.

   9.   A CE crashed due to unknown LFBSelector values.  This was a FE or code
        bug and was fixed.

   10.  With the remote connection there were a CE.

   All scenarios may lot of forces packet
        retransmittion.  The problem is that packets like Heartbeats
        were retransmitted.  This is a SCTP issue.  Perhaps SCTP-PR is
        needed to be tested in used.

   The implementers went beyond the interoperability test. call of duty.  The following ForCES protocol messages will be exchanged:

   o  Association Teardown Message

7. test was extended
   with another test for batching messages.  This test was also done
   successfully.

10.  Acknowledgements

   The authors of this draft would like to acknowledge and thank the
   chair of the ForCES working group Jamal Hadi Salim.

8.

   Also, the authors would like to acknowledge Professors Odysseas
   Koufopavlou and Spyros Denazis, as well as the Department of
   Electrical and Computer Engineering of the University of Patras for
   hosting the event.

11.  IANA Considerations

   This memo includes no request to IANA.

9.

12.  Security Considerations

   Section 9 of the FE-protocol [I-D.ietf-forces-protocol] specifies
   security considerations of the ForCES protocol.  For this
   interoperability test, no security MUST be chosen even for the
   distributed architecture.

10.

13.  References

10.1.

13.1.  Normative References

   [I-D.ietf-forces-model]
              Halpern, J. and J. Salim, "ForCES Forwarding Element
              Model", draft-ietf-forces-model-16 (work in progress),
              October 2008.

   [I-D.ietf-forces-protocol]
              Dong, L., Doria, A., Gopal, R., HAAS, R., Salim, J.,
              Khosravi, H., and W. Wang, "ForCES Protocol
              Specification", draft-ietf-forces-protocol-22 (work in
              progress), March 2009.

   [I-D.ietf-forces-sctptml]
              Salim, J. and K. Ogawa, "SCTP based TML (Transport Mapping
              Layer) for ForCES protocol", draft-ietf-forces-sctptml-02
              (work in progress), January 2009.

10.2.

13.2.  Informative References

   [RFC2119]  Bradner, S., "Key words for use in RFCs to Indicate
              Requirement Levels", BCP 14, RFC 2119, March 1997.

   [RFC2629]  Rose, M., "Writing I-Ds and RFCs using XML", RFC 2629,
              June 1999.

   [RFC3552]  Rescorla, E. and B. Korver, "Guidelines for Writing RFC
              Text on Security Considerations", BCP 72, RFC 3552,
              July 2003.

   [RFC3654]  Khosravi, H. and T. Anderson, "Requirements for Separation
              of IP Control and Forwarding", RFC 3654, November 2003.

   [RFC3746]  Yang, L., Dantu, R., Anderson, T., and R. Gopal,
              "Forwarding and Control Element Separation (ForCES)
              Framework", RFC 3746, April 2004.

   [RFC5226]  Narten, T. and H. Alvestrand, "Guidelines for Writing an
              IANA Considerations Section in RFCs", BCP 26, RFC 5226,
              May 2008.

   [ethereal]
              "Ethereal is a protocol analyzer. The specific ethereal
              that will be used is an updated Ethereal, by Fenggen Jia,
              that can analyze and decode the ForCES protocol
              messages.", <http://peach.ease.lsoft.com/scripts/
              wa.exe?A2=ind0906&L=FORCES&T=0&F=&S=&P=1048>.

   [tcpdump]  "Tcpdump is a linux protocol analyzer. The specific
              tcpdump that will be used is a modified tcpdump, by Jamal
              Hadi Salim, that can analyze and decode the ForCES
              protocol messages.", <http://peach.ease.lsoft.com/scripts/
              wa.exe?A2=ind0906&L=FORCES&T=0&F=&S=&P=2262>.

Authors' Addresses

   Evangelos Haleplidis
   University of Patras
   Patras,
   Greece

   Email: ehalep@ece.upatras.gr

   Kentaro Ogawa
   NTT Corporation
   Tokyo,
   Japan

   Email: ogawa.kentaro@lab.ntt.co.jp

   Xin-ping Wang
   Huawei Technologies Co., Ltd.
   China

   Email: carly.wang@huawei.com

   Chuanhuang Li
   Zhejiang Gongshang University
   18, Xuezheng Str., Xiasha University Town
   Hangzhou,   310018
   P.R.China

   Phone: +86-571-28877751
   Email: chuanhuang_li@pop.zjgsu.edu.cn