[Docs] [txt|pdf] [Tracker] [WG] [Email] [Diff1] [Diff2] [Nits] [IPR]
Versions: 00 01 02 03 04 05 06 07 08 09 10 11
12 RFC 3821
IPS Working Group M. Rajagopal, R. Bhagwat, R. A. Helland,
INTERNET-DRAFT LightSand Comm.
<draft-ietf-ips-fcovertcpip-05.txt> E. Rodriguez, Lucent Tech.
(Expires February, 2002) C. Carlson, QLogic
Category: standards-track D. Fraser, Compaq
D. Peterson, Cisco
L. Lamers, SAN Valley
V. Chau, G. Hecht, Gadzoox Networks
S. Wilson, B. Snively, R. Weber, Brocade Comm.
M. O'Donnell, A. Rijhsinghani, McDATA
S. Rupanagunta, Aarohi Comm.
V. Rangan, Rhapsody Networks
J. Nelson, K. Hirata, Vixel
M. Merhar, Pirus Networks
N. Wanamaker, Akara
Fibre Channel Over TCP/IP (FCIP)
Status of this Memo
This document is an Internet-Draft and is in full conformance with
all provisions of Section 10 of RFC 2026 [1].
Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF), its areas, and its working groups. Note that
other groups may also distribute working documents as Internet-Drafts.
Internet-Drafts are draft documents valid for a maximum of six
months and may be updated, replaced, or obsoleted by other documents
at any time. It is inappropriate to use Internet-Drafts as Reference
material or to cite them other than as "work in progress".
The list of current Internet-Drafts can be accessed at
http://www.ietf.org/ietf/lid-abstracts.txt
The list of Internet-Draft Shadow Directories can be accessed at
http://www.ietf.org/shadow.html
Abstract
Fibre Channel Over TCP/IP (FCIP) describes mechanisms that allow the
interconnection of islands of Fibre Channel storage area networks
over IP-based networks to form a unified storage area network in a
single Fibre Channel fabric. FCIP relies on IP-based network
services to provide the connectivity between the storage area
network islands over local area networks, metropolitan area
networks, or wide area networks.
Rajagopal, et al. Standards Track [Page 1]
Internet-Draft Fibre Channel Over TCP/IP (FCIP) August, 2001
Conventions used in this document
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
"SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
document are to be interpreted as described in RFC 2119 [2].
Table Of Contents
1. Purpose, Motivation and Objectives . . . . . . . . . . . . . . . 3
2. Relationship to Fibre Channel Standards . . . . . . . . . . . . 4
2.1 Relevant Fibre Channel Standards . . . . . . . . . . . . . . . 4
2.2 This Specification and Fibre Channel Standards . . . . . . . . 5
3. Terminology . . . . . . . . . . . . . . . . . . . . . . . . . . 5
4. Open Issues . . . . . . . . . . . . . . . . . . . . . . . . . . 7
5. Protocol Summary . . . . . . . . . . . . . . . . . . . . . . . . 7
6. The FCIP Model . . . . . . . . . . . . . . . . . . . . . . . . . 9
6.1 FCIP Protocol Model . . . . . . . . . . . . . . . . . . . . . . 9
6.2 FCIP Link . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
6.3 FC Entity . . . . . . . . . . . . . . . . . . . . . . . . . . 10
6.4 FCIP Entity . . . . . . . . . . . . . . . . . . . . . . . . . 11
6.5 FCIP Link Endpoint (FCIP_LEP) . . . . . . . . . . . . . . . . 12
6.6 FCIP Data Engine (FCIP_DE) . . . . . . . . . . . . . . . . . . 13
6.6.1 FCIP Encapsulation of FC Frames . . . . . . . . . . . . . . 15
6.6.2 FCIP Data Engine Error Detection and Recover . . . . . . . . 16
6.6.2.1 TCP Assistance With Error Detection and Recovery . . . . . 16
6.6.2.2 Errors in FCIP Headers and Discarding FCIP Frames . . . . 16
6.6.2.3 IP Network Transit Time Validation . . . . . . . . . . . . 17
6.6.2.4 Synchronization Failures . . . . . . . . . . . . . . . . . 18
7. Establishing and Maintaining a Synchronized Time Value . . . . 19
8. TCP Connection Management . . . . . . . . . . . . . . . . . . . 20
8.1 TCP Connection Establishment . . . . . . . . . . . . . . . . . 20
8.1.1 Connection Establishment Model . . . . . . . . . . . . . . . 20
8.1.2 Non-Dynamic Creation of a New TCP Connection . . . . . . . . 20
8.1.3 Dynamic Creation of New TCP Connection(s) . . . . . . . . . 21
8.1.4 Processing TCP Connect Requests . . . . . . . . . . . . . . 22
8.2 TCP Connection Parameters . . . . . . . . . . . . . . . . . . 23
8.2.1 TCP Selective Acknowledgement Option . . . . . . . . . . . . 24
8.2.2 TCP Window Scale Option . . . . . . . . . . . . . . . . . . 24
8.2.3 Protection against sequence number wrap . . . . . . . . . . 24
8.2.4 TCP No Delay Option . . . . . . . . . . . . . . . . . . . . 24
8.3 TCP Connection Considerations . . . . . . . . . . . . . . . . 24
8.4 Flow Control Mapping between TCP and FC . . . . . . . . . . . 24
9. Security . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
9.1 Considerations . . . . . . . . . . . . . . . . . . . . . . . . 25
9.2 IP Network Security Requirements . . . . . . . . . . . . . . . 26
9.3 Integrated Security . . . . . . . . . . . . . . . . . . . . . 26
9.4 External Security Gateway . . . . . . . . . . . . . . . . . . 27
9.5 Security Information Exchanged Between FC and FCIP Entities . 27
Rajagopal, et al. Standards Track [Page 2]
Internet-Draft Fibre Channel Over TCP/IP (FCIP) August, 2001
10. Performance . . . . . . . . . . . . . . . . . . . . . . . . . 27
10.1 Performance Considerations . . . . . . . . . . . . . . . . . 27
10.2 IP Quality of Service (QoS) Support . . . . . . . . . . . . . 28
10.3 QoS Information Exchanged Between FC and FCIP Entities . . . 29
11. References . . . . . . . . . . . . . . . . . . . . . . . . . . 29
12. Bibliography . . . . . . . . . . . . . . . . . . . . . . . . . 31
13. Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . 31
14. Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . 31
15. Full Copyright Statement . . . . . . . . . . . . . . . . . . . 34
Annex
A Example of synchronization recovery algorithm . . . . . . . . . 34
B Relationship between FCIP and IP over FC (IPFC) . . . . . . . . 39
C FC Frame Format . . . . . . . . . . . . . . . . . . . . . . . . 39
D FC Encapsulation Format . . . . . . . . . . . . . . . . . . . . 41
E FCIP Requirements on an FC Entity . . . . . . . . . . . . . . . 43
1. Purpose, Motivation and Objectives
Fibre Channel (FC) is a gigabit speed networking technology
primarily used to implement Storage Area Networks (SANs). See
section 2 for information about how Fibre Channel is standardized
and the relationship of this specification to Fibre Channel standards.
This specification describes mechanisms that allow the
interconnection of islands of Fibre Channel SANs over IP Networks to
form a unified SAN in a single Fibre Channel fabric. The motivation
behind defining these interconnection mechanisms is a desire to
connect physically remote FC sites allowing remote disk access, tape
backup, and live mirroring.
Fibre Channel standards have chosen nominal distances between switch
elements that are less than the distances available in an IP
Network. Since Fibre Channel and IP Networking technologies are
compatible, it is logical to turn to IP Networking for extending the
allowable distances between Fibre Channel switch elements.
The fundamental assumption made in this specification is that the
Fibre Channel traffic is carried over the IP Network in such a
manner that the Fibre Channel Fabric and all Fibre Channel devices
on the Fabric are unaware of the presence of the IP Network. This
means that the FC datagrams MUST be delivered in such time as to
comply with existing Fibre Channel specifications. The FC traffic
MAY span LANs, MANs and WANs, so long as this fundamental assumption
is adhered to.
Rajagopal, et al. Standards Track [Page 3]
Internet-Draft Fibre Channel Over TCP/IP (FCIP) August, 2001
The objectives of this document are to:
1) specify the encapsulation and mapping of Fibre Channel (FC)
frames employing FC Frame Encapsulation [23].
2) apply the mechanism described in 1) to an FC Fabric using an IP
network as an interconnect for two or more islands in an FC
Fabric.
3) address any FC concerns arising from tunneling FC traffic over
an IP-based network, including security, data integrity (loss),
congestion, and performance. This will be accomplished by
utilizing the existing IETF-specified suite of protocols.
4) be compatible with the referenced FC standards. While new work
may be undertaken in T11 [7] to optimize and enhance FC Fabrics,
this specification requires conformance only to the referenced
FC standards.
5) be compatible with all applicable IETF standards so that the IP
Network used to extend an FC Fabric can be used concurrently for
other reasonable purposes.
2. Relationship to Fibre Channel Standards
2.1 Relevant Fibre Channel Standards
FC is standardized under American National Standard for Information
Systems of the National Committee for Information Technology
Standards (ANSI-NCITS) in its T11 technical committee. T11 has
specified a number of documents describing FC protocols, operations,
and services. T11 documents of interest to readers of this
specification include (but are not limited to):
- FC-BB - Fibre Channel Backbone [3]
- FC-BB-2 - Fibre Channel Backbone -2 [4]
- FC-SW-2 - Fibre Channel Switch Fabric -2 [5]
- FC-FS - Fibre Channel Framing and Signaling [6]
FC-BB and FC-BB-2 describe the relationship between an FC Fabric and
interconnect technologies not defined in by Fibre Channel standards
(e.g., ATM and SONET). FC-BB-2 is the natural Fibre Channel home for
describing relationships to TCP/IP and FCIP.
FC-SW-2 describes the switch components of an FC Fabric and FC-FS
describes the FC Frame format and basic control features of Fibre
Channel.
Rajagopal, et al. Standards Track [Page 4]
Internet-Draft Fibre Channel Over TCP/IP (FCIP) August, 2001
Additional information regarding T11 activities is available on the
committee's web site [7].
2.2 This Specification and Fibre Channel Standards
When considering the challenge of transporting FC Frames over an IP
Network, it is logical to divide the standardization effort between
TCP/IP requirements and Fibre Channel requirements. This
specification covers the TCP/IP requirements for transporting FC
Frames and the Fibre Channel documents described in section 2.1
cover the Fibre Channel requirements.
This specification addresses only the requirements necessary to
properly utilize an IP Network as a conduit for FC Frames. The
result is a specification for an FCIP Entity (see section 6.4).
A product that tunnels an FC Fabric through an IP Network must
combine the FCIP Entity with an FC Entity (see section 6.3) using an
implementation specific interface. The requirements placed on an FC
Entity by this specification to achieve proper delivery of FC Frames
are summarized in annex E. More information about FC Entities can be
found in the Fibre Channel standards and an example of an FC Entity
can be found in FC-BB-2 [4].
No attempt is being made to define a specific API between an FCIP
Entity and an FC Entity at this time because doing so risks
compromising the performance and efficacy of the resulting products.
Current experience in this area is simply insufficient to guide
definition of the interface appropriately.
The objectives and motivations of this specification are not
impacted by the decision not to standardize a specific API between
FCIP Entities and FC Entities because fully functional and compliant
products can be built provided they contain both an FCIP Entity and
an FC Entity. The only products that cannot be built are those that
contain only one or the other and there is no urgent need for such
products at this time.
3. Terminology
Terms needed to clarify the concepts presented in FCIP are defined
here.
DLY_LIM - A time interval provided to the FCIP Entity by the FC
Entity when the FCIP_DE is created (see section 8.1). DLY_LIM is a
relative time, a time value that can be added to or subtracted from
an SNTP v4 time to be used by the FCIP_DE in computing the transit
time of a frame through the IP Network (see section 6.6.2.3).
Rajagopal, et al. Standards Track [Page 5]
Internet-Draft Fibre Channel Over TCP/IP (FCIP) August, 2001
FC End Node - A FC device that uses the connection services provided
by the FC Fabric.
FC Entity - The Fibre Channel specific element that combines with an
FCIP Entity to form an interface between an FC Fabric and an IP
Network (see section 6.3).
FC Fabric - An entity that interconnects various Nx_Ports (see [6])
attached to it, and is capable of routing FC Frames using only the
destination ID information in a FC Frame header (see annex C).
FC Frame - The basic unit of Fibre Channel data transfer (see annex
C).
FC Receiver Portal - The access point through which an FC Frame
enters an FCIP Data Engine from the FC Entity.
FC Transmitter Portal - The access point through which a
reconstituted FC Frame leaves an FCIP Data Engine to the FC Entity.
FCIP Data Engine (FCIP_DE) - The component of an FCIP Entity that
handles FC Frame encapsulation, de-encapsulation, and transmission
FCIP Frames through a single TCP connection (see section 6.6).
FCIP Entity - The principal FCIP interface point to the IP Network
(see section 6.4).
FCIP Frame - An FC Frame plus the FC Frame Encapsulation [23] header
and encoded EOF that contains the FC Frame (see section 6.6.1).
FCIP Link - One or more TCP connections that connect one FCIP_LEP to
another (see section 6.2).
FCIP Link Endpoint (FCIP_LEP) - The component of an FCIP Entity that
handles FC Frame encapsulation, de-encapsulation, and transmission
through a single FCIP Link (see section 6.5).
Encapsulated Frame Receiver Portal - The TCP access point through
which an FCIP Frame is received from the IP Network by an FCIP Data
Engine.
Encapsulated Frame Transmitter Portal - The TCP access point through
which an FCIP Frame is transmitted to the IP Network by an FCIP Data
Engine.
Rajagopal, et al. Standards Track [Page 6]
Internet-Draft Fibre Channel Over TCP/IP (FCIP) August, 2001
4. Open Issues
This draft is a work in progress and this section identifies areas
where the work is known to be incomplete and discusses the current
status of these efforts. This section will be removed before this
draft is considered for standardization.
- Security - In general, FCIP will follow or subset the security
mechanisms agreed for iSCSI. The basic principles of FCIP
security requirements are agreed and described in section 5.
Section 9 contains the latest information on the details of FCIP
security. It must be noted that the association between IP
Addresses and FCIP Entities is open to changes based on yet to
be finalized decisions about security. The point at which a TCP
connection is authorized to carry data is still being debated.
5. Protocol Summary
The FCIP protocol is summarized as follows:
1) The primary function of an FCIP Entity is forwarding FC Frames,
employing FC Frame Encapsulation described in [23].
2) Viewed from the IP Network perspective, FCIP Entities are peers
and communicate using TCP/IP. Each FCIP Entity is a TCP endpoint
in the IP-based network.
3) Viewed from the FC Fabric perspective, pairs of FCIP Entities,
in combination with their associated FC Entities, serve as an FC
Frame transmission component of the FC Fabric. The FC End Nodes
are unaware of the existence of the FCIP Link.
4) FC Primitive Signals, Primitive Sequences, and Class 1 FC Frames
are not transmitted across an FCIP Link because they cannot be
encoded using FC Frame Encapsulation [23].
5) The path (route) taken by an encapsulated FC Frame follows the
normal routing procedures of the IP Network.
6) At any instant in time, an FCIP Entity SHALL NOT have more than
one IP Address.
7) An FCIP Entity may contain multiple FCIP Link Endpoints, but
each FCIP Link Endpoint (FCIP_LEP) communicates with exactly one
other FCIP_LEP.
8) When multiple FCIP_LEPs with multiple FCIP_DEs are in use,
selection of which FCIP_DE to use for encapsulating and
Rajagopal, et al. Standards Track [Page 7]
Internet-Draft Fibre Channel Over TCP/IP (FCIP) August, 2001
transmitting a given FC Frame is outside the scope of this
document. FCIP Entities do not actively participate in FC Frame
routing.
9) The FCIP Control & Services function MAY use TCP/IP quality of
service features (see section 10.2) to support Fibre Channel
capabilities.
10) Each FCIP Entity is statically or dynamically configured with a
list of IP addresses and port numbers corresponding to
participating FCIP Entities. If dynamic discovery of
participating FCIP Entities is supported, the function SHALL be
performed using the Service Location Protocol (SLPv2) [21]. It
is outside the scope of this specification to describe any
static configuration method for participating FCIP Entity
discovery. Refer to section 8.1.3 for a detailed description of
dynamic discovery of participating FCIP Entities using SLPv2.
11) FCIP Entities do not actively participate in the discovery of FC
source and destination identifiers. Discovery of FC addresses
(accessible via the FCIP Entity) is provided by techniques and
protocols within the FC architecture as described in FC-FS [6]
and FC-SW-2 [5].
12) To support IP Network security, FCIP Entities MUST:
a) implement cryptographically protected authentication and
cryptographic data integrity keyed to the authentication
process, or
b) be capable of operating with external IP security mechanisms
that provide cryptographically protected authentication and
cryptographic data integrity keyed to the authentication
process.
FCIP entities MAY implement data privacy security features.
Security features and requirements are detailed in section 9.
13) On a given TCP connection, this specification relies on TCP/IP
to deliver a byte stream in the same order that it was sent.
14) This specification relies on both TCP and FC error recovery
mechanisms to detect and recover from data loss and corruption
within the IP Network.
Rajagopal, et al. Standards Track [Page 8]
Internet-Draft Fibre Channel Over TCP/IP (FCIP) August, 2001
6. The FCIP Model
6.1 FCIP Protocol Model
The relationship between FCIP and other protocols is illustrated in
figure 1.
+------------------------+ FCIP Link +------------------------+
| FCIP |===========| FCIP |
+--------+------+--------+ +--------+------+--------+
| FC-2 | | TCP | | TCP | | FC-2 |
+--------+ +--------+ +--------+ +--------+
| FC-1 | | IP | | IP | | FC-1 |
+--------+ +--------+ +--------+ +--------+
| FC-0 | | LINK | | LINK | | FC-0 |
+--------+ +--------+ +--------+ +--------+
| | PHY | | PHY | |
| +--------+ +--------+ |
| | | |
| | IP Network | |
V +--------------------+ V
to Fibre to Fibre
Channel Channel
Environment Environment
Fig. 1 FCIP Protocol Stack Model
Note that the objective of the FCIP Protocol is creation and
maintenance of one or more FCIP Links.
6.2 FCIP Link
The FCIP Link is the basic unit of service provided by the FCIP
Protocol to an FC Fabric. As shown in figure 2, an FCIP Link
connects two portions of an FC Fabric using an IP Network as a
transport to form a single FC Fabric.
/\/\/\/\/\/\ /\/\/\/\/\/\ /\/\/\/\/\/\
\ FC / FCIP \ IP / Link \ FC /
/ Fabric \=========/ Network \=========/ Fabric \
\/\/\/\/\/\/ \/\/\/\/\/\/ \/\/\/\/\/\/
Fig. 2 FCIP Link Model
At the points where the ends of the FCIP Link meet portions of the
FC Fabric, an FCIP Entity (see section 6.4) combines with an FC
Entity as described in section 6.3 to serve as the interface between
FC and IP.
Rajagopal, et al. Standards Track [Page 9]
Internet-Draft Fibre Channel Over TCP/IP (FCIP) August, 2001
An FCIP Link SHALL contain at least one TCP connection and MAY
contain more than one TCP connection. The endpoints of a single TCP
connection are FCIP Data Engines (see section 6.6). The endpoints of
a single FCIP Link are FCIP Link Endpoints (see section 6.5).
6.3 FC Entity
A product that tunnels an FC Fabric through an IP Network must
combine an FC Entity with an FCIP Entity (see section 6.4) to form a
complete interface between the FC Fabric and IP Network as shown in
figure 3.
+----------+ /\/\/\/\/\/\ +----------+
| FCIP | FCIP \ IP / Link | FCIP |
| Entity |=========/ Network \=========| Entity |
+----------+ \/\/\/\/\/\/ +----------+
| FC | | FC |
| Entity | | Entity |
+----------+ +----------+
| |
/\/\/\/\/\/\ /\/\/\/\/\/\
\ FC / \ FC /
/ Fabric \ / Fabric \
\/\/\/\/\/\/ \/\/\/\/\/\/
Fig. 3 FC Entity and FCIP Entity Model
In general, the combination of an FCIP Link and FC and FCIP Entities
is intended to replace a Fibre Channel defined connection between
Fibre Channel components. For example, this combination can be used
to replace a hard-wire connection between two Fibre Channel
switches. There are limitations on the generally intended usage of
the combination shown in figure 3. As another example, the
combination cannot be used to replace cable connections in a Fibre
Channel Arbitrated Loop because loop primitive signals cannot be
encapsulated for transmission over TCP.
The interface between the FC and FCIP Entities is implementation
specific. The minimum requirements placed on an FC Entity by this
specification are listed in annex E. More information about FC
Entities can be found in the Fibre Channel standards and an example
of an FC Entity can be found in FC-BB-2 [4].
Rajagopal, et al. Standards Track [Page 10]
Internet-Draft Fibre Channel Over TCP/IP (FCIP) August, 2001
6.4 FCIP Entity
The model for an FCIP Entity is shown in figure 4.
.......................................................
: FCIP Entity :
: :
: +-----------+ :
: | FCIP | :
: | Control & |------------------------------------+ :
: | Services | | :
: | Module | | :
: +-----------+ | :
: | +--------------------+ | :
: | +-------+--------------------+|----+ | :
: | |+-----+--------------------+|----+| | :
: | ||+----| FCIP Link Endpoint |----+|| | :
: | ||| +--------------------+ ||| | :
:.............................................|||.....:
| ||| ||| |
| ||| ||| o<--+
| ||| unique TCP ||| | |
| ||| connections-->||| | |
| ||| ||| | |
+----------+ /\/\/\/\/\/\ |
| FC | \ IP / |
| Entity | / Network \ |
+----------+ \/\/\/\/\/\/ |
| |
/\/\/\/\/\/\ +------------------+
\ FC / +->IP Address &
/ Fabric \ +->Well Known Port
\/\/\/\/\/\/
Fig. 4 FCIP Entity Model
The FCIP Entity is the connection interface point for the IP Network
and is the owner of the IP Address and Well Known Port used to form
TCP connections. An FC Fabric to IP Network interface product SHALL
contain one FCIP Entity for each IP Address assigned to the product.
An FCIP Entity contains an FCIP Control & Services Module to provide
the FC Entity with an interface to key IP Network features. The
interfaces to the IP Network features is implementation specific,
however, to maintain interoperability, the TCP/IP mechanisms used
are specified in this document as follows:
- TCP Connections - see section 8
Rajagopal, et al. Standards Track [Page 11]
Internet-Draft Fibre Channel Over TCP/IP (FCIP) August, 2001
- Security - see section 9
- Performance - see section 10
- Dynamic Discovery - see section 8.1.3
The FCIP Link Endpoints in an FCIP Entity provide the FC Frame
encapsulation and transmission features of FCIP.
6.5 FCIP Link Endpoint (FCIP_LEP)
Each time a TCP connection is formed to an IP Address for which no
TCP connection already exists, the FCIP Entity SHALL create a new
FCIP Link Endpoint containing one FCIP Data Engine.
An FCIP_LEP is a transparent data translation point between an FC
Entity and an IP Network. A pair of FCIP_LEPs communicating over one
or more TCP connections create an FCIP Link to join two islands of a
FC Fabric, producing a single FC Fabric.
The IP Network over which the two FCIP_LEPs communicate is not aware
of the FC payloads that it is carrying. Likewise, the FC End Nodes
connected to the FC Fabric are unaware of the TCP/IP based transport
employed in the structure of the FC Fabric.
As shown in figure 5, the FCIP Link Endpoint contains one FCIP Data
Engine for each TCP connection in the FCIP Link.
................................................
: FCIP Link Endpoint :
: +------------------+ :
: +-------+------------------+|----+ :
: |+-----+------------------+|----+| :
: ||+----| FCIP Data Engine |----+|| :
: ||| +------------------+ ||| :
:..............................................:
||| |||
+----------+ /\/\/\/\/\/\
| FC | \ IP /
| Entity | / Network \
+----------+ \/\/\/\/\/\/
|
/\/\/\/\/\/\
\ FC /
/ Fabric \
\/\/\/\/\/\/
Fig. 5 FCIP Link Endpoint Model
Rajagopal, et al. Standards Track [Page 12]
Internet-Draft Fibre Channel Over TCP/IP (FCIP) August, 2001
An FCIP_LEP uses normal TCP based flow control mechanisms for
managing its internal resources and matching them with the
advertised TCP Receiver Window Size (see section 8.4). An FCIP_LEP
MAY communicate with its FC Entity counterpart to coordinate flow
control.
6.6 FCIP Data Engine (FCIP_DE)
The model for one of the multiple FCIP_DEs that may be present in an
FCIP_LEP is shown in figure 6.
+--------------------------------+
| |
|-+ +------------------+ +-|
C |p| | Encapsulation | |p| N
F h -->|1|--->| Engine |--->|2|--> e
i a |-+ +------------------+ +-| t
b n | | I w
r n |-+ +------------------+ +-| P o
e e |p| | De-Encapsulation | |p| r
l <--|4|<---| Engine |<---|3|<-- k
|-+ +------------------+ +-|
| |
+--------------------------------+
Fig. 6 FCIP Data Engine Model
Data enters and leaves the FCIP_DE through four portals (p1 - p4).
The portals do not process or examine the data that passes through
them. They are only the named access points where the FCIP_DE
interfaces with external world. The names of the portals are as
follows:
p1) FC Receiver Portal - The interface through which an FC Frame
enters an FCIP_DE from the FC Entity.
p2) Encapsulated Frame Transmitter Portal - The TCP interface
through which an FCIP Frame is transmitted to the IP Network by
an FCIP_DE.
p3) Encapsulated Frame Receiver Portal - The TCP interface through
which an FCIP Frame is received from the IP Network by an FCIP_DE.
p4) FC Transmitter Portal - The interface through which a
reconstituted FC Frame exits an FCIP_DE to the FC Entity.
Rajagopal, et al. Standards Track [Page 13]
Internet-Draft Fibre Channel Over TCP/IP (FCIP) August, 2001
The work of the FCIP_DE is done by the Encapsulation and De-
Encapsulation Engines. The Engines have two functions:
1) Encapsulating and de-encapsulating FC Frames using the
encapsulation format described in FC Frame Encapsulation [23]
and in section 6.6.1 of this document, and
2) Detecting some data transmission errors and performing minimal
error recovery as described in section 6.6.2.
Data flows through the FCIP_DE as follows:
1) An FC Frame arrives at the FC Receiver Portal and is passed to
the Encapsulation Engine. The FC Frame is assumed to have been
processed by the FC Entity according to the applicable FC rules
and is not validated by the FCIP_DE.
2) In the Encapsulation Engine, the encapsulation format described
in FC Frame Encapsulation [23] and in section 6.6.1 of this
document SHALL be applied to prepare the FC Frame for
transmission over the IP Network. If the FC Entity has notified
the FCIP_DE that a properly synchronized time value is available
as described in section 7, that value SHALL be placed in the FC
Frame Encapsulation header time stamp fields. Otherwise, the FC
Frame Encapsulation headers time stamp fields SHALL be set to
zero.
3) The entire encapsulated FC Frame (a.k.a. the FCIP Frame) SHALL
be passed to the Encapsulated Frame Transmitter Portal where it
SHALL be inserted in the TCP byte stream.
4) Transmission of the FCIP Frame over the IP Network follows all
the TCP rules of operation. This includes but is not limited to
the in-order delivery of bytes in the stream, as specified by
TCP [8].
5) The FCIP Frame arrives at the partner FCIP Entity where it
enters the FCIP_DE through the Encapsulated Frame Receiver
Portal and is passed to the De-Encapsulation Engine for
processing.
6) The De-Encapsulation Engine SHALL validate the incoming TCP byte
stream as described in section 6.6.2 and SHALL de-encapsulate
the FC Frame according to the encapsulation format described in
FC Frame Encapsulation [23] and in section 6.6.1 of this document.
7) In the absence of errors, the de-encapsulated FC Frame SHALL be
passed to the FC Transmitter Portal for delivery to the FC Entity.
Rajagopal, et al. Standards Track [Page 14]
Internet-Draft Fibre Channel Over TCP/IP (FCIP) August, 2001
Every FC Frame that arrives at the FC Receiver Portal SHALL be
transmitted on the IP Network as described in steps 1 through 4
above. In the absence of errors, data bytes arriving at the
Encapsulated Frame Receiver Portal SHALL be de-encapsulated and
forwarded to the FC Transmitter Portal as described in steps 5
through 7.
6.6.1 FCIP Encapsulation of FC Frames
The FCIP encapsulation of FC Frames employs FC Frame Encapsulation
[23].
The features from FC Frame Encapsulation that are unique to
individual protocols SHALL be applied as follows for the FCIP
encapsulation of FC Frames.
The Protocol# field SHALL contain 1 in accordance with the IANA
Considerations annex of FC Frame Encapsulation [23].
The Protocol Specific field SHALL have the format shown in figure 7.
Note: the word numbers in figure 7 are relative to the complete FC
Frame Encapsulation header, not to the Protocol Specific field.
W|------------------------------Bit------------------------------|
o| |
r|3 3 2 2 2 2 2 2 2 2 2 2 1 1 1 1 1 1 1 1 1 1 |
d|1 0 9 8 7 6 5 4 3 2 1 0 9 8 7 6 5 4 3 2 1 0 9 8 7 6 5 4 3 2 1 0|
+---------------------------------------------------------------+
1| replication of encapsulation word 0 |
+-------------------------------+-------------------------------+
2| reserved | -reserved |
+-------------------------------+-------------------------------+
Fig. 7 FCIP Usage of FC Frame Encapsulation Protocol Specific
field
Word 1 of the Protocol Specific field SHALL contain an exact copy of
word 0 in FC Frame Encapsulation [23].
Word 2 of the Protocol Specific field is reserved for future
enhancements to the FCIP protocol.
The reserved field (bits 31-16 in word 2): SHALL contain 0.
The -reserved field (bits 15-0 in word 2): SHALL contain 65535 (or
0xFFFF).
Rajagopal, et al. Standards Track [Page 15]
Internet-Draft Fibre Channel Over TCP/IP (FCIP) August, 2001
The CRCV (CRC Valid) Flag SHALL be set to 0.
The CRC field SHALL be set to 0.
6.6.2 FCIP Data Engine Error Detection and Recover
6.6.2.1 TCP Assistance With Error Detection and Recovery
TCP [8] REQUIRES in order delivery, generation of TCP checksums, and
checking of TCP checksums. Thus, the byte stream passed from TCP to
the FCIP_LEP will be in order and free of errors detectable by the
TCP checksum. If TCP did not perform these functions, the FCIP_LEP
would have to.
6.6.2.2 Errors in FCIP Headers and Discarding FCIP Frames
Bytes delivered through the Encapsulated Frame Receiver Portal that
are not correctly delimited as defined by the FC Frame Encapsulation
[23] SHALL NOT be forwarded on to the FC Entity.
Synchronization of the FCIP_DE to the FCIP Frames in the data stream
entering the Encapsulated Frame Receiver Portal is maintained using
the FC Frame Encapsulation header's frame length field to determine
where in the data stream the next FC Encapsulation header is
located. Synchronization SHALL be verified on each FCIP Frame. The
validity and positioning of the following FCIP Frame information
SHOULD be used to verify synchronization:
a) Protocol # field and its ones complement (2 tests);
b) Version field and its ones complement (2 tests);
c) Replication of encapsulation word 0 in word 1 (1 test);
d) Reserved field and its ones complement (2 tests);
e) Flags field and its ones complement (2 tests);
f) Length field and its ones complement (2 tests);
g) Time stamp [integer] and time stamp [fraction] fields (2 tests);
h) CRC field is equal to zero (1 test);
i) SOF fields and ones complement fields (4 tests);
j) Format and values of FC header (1 test);
k) CRC of FC Frame (2 tests);
l) EOF fields and ones complement fields (4 tests); and/or
m) FC Frame Encapsulation header information in the next FCIP Frame
(1 test).
Verification SHALL be accomplished by performing the following tests:
a) Length field validation -- 63 < (Length * 4) < 2177 (f above);
b) Comparison of Length field to its ones complement (f above); and
c) At least 6 other of the 21 distinct tests listed above.
Rajagopal, et al. Standards Track [Page 16]
Internet-Draft Fibre Channel Over TCP/IP (FCIP) August, 2001
Errors in FCIP Frame headers SHOULD be considered carefully, since
some may be synchronization errors. For example, any failure of the
Length field tests described above SHALL be handled as a
synchronization error. Errors in FCIP Frames detected by the FCIP_DE
that affect synchronization with the Encapsulated Frame Receiver
Portal byte stream SHALL be handled as defined by section 6.6.2.4.
An error in an FCIP Frame that effects the synchronization MAY
require the FCIP Entity to notify the FC Entity that the previously
delivered FC Frame was invalid.
Whenever an FCIP_DE discards bytes delivered through the
Encapsulated Frame Receiver Portal, it SHALL cause the FCIP Entity
to notify the FC Entity of the condition and provide a suitable
description of the reason bytes were discarded.
The burden for recovering from discarded data falls on the FC Entity
and other components of the FC Fabric and is outside the scope of
this specification.
6.6.2.3 IP Network Transit Time Validation
Fibre Channel routing and error recovery protocols require FC Frames
to exit a receiving FCIP Entity within in a fixed interval from the
time they entered a sending FCIP Entity, including IP Network
transit time. If the FCIP Entity detects an FC Frame that has not
transited the IP Network in the fixed time interval required by
Fibre Channel, then that FC Frame MUST NOT exit the FCIP Entity. In
addition, the FCIP Entity MUST notify the FC Entity when it delivers
frames for which the IP Network transit time cannot be determined.
To implement this Fibre Channel requirement, the encapsulating
FCIP_DE places a time stamp in each FCIP Frame header transmitted
(see section 6.6).
The de-encapsulating FCIP_DE is REQUIRED to enforce the Fibre
Channel transit time requirement based on three time values:
a) The SNTP time stamp in the FCIP Frame header (see section 6.6);
b) The DLY_LIM relative time provided by the FC Entity when a
connection is created (see section 8.1); and
c) A synchronized SNTP time value for the FC Fabric maintained by
the FC Entity (see section 7).
Rajagopal, et al. Standards Track [Page 17]
Internet-Draft Fibre Channel Over TCP/IP (FCIP) August, 2001
The FCIP_DE SHALL NOT perform transit time validation on the
received FCIP Frames when either of the following conditions exist:
a) the FC Entity has notified the FCIP_DE that the synchronized
time value is not synchronized well enough for use as described
in section 7; or
b) the FC Frame Encapsulation [23] header does not contain a valid
time stamp. When this condition occurs, the FCIP Entity SHALL
notify the FC Entity that no IP Network transit time testing was
performed as part of delivering the FC Frame through the FC
Transmitter Portal.
Otherwise, the FCIP_DE SHALL use the valid time stamp information in
the FCIP Frame header to determine if received FCIP Frames have been
delayed by more than DLY_LIM in the IP Network by comparing the SNTP
time stamp in the FCIP Frame header adjusted by DLY_LIM to the
current FC Fabric synchronized SNTP time.
If an FCIP Frame has been delayed by more than DLY_LIM in the IP
Network, the FCIP_DE SHALL discard the FCIP Frame as described in
section 6.6.2.2. The discarding of delayed FCIP Frames SHALL
continue until a FCIP Frame is processed whose life in the IP
Network is smaller than DLY_LIM.
DLY_LIM is a time interval provided to the FCIP Entity by the FC
Entity when the FCIP_DE is created (see section 8.1). DLY_LIM is a
relative time, a time value that can be added to or subtracted from
an SNTP v4 time to get a new SNTP v4 time that is later (for
addition) or sooner (for subtraction) than the original SNTP v4 time.
The burden for recovering from discarded FCIP Frames falls on the FC
Entity and other components of the FC Fabric and is outside the
scope of this specification.
6.6.2.4 Synchronization Failures
If an FCIP_DE determines that it cannot find the next FCIP Frame
header in the byte stream entering through the Encapsulated Frame
Receiver Portal, the FCIP_DE SHALL either:
a) close the TCP connection [8] [9];
b) recover synchronization by searching the bytes delivered by the
Encapsulated Frame Receiver Portal for a valid FCIP Frame header
having the correct properties, and discarding bytes delivered by
the Encapsulated Frame Receiver Portal until a valid FCIP Frame
header is found; or
c) attempt to recover synchronization as described in b) and if
synchronization cannot be recovered close the TCP connection as
described in a).
Rajagopal, et al. Standards Track [Page 18]
Internet-Draft Fibre Channel Over TCP/IP (FCIP) August, 2001
If the FCIP_DE attempts to recover synchronization, the
resynchronization algorithm used SHALL meet the following
requirements:
a) discard or identify with an EOFa (see annex section C.1) those
FC Frames and fragments of FC Frames identified before
synchronization has again been completely verified. The number
of FC Frames not forwarded may vary based on the algorithm used;
b) return to sending valid FC Frames only after synchronization has
been verified; and
c) close the TCP/IP connection if the algorithm ends without
verifying successful synchronization. The probability of failing
to synchronize successfully and the time necessary to determine
whether or not synchronization was successful may vary with the
algorithm used.
An example algorithm meeting these requirements can be found in
annex A.
The burden for recovering from the discarding of FCIP Frames during
the optional resynchronization process described in this section
falls on the FC Entity and other components of the FC Fabric and is
outside the scope of this specification.
7. Establishing and Maintaining a Synchronized Time Value
The FC Entity SHALL use Fibre Channel services and suitable internal
clocks to establish and maintain a synchronized time value in Simple
Network Time Protocol (SNTP) Version 4 format [12] for use by the
FCIP_DE in:
a) building outgoing FC Frame Encapsulation headers, and
b) checking IP Network transit times in incoming FC Frame
Encapsulation headers.
The synchronized time value SHALL be maintained to an accuracy of at
least 0.01% of the smallest DLY_LIM (see section 6.6.2.3) value
passed to an FCIP_DE for the purpose of evaluating IP Network
transit time.
Note that since the FC Fabric is expected to have a single
synchronized time value throughout, only one synchronized time value
is needed for all FCIP_DEs regardless of their connection
characteristics.
If the FC Entity has not yet established a synchronized time value
or if events in the FC Fabric cast suspicion on the accuracy of a
Rajagopal, et al. Standards Track [Page 19]
Internet-Draft Fibre Channel Over TCP/IP (FCIP) August, 2001
previously established and maintained synchronized time value, the
FC Entity SHAL notify the FCIP_DEs not to use the synchronized time
value. Conversely, the FC Entity SHALL notify the FCIP_DEs when a
previously unreliable value has been corrected to a properly
synchronized time value. Since zero is an invalid time stamp value
in the FC Frame Encapsulation, it may be simplest to use a value of
zero to indicate the absence of a synchronized time value.
When there is an absence of a synchronized time value, the FCIP_DEs
are required not to validate the IP Network transit time of incoming
FC Frames (see section 6.6.2.3). Instead, all incoming FC Frames are
delivered to the FC Entity via the FC Transmitter Portal and the
decision regarding whether or not to forward the FC Frames to the FC
Fabric becomes the responsibility of the FC Entity.
8. TCP Connection Management
8.1 TCP Connection Establishment
8.1.1 Connection Establishment Model
The description of the connection establishment process in section
8.1 is a model for the interactions between an FC Entity and an FCIP
Entity during TCP connection establishment. For convenience, the
model is written in terms of routine calls from the FC Entity to the
FCIP Entity or vice versa with parameter passing to convey needed
information in either direction.
However, this description is only a model for the interactions
between an FC Entity and an FCIP Entity. Any implementation that has
the same effects on the FC Fabric and IP Network as those described
in the model meets the requirements of this specification. For
example, an implementation might use a shared data base and suitable
FCIP Entity algorithms to achieve the effects that the model
describes in terms of parameter passing.
8.1.2 Non-Dynamic Creation of a New TCP Connection
The FC Entity SHALL request creation of a new TCP Connection by
transmitting at least the following information to the FCIP Entity:
- IP Address
- DLY_LIM (see section 6.6.2.3) for the FCIP_DE
- TCP Connection Parameters (see section 8.2)
- Security Information (see section 9)
- Quality of Service Information (see section 10)
Rajagopal, et al. Standards Track [Page 20]
Internet-Draft Fibre Channel Over TCP/IP (FCIP) August, 2001
In response to a request from the FC Entity the FCIP Entity shall
generate a TCP connect request [8] to the FCIP Well-Known Port at
the specified IP Address. If the TCP connect request is rejected,
the FCIP Entity SHALL so inform the FC Entity.
If the TCP connect request is accepted, and the IP Address is one to
which no other TCP connections exist, the FCIP Entity SHALL:
1) Create a new FCIP_LEP for the new FCIP Link,
2) Create a new FCIP_DE within the newly created FCIP_LEP to
service the new TCP connection, and
3) Inform the FC Entity of the new FCIP_LEP and FCIP_DE.
If the TCP connect request is accepted, and the IP Address is one
for which a TCP connection already exists, the FCIP Entity SHALL:
1) Create a new FCIP_DE within the existing FCIP_LEP to service the
new TCP connection, and
2) Inform the FC Entity of the FCIP_LEP and new FCIP_DE.
8.1.3 Dynamic Creation of New TCP Connection(s)
If dynamic discovery of participating FCIP Entities is supported the
function SHALL be performed using the Service Location Protocol
(SLPv2) [21] in the manner defined for FCIP usage [24].
The FC Entity SHALL initiate the dynamic discovery process by
informing the FCIP Entity of one or more FCIP Discovery Domain(s) to
be used in the dynamic discovery process. Upon receiving a request
to initiate dynamic discovery, the FCIP Entity SHALL:
1) Establish an SLPv2 Service Agent to advertise the availability
of this FCIP Entity to peer FCIP Entities in the FCIP Discovery
Domain(s) identified by the FC Entity.
2) establish an SLPv2 User Agent to locate service advertisements
for peer FCIP Entities in the FCIP Discovery Domain(s)
identified by the FC Entity.
For each peer FCIP Entity dynamically discovered through the SLPv2
User Agent, the FCIP Entity SHALL request authorization from the FC
Entity before creating a TCP connection. When requesting this
Rajagopal, et al. Standards Track [Page 21]
Internet-Draft Fibre Channel Over TCP/IP (FCIP) August, 2001
authorization, the FCIP Entity SHALL deliver to the FC Entity at
least the following information:
- Information about the FCIP_LEP, new or existing
- Information about the FCIP_DE for the new TCP connection
- any information from the service advertisement that might aid in
the FC Entity's response
When responding affirmatively to an FCIP Entity request to authorize
creating a TCP connection to a dynamically discovered peer FCIP
Entity, the FC Entity SHALL provide at least the following
information:
- DLY_LIM (see section 6.6.2.3) for the FCIP_DE
- TCP Connection Parameters (see section 8.2)
- Security Information (see section 9)
- Quality of Service Information (see section 10)
If the FC Entity authorizes creating the TCP connection, the FCIP
Entity shall generate a TCP connect request [8] to the FCIP Well-
Known Port at the IP Address specified by the service advertisement.
If the TCP connect request is rejected, the FCIP Entity SHALL so
inform the FC Entity.
If the TCP connect request is accepted, and the IP Address is one to
which no other TCP connections exist, the FCIP Entity SHALL:
1) Finalize creation of a new FCIP_LEP for the new FCIP Link, and
2) Finalize creation of a new FCIP_DE within the newly created
FCIP_LEP to service the new TCP connection.
If the TCP connect request is accepted, and the IP Address is one
for which a TCP connection already exists, the FCIP Entity SHALL
finalize creation of a new FCIP_DE within the existing FCIP_LEP to
service the new TCP connection.
8.1.4 Processing TCP Connect Requests
The FCIP Entity SHALL listen for new TCP connection requests [8] on
the FCIP Well-Known Port. An FCIP Entity MAY also accept and
establish TCP connections to a TCP port number other than the FCIP
Well-Known Port, as configured by the network administrator.
Upon receipt of a TCP connect request, the FCIP Entity SHALL
determine if a TCP connection already exists for the IP Address
making the TCP connect request. The FCIP Entity SHALL notify the FC
Rajagopal, et al. Standards Track [Page 22]
Internet-Draft Fibre Channel Over TCP/IP (FCIP) August, 2001
Entity of the TCP connect request, transmitting at least the
following information:
- IP Address
- Information about the FCIP_LEP, new or existing
- Information about the FCIP_DE for the new TCP connection
- Those TCP Connection Parameters (see section 8.2) known to the
FCIP Entity
- Security Information (see section 9)
In response to the information provided by the FCIP Entity, the FC
Entity MUST either accept or reject the TCP connect request. If the
FC Entity rejects the TCP connect request, the FCIP Entity SHALL
terminate the TCP connect request [8].
If the FC Entity accepts the TCP connect request, the FC Entity
SHALL inform the FCIP Entity of at least the following information
to be applied to the new connection:
- DLY_LIM (see section 6.6.2.3) for the FCIP_DE
- Quality of Service Information (see section 10)
Upon being informed by the FC Entity that a TCP connect request is
to be accepted, the FCIP Entity SHALL:
1) Accept the TCP connect request,
2) Finalize creation of the new FCIP_DE for the new TCP connection,
and
3) If the new TCP connection is to an IP Address for which no other
TCP connection exists, finalize the creation of the FCIP_LEP.
8.2 TCP Connection Parameters
In order to provide efficient management of FCIP_LEP resources as
well as FCIP Link resources, coordination of certain TCP connection
parameters between the FC Entity and FCIP Entity is RECOMMENDED,
with the FC Entity providing instructions to the FCIP Entity
regarding how the TCP Parameters should be set for each newly
created TCP connection. Note that FC Entity instructions change
optional TCP connection parameter settings to required settings.
Rajagopal, et al. Standards Track [Page 23]
Internet-Draft Fibre Channel Over TCP/IP (FCIP) August, 2001
8.2.1 TCP Selective Acknowledgement Option
The Selective Acknowledgement option RFC 2883 [22] allows the
receiver to acknowledge multiple lost packets in a single ACK,
enabling faster recovery. An FCIP Entity MAY negotiate use of TCP
SACK and use it for faster recovery from lost packets and holes in
TCP sequence number space.
8.2.2 TCP Window Scale Option
This option allows TCP window sizes larger than 16-bit limits to be
advertised by the receiver. It is necessary to allow data in long
fat networks to fill the available pipe. This also implies buffering
on the TCP sender that matches the (bandwidth*delay) product of the
TCP connection. An FCIP_LEP uses locally available mechanisms to set
a window size that matches the available local buffer resources and
the desired throughput.
8.2.3 Protection against sequence number wrap
It is RECOMMENDED that FCIP Entities implement protection against
sequence number wrap. It is quite possible that within a single
connection, TCP sequence numbers wrap within a timeout window.
8.2.4 TCP No Delay Option
FCIP Entities SHALL disable the Nagle TCP No Delay option. This
option is designed for usage in a telnet environment.
8.3 TCP Connection Considerations
In idle mode, a TCP connection "keep alive" option of TCP is
normally used to keep a connection alive. However, this timeout is
fairly large and may prevent early detection of loss of
connectivity. In order to facilitate faster detection of loss of
connectivity, FC Entities SHOULD implement some form of Fibre
Channel connection failure detection.
8.4 Flow Control Mapping between TCP and FC
The FCIP Entity and FC Entity are connected to the IP Network and FC
Fabric, respectively, and they need to follow the flow control
mechanisms of both TCP and FC, which work independent of each other.
This section provides guidelines as to how the FCIP Entity can map
TCP flow control to status notifications to the FC Entity.
Rajagopal, et al. Standards Track [Page 24]
Internet-Draft Fibre Channel Over TCP/IP (FCIP) August, 2001
There are two scenarios when the flow control management becomes
crucial:
1) When there is line speed mismatch between the FC and IP
interfaces.
Even though it is RECOMMENDED that both the FC and IP interfaces
to the FC Entity and FCIP Entity, respectively, be of comparable
speeds, it is possible to carry FC traffic over an IP Network
that has a different line speed and bit error rate.
2) When the FC Fabric or IP Network encounters congestion.
Even when both the FC Fabric or IP network are of comparable
speeds, during the course of operation the FC Fabric or the IP
Network could encounter congestion due to transient conditions.
The FC Entity uses Fibre Channel mechanisms for flow control at the
FC Receiver Portal based on information supplied by the FCIP Entity
regarding flow constraints at the Encapsulated Frame Transmitter
Portal. The FCIP Entity uses TCP mechanisms for flow control at the
Encapsulated Frame Receiver Portal portal based on information
supplied by the FC Entity regarding flow constraints at the FC
Transmitter Portal. Coordination of these flow control mechanisms is
outside the scope of this specification.
9. Security
9.1 Considerations
Using a wide-area, general purpose network such as an IP Network in
a position normally occupied by physical cabling introduces some
security problems not normally encountered in Fibre Channel Fabrics.
FC transport media are typically protected physically from outside
access; IP Networks typically invite outside access.
The general effect is that the security of the entire FC Fabric is
only as good as the security of the entire IP Network through which
it tunnels. The following broad classes of attacks are possible:
1) Unauthorized Fibre Channel elements can gain access to resources
through normal Fibre Channel processes.
2) Unauthorized agents can monitor and manipulate Fibre Channel
traffic flowing over physical media used by the IP Network and
under control of the agent.
Rajagopal, et al. Standards Track [Page 25]
Internet-Draft Fibre Channel Over TCP/IP (FCIP) August, 2001
To a large extent, these security risks are typical of the risks
facing any other application using an IP Network. They are mentioned
here only because Fibre Channel storage networks are not normally
suspicious of the media. Fibre Channel Fabric administrators will
need to be aware of these additional security risks.
9.2 IP Network Security Requirements
Security protocols and procedures used in other IP applications MAY
be used for FCIP. FCIP Entities MUST ensure secure operation of FCIP
Links by implementing one of the following two methods:
1) by using ESP [14] from the IPSec Security Protocol Suite with
NULL encryption [15] for cryptographic data integrity and
integrity of authentication. Authentication is performed using
SRP [RFC2945]. This method is discussed in section 9.3; or
2) by appropriate configuration of an external entity that
implements IP security using mechanisms such as IPSec and
Virtual Private Networks. This method is discussed in section 9.4.
The mechanism for configuring whether a particular deployment uses
1) or 2) is outside the scope of this document.
Note: Two overviews of the IPSec Security Protocol Suite are
available in RFC 2401 [13] and RFC 2411 [16].
9.3 Integrated Security
When both FCIP Entity and IP Security implementations are integrated
into a single device, IPSec ESP (in transport mode) MUST be
implemented.
Upon receiving a TCP connection request, the receiving FCIP Entity
SHALL identify the FCIP Link per the IP address pair of the
connection. It SHALL then verify that the FCIP Link has been
previously authenticated. If not, the FCIP Entity SHALL authenticate
a new peer using a separate TCP connection. This TCP connection is
used for negotiation of SRP related parameters.
<SRP message negotiation will be per iSCSI discussions>
If authentication fails, the original TCP connection that initiated
the authentication exchange is terminated and the FC Entity is not
informed that a TCP connect request was received.
If the authentication is successful, a new FCIP_LEP is created, with
the authenticated FCIP Link as described in section 8.1.4.
Rajagopal, et al. Standards Track [Page 26]
Internet-Draft Fibre Channel Over TCP/IP (FCIP) August, 2001
The FCIP Entity remembers the IP pair and the key material for
authentication, so that any future TCP connections for that IP
address pair bypasses this authentication step. The key material is
then used as part of the ESP Security Parameters.
<association of SRP key material with ESP header will be per iSCSI
discussions>
9.4 External Security Gateway
Figure 8 illustrates the use of an externally supplied security
gateway for securing the FCIP Link.
+--------+ Insecure +--------+ Secure +--------+ Insecure +-------+
| FCIP | Network | IPSec | Network | IPSec | Network |FCIP |
| Entity |----------| Device |---------| Device |----------|Entity |
+--------+ +--------+ +--------+ +-------+
Fig. 8 External Security Gateway Model
In this deployment, only certain parts of the FCIP Link are exposed
to security threats and so only these specific parts of the FCIP
Link need to be secured. The part of the network between the two
security gateways is secured using devices implementing IPSec.
The IPSec Device or any other equivalent gateway is required to
operate in tunnel mode, so that the IP addresses of the two FCIP
Entities are visible through the security devices that are
implemented.
9.5 Security Information Exchanged Between FC and FCIP Entities
TBD
10. Performance
10.1 Performance Considerations
Traditionally, the links between FC Fabric components have been
characterized by low latency and high throughput. The purpose of
FCIP is to replace some of these links with an IP Network, where low
latency and high throughput are not as certain. It follows that FCIP
Entities and their counterpart FC Entities probably will be
interested in optimal use of the IP Network.
Many options exist for ensuring low latency and high throughput in
an IP Network. For example, a private IP Network might be
Rajagopal, et al. Standards Track [Page 27]
Internet-Draft Fibre Channel Over TCP/IP (FCIP) August, 2001
constructed for the sole use of FCIP Entities. The options that are
within the scope of this specification are discussed here.
One option for increasing the probability that FCIP data streams
will experience low latency and high throughput is the IP QoS
techniques discussed in section 10.2. This option can have value
when applied to a single TCP connection. Depending on the
sophistication of the FC Entity, further value may be obtained by
having multiple TCP connections with differing QoS characteristics.
There are many reasons why an FC Entity might request creation of
multiple TCP connections within an FCIP_LEP. These reasons include a
desire to provide differentiated service for different TCP data
connections between FCIP_LEPs or a preference to separately queue
different streams of traffic not having a common in-order delivery
requirement.
At the time a new TCP connection is created, the FC Entity SHALL
specify to the FCIP Entity the QoS characteristics (including but
not limited to IP per-hop-behavior) to be used for the lifetime of
that connection. This MAY be achieved by having:
a) only one set of QoS characteristics for all TCP connections;
b) a default set of QoS characteristics that the FCIP Entity
applies in the absence of differing instructions from the FC
Entity; or
c) a sophisticated mechanism for exchanging QoS requirements
information between the FC Entity and FCIP Entity each time a
new TCP connection is created.
Once established, the QoS characteristics of a TCP connection SHALL
NOT be changed, since this specification provides no mechanism for
the FC Entity to control such changes. The mechanism for providing
different QoS characteristics in FCIP is the establishment of a
different TCP connections and associated FCIP_DEs.
When FCIP is used with a network with a large (bandwidth*delay)
product, it is RECOMMENDED that FCIP_LEPs use the TCP mechanisms
(window scaling and wrapped sequence protection) for Long Fat
Networks (LFNs) as defined in RFC 1323 [10].
10.2 IP Quality of Service (QoS) Support
Many methods of providing QoS have been devised or proposed. These
include (but are not limited to) the following:
- Multi-Protocol Label Switching (MPLS)
- Differentiated Services Architecture (diffserv) -- RFC 2474
[17], RFC 2475 [18], RFC 2597 [19], and RFC 2598 [20] -- and
Rajagopal, et al. Standards Track [Page 28]
Internet-Draft Fibre Channel Over TCP/IP (FCIP) August, 2001
other forms of per-hop-behavior (PHB)
- Integrated Services, RFC 1633 [11]
- IEEE 802.1p
The purpose of this specification is not to specify any particular
form of IP QoS but rather to specify only those issues that must be
addressed in order to maximize interoperability between FCIP
equipment that has been manufactured by different vendors.
It is RECOMMENDED that some form of preferential QoS be used for
FCIP traffic to minimize latency and drop precedence. No particular
form of QoS is recommended.
If a PHB IP QoS is implemented, it is RECOMMENDED that it
interoperate with diffserv (see RFC 2474 [17], RFC 2475 [18], RFC
2597 [19], and RFC 2598 [20]).
If diffserv/PHB QoS is NOT implemented, the DSCP field for all IP
packets SHALL be set to '000000'.
10.3 QoS Information Exchanged Between FC and FCIP Entities
At the time a new TCP connection is created, the FC Entity SHALL
specify to the FCIP Entity the QoS characteristics to be used for
the lifetime of that connection. The nature of the information
exchanged depends on the QoS method (see section 10.2) implemented
by the FC Entity and FCIP Entity and on whether the FCIP Entity is
designed to provide differing QoS characteristics for different TCP
connections.
The FCIP Entity SHALL instantiate the QoS characteristics specified
by the FC Entity when it creates the TCP connection.
11. References
The references in this section were current as of the time this
specification was approved. This specification is intended to
operate with newer version of the referenced documents and looking
for newer reference documents is recommended.
[1] Bradner, S., "The Internet Standards Process -- Revision 3", BCP
9, RFC 2026, October 1996.
[2] Bradner, S., "Key words for use in RFCs to Indicate Requirement
Levels", BCP 14, RFC 2119, March 1997.
[3] Fibre Channel Backbone (FC-BB), ANSI NCITS.342:200x, March 5,
2001 (www.t11.org).
Rajagopal, et al. Standards Track [Page 29]
Internet-Draft Fibre Channel Over TCP/IP (FCIP) August, 2001
[4] Fibre Channel Backbone -2 (FC-BB-2), T11 Project 1466-D,
(www.t11.org).
[5] Fibre Channel Switch Fabric -2 (FC-SW-2), ANSI NCITS.355:200x,
May 23, 2001 (www.t11.org).
[6] Fibre Channel Framing and Signaling (FC-FS), T11 Project 1331-D,
Rev 1.2, February 16, 2001 (www.t11.org).
[7] http://www.t11.org
[8] "Transmission Control Protocol", RFC 793, Sept. 1981.
[9] Braden, R., "Requirements for Internet Hosts -- Communication
Layers", RFC 1122, October 1989
[10] Jacobson, V., Braden, R. and Borman, D., "TCP Extensions for
High Performance", RFC 1323, May 1992.
[11] R. Braden, et. al., ISI, "Integrated Services in the Internet
Architecture: an Overview", RFC 1633, June 1994
[12] Mills, D., "Simple Network Time Protocol (SNTP) Version 4 for
IPv4, IPv6 and OSI", RFC 2030, October 1996.
[13] Kent, S. and Atkinson, R., "Security Architecture for the
Internet Protocol", RFC 2401, Nov. 1998.
[14] Kent, S. and Atkinson, R., "IP Encapsulating Security Payload
(ESP)", RFC 2406, Nov. 1998.
[15] Glenn, R., Kent, S., "The NULL Encryption Algorithm and Its Use
With IPsec", RFC 2410, Nov. 1998
[16] Thayer, R., Glenn, R., and Doraswamy, N., "IP Security Document
Roadmap", RFC 2411, Nov. 1998.
[17] Nichols, K., Blake, S., Baker, F. and D. Black, "Definition of
the Differentiated Services Field (DS Field) in the IPv4 and
Ipv6 Headers", RFC 2474, December 1998.
[18] Blake, S., Black, D., Carlson, M., Davies, E., Wang, Z., Weiss,
W., "An Architecture for Differentiated Services", RFC 2475,
Dec. 1998.
[19] Heinanen, J., Baker, F., Weiss, W., Wroclawski, J., "An Assured
Forwarding PHB", RFC 2597, June 1999.
Rajagopal, et al. Standards Track [Page 30]
Internet-Draft Fibre Channel Over TCP/IP (FCIP) August, 2001
[20] Jacobson, V., Nichols, K., Poduri, K., "An Expedited Forwarding
PHB Group", RFC 2598, June 1999.
[21] E.Guttman, C. Perkins, J. Veizades, M. Day. Service Location
Protocol, version 2, RFC 2608, July, 1999.
[22] Floyd, et al, "SACK Extension", RFC 2883, July 2000.
[23] Weber, Rajagopal, Travostino, Chau, O'Donnell, Monia Merhar,
"FC Frame Encapsulation", draft-ietf-ips-fcencapsulation-__.txt
(RFC reference and date to be added during standards action).
[24] Peterson, "Blah Blah FCIP Usage of SLPv2 For Dynamic
Configuration", draft-ietf-ips-...-___.txt (RFC reference and
date to be added during standards action).
12. Bibliography
The following references may prove informative to readers unfamiliar
with Fibre Channel.
Kembel, R., "The Fibre Channel Consultant: A Comprehensive
Introduction", Northwest Learning Associates, 1998
13. Acknowledgments
Funding for the RFC Editor function is currently provided by the
Internet Society.
14. Authors' Addresses
Murali Rajagopal Raj Bhagwat
LightSand Communications, Inc. LightSand Communications, Inc.
24411 Ridge Route Dr. 24411 Ridge Route Dr.
Suite 135 Suite 135
Laguna Hills, CA 92653 Laguna Hills, CA 92653
USA USA
Phone: +1 949 837 1733 x101 Phone: +1 949 837 1733 x104
Email: muralir@lightsand.com Email: rajb@lightsand.com
Rajagopal, et al. Standards Track [Page 31]
Internet-Draft Fibre Channel Over TCP/IP (FCIP) August, 2001
R. Andy Helland Elizabeth G. Rodriguez
LightSand Communications, Inc. Lucent Technologies
375 Los Coches Street 1202 Richardson Drive, Suite 104
Milpitas, CA 95035 Richardson, TX 75080
USA USA
Phone: +1 408 404 3119 Phone: +1 972 231 0672
Fax: +1 408 941 2166 Fax: +1 972 644 5857
Email: andyh@lightsand.com Email: egrodriguez@lucent.com
Sriram Rupanagunta Neil Wanamaker
Aarohi Communications Akara
3200 Montelena Drive 10624 Icarus Court
San Jose, CA 95135 Austin, TX 78726
USA USA
Phone: +1 408 966 8309 Phone: +1 512 257 7633
Email: sriramr@aarohi-inc.com Fax: +1 512 257 7877
Email: nwanamaker@akara.com
Steve Wilson Bob Snively
Brocade Comm. Systems, Inc. Brocade Comm. Systems, Inc.
1745 Technology Drive 1745 Technology Drive
San Jose, CA. 95110 San Jose, CA 95110
USA USA
Phone: +1 408 487 8128 Phone: +1 408 487 8135
Fax: +1 408 487 8101 Email: rsnively@brocade.com
email: swilson@brocade.com
Ralph Weber
ENDL Texas, representing Brocade
Suite 102 PMB 178
18484 Preston Road
Dallas, TX 75252
USA
Phone: +1 214 912 1373
Email: roweber@acm.org
David Peterson Donald R. Fraser
Cisco Systems - SRBU Compaq Computer Corporation
6450 Wedgwood Road 301 Rockrimmon Blvd., Bldg. 5
Maple Grove, MN 55311 Colorado Springs, CO 80919
USA USA
Phone: +1 763 398 1007 Phone: +1 719 548 3272
Cell: +1 612 802 3299 Email: don.fraser@compaq.com
Email: dap@cisco.com
Rajagopal, et al. Standards Track [Page 32]
Internet-Draft Fibre Channel Over TCP/IP (FCIP) August, 2001
Vi Chau Gaby Hecht
Gadzoox Networks, Inc. Gadzoox Network, Inc.
16241 Laguna Canyon Road 16241 Laguna Canyon Road
Suite 100 Suite 100
Irvine, CA 92618 Irvine, CA 92618
USA USA
Phone: +1 949 789 4639 Phone: +1 949 789 4642
Fax: +1 949 453 1271 Fax: +1 949 453 1271
Email: vchau@gadzoox.com Email: ghecht@Gadzoox.com
Ken Hirata Jim Nelson
Vixel Corporation Vixel Corporation
15245 Alton Parkway, Suite 100 15245 Alton Parkway, Suite 100
Irvine, CA 92618 Irvine, CA 92618
USA USA
Phone: +1 949 788 6368 Phone: +1 949 450 6159
Fax: +1 949 753 9500 Fax: +1 949 753 9500
Email: ken.hirata@vixel.com Email: Jim.Nelson@vixel.com
Michael E. O'Donnell Anil Rijhsinghani
McDATA Corporation McDATA Corporation
310 Interlocken Parkway 5 Brickyard lane
Broomfield, Co. 80021 Westboro, MA 01581
USA USA
Phone: +1 303 460 4142 Phone: +1 508 870 6593
Fax: +1 303 465 4996 Email:
Email: modonnell@mcdata.com anil.rijhsinghani@mcdata.com
Milan J. Merhar Craig W. Carlson
43 Nagog Park QLogic Corporation
Pirus Networks 6321 Bury Drive
Acton, MA 01720 Eden Prairie, MN 55346
USA USA
Phone: +1 978 206 9124 Phone: +1 952 932 4064
Email: Milan@pirus.com Email: craig.carlson@qlogic.com
Venkat Rangan Larwrence J. Lamers
Rhapsody Networks Inc. SAN Valley
3450 W. Warren Ave. 4611 Park Norton Place
Fremont, CA 94538 San Jose, CA 95136-2523
USA USA
Phone: +1 510 743 3018 Phone: +1 408 626 1285
Fax: +1 510 687 0136 Email: ljlamers@ieee.org
Email: venkat@rhapsodynetworks.com
Rajagopal, et al. Standards Track [Page 33]
Internet-Draft Fibre Channel Over TCP/IP (FCIP) August, 2001
15. Full Copyright Statement
Copyright (C) The Internet Society (2001). All Rights Reserved.
This document and translations of it may be copied and furnished to
others, and derivative works that comment on or otherwise explain it
or assist in its implementation may be prepared, copied, published
and distributed, in whole or in part, without restriction of any
kind, provided that the above copyright notice and this paragraph
are included on all such copies and derivative works. However, this
document itself may not be modified in any way, such as by removing
the copyright notice or references to the Internet Society or other
Internet organizations, except as needed for the purpose of
developing Internet standards in which case the procedures for
copyrights defined in the Internet Standards process must be
followed, or as required to translate it into languages other than
English.
The limited permissions granted above are perpetual and will not be
revoked by the Internet Society or its successors or assigns.
This document and the information contained herein is provided on an
"AS IS" basis and THE INTERNET SOCIETY AND THE INTERNET ENGINEERING
TASK FORCE DISCLAIMS ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING
BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION
HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF
MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE.
ANNEX A - Example of synchronization recovery algorithm
The contents of this annex are informative.
Synchronization may be recovered as specified in section 6.6.2.4. An
example of an algorithm for searching the bytes delivered to the
Encapsulated Frame Receiver Portal for a valid FCIP Frame header is
provided in this annex.
This resynchronization uses the principle that a valid FCIP data
stream must contain at least one valid header every 2176 bytes (the
maximum length of an encapsulated FC Frame). Although other data
patterns containing apparently valid headers may be contained in the
stream, the FC CRC or FCIP Frame validity of the data patterns
contained in the data stream will always be either interrupted by or
resynchronized with the valid FCIP Frame headers.
Consider the case shown in figure 9. A series of short FCIP Frames,
perhaps from a trace, are embedded in larger FCIP Frames, say as a
result of a trace file being transferred from one disk to another.
Rajagopal, et al. Standards Track [Page 34]
Internet-Draft Fibre Channel Over TCP/IP (FCIP) August, 2001
The headers for the short FCIP Frames are denoted SFH and the long
FCIP Frame headers are marked as LFH.
+-+--+-+----+-+----+-+----+-+-+-+---+-+---
|L| |S| |S| |S| |S| |L| |S|
|F| |F| |F| |F| |F| |F| |F|...
|H| |H| |H| |H| |H| |H| |H|
+-+--+-+----+-+----+-+----+-+-+-+---+-+---
| |
|<---------2176 bytes-------->|
Fig. 9 Example of resynchronization data stream
A resynchronization attempt that starts just to the right of an LFH
will find several SFH FCIP Frames before discovering that they do
not represent the transmitted stream of FCIP Frames. Within 2176
bytes plus or minus, however, the resynchronization attempt will
encounter an SFH whose length does not match up with the next SFH
because the LFH will fall in the middle of the short FCIP Frame
pushing the next header farther out in the byte stream.
Note that the resynchronization algorithm cannot forward any
prospective FC Frames to the FC Transmitter Portal because until
synchronization is completely established there is no certainty that
anything that looked like an FCIP Frame really was one. For example,
an SFH might fortuitously contain a length that points exactly to
the beginning of an LFH. The LFH would identify the correct
beginning of a transmitted FCIP Frame, but that in no way guarantees
that the SFH was also a correct FCIP Frame header.
There exist some data streams that cannot be resynchronized by this
algorithm. If such a data stream is encountered, the algorithm
causes the TCP connection to be closed.
The resynchronization assumes that security and authentication
procedures outside the FCIP Entity are protecting the valid data
stream from being replaced by an intruding data stream containing
valid FCIP data.
The following steps are one example of how an FCIP_DE might
resynchronize with the data stream entering the Encapsulated Frame
Receiver Portal.
Rajagopal, et al. Standards Track [Page 35]
Internet-Draft Fibre Channel Over TCP/IP (FCIP) August, 2001
1) Search for candidate and strong headers:
The data stream entering the Encapsulated Frame Receiver Portal
is searched for 12 bytes in a row containing the required values
for:
a) Protocol field,
b) Version field,
c) ones complement of the Protocol field,
d) ones complement of the Version field,
e) replication of encapsulation word 0 in word 1, and
f) Reserved field and its ones complement.
If such a 12-byte grouping is found, the FCIP_DE assumes that it
has identified bytes 0-2 of a candidate FCIP encapsulation header.
All bytes up to and including the candidate header byte are
discarded.
If no candidate header has been found after searching a
specified number of bytes greater than some multiple of 2176
(the maximum length of an FCIP Frame), resynchronization has
failed and the TCP/IP connection is closed.
Word 3 of the candidate header contains the Frame Length and
Flags fields and their ones complements. If the fields are
consistent with their ones complements, the candidate header is
considered a strong candidate header. The Frame Length field is
used to determine where in byte stream the next strong candidate
header should be and processing continues at step 2).
2) Use multiple strong candidate headers to locate a verified
candidate header:
The Frame Length in one strong candidate header is used to skip
incoming bytes until the expected location of the next strong
candidate header is reached. Then the tests described in step 1)
are applied to see if another strong candidate header has
successfully been located.
All bytes skipped and all bytes in all strong candidate headers
processed are discarded.
Strong candidate headers continue to be verified in this way for
at least 4352 bytes (twice the maximum length of an FCIP Frame).
If at anytime a verification test fails, processing restarts at
step 1 and a retry counter is incremented. If the retry counter
exceeds 3 retries, resynchronization has failed and the TCP
connection is closed.
Rajagopal, et al. Standards Track [Page 36]
Internet-Draft Fibre Channel Over TCP/IP (FCIP) August, 2001
After strong candidate headers haves been verified for at least
4352 bytes, the next header identified is a verified candidate
header and processing continues at step 3).
Note: If a strong candidate header was part of the data content
of an FCIP Frame, the FCIP Frame defined by that or a subsequent
strong candidate header will eventually cross an actual header
in the byte stream. As a result it will either identify the
actual header as a strong candidate header or it will lose
synchronization again because of the extra 28 bytes in the
length, returning to step 1 as described above.
3) Use multiple strong candidate headers to locate a verified
candidate header:
Incoming bytes are skipped and discarded until the next verified
candidate header is reached. Each verified candidate header is
tested against the full collection of tests listed in section
6.6.2.2 as would normally be the case.
Verified candidate headers continue to be located and tested in
this way for a minimum of 4352 bytes (twice the maximum length
of an FCIP Frame). If all verified candidate headers encountered
are valid, the last verified candidate header is a valid header.
At this point the FCIP_DE stops discarding bytes and begins
normal FCIP de-encapsulation begins, including for the first
time since synchronization was lost, delivery of FC frames
through the FC Transmitter Portal according to normal FCIP rules.
If any verified candidate headers are invalid but meet all the
requirements of a strong candidate header, increment the retry
counter and return to step 2). If any verified candidate headers
are invalid and fail to meet the tests for a strong candidate
header, increment the retry counter and return to step 1. If the
retry counter exceeds 4 retries, resynchronization has failed
and the TCP/IP connection is closed.
A flowchart for this algorithm can be found in figure 10.
Rajagopal, et al. Standards Track [Page 37]
Internet-Draft Fibre Channel Over TCP/IP (FCIP) August, 2001
Synchronization is lost
|
_____________v_______________
| |
| Search for candidate header |
+----------->| |
| | Found Not Found |
| | (Strong candidate) |
| |_____________________________|
| | |
| | + --------->Close TCP/IP
| _______v_____________________ Connection
| | |
| | Enough strong candidate |
| +---->| headers identified? |
| | | |
| | | No Yes |
| | | (Verified candidate) |
| | |_____________________________|
|___________________| |
^ | |
| | |
| | _______________________v_____
| | | |
| | | Enough verified candidate |
| | | headers validated? |
| | | |
| | | No Yes |
| | | (Resynchronized) |
| | |_____________________________|
| | | |
| | ______v__________ | Resume
| | | | + ---> Normal
| | | Synchronization | De-encapsulation
| | | Lost? |
| | | |
| | | No Yes |
| | |_________________|
| | | |
| |________| |
|___________________________|
Fig. 10 Flow diagram of simple synchronization example
Rajagopal, et al. Standards Track [Page 38]
Internet-Draft Fibre Channel Over TCP/IP (FCIP) August, 2001
ANNEX B - Relationship between FCIP and IP over FC (IPFC)
The contents of this annex are informative.
IPFC (RFC 2625) describes the encapsulation of IP packets in FC
Frames. It is intended to facilitate IP communication over an FC
network.
FCIP describes the encapsulation of FC Frames in TCP segments which
in turn are encapsulated inside IP packets for transporting over an
IP network. It gives no consideration to the type of FC Frame that
is being encapsulated. Therefore, the FC Frame may actually contain
an IP packet as described in the IP over FC specification (RFC
2625). In such a case, the data packet would have:
- Data Link Header
- IP Header
- TCP Header
- FCIP Header
- FC Header
- IP Header
Note: The two IP headers would not be identical to each other. One
would have information pertaining to the final destination while the
other would have information pertaining to the FCIP Entity.
The two documents focus on different objectives. As mentioned above,
implementation of FCIP will lead to IP encapsulation within IP.
While perhaps inefficient, this should not lead to issues with IP
communication. One caveat: if a Fibre Channel device is
encapsulating IP packets in an FC Frame (e.g. an IPFC device), and
that device is communicating with a device running IP over a non-FC
medium, a second IPFC device may need to act as a gateway between
the two networks. This scenario is not specifically addressed by FCIP.
There is nothing in either of the specifications to prevent a single
device from implementing both FCIP and IP-over-FC (IPFC), but this
is implementation specific, and is beyond the scope of this document.
ANNEX C - FC Frame Format
The contents of this annex are informative.
All FC Frames have a standard format (see FC-FS [6]) much like LAN's
802.x protocols. However, the exact size of each FC Frame varies
depending on the size of the variable fields. The size of the
variable field ranges from 0 to 2112-bytes as shown in the FC Frame
Rajagopal, et al. Standards Track [Page 39]
Internet-Draft Fibre Channel Over TCP/IP (FCIP) August, 2001
Format in figure 11 resulting in the minimum size FC Frame of 36
bytes and the maximum size FC Frame of 2176 bytes. Valid FC Frame
lengths are always a multiple of four bytes.
+------+--------+-----------+----//-------+------+------+
| SOF |Frame |Optional | Frame | CRC | EOF |
| (4B) |Header |Header | Payload | (4B) | (4B) |
| |(24B) |<----------------------->| | |
| | | Data Field = (0-2112B) | | |
+------+--------+-----------+----//-------+------+------+
Fig. 11 FC Frame Format
C.1 SOF and EOF Delimiters
On an FC link, Start-of-Frame (SOF) and End-Of-Frame (EOF) are
called Ordered Sets and are sent as special words constructed from
the 8B/10B comma character (K28.5) followed by three additional 8B/
10B data characters making them uniquely identifiable in the data
stream.
On an FC link the SOF delimiter serves to identify the beginning of
an FC Frame and prepares the receiver for FC Frame reception. The
SOF contains information about the FC Frame's Class of Service,
position within a sequence, and in some cases, connection status.
The EOF delimiter identifies the end of the FC Frame and the final
FC Frame of a sequence. In addition, it serves to force the running
disparity to negative. The EOF is used to end the connection in
connection-oriented classes of service.
A special EOF delimiter called EOFa (End Of Frame - Abort) is used
to terminate a partial FC Frame resulting from a malfunction in a
link facility during transmission. Since an FCIP Entity functions
like a transmission link with respect to the rest of the FC Fabric,
FCIP_DEs may use EOFa in their error recovery procedures.
It is therefore important to preserve the information conveyed by
the delimiters across the IP-based network, so that the receiving
FCIP Entity can correctly reconstruct the FC Frame in its original
SOF and EOF format before forwarding it to its ultimate FC
destination on the FC link.
When an FC Frame is encapsulated and sent over a byte-oriented
interface, the SOF and EOF delimiters are represented as sequences
of four consecutive bytes, which carry the equivalent Class of
Service and FC Frame termination information as the FC ordered sets.
The representation of SOF and EOF in an encapsulation FC Frame is
Rajagopal, et al. Standards Track [Page 40]
Internet-Draft Fibre Channel Over TCP/IP (FCIP) August, 2001
described in FC Frame Encapsulation [23].
C.2 Frame Header
The FC Frame Header is transparent to the FCIP Entity. The FC Frame
Header is 24 bytes long and has several fields that are associated
with the identification and control of the payload. Current FC
Standards allow up to 3 Optional Header fields [6]:
- Network_Header (16-bytes)
- Association_Header (32-bytes)
- Device_Header (up to 64-bytes).
C.3 Frame Payload
The FC Frame Payload is transparent to the FCIP Entity. An FC
application level payload is called an Information Unit at the FC-4
Level. This is mapped into the FC Frame Payload of the FC Frame. A
large Information Unit is segmented using a structure consisting of
FC Sequences. Typically, a Sequence consists of more than one FC
Frame. FCIP does not maintain any state information regarding the
relationship of FC Frames within a FC Sequence.
C.4 CRC
The FC CRC is 4 bytes long and uses the same 32-bit polynomial used
in FDDI and is specified in ANSI X3.139 Fiber Distributed Data
Interface. This CRC value is calculated over the entire FC header
and the FC payload; it does not include the SOF and EOF delimiters.
Note: When FC Frames are encapsulated into FCIP Frames, the FC Frame
CRC is untouched by the FCIP Entity.
ANNEX D - FC Encapsulation Format
This annex contains a reproduction of the FC Encapsulation Format
[23] as it applies to FCIP Frames. The information in this annex was
correct as of the time this specification was approved. The
information in this annex is informative only.
If there are any differences between the information here and the FC
Encapsulation Format specification [23], the FC Encapsulation Format
specification takes precedence.
If there are any differences between the information here and the
contents of section 6.6.1, then the contents of section 6.6.1 take
precedence.
Rajagopal, et al. Standards Track [Page 41]
Internet-Draft Fibre Channel Over TCP/IP (FCIP) August, 2001
Figure 12 applies the requirements stated in section 6.6.1 and in
the FC Encapsulation Frame format resulting in a summary of the FCIP
frame format. Where FCIP requires specific values, those values are
shown in hexadecimal in parentheses. Detailed requirements for the
FCIP usage of the FC Encapsulation Format are in section 6.6.1.
W|------------------------------Bit------------------------------|
o| |
r|3 3 2 2 2 2 2 2 2 2 2 2 1 1 1 1 1 1 1 1 1 1 |
d|1 0 9 8 7 6 5 4 3 2 1 0 9 8 7 6 5 4 3 2 1 0 9 8 7 6 5 4 3 2 1 0|
+---------------+---------------+---------------+---------------+
0| Protocol# | Version | -Protocol# | -Version |
| (0x01) | (0x01) | (0xFE) | (0xFE) |
+---------------+---------------+---------------+---------------+
1| Protocol# | Version | -Protocol# | -Version |
| (0x01) | (0x01) | (0xFE) | (0xFE) |
+---------------+---------------+---------------+---------------+
2| Reserved | -Reserved |
| (0x00-00) | (0xFF-FF) |
+-----------+-------------------+-----------+-------------------+
3| Flags | Frame Length | -Flags | -Frame Length |
| (0x00) | | (0x3F) | |
+-----------+-------------------+-----------+-------------------+
4| Time Stamp [integer] |
+---------------------------------------------------------------+
5| Time Stamp [fraction] |
+---------------------------------------------------------------+
6| CRC |
| (0x00-00-00-00) |
+---------------+---------------+-------------------------------+
7| SOF | SOF | -SOF | -SOF |
+---------------+---------------+-------------------------------+
9| |
+----- FC frame content (see annex C) -----+
| |
+---------------+---------------+-------------------------------+
n| EOF | EOF | -EOF | -EOF |
+---------------+---------------+-------------------------------+
Fig. 12 FCIP Frame Format
The names of fields are generally descriptive on their contents and
the FC Encapsulation Format specification [23] is referenced for
details. Field names preceded by a minus sign are one's complement
values of the named field.
Rajagopal, et al. Standards Track [Page 42]
Internet-Draft Fibre Channel Over TCP/IP (FCIP) August, 2001
ANNEX E - FCIP Requirements on an FC Entity
The contents of this annex are informative for FCIP but might be
considered normative on FC-BB-2.
The capabilities that FCIP requires of an FC Entity include:
1) The FC Entity must deliver FC Frames to the correct FCIP Data
Engine (in the correct FCIP Link Endpoint).
2) When FC Frames exit FCIP Data Engine(s) via the FC Transmitter
Portal(s), the FC Entity should forward them to the FC Fabric.
However, two conditions cause FC Frames to exit via the FC
Transmitter Portal without having been checked for proper IP
Network transit time (as specified by the FC Entity in DLY_LIM).
In both of the following cases, the FC Entity must decide
whether or not FC Frames that are not known to have transited
the IP Network in the specified time should be forwarded to the
FC Fabric:
a) When the reliability of the synchronized time value (see
section 7) is uncertain, the FCIP Data Engine does not check
the IP Network transit time of any FC Frames delivered via
FC Transmitter Portal; and
b) When incoming FCIP Frames do not contain a valid time stamp
(see section 6.6.2.3), the FCIP Data Engine does not check
the IP Network transit time and notifies the FC Entity that
the IP Network transit time of the resulting FC Frame is
unknown.
Note: FC Frames that are known to have not transited the IP
Network in the specified time will not exit an FCIP Data Engine.
3) The only delivery ordering guarantee provided by FCIP is
correctly ordered delivery of FC Frames between a pair of FCIP
Data Engines. FCIP expects the FC Entity to implement all other
FC Frame delivery ordering requirements.
4) The FC Entity must support the FCIP Entity in the processing of
incoming connect requests by deciding to accept a connect request.
5) The FC Entity may require the FCIP Entity to perform TCP Connect
and Close requests.
6) The FC Entity may instruct the FCIP Entity regarding TCP
connection parameter settings and the DLY_LIM (see section
6.6.2.3) to be applied to FCIP Data Engines.
7) The FC Entity must provide the FCIP_DE with a synchronized time
value for use in building outgoing FCIP Encapsulated Frame
Rajagopal, et al. Standards Track [Page 43]
Internet-Draft Fibre Channel Over TCP/IP (FCIP) August, 2001
headers and in checking incoming time stamp values. The FC
Entity must notify the FCIP_DE when that value is not
synchronized well enough to be used for these purposes.
8) The FC Entity may recover from connection failures.
9) The FC Entity must recover from events that the FCIP Entity
cannot handle, such as:
a) loss of synchronization with FCIP Frame headers from the
Encapsulated Frame Receiver Portal requiring resetting the
TCP connection;
b) recovering from FCIP Frames that are discarded as a result
of synchronization problems (see section 6.6.2.2 and section
6.6.2.4) or delays in transiting the IP network (see section
6.6.2.3);
c) additional examples, TBD
10) The FC Entity must work cooperatively with the FCIP Entity to
manage flow control problems in either the IP Network or FC
Fabric.
11) The FC Entity may test for failed TCP connections.
12) The FC Entity must initiate the dynamic discovery of peer FCIP
Entities by informing the FCIP Entity of the FCIP Discovery
Domain to be used (see section 8.1.3). The FC Entity also must
coordinate with the FCIP Entity in selecting which of the
discovered peers should have TCP connections and how those
connections are to be configured.
13) TBD support for security
14) Communicate QoS needs when a TCP connection is created as
described in section 10.3.
15) TBD support for monitoring
Note that the Fibre Channel standards MUST be consulted for a
complete understanding of the requirements placed on an FC Entity.
Rajagopal, et al. Standards Track [Page 44]
Html markup produced by rfcmarkup 1.129d, available from
https://tools.ietf.org/tools/rfcmarkup/