[Docs] [txt|pdf] [Tracker] [WG] [Email] [Diff1] [Diff2] [Nits]

Versions: 03

IP Storage Working Group                               M. Krueger
                                                       R. Haagens
Internet Draft                                    Hewlett-Packard
                                                      Corporation
Category: Informational
                                                   C. Sapuntzakis
                                                         M. Bakke
                                                    Cisco Systems

Document: draft-ietf-ips-iscsi-reqmts-03.txt           April 2001


              iSCSI Requirements and Design Considerations


Status of this Memo

   This document is an Internet-Draft and is in full conformance with
   all provisions of Section 10 of RFC2026 [1].

   Internet-Drafts are working documents of the Internet Engineering
   Task Force (IETF), its areas, and its working groups.  Note that
   other groups may also distribute working documents as Internet-
   Drafts.

   Internet-Drafts are draft documents valid for a maximum of six
   months and may be updated, replaced, or obsoleted by other documents
   at any time.  It is inappropriate to use Internet-Drafts as
   reference material or to cite them other than as "work in progress."

   The list of current Internet-Drafts can be accessed at
   http://www.ietf.org/ietf/1id-abstracts.txt

   The list of Internet-Draft Shadow Directories can be accessed at
   http://www.ietf.org/shadow.html.


Abstract

   The IP Storage Working group is chartered with developing
   comprehensive technology to transport block storage data over IP
   protocols.  This effort includes a protocol to transport the Small
   Computer Systems Interface (SCSI) protocol over the internet
   (iSCSI).  The initial version of the iSCSI protocol will define a
   mapping of SCSI transport protocol over TCP/IP so that SCSI storage
   controllers (principally disk and tape arrays and libraries) can be
   attached to IP networks, notably Gigabit Ethernet (GbE) and 10
   Gigabit Ethernet (10 GbE).

   This document specifies the requirements iSCSI and it's related
   infrastructure should satisfy and the design considerations guiding
   the iSCSI protocol development efforts. In the interest of timely
   adoption of the iSCSI protocol, this group has chosen to focus the


Krueger          Informational  Expires August 2001                1

               iSCSI Reqmnts and Design Considerations      Nov. 2000


   first version of the protocol to work with the existing SCSI
   architecture and commands, and the existing TCP/IP transport layer.
   Both these protocols are widely-deployed and well-understood.  The
   thought is that using these mature protocols will entail a minimum
   of new invention, the most rapid possible adoption, and the greatest
   compatibility with Internet architecture, protocols, and equipment.

   The iSCSI protocol is a mapping of SCSI to TCP, and constitutes a
   "SCSI transport" as defined by the ANSI T10 document SCSI SAM-2
   document [SAM2, p. 3, "Transport Protocols"].

Conventions used in this document

   The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
   "SHOULD", "SHOULD NOT", "RECOMMENDED",  "MAY", and "OPTIONAL" in
   this document are to be interpreted as described in RFC-2119 [2].

Table of Contents

1.  Summary of Requirements...........................................2
2.  iSCSI Design Considerations.......................................6
 2.1. General Discussion..............................................6
 2.2. Performance/Cost................................................9
 2.3. Framing........................................................10
 2.4. High bandwidth, bandwidth aggregation..........................11
3.  Ease of implementation/complexity of protocol....................13
4.  Reliability and Availability.....................................13
 4.1. Detection of Data Corruption...................................13
 4.2. Recovery.......................................................14
5.  Interoperability.................................................15
 5.1. Internet infrastructure........................................15
 5.2. SCSI...........................................................15
6.  Security Considerations..........................................16
 6.1. Extensible Security............................................16
 6.2. Authentication.................................................17
 6.3. Data Integrity.................................................17
 6.4. Data Privacy...................................................17
7.  Management.......................................................18
 7.1. Naming.........................................................18
 7.2. Discovery......................................................19
8.  Internet Accessibility...........................................20
 8.1. Denial of Service..............................................20
 8.2. Firewalls and Proxy servers....................................20
 8.3. Congestion Control and Transport Selection.....................20
9.  Definitions......................................................20
10.   References.....................................................21
11.   Acknowledgements...............................................22
12.   Author's Addresses.............................................22





Krueger             Informational   Exp. May 2001                   2

               iSCSI Reqmnts and Design Considerations      Nov. 2000

1. Summary of Requirements

   The iSCSI standard:

>From section 2.1 General Discussion:

   MUST NOT require modifications to the current IP and Ethernet
   infrastructure to support storage traffic over TCP.

>From section 2.2 Performance/Cost:
   MUST allow implementations to equal or improve on the current state
   of the art for SCSI interconnects.

   MUST enable cost competitive implementations.

   SHOULD minimize control overhead  to enable low delay
   communications.

   MUST provide high bandwidth and bandwidth aggregation.

   MUST have low host CPU utilizations, equal to or better than current
   technology.

   MUST be possible to build I/O adapters that handle the entire SCSI
   task.

   MUST permit zero-copy memory architectures.

   MUST NOT impose complex operations on host software.

   MUST be cost competitive with alternative storage networking
   technologies.

>From section 2.4 High Bandwidth/Bandwidth Agreggation:
   MUST allow the initiator and target to use multiple network
   interfaces and multiple paths through the network.

   MUST operate over a single TCP connection.

   MUST provide a FIFO transport of SCSI commands, even when commands
   are sent along different paths. This command ordering mechanism
   SHOULD seek to minimize the amount of communication necessary across
   multiple adapters doing transport off-load.

   SHOULD support connection binding, and it MUST be optional to
   implement.

>From section 3 Ease of Implementation/Complexity of Protocol:
   SHOULD keep the protocol simple.

   SHOULD minimize optional features.

   SHOULD negotiate optional features at session setup.

>From section 4.1 Detection of Data Corruption:


Krueger             Informational   Exp. May 2001                   3

               iSCSI Reqmnts and Design Considerations      Nov. 2000


   MUST support a data integrity check format for use in digest
   generation.

   MAY use separate digest for data and headers.

   iSCSI header format SHOULD be extensible to include other data
   integrity digest calculation methods.

>From section 4.2 Recovery:
   MUST specify mechanisms to recover in a timely fashion from
   failures on the initiator, target, or connecting infrastructure.

   MUST specify recovery methods for non-idempotent requests.

   SHOULD take into account fail-over schemes for mirrored targets or
   highly available storage configurations.

   SHOULD provide a method for sessions to be gracefully terminated and
   restarted that can be initiated by either the initiator or target.

>From section 5 Interoperability:
   iSCSI protocol document MUST be clear and unambiguous.

>From section 5.1 Internet Infrastructure:
   MUST:
    -- be compatible with both IPv4 and IPv6
    -- use TCP connections conservatively, keeping in mind there may be
       many other users of TCP on a given machine.

   MUST NOT require changes to existing internet protocols.

>From section 5.2 SCSI:
   Any feature SAM2 requires in a valid transport mapping MUST be
   specified by iSCSI and the specification SHOULD make such a feature
   either RECOMMENDED or REQUIRED in implementations.

   MUST NOT require changes to the SCSI-3 command sets and SCSI client
   code except to reflect lengthier iSCSI target names.

   SHOULD track changes to SCSI and the SCSI Architecture Model.

   MUST be capable of supporting all SCSI-3 command sets and device
   types.

   MUST allow for the construction of gateways to other SCSI transports

   MUST reliably transport SCSI commands from the initiator to the
   target.  MUST correctly deal with iSCSI packet drop, duplication,
   corruption, stale packets, and re-ordering.

   MUST support ordered delivery of SCSI commands from the initiator to
   the target, to support SCSI Task Queuing.


Krueger             Informational   Exp. May 2001                   4

               iSCSI Reqmnts and Design Considerations      Nov. 2000


>From section 6.1 Extensible Security:
   SHOULD require minimal configuration and overhead in the insecure
   operation.

   SHOULD provide for strong authentication when increased security is
   required.

   SHOULD allow integration of new security mechanisms without breaking
   backwards compatible operation.

>From section 6.2 Authentication:
   MAY support various levels of authentication security.

   MUST support private authenticated login.

   iSCSI authenticated login MUST be resilient against passive attacks.

   MUST NOT preclude optional data origin authentication of its
   communications.

>From section 6.3 Data Integrity:
   SHOULD NOT preclude use of additional data integrity protection
   protocols (IPSec, TLS).

>From section 6.4 Data Privacy:
   MAY use a data encryption protocol such as TLS or IPsec ESP to
   provide data privacy between iSCSI endpoints.

>From section 7 Management:
   SHOULD be manageable using IP-based management protocols (eg. SNMP,
   RMI).

>From section 7.1 Naming:
   MUST support the naming architecture of SAM-2.

   The means by which an iSCSI resource is located MUST use or extend
   existing internet standard resource location methods (URL).

   MUST provide a means of identifying iSCSI targets by a unique
   identifier that is independent of the path on which it is found.

   The format for the iSCSI names MUST use existing naming authorities.

   An iSCSI name SHOULD be a human readable string in an international
   character set encoding.

   Standard internet lookup services SHOULD be used to resolve iSCSI
   names.

   SHOULD deal with the complications of the new SCSI security
   architecture.

   MUST support SCSI 3rd party operations such as EXTENDED COPY.

Krueger             Informational   Exp. May 2001                   5

               iSCSI Reqmnts and Design Considerations      Nov. 2000



>From section 7.2 Discovery:

   MUST have no impact on the use of current IP network discovery
   techniques.

   MUST provide some means of determining whether an iSCSI service is
   available through an IP address.

   SCSI protocol-dependent techniques SHOULD be used for further
   discovery beyond the iSCSI layer.

   MUST provide a method of discovering, given an IP end point on its
   well-known port, the list of SCSI targets available to the
   requestor.  The use of this discovery service MUST be optional.

>From section 8 Internet Accessability.

   SHOULD be scrutinized for denial of service issues and they should
   be addressed.

>From section 8.2 Firewalls and Proxy Servers

   SHOULD allow deployment where functional and optimizing middle-boxes
   such as firewalls, proxy servers and NATs are present.

   use of IP addresses and TCP ports SHOULD be firewall friendly.

>From section 8.3 Congestion Control and Transport Selection

   MUST be a good network citizen with TCP-compatible congestion
   control (as defined in RFC 2309).

   iSCSI implementations MUST NOT use multiple connections as a means
   to avoid transport-layer congestion control.


2. iSCSI Design Considerations

  2.1. General Discussion

   Traditionally, storage controllers (e.g., disk array controllers,
   tape library controllers) have supported the SCSI-3 protocol, and
   have been attached to computers through the SCSI parallel bus or
   through Fibre Channel.  File-oriented storage controllers have
   supported the NFS and/or CIFS protocols, and have been attached
   directly to IP networks such as Ethernet.

   The IP infrastructure offers compelling advantages for volume/block-
   oriented storage attachment compared to current approaches.  It
   offers the opportunity to take advantage of the performance/cost


Krueger             Informational   Exp. May 2001                   6

               iSCSI Reqmnts and Design Considerations      Nov. 2000


   benefits provided by competition in the internet marketplace. This
   reduces the cost of storage infrastructure by:

    -- Increasing performance (market driven by networking demand)
    -- Offers richer array of management, security and QoS solutions
    -- Economies arising from the need to install and operate only
       single type of network

   In addition, mapping SCSI over IP provides:

    -- Extended distance ranges
    -- Connectivity to "carrier class" services that support IP

   The following applications for iSCSI are contemplated:

    -- Local storage access, consolidation, clustering and pooling (as
       in the data center)
    -- Client access to remote storage (eg. a "storage service
       provider")
    -- Local and remote synchronous and asynchronous mirroring between
       storage controllers
    -- Local and remote backup and recovery

   iSCSI MUST support the following topologies:

    -- Point-to-point direct connections
    -- Dedicated storage LAN, consisting of one or more LAN segments
    -- Shared LAN, carrying a mix of traditional LAN traffic plus
       storage traffic
    -- LAN-to-WAN extension using IP routers or carrier-provided "IP
       Datatone"
    -- Private networks and the public Internet

   iSCSI protocols will enable local-area storage networks built using
   Ethernet LAN switches.  These networks may be dedicated to storage,
   or shared with traditional Ethernet uses, as determined by cost,
   performance, administration, and security considerations.  In the
   local area, TCP's adaptive retransmission timers provide for
   automatic and rapid error detection and recovery.

   IP LAN-WAN routers may be used to extend the IP storage network to
   the wide area, permitting remote disk access (as for a storage
   utility), synchronous and asynchronous remote mirroring, and remote
   backup and restore (as for tape vaulting).  In the WAN,  using TCP
   end-to-end avoids the need for specialized equipment for protocol
   conversion, ensures data reliability, copes with network congestion,
   and provides adaptive retransmission strategies to WAN delays.

   The iSCSI technology deployment will involve the following elements:
    (1)  Conclusion of a complete, unambigous protocol standard and
         supporting implementations;


Krueger             Informational   Exp. May 2001                   7

               iSCSI Reqmnts and Design Considerations      Nov. 2000


    (2)  Development of Ethernet storage NICs and related driver and
         protocol software; [NOTE: high-speed applications of iSCSI are
         expected to require significant portions of the iSCSI/TCP/IP
         implementation in hardware to achieve the necessary
         throughput.]
    (3)  Development of compatible storage controllers; and
    (4)  The likely development of translating gateways to provide
         connectivity between the Ethernet storage network and the
         Fibre Channel and/or parallel-bus SCSI domains.
    (5)  Development of specifications for iSCSI device management such
         as MIBs, LDAP or XML schemas, etc.
    (6)  Development of management and directory service applications
         to support a robust SAN infrastructure.

   Products could initially be offered for Gigabit Ethernet attachment,
   with rapid migration to 10 GbE.  For performance competitive with
   alternative SCSI transports, it will be necessary to implement the
   performance path of the full protocol stack in hardware.  These new
   storage NICs might perform full-stack processing of a complete SCSI
   task, analogous to today's SCSI and Fibre Channel HBAs, and might
   also support all host protocols that use TCP (NFS, CIFS, HTTP, etc).

   The iSCSI protocol MUST NOT require modifications to the current IP
   and Ethernet infrastructure to support storage traffic over TCP.
   Nevertheless, the performance and security requirements of storage
   creates opportunities for improvement in security protocols and QoS
   implementations.  The addition of storage traffic to local and wide-
   area internets (and even to the public Internet) may introduce
   increased requirements for traffic monitoring and engineering in
   those environments.

   Organizations may initially choose to operate storage networks based
   on iSCSI that are independent of (isolated from) their current data
   networks except for secure routing of storage management traffic.
   These organizations will benefit from the high performance/cost of
   IP equipment and a unified management architecture, compared to
   alternative means of building storage networks.  As security and QoS
   evolve, it may become reasonable to build combined networks with
   shared infrastructure; nevertheless, it is likely that sophisticated
   users will choose to keep their storage sub-networks isolated to
   afford the best control of security and QoS.

   The charter of the IETF IP Storage Working Group (IPSWG) describes
   the broad goal of mapping SCSI to IP using a transport that has
   proven congestion avoidance behavior and broad implementation on a
   variety of platforms.  Within that broad charter, several transport
   alternatives may be considered.  Initial IPS work focuses on TCP,
   and this requirements document is restricted to that domain of
   interest.





Krueger             Informational   Exp. May 2001                   8

               iSCSI Reqmnts and Design Considerations      Nov. 2000


  2.2. Performance/Cost

   In general, iSCSI MUST allow implementations to equal or improve on
   the current state of the art for SCSI interconnects.  This goal
   breaks down into several types of requirement:

   Cost competitive with alternative storage network technologies:

   In order to be adopted by vendors and the user community, the iSCSI
   protocol MUST enable cost competitive implementations when compared
   to other SCSI transports (Fibre Channel).

   Low delay communication:

   Conventional storage access is of a stop-and-wait or remote
   procedure call type.  Applications typically employ very little
   pipelining of their storage accesses, and so storage access delay
   directly impacts performance.  The delay imposed by current storage
   interconnects, including protocol processing, is generally in the
   range of 100 microseconds.  The use of caching in storage
   controllers means that many storage accesses complete almost
   instantly, and so the delay of the interconnect can have a high
   relative impact on overall performance.  When stop-and-wait IO is
   used, the delay of the interconnect will affect performance.  The
   iSCSI protocol SHOULD minimize control overhead, which adds to
   delay.

   Low host CPU utilization, equal to or better than current
   technology:

   For competitive performance, the iSCSI protocol MUST allow three key
   implementation goals to be realized:

   (1)  iSCSI MUST make it possible to build I/O adapters that handle
        an entire SCSI task, as alternative SCSI transport
        implementations do.
   (2)  The protocol MUST permit "zero-copy" memory architectures,
        where the I/O adapter reads or writes host memory exactly once
        per disk transaction.
   (3)  The protocol SHOULD NOT impose complex operations on the host
        software, which would increase host instruction path length
        relative to alternatives.

   Direct data placement (zero-copy iSCSI):

   This is an important implementation goal.  In an iSCSI system, each
   of the end nodes (for example host computer and storage controller)
   has ample memory; but the intervening nodes (NIC, switches) do not.
   Assume a WAN-scale retransmission requirement of 25 MB (1 Gbps) or
   250 MB (10 Gbps, see Framing discussion).  Therefore, intervening
   nodes MUST NOT be required to buffer data.


Krueger             Informational   Exp. May 2001                   9

               iSCSI Reqmnts and Design Considerations      Nov. 2000


   High bandwidth, bandwidth aggregation:

   The bandwidth (transfer rate, MB/sec) supported by storage
   controllers is rapidly increasing, due to several factors:

     1. Increase in disk spindle and controller performance;
     2. Use of ever-larger caches, and improved caching algorithms;
     3. Increased scale of storage controllers (number of supported
        spindles, speed of interconnects).

   The iSCSI protocol MUST provide for full utilization of available
   link bandwidth.  The protocol MUST also allow an implementation to
   exploit parallelism (multiple connections) at the device interfaces
   and within the interconnect fabric.

   The next two sections further discuss the need for direct data
   placement and high bandwidth.


  2.3. Framing

   Framing refers to the addition of information in a header, or the
   data stream to allow implementations to locate the boundaries of an
   iSCSI protocol data unit (PDU) within the TCP byte stream.  There
   are two technical requirements driving framing: interfacing needs,
   and accelerated processing needs.

   A framing solution that addresses the "interfacing needs" of the
   iSCSI protocol will facilitate the implementation of a message-based
   upper layer protocol (iSCSI) on top of an underlying byte streaming
   protocol (TCP).  Since TCP is a reliable transport, this can be
   accomplished by including a length field in the iSCSI header.
   Finding the protocol frame assumes that the receiver will parse from
   the beginning of the TCP data stream, and never make a mistake (lose
   alignment on packet headers).

   The other technical requirement for framing, "accelerated
   processing", stems from the need to handle increasingly higher data
   rates in the physical media interface.  Two needs arise from higher
   data rates:

   (1)  LAN environment - NIC vendors seek ways to provide "zero-copy"
        methods of moving data directly from the wire into application
        buffers.

   (2)  WAN environment- the emergence of high bandwidth, high latency,
        low bit error rate physical media places huge buffer
        requirements on the physical interface solutions.

   First, vendors are producing network processing hardware that
   offloads network protocols to hardware solutions to achieve higher
   data rates.  The concept of "zero-copy" seeks to store blocks of

Krueger             Informational   Exp. May 2001                  10

               iSCSI Reqmnts and Design Considerations      Nov. 2000


   data in appropriate memory locations (aligned) directly off the
   wire, even in when data is reordered due to packet loss.  This is
   necessary to drive actual data rates of 10 Gigabits and beyond.

   Secondly, in order for iSCSI to be successful in the WAN arena it
   MUST be possible to operate efficiently in high bandwidth, high
   delay networks.  The emergence of multi-gigabit IP networks with
   latencies in the tens to hundreds of milliseconds presents a
   challenge. To fill such large pipes, tens of megabytes of
   outstanding requests from the application are needed. In addition,
   some protocols potentially require tens of megabytes at the
   transport layer to deal with buffering for reassembly of data when
   packets are received out-of-order.

   Consider that a network pipe at 10 Gbps x 200 msec holds 250 MB.
   [Assume land-based communication with a spot half way around the
   world at the equator.  Ignore additional distance due to cable
   routing.  Ignore repeater and switching delays; consider only a
   speed-of-light delay of 5 microsec/km.  The circumference of the
   globe at the equator is approx. 40000 km (round-trip delay must be
   considered to keep the pipe full).  10 Gb/sec x 40000 km x 5
   microsec/km x B / 8b = 250 MB].  In a conventional TCP
   implementation, loss of a TCP segment means that stream processing
   MUST stop until that segment is recovered, which takes at least a
   time of <network round trip> to accomplish.  Following the example
   above, an implementation would be obliged to catch 250 MB of data
   into an anonymous buffer before resuming stream processing; later,
   this data would need to be moved to its proper location.  Some
   proponents of iSCSI seek some means of putting data directly where
   it belongs, and avoiding extra data movement in the case of segment
   drop.  This is a key concept in understanding the debate behind
   framing methodologies.

   The framing of the iSCSI protocol impacts both the "interfacing
   needs" and the "accelerated processing needs", however, while
   including a length in a header may suffice for the "interfacing
   needs", it will not serve the "accelerated processing needs". The
   framing mechanism developed should allow resynchronization of packet
   boundaries even in the case where a packet is temporarily missing in
   the incoming data stream.


  2.4. High bandwidth, bandwidth aggregation

   Experience has shown that any single link can be saturated by
   storage traffic. Scientific data applications, asynchronous and
   synchronous data replication are examples of storage applications
   that push the limits of throughput.

   The iSCSI protocol MUST allow the initiator and target to use
   multiple network interfaces and multiple paths through the network
   for increased throughput.  However, in order to provide for lower

Krueger             Informational   Exp. May 2001                  11

               iSCSI Reqmnts and Design Considerations      Nov. 2000


   cost implementations, the protocol MUST operate over a single TCP
   connection.

   Some applications, such as log updates, streaming tape, and
   replication, require ordering of updates and thus ordering of SCSI
   commands. An initiator may maintain ordering by waiting for each
   update to complete before issuing the next (a.k.a. synchronous
   updates). However, the throughput of synchronous updates decreases
   inversely with increases in latency of the operation.

   For greater throughput, the SCSI task queuing mechanism allows an
   initiator to have multiple commands outstanding at the target
   simultaneously and to express ordering constraints on the execution
   of those commands. The task queuing mechanism is only effective if
   the commands arrive at the target in the order they were presented
   to the initiator (FIFO order).

   The iSCSI standard MUST provide a FIFO transport of SCSI commands,
   even when commands are sent along different paths.  This is referred
   to as "command ordering".  This command ordering mechanism SHOULD
   seek to minimize the amount of communication necessary across
   multiple adapters doing transport off-load.

   There are a few potential ways to satisfy the multiple path and
   ordering requirements.

   A popular way to satisfy the multiple-path requirement is to have a
   driver above the SCSI layer instantiate multiple copies of the SCSI
   transport, each communicating to the target along a different path.
   "Wedge" drivers use this technique today to attain high performance.
   Unfortunately, wedge drivers must wait for acknowledgement of
   completion of each request (stop-and-wait) to ensure ordered
   updates.

   Another approach might be for iSCSI protocol to use multiple
   instances of its underlying transport (e.g. TCP). The iSCSI layer
   would make these independent transport instances appear as one SCSI
   transport instance and maintain the ability to do ordered SCSI
   command queuing. The document will refer to this technique as
   "connection binding" for convenience.

   The iSCSI protocol SHOULD support connection binding, and it MUST be
   optional to implement.

   In the presence of connection binding, there are two ways to assign
   features to connections. In the symmetric approach, all the
   connections are identical from a feature standpoint. In the
   asymmetric model, connections have different features. For example,
   some connections may be used primarily for data transfers whereas
   others are used primarily for SCSI commands.




Krueger             Informational   Exp. May 2001                  12

               iSCSI Reqmnts and Design Considerations      Nov. 2000


   Since the iSCSI protocol must support the case where there was only
   one transport connection, the protocol must have command, data, and
   status travel over the same connection.

   In the case of multiple connections, the iSCSI protocol MUST keep
   the command and its associated data and status on the same
   connection (connection allegiance). Sending data and status on the
   same connection is desirable because this guarantees that status is
   received after the data (TCP provides ordered delivery). In the case
   where each connection is managed by a separate processor, allegiance
   decreases the need for inter-processor communication.  This
   symmetric approach is a natural extension of the single connection
   approach.

   An alternate approach that was extensively discussed involved
   sending all commands on a single connection and the associated data
   and status on a different connection (asymetric approach). In this
   scheme, the transport ensures the commands arrive in order. The
   protocol on the data and status connections is simpler, perhaps
   lending itself to a simpler realization in hardware.  One
   disadvantage of this approach is that the recovery procedure is
   different if a command connection fails vs. a data connection. Some
   argued that this approach would require greater inter-processor
   communication when connections are spread across processors.
   The reader may reference the mail archives of the IPS mailing list
   between June and September of 2000 for extensive discussions on
   symmetric vs asymmetric connection models.


3. Ease of implementation/complexity of protocol

   Experience has shown that adoption of a protocol by the internet
   community is inversely proportional to its complexity.  In addition,
   the simpler the protocol, the easier it is to diagnose problems.
   The designers of iSCSI SHOULD strive to fulfill the requirements of
   the creating a SCSI transport over IP, while keeping the protocol as
   simple as possible.

   In the interest of simplicity, iSCSI SHOULD minimize optional
   features.  When features are deemed necessary, the protocol SHOULD
   allow for feature negotiation at session establishment (login) and
   provide for rejection when an implementation does not support a
   requested feature.


4. Reliability and Availability


  4.1. Detection of Data Corruption




Krueger             Informational   Exp. May 2001                  13

               iSCSI Reqmnts and Design Considerations      Nov. 2000


   There have been several research papers that suggest that the TCP
   checksum calculation allows a certain number of bit errors to pass
   undetected [10] [11].

   In order to protect against data corruption, the iSCSI protocol MUST
   support a data integrity check format for use in digest generation.

   The iSCSI protocol MAY use separate digests for data and headers. In
   an iSCSI proxy or gateway situation, the iSCSI headers are removed
   and re-built, and the TCP stream is terminated on either side.  This
   means that even the TCP checksum is removed and recomputed within
   the gateway.  To ensure the protection of commands, data, and status
   the iSCSI protocol MUST include a CRC or other digest mechanism that
   is computed on the SCSI data block itself, as well as on each
   command and status message.  Since gateways may strip iSCSI headers
   and rebuild them, a separate header CRC is required.  Two header
   digests, one for invariant portions of the header (addresses) and
   one for the variant portion would provide protection against changes
   to portions of the header that should never be changed by middle
   boxes (eg, addresses).

   The iSCSI header format SHOULD be extensible to include other digest
   calculation methods.


  4.2. Recovery

   The SCSI protocol was originally designed for a parallel bus
   transport that was highly reliable.  SCSI applications tend to
   assume that transport errors never happen, and when they do, SCSI
   application recovery tends to be expensive in terms of time and
   computational resources.

   iSCSI protocol design, while placing an emphasis on simplicity, MUST
   lead to timely recovery from failure of initiator, target, or
   connecting internet infrastructure (cabling, data path equipment
   such as routers, etc).

   iSCSI MUST specify recovery methods for non-idempotent requests,
   such as operations on tape drives.

   The iSCSI protocol error recover mechanism SHOULD take into account
   fail-over schemes for mirrored targets or highly available storage
   configurations that provide paths to target data through multiple
   "storage servers".  This would provide a basis for layered
   technologies like high availability and clustering.

   The iSCSI protocol SHOULD also provide a method for sessions to be
   gracefully terminated and restarted that can be initiated by either
   the initiator or target.  This provides the ability to gracefully
   fail over an initiator or target, or reset a target after performing
   maintenance tasks such as upgrading software.

Krueger             Informational   Exp. May 2001                  14

               iSCSI Reqmnts and Design Considerations      Nov. 2000




5. Interoperability

   It must be possible for initiators and targets that implement the
   required portions of the iSCSI specification to interoperate.  While
   this requirement is so obvious that it doesn't seem worth
   mentioning, if the protocol specification contains ambiguous
   wording, different implementations may not interoperate.  The iSCSI
   protocol document MUST be clear and unambiguous.


  5.1. Internet infrastructure

   The iSCSI protocol MUST:
    -- be compatible with both IPv4 and IPv6
    -- use TCP connections conservatively, keeping in mind there may be
       many other users of TCP on a given machine.

   The iSCSI protocol MUST NOT require changes to existing internet
   protocols


  5.2. SCSI

   In order to be considered a SCSI transport, the iSCSI standard must
   comply with the requirements of the SCSI Architecture Model [SAM2]
   for a SCSI transport.  Any feature SAM2 requires in a valid
   transport mapping MUST be specified by iSCSI and the specification
   SHOULD make such a feature either RECOMMENDED or REQUIRED in
   implementations.

   iSCSI is intended to be a new SCSI "transport" [SAM2].  As a mapping
   of SCSI over TCP, iSCSI requires interaction with both T10 and IETF.
   However, the iSCSI protocol MUST NOT require changes to the SCSI-3
   command sets and SCSI client code except to reflect lengthier iSCSI
   target names and potentially lengthier timeouts.  Collaboration with
   T10 will be necessary to achieve this requirement.  Storage
   attachment to IP networks will engender an unprecedented potential
   for device sharing.  This alone may impact future T10 work.

   The iSCSI protocol SHOULD track changes to SCSI and the SCSI
   Architecture Model.

   The iSCSI protocol MUST be capable of supporting all SCSI-3 command
   sets and device types. The primary focus is on supporting 'larger'
   devices: host computers and storage controllers (disk arrays, tape
   libraries).  However, other command sets (printers, scanners) MUST
   be supported.  These requirements MUST NOT be construed to mean that
   iSCSI must be natively implementable on all of today's SCSI devices,
   which might have limited processing power or memory.


Krueger             Informational   Exp. May 2001                  15

               iSCSI Reqmnts and Design Considerations      Nov. 2000



   ACA (Auto Contingent Allegiance) is an optional SCSI mechanism that
   stops execution of a sequence of dependent SCSI commands when one of
   them fails.  The situation surrounding it is complex - T10 specifies
   ACA in SAM2, and hence iSCSI must support it and endeavor to make
   sure that ACA gets implemented sufficiently (two independent
   interoperable implementations) to avoid dropping ACA in the
   transition from Proposed Standard to Draft Standard.  This implies
   iSCSI SHOULD support ACA implementation.  There is considerable
   discussion on the mailing list archives as to whether or not ACA is
   ever used by a SCSI application.

   The iSCSI protocol MUST allow for the construction of gateways to
   other SCSI transports, including parallel SCSI [SPI-X] and to SCSI-
   FCP[FCP, FCP-2].  It MUST be possible to construct "translating"
   gateways so that iSCSI hosts can interoperate with SCSI-X devices;
   so that SCSI-X devices can communicate over an iSCSI network; and so
   that SCSI-X hosts can use iSCSI targets (where SCSI-X refers to
   parallel SCSI, SCSI-FCP, or SCSI over any other transport).  This
   requirement is implied by support for SAM-2, but is worthy of
   emphasis. These are true application protocol gateways, and not just
   bridge/routers.  The different standards have only the SCSI-3
   command set layer in common.  These gateways are not mere packet
   forwarders.

   The iSCSI protocol MUST reliably transport SCSI commands from the
   initiator to the target.  According to [SAM-2, p. 17.] "The function
   of the service delivery subsystem is to transport an error-free copy
   of the request or response between the sender and the receiver"
   [SAM-2, p. 22]. The iSCSI protocol MUST correctly deal with iSCSI
   packet drop, duplication, corruption, stale packets, and re-
   ordering.

   iSCSI MUST support ordered delivery of SCSI commands from the
   initiator to the target, to support SCSI Task Queuing.


6. Security Considerations

   In the past, directly attached storage systems have implemented
   minimal security checks because the physical connection offered
   little chance for attack.   Transporting block storage (SCSI) over
   IP opens a whole new opportunity for a variety of malicious attacks.
   Attacks can take the active form (identity spoofing, man-in-the-
   middle) or the passive form (eavesdropping).


  6.1. Extensible Security

   The security services required for communications depends on the
   individual network configurations and environments.  Organizations
   are setting up Virtual Private Networks(VPN), also known as

Krueger             Informational   Exp. May 2001                  16

               iSCSI Reqmnts and Design Considerations      Nov. 2000


   Intranets, that will require one set of security functions for
   communications within the VPN and possibly many different security
   functions for communications outside the VPN to support
   geographically separate components.  The iSCSI protocol is
   applicable to a wide range of internetworking environments that may
   employ different security policies.  The protocol SHOULD require
   minimal configuration and overhead in the insecure operation,
   provide for strong authentication when increased security is
   required, and allow integration of new security mechanisms without
   breaking backwards compatible operation.


  6.2. Authentication

   The iSCSI protocol MAY support various levels of authentication
   security, ranging from no authentication to secure authentication
   using public or private keys.

   The iSCSI protocol MUST support private authenticated login.
   Authenticated login aids the target in blocking the unauthorized use
   of SCSI resources.  "Private" authenticated login mandates protected
   identity exchange (no clear text passwords at a minimum).  Since
   block storage privacy is considered critical in enterprises and many
   IP networks may have access holes, organizations will want to
   protect their IP SCSI resources.

   The iSCSI authenticated login MUST be resilient against passive
   attacks since many IP networks are vulnerable to packet inspection.
   Simple, US-exportable techniques exist to satisfy this requirement.

   In addition, the iSCSI protocol MUST NOT preclude optional data
   origin authentication of its communications. Data origin
   authentication is critical since IP networks are vulnerable to
   source spoofing, where a malicious third party can pretend to send
   packets from the initiator’s IP address.

   These requirements should be met using internet protocols such as
   IPsec or TLS. The endpoints may negotiate the authentication method,
   optionally none.


  6.3. Data Integrity

   The iSCSI protocol SHOULD NOT preclude use of additional data
   integrity protection protocols (IPSec, TLS).


  6.4. Data Privacy

   Block storage is used for storing sensitive information, where data
   privacy is critical.  An application may encrypt the data blocks


Krueger             Informational   Exp. May 2001                  17

               iSCSI Reqmnts and Design Considerations      Nov. 2000


   before writing them to storage - this provides the best protection
   for the application. Even if the storage or communications are
   compromised, the attacker will have difficulty reading the data.

   In certain environments, encryption may be desired to provide an
   extra assurance of privacy. An iSCSI implementation MAY use a data
   encryption protocol such as TLS or IPsec ESP to provide data privacy
   between iSCSI endpoints.


7. Management

   The iSCSI protocol layer SHOULD be manageable using IP-based
   management protocols (eg. SNMP, RMI).

   The iSCSI protocol document SHOULD NOT define the management
   architecture for iSCSI within the network infrastructure.


  7.1. Naming

   Whenever possible, iSCSI MUST support the naming architecture of
   SAM-2.  Deviations and uncertainties MUST be made explicit, and
   comments and resolutions worked out between ANSI T10 and the IPS
   working group.

   The means by which an iSCSI resource is located MUST use or extend
   existing internet standard resource location methods.  RFC 1783 [12]
   specifies URL syntax and semantics which should be sufficiently
   extensible for the iSCSI resource.

   The iSCSI protocol MUST provide a means of identifying an iSCSI
   storage device by a unique identifier that is independent of the
   path on which it is found.  This name will be used to correlate
   alternate paths to the same device.  The format for the iSCSI names
   MUST use existing naming authorities, to avoid creating new central
   administrative tasks.  An iSCSI name SHOULD be a human readable
   string in an international character set encoding.

   Note that LU names are discovered through SCSI-level inquiries, and
   are not just for Fibre Channel.  There is nothing to prevent iSCSI
   (or parallel SCSI) from implementing the LU WWN.  As such, this is
   outside the scope of the iSCSI protocol specification.

   Standard internet lookup services SHOULD be used to resolve names.
   For example, Domain Name Services (DNS) MAY be used to resolve the
   <hostname> portion of the URL to one or multiple IP addresses.  When
   a hostname resolves to multiple addresses, these addresses should be
   equivalent for functional (possibly not performance) purposes.  This
   means that the addresses can be used interchangeably as long as
   performance isn’t a concern.  For example, the same set of SCSI
   targets MUST be accessible from each of these addresses.

Krueger             Informational   Exp. May 2001                  18

               iSCSI Reqmnts and Design Considerations      Nov. 2000



   An iSCSI device naming scheme MUST interact correctly with the
   proposed SCSI security architecture [99-245r9].  Particular
   attention must be directed to the proxy naming architecture defined
   by the new security model.  In this new model,  a host is identified
   by an Access ID, and SCSI Logical Unit Numbers (LUNs) can be mapped
   in a manner that gives each AccessID a unique LU map.  Thus, a given
   LU within a target may be addressed by different LUNs.

   The iSCSI naming architecture MUST support SCSI 3rd party operations
   such as EXTENDED COPY.  The key issue here relates to the naming
   architecture for SCSI LUs - iSCSI must provide a means of passing a
   name or handle between parties. iSCSI must provide a name that could
   be used in the XCOPY command and fit within the available space
   allocated by that command.  And it must be possible, of course, for
   the XCOPY target (the third party) to dereference the name to the
   correct target and LU.


  7.2. Discovery

   iSCSI MUST have no impact on the use of current IP network discovery
   techniques.  Network management platforms discover IP addresses and
   have various methods of probing the services available through these
   IP addresses.  An iSCSI service should be evident using similar
   techniques.

   The iSCSI specifications MUST provide some means of determining
   whether an iSCSI service is available through an IP address.  It is
   expected that iSCSI will be a point of service in a host, just as
   SNMP, etc are points of services, associated with a well known port
   number.

   SCSI protocol-dependent techniques SHOULD be used for further
   discovery beyond the iSCSI layer.  Discovery is a complex, multi-
   layered process.  The SCSI protocol specifications provide specific
   commands for discovering LUs and the commands associated with this
   process will also work over iSCSI.

   The iSCSI protocol MUST provide a method of discovering, given an IP
   end point on its well-known port, the list of SCSI targets available
   to the requestor.  The use of this discovery service MUST be
   optional.

   Further discovery guidelines are outside the scope of this document
   and may be addressed in separate Informational drafts.







Krueger             Informational   Exp. May 2001                  19

               iSCSI Reqmnts and Design Considerations      Nov. 2000


8. Internet Accessibility

  8.1. Denial of Service

   As with all services, the denial of service by either incorrect
   implementations or malicious agents is always a concern.  All
   aspects of the iSCSI protocol SHOULD be scrutinized for potential
   denial of service issues, and guarded against as much as possible.


  8.2. NATs, Firewalls and Proxy servers

   NATs (Network Address Translator), firewalls, and proxy servers are
   a reality in today's internet.  These devices present a number of
   challenges to device access methods being developed for iSCSI.  For
   example, specifying a URL syntax for iSCSI resource connection
   allows an initiator to address an iSCSI target device both directly
   and through an iSCSI proxy server or NAT.  iSCSI SHOULD allow
   deployment where functional and optimizing middle-boxes such as
   firewalls, proxy servers and NATs are present.


   The iSCSI protocols use of IP addressing and TCP port numbers MUST
   be firewall friendly. This means that all connection requests should
   normally be addressed to a specific, well-known TCP port.  That way,
   firewalls can filter based on source and destination IP addresses,
   and destination (target) port number.  Additional TCP connections
   would require different source port numbers (for uniqueness), but
   could be opened after a security dialogue on the control channel.

   It’s important that iSCSI operate through a firewall to provide a
   possible means of defending against Denial of Service (DoS) assaults
   from less-trusted areas of the network.  It is assumed that a
   firewall will have much greater processing power for dismissing
   bogus connection requests than end nodes.


  8.3. Congestion Control and Transport Selection

   The iSCSI protocol MUST be a good network citizen with proven
   congestion control (as defined in RFC 2309). In addition, iSCSI
   implementations MUST NOT use multiple connections as a means to
   avoid transport-layer congestion control.


9. Definitions

   Certain definitions are offered here, with references to the
   original document where applicable, in order to clarify the
   discussion of requirements.  Definitions without references are the
   work of the authors and reviewers of this document.

Krueger             Informational   Exp. May 2001                  20

               iSCSI Reqmnts and Design Considerations      Nov. 2000



   Logical Unit (LU): A target-resident entity that implements a device
   model and executes SCSI commands sent by an application client [SAM-
   2, sec. 3.1.50, p. 7].

   Logical Unit Number (LUN): A 64-bit identifier for a logical unit
   [SAM-2, sec. 3.1.52, p. 7].

   SCSI Device:  A device that is connected to a service delivery
   subsystem and supports a SCSI application protocol [SAM-2, sec.
   3.1.78, p. 9].

   Service Delivery Port (SDP): A device-resident interface used by the
   application client, device server, or task manager to enter and
   retrieve requests and responses from the service delivery subsystem.
   Synonymous with port (SAM-2 sec. 3.1.61) [SAM-2, sec. 3.1.89, p. 9].

   Target: A SCSI device that receives a SCSI command and directs it to
   one or more logical units for execution [SAM-2 sec. 3.1.97, p. 10].

   Task: An object within the logical unit representing the work
   associated with a command or a group of linked commands [SAM-2, sec.
   3.1.98, p. 10].

   Transaction: A cooperative interaction between two objects,
   involving the exchange of information or the execution of some
   service by one object on behalf of the other [SAM-2, sec. 3.1.109,
   p. 10].


10.     References

   1  Bradner, S., "The Internet Standards Process -- Revision 3", BCP
      9, RFC 2026, October 1996.

   2  Bradner, S., "Key words for use in RFCs to Indicate Requirement
      Levels", BCP 14, RFC 2119, March 1997

   3 [SAM-2] ANSI NCITS.  Weber, Ralph O., editor.  SCSI Architecture
     Model -2 (SAM-2).  T10 Project 1157-D.  rev 13, 22 Mar 2000.

   4 [SPC-2] ANSI NCITS.  Weber, Ralph O., editor.  SCSI Primary
     Commands   2 (SPC-2).  T10 Project 1236-D.  rev 18, 21 May 2000.

   5 [CAM-3] ANSI NCITS.  Dallas, William D., editor.  Information
     Technology - Common Access Method - 3 (CAM-3)).  X3T10 Project
     990D.  rev 3, 16 Mar 1998.

   6 [99-245r8] Hafner, Jim.  A Detailed Proposal for Access Controls.
     T10/99-245 revision 8, 26 Apr 2000.

   7 [SPI-X] ANSI NCITS.  SCSI Parallel Interface - X.

   8 [FCP] ANSI NCITS.  SCSI-3 Fibre Channel Protocol [ANSI
     X3.269:1996].

   9 [FCP-2] ANSI NCITS.  SCSI-3 Fibre Channel Protocol - 2 [T10/1144-
     D].


Krueger             Informational   Exp. May 2001                  21

               iSCSI Reqmnts and Design Considerations      Nov. 2000


   10 Paxon, V. End-to-end internet packet dynamics, IEEE Transactions
     on Networking 7,3 (June 1999) pg 277-292.

   11 Stone J., Partridge, C. When the CRC and TCP checksum disagree,
     ACM Sigcomm (Sept. 2000).

   12 [RFC1783] Berners-Lee, t., et.al.,"Uniform Resource Locators", RFC
     1783, December 1994.



11.     Acknowledgements

   Special thanks to Julian Satran, IBM and David Black, EMC for their
   extensive review comments.


12.     Author's Addresses

   Address comments to:

   Marjorie Krueger
   Hewlett-Packard Corporation
   8000 Foothills Blvd
   Roseville, CA 95747-5668, USA
   Phone: +1 916 785-2656
   Email: marjorie_krueger@hp.com

   Randy Haagens
   Hewlett-Packard Corporation
   8000 Foothills Blvd
   Roseville, CA 95747-5668, USA
   Phone: +1 916 785-4578
   Email: Randy_Haagens@hp.com

   Costa Sapuntzakis
   Cisco Systems, Inc.
   170 W. Tasman Dr.
   San Jose, CA 95134, USA
   Phone: +1 408 525-5497
   Email: csapuntz@cisco.com

   Mark Bakke
   Cisco Systems, Inc.
   6450 Wedgwood Road
   Maple Grove, MN 55311
   Phone: +1 763 398-1054
   Email: mbakke@cisco.com





Krueger             Informational   Exp. May 2001                  22

               iSCSI Reqmnts and Design Considerations      Nov. 2000



Full Copyright Statement

   "Copyright (C) The Internet Society (2001). All Rights Reserved.
   This document and translations of it may be copied and furnished to
   others, and derivative works that comment on or otherwise explain it
   or assist in its implementation may be prepared, copied, published
   and distributed, in whole or in part, without restriction of any
   kind, provided that the above copyright notice and this paragraph
   are included on all such copies and derivative works. However, this
   document itself may not be modified in any way, such as by removing
   the copyright notice or references to the Internet Society or other
   Internet organizations, except as needed for the purpose of
   developing Internet standards in which case the procedures for
   copyrights defined in the Internet Standards process must be
   followed, or as required to translate it into languages other than
   English.

   The limited permissions granted above are perpetual and will not be
   revoked by the Internet Society or its successors or assigns.





























Krueger             Informational   Exp. May 2001                  23


Html markup produced by rfcmarkup 1.129c, available from https://tools.ietf.org/tools/rfcmarkup/