[Docs] [txt|pdf] [Tracker] [WG] [Email] [Nits] [IPR]

Versions: 00 01 02 03 04 05 06 07 08 09 10 11 12 13 14 15 16 17 18 19 20 RFC 3720

IPS                                                       Julian Satran
Internet Draft                                             Daniel Smith
Document: draft-ietf-ips-iscsi-00.txt                       Kalman Meth
Category: standards-track                                           IBM

                                                 Constantin Sapuntzakis
                                                          Cisco Systems

                                                           Matt Wakeley
                                                   Agilent Technologies

                                                      Paul Von Stamwitz
                                                                Adaptec

                                                          Randy Haagens
                                                    Hewlett-Packard Co.

                                                           Efri Zeidner
                                                                SANGate

                                                      Luciano Dalle Ore
                                                                Quantum

                                                            Yaron Klein
                                                                 SANRAD


                                 iSCSI










Julian Satran      Standards-Track, Expire May 2001                  1

                                iSCSI                  November, 2000



Status of this Memo


   This document is an Internet-Draft and is in full conformance with
   all provisions of Section 10 of RFC2026 [1].

   Internet-Drafts are working documents of the Internet Engineering
   Task Force (IETF), its areas, and its working groups. Note that other
   groups may also distribute working documents as Internet-Drafts.
   Internet-Drafts are draft documents valid for a maximum of six months
   and may be updated, replaced, or made obsolete by other documents at
   any time. It is inappropriate to use Internet- Drafts as reference
   material or to cite them other than as "work in progress."
   The list of current Internet-Drafts can be accessed at
   http://www.ietf.org/ietf/1id-abstracts.txt
   The list of Internet-Draft Shadow Directories can be accessed at
   http://www.ietf.org/shadow.html.


Abstract

   The Small Computer Systems Interface (SCSI) is a popular family of
   protocols for communicating with I/O devices, especially storage
   devices.  This memo describes a transport protocol for SCSI that
   operates on top of TCP.  The iSCSI protocol aims to be fully
   compliant with the requirements laid out in the SCSI Architecture
   Model - 2 [SAM2] document.

Acknowledgements

   Besides the authors a large group of people contributed through their
   review, comments and valuable insights to the creation of this
   document - too many to mention them all. Nevertheless, we are
   grateful to all of them.  We are especially grateful to those that
   found the time and patience to participate in our weekly phone
   conferences and intermediate meetings in Almaden and Haifa and thus
   helped shape this document: Jim Hafner, John Hufferd, Prasenjit
   Sarkar, Meir Toledano, John Dowdy, Steve Legg, Alain Azagury (IBM),
   Dave Nagle (CMU), David Black (EMC), John Matze (Veritas), Mark
   Bakke, Steve DeGroote, Mark Shrandt (NuSpeed), Gabi Hecht (Gadzoox),
   Robert Snively (Brocade), Nelson Nachum (StorAge).  Many more helped
   clean and improve this document within the IPS working group. We are
   especially grateful to David Robinson (Sun), Charles Monia, Joshua
   Tseng (Nishan), Somesh Gupta, Mallikarjun C., Michael Krause (HP),
   Stephen Byan (Genroco). And last but not least thanks Ralph Weber for
   keeping us in-line with T10 (SCSI) standardization.

Satran                Standards-Track, May 2001                     2

                                iSCSI                  November, 2000






Conventions used in this document


   In examples, "I->" and "T->" indicate iSCSI PDUs sent by the
   initiator and target respectively.

   The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
   "SHOULD", "SHOULD NOT", "RECOMMENDED",  "MAY", and "OPTIONAL" in this
   document are to be interpreted as described in RFC-2119 [2].







































Satran                Standards-Track, May 2001                     3

                                iSCSI                  November, 2000


1. Overview

1.1 SCSI Concepts

   The SCSI Architecture Model-2 [SAM2] describes in detail the
   architecture of the SCSI family of I/O protocols. This section
   provides a brief background to situate readers in the vocabulary of
   the SCSI architecture.

   At the highest level, SCSI is a family of interfaces for requesting
   services from I/O devices, including hard drives, tape drives, CD and
   DVD drives, printers, and scanners. In SCSI parlance, an individual
   I/O device is called a ôlogical unitö.

   SCSI is a client-server architecture. Clients of a SCSI interface are
   called ôinitiatorsö. Initiators issue SCSI ôcommandsö to request
   service from a logical unit. The ôdevice serverö on the logical unit
   accepts SCSI commands and executes them.

   A ôSCSI transportö maps the client-server SCSI protocol to a specific
   interconnect. Initiators are one endpoint of a SCSI transport. The
   ôtargetö is the other endpoint. A ôtargetö can have multiple LUs
   behind it. Each logical unit has a number called a LUN.

   A SCSI task is a SCSI command or possibly a linked set of SCSI
   commands. Some LUNs support multiple pending (queued) tasks. The
   queue of tasks is managed by the target, though. The target uses an
   initiator provided "task tag" to distinguish between tasks. Only one
   command in a task can be outstanding at any given time.

   Each SCSI command results in an optional data phase and a response
   phase. In the data phase, information can travel from the initiator
   to target (e.g. WRITE), target to initiator (e.g. READ), or in both
   directions. In the response phase, the target returns the final
   status of the operation, including any errors. A response terminates
   a SCSI command.

   Command Data Blocks (CDB) are the data structures used to contain the
   command parameters to be handed by an initiator to a target. The CDB
   content and structure is defined by [SAM] and device class specific
   SCSI standards.


1.2 iSCSI Concepts &  Functional Overview

   The iSCSI protocol is a mapping of the SCSI RPC model on top of the
   TCP protocol.

Satran                Standards-Track, May 2001                     4

                                iSCSI                  November, 2000



   In keeping with similar protocols, the initiator and target divide
   their communications into messages. This document will use the term
   ôiSCSI protocol data unitö (iSCSI PDU) for these messages.

1.2.1 Layers & Sessions

   The following conceptual layering model is used in this document to
   specify initiator and target actions and how those relate to
   transmitted and received Protocol Data Units:

      -the SCSI layer builds/receives SCSI CDB (Command Data Blocks)
      and relays/receives them with the remaining command execute
      parameters (cf. SAM-2) to/from the
      -the iSCSI layer that builds/receives iSCSI PDUs and
      relays/receives them to/from - one or more TCP connections that
      form an initiator-target "session".

   Communication between initiator and target occurs over one or more
   TCP connections.  The TCP connections carry control messages, SCSI
   commands, parameters and data within iSCSI Protocol Data Units (iSCSI
   PDUs).  The group of TCP connections linking an initiator with a
   target form a session (loosely equivalent to a SCSI I-T nexus). A
   session is defined by a session ID (composed of an initiator part and
   a target part). TCP connections can be added and removed from a
   session.  Connections within a session are identified by a connection
   ID (CID). Across all connections within a session an initiator will
   see one "target image" - all target identifying elements like LUN are
   the same. Also across all connections within a session a target will
   see one "initiator image" - all initiator identifying elements like
   Initiator Task Tag are the same.

   An iSCSI target MUST support at least one TCP connection.  An iSCSI
   initiator SHOULD support several connections in a session.

1.2.2 Ordering and iSCSI numbering

   iSCSI uses Command, Status and Data numbering schemes.

   Command numbering is session wide and is used for ordered command
   delivery over multiple connections.  It can also be used as a
   mechanism for command flow control over a session.

   Status numbering is per connection and is used to enable recovery
   in case of connection failure.



Satran                Standards-Track, May 2001                     5

                                iSCSI                  November, 2000


   Data numbering is per command and is meant to reduce the amount of
   memory needed by a target sending unrecoverable data for command
   restart.

   Normally, fields in the iSCSI PDUs communicate the reference numbers
   between the initiator and target.  During periods when traffic on a
   connection is unidirectional, iSCSI NOP-message PDUs may be utilized
   to synchronize the command and status ordering registers of the
   target and initiator.

   iSCSI NOP command PDUs are used as acknowledgements for data
   numbering.


1.2.2.1 Command numbering

   iSCSI supports ordered command delivery within a session.  All
   commands (initiator-to-target) and responses (target-to-initiator)
   are numbered.  Any SCSI activity is related to a task (SAM-2). The
   task is identified by the Initiator Task Tag for the life of the
   task.  Commands in transit from the initiator SCSI layer to the
   target SCSI layer are numbered by iSCSI and the number is carried by
   the iSCSI PDU as CmdRN (Command-Reference-Number).
   The numbering is session-wide.  All iSCSI PDUs that have a task
   association carry this number. CmdRNs are allocated by the initiator
   iSCSI within a 32 bit unsigned counter (modulo 2**32).  The value 0
   is reserved and used to mean immediate delivery. Comparisons and
   arithmetic on CmdRN SHOULD use Serial Number Arithmetic as defined in
   [RFC1982] where SERIAL_BITS = 32.
   The target may choose to deliver some task management commands for
   immediate delivery.  The means by which the SCSI layer may request
   immediate delivery for a command or by which iSCSI will decide by
   itself to mark a PDU for immediate delivery are outside the scope of
   this document.
   CmdRNs are significant only during command delivery to the target.
   Once the device serving part of the target SCSI has received a
   command, CmdRN ceases to be significant.  During command delivery to
   the target, the allocated numbers are unique session wide.  The
   initiator and target are assumed to have three registers that define
   the allocation mechanism - CmdRN - the current command reference
   number advanced by 1 on each command shipped; ExpCmdRN - the next
   expected command by the target - acknowledges all commands up to it;
   MaxCmdRN - the maximum number to be shipped - MaxCmdRN - ExpCmdRN
   defines the queuing capacity of the receiving iSCSI layer.
   The target SHOULD NOT transmit a MaxCmdRN that is more than 2**31 - 1
   above the last ExpCmdRN.  CmdRN can take any value from ExpCmdRN to


Satran                Standards-Track, May 2001                     6

                                iSCSI                  November, 2000


   MaxCmdRN except 0. The target MUST silently ignore any command
   outside this range or duplicates within the range not flagged with
   the retry bit (the X bit in the opcode).  The target and initiator
   registers MUST uphold causal ordering.

   iSCSI initiators MUST implement the command/request numbering scheme
   only if they support more than one connection per session (as even
   sessions with a single connection may be expanded beyond one
   connection).

   Command numbering for sessions that will only be made up of one
   connection is optional. iSCSI initiators utilizing a single
   connection for a session and not utilizing command numbering MUST
   indicate that they will not support command numbering by setting
   InitCmdRN to 0 in Login command.

   Whenever an initiator indicates support for command numbering, by
   setting InitCmdRN to a non-zero value at Login, the target MUST
   provide ExpCmdRN and MaxCmdRN values that will enable the initiator
   to make progress.


1.2.2.2 Response/Status numbering

   Responses in transit from the target to the initiator are numbered.
   The StatRN (Status Reference Number) is used for this purpose. StatRN
   is a counter maintained per connection.  ExpStatRN is used by the
   initiator to acknowledge status.
   To enable command recovery the target MAY maintain enough state to
   enable data and status recovery after a connection failure.
   A target can discard all the state information maintained for
   recovery after the status delivery is acknowledged through ExpStatRN.
   A large difference between StatRN and ExpStatRN may indicate a failed
   connection.

   Initiators and Targets MUST support the response-numbering scheme
   regardless of the support for command recovery.

1.2.2.3 Data PDU numbering

   Incoming Data PDUs MAY be numbered by a target to enable fast
   recovery of long running READ commands.
   Data PDUs are numbered with DataRN.  NOP command PDUs carrying the
   same Initiator Tag as the Data PDUs are used to acknowledge the
   incoming Data PDUs with ExpDataRN.  Support for Data PDU
   acknowledgement and the maximum number of unacknowledged data PDUs
   are negotiated at login.

Satran                Standards-Track, May 2001                     7

                                iSCSI                  November, 2000


   In a PDU carrying both data and status, the field is used for StatRN
   and the last set of data blocks is implicitly acknowledged when
   Status is acknowledged.

1.2.3 Timers and timeouts

   Initiators MUST implement the following timers:

      - T1 - Command delivery timer
      - T2 - Status delivery timer
      - T3 - Data delivery timer

   The T1 timer is started when a SCSI command or task management PDU is
   sent to a target and is reset by the delivery being acknowledged or
   the first data block is received or the status is received.

   The T2 timer is started when the last outgoing iSCSI PDU of a command
   is sent and reset when status is received.

   The T3 timer is started/restarted after an incoming PDU is received
   and reset when status is received.

   The timer values are target and link dependent and are negotiated at
   login.

   At any timer expiration (timeout) the initiator MUST resend the
   command or task management PDU with the restart bit set.

   Timers support recovery for gateways between an iSCSI transport and
   an unreliable transport (e.g., FCP).


1.2.4 iSCSI Login

   The purpose of iSCSI login is to enable a TCP connection for iSCSI
   use, authenticate the parties, negotiate the sessionÆs parameters,
   open a security association protocol and mark the connection as
   belonging to an iSCSI session.

   A session is used to identify to a target all the connections with a
   given initiator. The targets listen on a well-known TCP port for
   incoming connections. The initiator begins the login process by
   connecting to that well-known TCP port.

   As part of the login process, the initiator and target MAY wish to
   authenticate each other and set a security association protocol for
   the session. This can occur in many different ways and is subject to

Satran                Standards-Track, May 2001                     8

                                iSCSI                  November, 2000


   negotiation. Negotiation and security associations executed before
   the Login Command are outside the scope of this document although
   they might realize a related function (e.g., establish a IPsec or TLS
   session). The Login Command starts the iSCSI Login Phase. Within the
   Login Phase, negotiation is carried on through parameters of the
   Login Command and Response and optionally through intervening Text
   Commands and Responses. The Login Response concludes the Login Phase.
   Once suitable authentication has occurred, the target MAY authorize
   the initiator to send SCSI commands. How the target chooses to
   authorize an initiator is beyond the scope of this document. The
   target indicates a successful authentication and authorization by
   sending a login response with "accept login". Otherwise, it sends a
   response with a ôlogin rejectö, indicating a session is not
   established.
   It is expected that iSCSI parameters will be negotiated after the
   security association protocol is established if there is a security
   association.


   The login message includes a session ID - composed with an initiator
   part ISID and a target part TSID. For a new session, the TSID is
   null. As part of the response, the target will generate a TSID.
   Session specific parameters can be specified only for the first login
   of a session (TSID null)(e.g., the maximum number of connections that
   can be used for this session). Connection specific parameters (if
   any) can be specified for any login. Thus, a session is operational
   once it has at least one connection.

   Any message except login and text sent on a TCP connection before
   this connection gets into full feature phase at the initiator SHOULD
   be ignored by the initiator. Any message except login and text
   reaching a target on a TCP connection before the full feature phase
   MUST be silently ignored by the target.

1.2.5 Text mode negotiation

   During login and thereafter some session or connection parameters are
   negotiated through an exchange of textual information.

   In negotiation, the offering party will send a list of values for a
   key in its order of preference.

   The responding party will answer with a value from the list.





Satran                Standards-Track, May 2001                     9

                                iSCSI                  November, 2000


   The value "none" MUST always be used to indicate a missing function.
   However, none is a valid selection only if it was explicitly offered
   and it MAY be selected by omission (i.e. <key>:none MAY be omitted).

   The general format is:

      Offer-> <key>:(<value1>,<value2>,...,<valuen>)
      Answer-> <key>:<valuex>


1.2.6 iSCSI Full Feature Phase

   Once the initiator is authorized to do so, the iSCSI session is in
   iSCSI full feature phase. The initiator may send SCSI commands and
   data to the various LUNs on the target by wrapping them in iSCSI
   messages that go over the established iSCSI session.  For SCSI
   commands that require data and/or parameter transfer, the (optional)
   data and the status for a command must be sent over the same TCP
   connection that was used to deliver the SCSI command (we call this
   "connection allegiance").  Thus if an initiator issues a READ
   command, the target must send the requested data, if any, followed by
   the status to the initiator over the same TCP connection that was
   used to deliver the SCSI command.  If an initiator issues a WRITE
   command, the initiator must send the data, if any, for that command
   and the target MUST return the status over the same TCP connection
   that was used to deliver the SCSI command.  However consecutive
   commands that are part of a SCSI linked commands task MAY use
   different connections - connection allegiance is strictly per-command
   and not per-task. During iSCSI Full Feature Phase, the initiator and
   target MAY interleave unrelated SCSI commands, their SCSI Data and
   responses, over the session.  Outgoing SCSI data (initiator to target
   - user data or command parameters) is sent as either unsolicited data
   or solicited data.  Unsolicited data can be part of an iSCSI command
   PDU ("immediate data") or an iSCSI data PDU.  An initiator may send
   only one unsolicited data item (immediate or in a separate PDU) - all
   subsequent data items have to be solicited.  Solicited data are sent
   in response to Ready To Transfer (R2T) PDUs.  Targets operate in
   either solicited (R2T) data mode or unsolicited (non R2T) data mode.
   An initiator MUST always honor an R2T data request.  It is considered
   an error for an initiator to send unsolicited data PDUs to a target
   operating in R2T mode (only solicited data).  It is also an error for
   an initiator to send more than one unsolicited data PDU (whether
   immediate or as a separate PDU).  An initiator MAY request, at login,
   to send immediate data blocks of any size. If the initiator requests
   a specific block size the target MUST indicate the size of immediate
   data blocks it is ready to accept in its response.  Beside iSCSI,
   SCSI also imposes a limit on the amount of unsolicited data a target

Satran                Standards-Track, May 2001                    10

                                iSCSI                  November, 2000


   is willing to accept. The iSCSI immediate data limit MUST not exceed
   the SCSI limit.

   A target SHOULD NOT silently discard data and request retransmission
   through R2T.  Initiators MUST NOT perform any score boarding for data
   and the residual count calculation is to be performed by the targets.
   Incoming data is always solicited. SCSI Data packets are matched to
   their corresponding SCSI commands by using Tags that are specified in
   the protocol.

   Initiator tags for pending commands are unique initiator-wide for a
   session.  Target tags for pending commands are unique LU-wide for the
   session; together with the LUN they form a target-wide unique
   composite tag for a session.  The above mechanisms are designed to
   accomplish efficient data delivery and a large degree of control over
   the data flow.  iSCSI initiators and targets MUST also enforce some
   ordering rules to achieve deadlock-free operation.
   Unsolicited data MUST be sent on every connection in the same order
   in which commands were sent. If the amount of data exceeds the amount
   allowed for unsolicited write data, the specific connection MUST be
   stalled - no new data will be sent on the specific connection until
   initiator receives an R2T iSCSI PDU from the target.  A target
   receiving data out of order or observing a connection violating the
   above rules MUST terminate the session.

   Each iSCSI session to a target is treated as if it originated from a
   different initiator.

1.2.7 iSCSI Connection Termination

   Connection termination is assumed an exceptional event.
   Graceful TCP connection shutdowns are done by sending TCP FINs.
   Graceful connection shutdowns MUST only occur when there are no
   outstanding tasks that have allegiance to the connection.  A target
   SHOULD respond rapidly to a FIN from the initiator by closing it's
   half of the connection as soon as it has finished all outstanding
   tasks that have allegiance to the connection.  Connection termination
   with outstanding tasks may require recovery actions.

   Connection termination is also required as prelude to recovery.  By
   terminating a connection before starting recovery, initiator and
   target can avoid having stale PDUs being received after recovery.  In
   this case, the initiator will send a LOGOUT request on any of the
   operational connections of a session indicating what connection
   should be terminated.



Satran                Standards-Track, May 2001                    11

                                iSCSI                  November, 2000



1.2.8 Naming & mapping

   Text string names are used in iSCSI to:

      - provide explicitly a transportID for the target to enable the
      later to recognize the initiator because the conventional IP-
      address and port tuple is inaccurate behind firewalls and NAT
      devices (key - initiator)
      - provide a target selector - targetID for simple
      configurations hiding several targets behind an IP-address and
      port (key - target)
      - provide a symbolic address for source and destination targets
      in third party commands (through the map command)

   The targetID MUST be presented within the login phase.

   The names do not require handling within iSCSI  - i.e. are opaque
   entities within this document.  In order to enable implementers to
   relate them to other names and name handling mechanisms the following
   syntax for names SHOULD be used

      <domain-name>[/modifier]

   Where domain-name follows DNS rules and the modifier is an
   alphanumeric string (N.B. the whole pattern follows the URL
   structure)

   Some mapped names for third party command use might have to include a
   port number.  For those the following syntax SHOULD be used:

      <domain-name>[[/modifier]:[port]]

   The text to address transformation, wherever needed, will be
   performed through available name translation services (DNS servers,
   LDAP accessible directories etc.)

   To enable simple devices to operate without name-to-address
   conversion services the following conventions SHOULD be used:

      A domain name that contains exactly four numbers separated by
      dots (.), where each number is in the range 0 through 255, will
      be interpreted as an IPv4 address.
      A domain name that contains more than four, but at most 16
      numbers separated by dots (.), where each number is in the
      range 0 through 255, will be interpreted as an Ipv6 address.


Satran                Standards-Track, May 2001                    12

                                iSCSI                  November, 2000


   Examples of IPv4 addresses/names:

      10.0.0.1/diskfarm1
      10.0.0.2

   Examples of IPv6 addresses/names


      12.5.7.10.0.0.1/tapefarm1
      12.5.6.10.0.0.2


   For management/support tools as well as naming services that use a
   text prefix to express the protocol intended (as in http:// or
   ftp://) the following form MAY be used:

      iSCSI://<domain-name>[[/modifier]:[port]]

   Examples:


      iSCSI://diskfarm1.acme.com
      iSCSI://computingcenter.acme.com/diskfarm1



   When a target has to act as an initiator for a third party command,
   it MAY use the initiator name it learned during login as required by
   the authentication mechanism to the third party.

   To address targets and logical units within a target, SCSI uses a
   fixed length (8 bytes) uniform addressing scheme; in this document,
   we call those addresses SCSI reference addresses (SRA).

   To provide the target with the protocol specific addresses (iSCSI or
   FC) iSCSI uses a Map Command; the Map command sends the managing
   target the protocol specific addresses and gets from the target the
   SRAs to use in subsequent commands.  For iSCSI, a protocol specific
   address is a TCP address and a modifier.  After mapping, iSCSI will
   be provided with a handle to the address in standard SCSI format.

1.2.9 Message Framing

   iSCSI presents an mapping of the SCSI protocol onto TCP.  This
   encapsulation is accomplished by sending iSCSI PDUs that are of
   varying length. Unfortunately, TCP does not have a built-in mechanism


Satran                Standards-Track, May 2001                    13

                                iSCSI                  November, 2000


   for signaling message boundaries at the TCP layer.  iSCSI overcomes
   this obstacle by placing the message length in the iSCSI message
   header. This serves to delineate the end of the current message as
   well as the beginning of the next message.

   In situations where IP packets are delivered in-order from the
   network, message framing is not an issue (messages are processed one
   after the other). In the presence of IP packet reordering (mostly due
   to frames being dropped), it is best to minimize the dependencies
   between TCP segments, to enable as much processing of the received
   out-of-order segments as possible. Such processing ensures that data
   can be copied to correct buffers the first time, decreasing the need
   for dedicated reassembly buffers as well as the latency and bandwidth
   related to extra copies.

   Another area where it is extremely helpful to delineate iSCSI
   messages is when using a protocol analyzer to monitor or debug an
   iSCSI session.  Typically, an analyzer will not be snooping
   continuously from the time the session is established, and thus it
   will not be "aligned" with the iSCSI messages.  An iSCSI message
   delimiter would enable the analyzer to discover and decode iSCSI
   messages.

   Unfortunately, when relying solely on the "message length in the
   iSCSI message" scheme to delineate iSCSI messages, a missing TCP
   segment that contains an iSCSI message header (with the message
   length) makes it impossible to find message boundaries in subsequent
   TCP segments. The missing TCP segment must be received before any
   following segments can be processed.

   The iSCSI protocol uses the urgent bit in the TCP header to delineate
   iSCSI messages. The first byte of every iSCSI message MUST be marked
   "urgent".  The result is the TCP urgent pointer will point to the
   first byte of the iSCSI message in the TCP segment.

   When a large iSCSI message is sent, the first TCP segment will
   contain the iSCSI header, but the remaining TCP segments will not
   contain any iSCSI framing information.  To minimize the amount of
   buffering required when an iSCSI header is lost, it is recommended
   that the iSCSI Data PDU size be restricted to a small value (perhaps
   a few TCP segments in length).

   There are differing interpretations of whether the Urgent pointer
   points to the last (only) byte of urgent data (as defined by
   RFC1122), or the byte after the urgent data (typically BSD
   implementations). iSCSI has implemented a mechanism to resolve which


Satran                Standards-Track, May 2001                    14

                                iSCSI                  November, 2000


   interpretation is being used on the data received. Bit 7 in the first
   byte of the iSCSI message (F bit in the opcode field) that shall
   always be zero.  Bit 7 in the following byte (opcode specific fields)
   shall always be one.  When an iSCSI implementation receives an out of
   order TCP segment with the Urgent pointer defined, it shall look at
   the byte pointed to by the Urgent pointer.  If the bit is clear, the
   sender is RFC1122 compliant.  If the bit is set, the sender has
   implemented the BSD interpretation, and must "back up" one byte to
   find the beginning of the iSCSI message.











































Satran                Standards-Track, May 2001                    15

                                iSCSI                  November, 2000


2. iSCSI PDU Formats

   All multi-byte integers specified in formats defined in this document
   are to be represented in network byte order (i.e., big endian).  Any
   bits not defined should be set to zero.

2.1 Template Header and Opcodes

   All iSCSI PDUs  begin with a 48-byte header. Additional data appears,
   as necessary, beginning with byte 48. The fields of Opcode and Length
   appear in all iSCSI PDUs. In addition, the Initiator Task tag,
   Logical Unit Number, and Flags fields, when used, always appear in
   the same location in the header.



   Byte /    0       |       1       |       2       |       3       |
      /              |               |               |               |
     |7 6 5 4 3 2 1 0|7 6 5 4 3 2 1 0|7 6 5 4 3 2 1 0|7 6 5 4 3 2 1 0|
     +---------------+---------------+---------------+---------------+
    0|F| Opcode      |1|X| Opcode-specific fields                    |
     |               | |P|                                           |
     +---------------+---------------+---------------+---------------+
    4| Length of Data (after 48 byte Header)                         |
     +---------------+---------------+---------------+---------------+
    8| LUN or Opcode-specific fields                                 |
     +                                                               +
   12|                                                               |
     +---------------+---------------+---------------+---------------+
   16| Initiator Task Tag or Opcode-specific fields                  |
     +---------------+---------------+---------------+---------------+
   20/ Opcode-specific fields                                        /
    +/                                                               /
     +---------------+---------------+---------------+---------------+
   48| Header digest (optional-constant-length)                      |
     +---------------------------------------------------------------+
   +n/                                                               /
    +/ Data (optional)                                               /
     +---------------------------------------------------------------+
    m| Data digest (optional-variable-length)    |
     +---------------------------------------------------------------+

2.1.1 F bit

   If set to 1 indicates BSD semantic for the urgent pointer.
   If set to 0 indicates RFC 1122 semantic for the urgent pointer.


Satran                Standards-Track, May 2001                    16

                                iSCSI                  November, 2000




2.1.2 Opcode

   The Opcode indicates what type of iSCSI PDU the header encapsulates.
   The Opcode is further encoded as follows:

      b6   Response
      b5-0 Operation

   The opcodes are divided into two categories: initiator opcodes and
   target opcodes. Initiator opcodes are in PDUs sent by the initiators,
   and target opcodes are in PDUs sent by the target. The initiator MUST
   NOT send target opcodes and the target MUST NOT send initiator
   opcodes.  Target opcodes are also called responses and are
   distinguished by having the Response bit (bit 6) set to 1.

   Valid initiator opcodes defined in this specification are:


      0x00 NOP-Out Message (from initiator to target)
      0x01 SCSI Command (encapsulates a SCSI Command Descriptor
      Block)
      0x02 SCSI Task Management Command
      0x03 Login Command
      0x04 Text Command
      0x05 SCSI Data (for WRITE operation)
      0x06 NOP Command (from initiator to target)
      0x07 Map Command
      0x08 Logout Command


   Valid target opcodes are:


      0x40 NOP-In Message (from target to initiator)
      0x41 SCSI Response (contains SCSI status and possibly sense
      information or other response information)
      0x42 SCSI Task Management Response
      0x43 Login Response
      0x44 Text Response
      0x45 SCSI Data (for READ operation)
      0x46 NOP Response (from target to initiator)
      0x47 Map Response
      0x48 Logout Response



Satran                Standards-Track, May 2001                    17

                                iSCSI                  November, 2000


      0x50 Ready To Transfer (R2T - sent by target to initiator when
      it is ready to receive data from initiator)
      0x51 Asynchronous Event (sent by target to initiator to
      indicate certain special conditions)
      0x7f Opcode Not Understood

2.1.3 Opcode-specific fields

   These fields have different meanings for different messages.
   Bit 7 of the second byte MUST be 1 and bit 6 of the second byte is
   used as a retry indicator for commands (X bit) or Poll bit and must
   be 0 in all other iSCSI PDUs

2.1.4 Length

   The Length field indicates the number of bytes, beyond the first 48
   bytes, that are being sent together with this message header. The
   length includes the header and data digests if any. It is anticipated
   that most iSCSI PDUs (not counting data transfer PDUs) will not need
   more than the 48 byte header.


2.1.5 LUN

   Some opcodes operate on a specific Logical Unit. The Logical Unit
   Number (LUN) field identifies which  Logical Unit..  If the opcode
   does not relate to a Logical Unit, this field either is ignored or
   may be used for some other purpose.  The LUN field is 64-bits in
   accordance with [SAM2]. The exact format of this field can be found
   in the [SAM2] document.

2.1.6 Initiator Task Tag

   The initiator assigns a Task Tag to each SCSI task that it issues.
   This tag is a session-wide unique identifier that can be used to
   uniquely identify the Task.

   To enable gateways to older networks to operate without retaining
   per/LU state the target may specify, during the login phase, the use
   of a limited number of bits within the Initiator Task Tag. Those will
   be the least significant n-bits of the Initiator Task Tag. For
   example:

      tag:16

   means that only the last 16 bits of the Initiator Task Tag will be
   used (the first 16 have to be 0).

Satran                Standards-Track, May 2001                    18

                                iSCSI                  November, 2000



   Even when using a limited number of bits in the Initiator Task Tag it
   has to remain an session-wide unique identifier.


2.1.7 Header Digest and Data Digest

   Optional header and data digests protect the integrity and
   authenticity of header and data, respectively. The digests, if
   present, appear as trailers located, respectively, after the header
   and PDU-specific data.

   The digest type and length are negotiated during the login phase.

   The separation of the header and data digests is useful in iSCSI
   routing applications, where only the header changes when a message is
   forwarded. In this case, only the header digest should be re-
   calculated.

   Note that the digest size may vary. For example, as CRC32 is
   efficient for small data segments (2K), this iSCSI enables to use a
   CRC scheme including a CRC32 for every 2K of data. The receiver
   should be aware of the variable size.




























Satran                Standards-Track, May 2001                    19

                                iSCSI                  November, 2000



2.2 SCSI Command

   Byte /    0       |       1       |       2       |       3       |
      /              |               |               |               |
     |7 6 5 4 3 2 1 0|7 6 5 4 3 2 1 0|7 6 5 4 3 2 1 0|7 6 5 4 3 2 1 0|
     +---------------+---------------+---------------+---------------+
    0|F|  0x01       |1|X|R|W|0|ATTR | Reserved (0)  | AddCDB
     +---------------+---------------+---------------+---------------+
    4| Length                                                        |
     +---------------+---------------+---------------+---------------+
    8| Logical Unit Number (LUN)                                     |
     +                                                               +
   12|                                                               |
     +---------------+---------------+---------------+---------------+
   16| Initiator Task Tag                                            |
     +---------------+---------------+---------------+---------------+
   20| Expected Data Transfer Length                                 |
     +---------------+---------------+---------------+---------------+
   24| CmdRN                                                         |
     +---------------+---------------+---------------+---------------+
   28| ExpStatRN                                                     |
     +---------------+---------------+---------------+---------------+
   32/ SCSI Command Descriptor Block (CDB)                           /
    +/                                                               /
     +---------------+---------------+---------------+---------------+
   48/ Command Data (Command Dependent)                              /
    +/                                                               /
     +---------------+---------------+---------------+---------------+


2.2.1 Flags & Task Attributes

      The flags field for a SCSI Command is:


      b7   1   MUST be 1 for framing
      b6   Retry (X)
      b5   (R) set to 1 when input data is expected
      b4   (W) set to 1 when output data is expected
      b3   Reserved (MUST be 0)
      b0-2 used to indicate Task Attributes

   The Task Attributes (ATTR) can have one of the following integer
   values (see [SAM2] for details):

      0    Untagged

Satran                Standards-Track, May 2001                    20

                                iSCSI                  November, 2000


      1    Simple
      2    Ordered
      3    Head of Queue
      4    ACA

2.2.2 AddCDB

   Additional CDB length (over 16) in units of 4 bytes.

2.2.3 CmdRN - Command Reference Number

   Enables ordered delivery across multiple connections in a single
   session.

2.2.4 ExpStatRN - Expected Status Reference Number

   Command responses up to ExpStatRN -1 (mod 2**32) have been received
   (acknowledges status) on the connection.

2.2.5 Expected Data Transfer Length

   For unidirectional operations, the Expected Data Transfer Length
   field states the number of bytes of data involved in this SCSI
   operation.  For a WRITE operation, the initiator uses this field to
   specify the number of bytes of data it expects to transfer for this
   operation.  For a READ operation, the initiator uses this field to
   specify the number of bytes of data it expects the target to transfer
   to the initiator.  It corresponds to the SAM-2 byte count.

   For bi-directional operations, this field states the number of data
   bytes involved in the outbound transfer. For bi-directional
   operations, an additional field indicating the Expected Bidi-Read
   Data Transfer Length is following the (possibly extended) CDB as
   shown bellow:

     +---------------+---------------+---------------+---------------+
   48/ Additional CDB (if any)                                       /
    +/                                                               /
     +---------------+---------------+---------------+---------------+
   +n| Expected Bidi-Read Data Transfer Length                       |
     +---------------------------------------------------------------+
   +4/ Immediate data (optional)                                     /
     /                                                               /
     +---------------------------------------------------------------+




Satran                Standards-Track, May 2001                    21

                                iSCSI                  November, 2000






   If no data will be transferred in SCSI Data packets for this SCSI
   operation, this field should be set to zero.

   Upon completion of a data transfer, the target will inform the
   initiator of how many bytes were actually processed (sent or
   received) by the target.

2.2.6 CDB - SCSI Command Descriptor Block

   There are 16 bytes in the CDB field to accommodate the largest
   currently defined CDB.  Whenever larger CDBs are used, the CDB
   spillover MAY extend beyond the 48-byte header.

2.2.7 Command-Data

   Some SCSI commands require additional parameter data to accompany the
   SCSI command. This data may be placed beyond the 48-byte boundary of
   the iSCSI header.  Alternatively, user data (as from a WRITE
   operation) can be placed in the same PDU (both cases referred to as
   immediate data).

2.3 SCSI Response

   Byte /    0       |       1       |       2       |       3       |
      /              |               |               |               |
     |7 6 5 4 3 2 1 0|7 6 5 4 3 2 1 0|7 6 5 4 3 2 1 0|7 6 5 4 3 2 1 0|
     +---------------+---------------+---------------+---------------+
    0|F|  0x41       |1|Rsvd |o|u|O|U| Reserved (0)                  |
     +---------------+---------------+---------------+---------------+
    4| Length                                                        |
     +---------------+---------------+---------------+---------------+
    8| Reserved (0)                                                  |
     +                                                               +
   12|                                                               |
     +---------------+---------------+---------------+---------------+
   16| Initiator Task Tag                                            |
     +---------------+---------------+---------------+---------------+
   20| Basic Residual Count                                          |
     +---------------+---------------+---------------+---------------+
   24| StatRN                                                        |
     +---------------+---------------+---------------+---------------+
   28| ExpCmdRN                                                      |
     +---------------+---------------+---------------+---------------+

Satran                Standards-Track, May 2001                    22

                                iSCSI                  November, 2000


   32| MaxCmdRN                                                      |
     +---------------+---------------+---------------+---------------+
   36| Command Status| Reserved (0)                                  |
     +---------------+---------------+---------------+---------------+
   40| Resp_length                   | Sense_length                  |
     +---------------+---------------+---------------+---------------+
   44| Bidi-Read Residual Count                                      |
     +---------------+---------------+---------------+---------------+
   48/ Response and/or sense Data (optional)                         /
    +/                                                               /
     +---------------+---------------+---------------+---------------+

2.3.1 Byte 1 - Flags

      b0   (U) set for Residual Underflow. In this case, the Basic
      Residual Count indicates how many bytes were not transferred
      out of those expected to be transferred.
      b1   (O) set for Residual Overflow. In this case, the Bsic
      Residual Count indicates how many bytes could not be
      transferred because the initiator's Expected Data Transfer
      Length was too small.
      b2   (u) same as b0 but for the read-part of a bi-directional
      operation
      b3   (o) same as b1 but for the read-part of a bi-directional
      operation
      b4-6 not used (SHOULD be set to 0)

   Bits O and U are mutually exclusive and so are bits o and u.

2.3.2 Basic Residual Count

   The Basic Residual Count field is valid only in case either the U bit
   or the O bit is set. If neither bit is set, the Basic Residual Count
   field SHOULD be zero.  If the U bit is set, the Basic Residual Count
   indicates how many bytes were not transferred out of those expected
   to be transferred.  If the O bit is set, the Basic Residual Count
   indicates how many bytes could not be transferred because the
   initiator's Expected Data Transfer Length was too small.

2.3.3 Bidi-Read Residual Count

   The Bidi-Read Residual Count field is valid only in case either the u
   bit or the o bit is set. If neither bit is set, the Bidi-Read
   Residual Count field SHOULD be zero.  If the u bit is set, the Bidi-
   Read Residual Count indicates how many bytes were not transferred in
   out of those expected to be transferred.  If the o bit is set, the
   Bidi-Read Residual Count indicates how many bytes could not be

Satran                Standards-Track, May 2001                    23

                                iSCSI                  November, 2000


   transferred in because the initiator's Expected Bidi-Read Transfer
   Length was too small.

2.3.4 Command Status

   The Command Status field is used to report the SCSI status of the
   command (as specified in [SAM2]).

2.3.5 Resp_length - Response length

2.3.6 Sense_length - Length of sense data

2.3.7 Response and/or Sense Data

   iSCSI targets MUST support and enable autosense.  If the Command
   Status was CHECK CONDITION (0x02), then the Response and/or Sense
   Data field will contain sense data for the failed command after the
   response data.  Some sense codes will relate to iSCSI check
   conditions (e.g. excessive number of outstanding commands, immediate
   data blocks too large etc.).  The Length parameters specify the
   number of bytes in each section of this field.  If no error occurred,
   and no data is needed for the response to the SCSI Command the length
   field is zero.  If both Response Data and Sense Data are present, the
   Response Data precedes the Sense Data.

2.3.8 StatRN - Status Reference Number

   StatRN is a reference number that the target iSCSI layer generates
   per connection and that in turn enables the initiator to acknowledge
   status reception. StatRN is incremented by 1 for every
   response/status sent on a connection.

2.3.9 ExpCmdRN - next expected CmdRN from this initiator

   ExpCmdRN is a reference number that the target iSCSI returns to the
   initiator to acknowledge command reception. The initiator must ignore
   values not between the current value of the ExpCmdRN and MaxCmdRN;
   this may be required when updates arrive out of order (they travel on
   different TCP connections).

2.3.10 MaxCmdRN - maximum CmdRN acceptable from this initiator

   MaxCmdRN is a reference number that the target iSCSI returns to the
   initiator to indicate the maximum CmdRN the initiator can send. The
   initiator must ignore values not between the current value of the
   ExpCmdRN and MaxCmdRN; this may be required when updates arrive out
   of order (they travel on different TCP connections).

Satran                Standards-Track, May 2001                    24

                                iSCSI                  November, 2000



   Update order is MaxCmdRN, ExpCmdRN to allow checking the above rules.



















































Satran                Standards-Track, May 2001                    25

                                iSCSI                  November, 2000



2.4 NOP-Out Message

   An initiator to target message.

   Byte /    0       |       1       |       2       |       3       |
      /              |               |               |               |
     |7 6 5 4 3 2 1 0|7 6 5 4 3 2 1 0|7 6 5 4 3 2 1 0|7 6 5 4 3 2 1 0|
     +---------------+---------------+---------------+---------------+
    0|F|  0x00       |1|P| Reserved (0)                              |
     +---------------+---------------+---------------+---------------+
    4| Length                                                        |
     +---------------+---------------+---------------+---------------+
    8/ Reserved (0)                                                  /
    +/                                                               /
     +---------------+---------------+---------------+---------------+
   28| ExpStatRN                                                     |
     +---------------+---------------+---------------+---------------+
   32/ Reserved (0)                                                  /
    +/                                                               /
     +---------------+---------------+---------------+---------------+
   48

2.4.1 P - poll bit

   Request a NOP-In message
























Satran                Standards-Track, May 2001                    26

                                iSCSI                  November, 2000



2.5 NOP-In Message

   A target to initiator message.

   Byte /    0       |       1       |       2       |       3       |
      /              |               |               |               |
     |7 6 5 4 3 2 1 0|7 6 5 4 3 2 1 0|7 6 5 4 3 2 1 0|7 6 5 4 3 2 1 0|
     +---------------+---------------+---------------+---------------+
    0|F|  0x40       |1|P| Reserved (0)                              |
     +---------------+---------------+---------------+---------------+
    4| Length                                                        |
     +---------------+---------------+---------------+---------------+
    8/ Reserved (0)                                                  /
    +/                                                               /
     +---------------+---------------+---------------+---------------+
   28| ExpCmdRN                                                      |
     +---------------+---------------+---------------+---------------+
   32| MaxCmdRN                                                      |
     +---------------+---------------+---------------+---------------+
   36/ Reserved (0)                                                  /
    +/                                                               /
     +---------------+---------------+---------------+---------------+
   48

2.5.1 P - poll bit

   Request a NOP-Out message






















Satran                Standards-Track, May 2001                    27

                                iSCSI                  November, 2000


2.6  SCSI Task Management Command

   Byte /    0       |       1       |       2       |       3       |
      /              |               |               |               |
     |7 6 5 4 3 2 1 0|7 6 5 4 3 2 1 0|7 6 5 4 3 2 1 0|7 6 5 4 3 2 1 0|
     +---------------+---------------+---------------+---------------+
    0|F|  0x02       |1|0| Function  | Reserved (0)                  |
     +---------------+---------------+---------------+---------------+
    4| Length                                                        |
     +---------------+---------------+---------------+---------------+
    8| Logical Unit Number (LUN)                                     |
     +                                                               +
   12|                                                               |
     +---------------+---------------+---------------+---------------+
   16| Initiator Task Tag                                            |
     +---------------+---------------+---------------+---------------+
   20| Referenced Task Tag or Reserved (0)                           |
     +---------------+---------------+---------------+---------------+
   24| CmdRN                                                         |
     +---------------+---------------+---------------+---------------+
   28| ExpStatRN                                                     |
     +---------------+---------------+---------------+---------------+
   32/ Reserved (0)                                                  /
    +/                                                               /
     +---------------+---------------+---------------+---------------+
   48

2.6.1 Function

   The Task Management functions provide an initiator with a way to
   explicitly control the execution of one or more Tasks. The Task
   Management functions are summarized as follows (for a more detailed
   description see the [SAM2] document):

      1    Abort Task---aborts the task identified by the Referenced
      Task Tag field.
      2    Abort Task Set---aborts all Tasks issued by this initiator
      on the Logical Unit.
      3    Clear ACA---clears the Auto Contingent Allegiance
      condition.
      4    Clear Task Set---Aborts all Tasks (from all initiators)
      for the Logical Unit.
      5    Logical Unit Reset
      6    Target Warm Reset
      7    Target Cold Reset



Satran                Standards-Track, May 2001                    28

                                iSCSI                  November, 2000


   For the functions above a SCSI Task Management Response MUST be
   returned, using the Initiator Task Tag to identify the operation for
   which it is responding.
   For the <Clear Task Set> the target MUST send an Asynchronous Event
   to all other attached initiators to inform them that all pending
   tasks are cancelled and then enter the ACA state for any initiator
   for which it had pending tasks.
   For the <Target Warm Reset> and <Target Cold Reset> functions, the
   target cancels all pending operations. The target MUST send an
   Asynchronous Event to all attached initiators notifying them that the
   target is being reset.

   In addition, for the <Target Warm Reset> the target will enter the
   ACA state on all sessions and all LUs on which an AE was sent.

   In addition, for the <Target Cold Reset> the target then MUST
   terminate all of its TCP connections to all initiators (all sessions
   are terminated). However, if the target finds that it cannot send the
   required response or AE it MUST continue the reset operation and it
   SHOULD log the condition for later retrieval.

   Further actions on reset functions are specified in the relevant SCSI
   documents for the specific class of devices.




























Satran                Standards-Track, May 2001                    29

                                iSCSI                  November, 2000



2.7 SCSI Task Management Response


   Byte /    0       |       1       |       2       |       3       |
      /              |               |               |               |
     |7 6 5 4 3 2 1 0|7 6 5 4 3 2 1 0|7 6 5 4 3 2 1 0|7 6 5 4 3 2 1 0|
     +---------------+---------------+---------------+---------------+
    0|F| 0x42        |1|0| Reserved (0)                              |
     +---------------+---------------+---------------+---------------+
    4| Length                                                        |
     +---------------+---------------+---------------+---------------+
    8| Logical Unit Number (LUN)                                     |
     +                                                               +
   12|                                                               |
     +---------------+---------------+---------------+---------------+
   16| Initiator Task Tag                                            |
     +---------------+---------------+---------------+---------------+
   20| Referenced Task Tag or Reserved (0)                           |
     +---------------+---------------+---------------+---------------+
   24| StatRN                                                        |
     +---------------+---------------+---------------+---------------+
   28| ExpCmdRN                                                      |
     +---------------+---------------+---------------+---------------+
   32| MaxCmdRN                                                      |
     +---------------+---------------+---------------+---------------+
   36| Response      | Reserved (0)                                  |
     +---------------+---------------+---------------+---------------+
   40/ Reserved (0)                                                  /
    +/                                                               /
     +---------------+---------------+---------------+---------------+
   48

   For the functions <Abort Task, Abort Task Set, Clear ACA, Clear Task
   Set, Logical Unit reset>, the target performs the requested Task
   Management function and sends a SCSI Task Management Response back to
   the initiator. The target provides a Response, which may take on the
   following values:

       0    Function Complete
       1    Function Rejected

   For the <Target Cold Reset> and <Target Warm Reset> functions, the
   target cancels all pending operations. The target MUST send an
   Asynchronous Event to all attached initiators notifying them that the
   target is being reset.  For the <Target Cold Reset> the target MUST


Satran                Standards-Track, May 2001                    30

                                iSCSI                  November, 2000


   then close all of its TCP connections to all initiators (terminates
   all sessions).



















































Satran                Standards-Track, May 2001                    31

                                iSCSI                  November, 2000



2.8 SCSI Data

   The typical data transfer specifies the length of the data payload,
   the Transfer Tag provided by the receiver for this data transfer, and
   a buffer offset.  The typical SCSI Data packet for WRITE (from
   initiator to target) has the following format:


   Byte /    0       |       1       |       2       |       3       |
      /              |               |               |               |
     |7 6 5 4 3 2 1 0|7 6 5 4 3 2 1 0|7 6 5 4 3 2 1 0|7 6 5 4 3 2 1 0|
     +---------------+---------------+---------------+---------------+
    0|F| 0x05        |1|0| Reserved (0)                              |
     +---------------+---------------+---------------+---------------+
    4| Length                                                        |
     +---------------+---------------+---------------+---------------+
    8| LUN or Reserved (0)                                           |
   12|                                                               |
     +---------------+---------------+---------------+---------------+
   16| Initiator Task Tag                                            |
     +---------------+---------------+---------------+---------------+
   20| Target Task Tag (solicited) or Reserved (0) (unsolicited)     |
     +---------------+---------------+---------------+---------------+
   24| Reserved (0)                                                  |
     +---------------+---------------+---------------+---------------+
   28| ExpStatRN                                                     |
     +---------------+---------------+---------------+---------------+
   32/ Reserved (0)                                                  /
     /                                                               /
     +---------------+---------------+---------------+---------------+
   40| Buffer Offset                                                 |
     +---------------+---------------+---------------+---------------+
   44| Reserved (0)                                                  |
     +---------------+---------------+---------------+---------------+
   48/ Payload                                                       /
    +/                                                               /
     +---------------+---------------+---------------+---------------+











Satran                Standards-Track, May 2001                    32

                                iSCSI                  November, 2000



   The typical SCSI Data packet for READ (from target to initiator) has
   the following format:


   Byte /    0       |       1       |       2       |       3       |
      /              |               |               |               |
     |7 6 5 4 3 2 1 0|7 6 5 4 3 2 1 0|7 6 5 4 3 2 1 0|7 6 5 4 3 2 1 0|
     +---------------+---------------+---------------+---------------+
    0|F| 0x45        |1|P| (0) |S|O|U| Reserved (0)                  |
     +---------------+---------------+---------------+---------------+
    4| Length                                                        |
     +---------------+---------------+---------------+---------------+
    8| Reserved (0)                                                  |
     +---------------+---------------+---------------+---------------+
   12| Reserved (0)                                                  |
     +---------------+---------------+---------------+---------------+
   16| Initiator Task Tag                                            |
     +---------------+---------------+---------------+---------------+
   20| Residual Count                                                |
     +---------------+---------------+---------------+---------------+
   24| DataRN                                                        |
     +---------------+---------------+---------------+---------------+
   28| ExpCmdRN                                                      |
     +---------------+---------------+---------------+---------------+
   32| MaxCmdRN                                                      |
     +---------------+---------------+---------------+---------------+
   36| Command Status|iSCSI Status   | Reserved (0)                  |
     +---------------+---------------+---------------+---------------+
   40| Buffer Offset                                                 |
     +---------------+---------------+---------------+---------------+
   44| Reserved (0)                                                  |
     +---------------+---------------+---------------+---------------+
   48/ Payload                                                       /
    +/                                                               /
     +---------------+---------------+---------------+---------------+


2.8.1 Length

   The length field specifies the total number of bytes in the following
   payload.

2.8.2 Target Task Tag




Satran                Standards-Track, May 2001                    33

                                iSCSI                  November, 2000


   The Target Task Tag is provided to the target if the transfer is
   honoring a R2T. In this case, the Target Task Tag field is a replica
   of the Target Task Tag provided with the R2T.
   The Target Task Field and LUN tuple SHOULD be unique per session.

2.8.3 Buffer Offset

   The Buffer Offset field contains the offset of the following data
   against the complete data transfer. The sum of the buffer offset and
   length should not exceed the expected transfer length for the
   command.

2.8.4 Flags

   The last SCSI Data packet sent from a target to an initiator for a
   particular SCSI command that completed successfully may optionally
   also contain the Command Status for the data transfer.  In this case
   Sense Data cannot be sent together with the Command Status.  If the
   command completed with an error, then the response and sense data
   must be sent in a SCSI Response packet and must not be sent in a SCSI
   Data packet.

      b0-1 as in an ordinary SCSI Response
      b2   S (status)- set to indicate that the Command Status field
      contains status
      b3-5 not used (should be set to 0)
      b6   P (poll) - set to indicate data acknowledgement is
      requested; b7 and b2 are mutually exclusive - if S bit is set P
      bit MUST be ignored

   If the S bit is set, then there is meaning to the extra fields in the
   SCSI Data packet (StatRN, Command Status, Residual Count).

2.8.5 Data numbering (DataRN)

   On inbound data, the target MAY number (sequence) the data packets to
   enable shorter recovery on connection failure.  In case the target
   numbers data packets, the initiator MUST acknowledge them by
   specifying the next expected packet in a NOP command with the same
   Initiator Tag. Acknowledging NOP PDUs MAY be postponed for a maximum
   of 32 incoming data PDUs.  An explicit request for acknowledgement
   made by setting the P bit MUST be honored.






Satran                Standards-Track, May 2001                    34

                                iSCSI                  November, 2000



2.9 Text Command

   The Text Command is provided to allow the exchange of information and
   for future extensions. It permits the initiator to inform a target of
   its capabilities or to request some special operations.


   Byte /    0       |       1       |       2       |       3       |
      /              |               |               |               |
     |7 6 5 4 3 2 1 0|7 6 5 4 3 2 1 0|7 6 5 4 3 2 1 0|7 6 5 4 3 2 1 0|
     +---------------+---------------+---------------+---------------+
    0|F| 0x04        |1|0| Type      | Reserved (0)                  |
     +---------------+---------------+---------------+---------------+
    4| Length                                                        |
     +---------------+---------------+---------------+---------------+
    8/ Reserved (0)                                                  /
    +/                                                               /
     +---------------+---------------+---------------+---------------+
   16| Initiator Task Tag                                            |
     +---------------+---------------+---------------+---------------+
   20| Reserved (0)                                                  |
     +---------------+---------------+---------------+---------------+
   24| CmdRN                                                         |
     +---------------+---------------+---------------+---------------+
   28| ExpStatRN                                                     |
     +---------------+---------------+---------------+---------------+
   32/ Reserved (0)                                                  /
    +/                                                               /
     +---------------+---------------+---------------+---------------+
   48/ Text                                                          /
    +/                                                               /
     +---------------+---------------+---------------+---------------+

2.9.1 Type

      0 outside login phase
      1 within login


2.9.2 Length

   This is the length, in bytes, of the Text field.


2.9.3 Initiator Task Tag


Satran                Standards-Track, May 2001                    35

                                iSCSI                  November, 2000


   The initiator assigned identifier for this Text Command.
   If the command is sent as part of the Login Phase the Initiator Task
   Tag MUST be the same as the one sent with the Login Command.

2.9.4 Text

   The initiator sends the target a set of key:value or key:(list) pairs
   encoded in UTF-8 Unicode. The key and value are separated by a ':'
   (0x3A) delimiter. Many key:value pairs can be included in the Text
   block by separating them with null ' ' (0x00) delimiters. Some basic
   key:value pairs are described in Appendix A & C.  The target responds
   by sending its response back to the initiator. The target and
   initiator can then perform some advanced operations based on their
   common capabilities.

   Manufacturers may introduce new keys by prefixing them with their
   (reversed) domain name, for example,

      com.foo.bar.do_something:0000000000000003

   Any key that the target does not understand may be ignored without
   affecting basic function. Once the target has processed all the
   key:value or key:(list) pairs, it responds with the Text Response
   command, listing the parameters that it supports. It is recommended
   that Text operations that will take a long time should be placed in
   their own Text command.  If the Text Response does not contain a key
   that was requested, the initiator must assume that the key was not
   understood by the target.
   Targets and initiators may limit the size of the text accepted in a
   text command and text response as well as the size of key:value
   pairs.  Such limits should be indicated at login.
   The default limit is 16384 UTF8 characters.

















Satran                Standards-Track, May 2001                    36

                                iSCSI                  November, 2000



2.10 Text Response

   The Text Response message contains the responses of the target to the
   initiator's Text Command. The format of the Text field matches that
   of the Text Command.


   Byte /    0       |       1       |       2       |       3       |
      /              |               |               |               |
     |7 6 5 4 3 2 1 0|7 6 5 4 3 2 1 0|7 6 5 4 3 2 1 0|7 6 5 4 3 2 1 0|
     +---------------+---------------+---------------+---------------+
    0|F| 0x44        |1|0| Type      | Reserved (0)                  |
     +---------------+---------------+---------------+---------------+
    4| Length                                                        |
     +---------------+---------------+---------------+---------------+
    8/ Reserved (0)                                                  /
    +/                                                               /
     +---------------+---------------+---------------+---------------+
   16| Initiator Task Tag                                            |
     +---------------+---------------+---------------+---------------+
   20| Reserved (0)                                                  |
     +---------------+---------------+---------------+---------------+
   24| StatRN                                                        |
     +---------------+---------------+---------------+---------------+
   28| ExpCmdRN                                                      |
     +---------------+---------------+---------------+---------------+
   32| MaxCmdRN                                                      |
     +---------------+---------------+---------------+---------------+
   36/ Reserved (0)                                                  /
    +/                                                               /
     +---------------+---------------+---------------+---------------+
   48/ Text Response                                                 /
    +/                                                               /
     +---------------+---------------+---------------+---------------+

2.10.1 Type

      0 outside login phase
      1 within login

2.10.2 Length

   This is the length, in bytes, of the Text Response field.

2.10.3 Initiator Task Tag


Satran                Standards-Track, May 2001                    37

                                iSCSI                  November, 2000


   The Initiator Task Tag matches the tag used in the initial Text
   Command or the Login Initiator Task Tag.

2.10.4 Text Response

   The Text Response field contains responses in the same key:value
   format as the Text Command. Appendix C lists some basic Text Commands
   and their Responses.  If the Text Response does not contain a key
   that was requested, the initiator must assume that the key was not
   understood by the target or that the answer is <key>:none and the two
   MUST be equivalent where applicable.









































Satran                Standards-Track, May 2001                    38

                                iSCSI                  November, 2000



2.11 Login Command

   After establishing a TCP connection between an initiator and a
   target, the initiator MUST issue a Login Command to gain further
   access to the target's resources.


   Byte /    0       |       1       |       2       |       3       |
      /              |               |               |               |
     |7 6 5 4 3 2 1 0|7 6 5 4 3 2 1 0|7 6 5 4 3 2 1 0|7 6 5 4 3 2 1 0|
     +---------------+---------------+---------------+---------------+
    0|F| 0x03        |1|0| Rsrvd (0) | Version-major | Version-minor |
     +---------------+---------------+---------------+---------------+
    4| Length                                                        |
     +---------------+---------------+---------------+---------------+
    8| CID                           | Reserved (0)                  |
     +---------------+---------------+---------------+---------------+
   12| ISID                          |TSID                           |
     +---------------+---------------+---------------+---------------+
   16| Initiator Task Tag                                            |
     +---------------+---------------+---------------+---------------+
   20| Reserved (0)                                                  |
     +---------------+---------------+---------------+---------------+
   24| InitCmdRN   or   0                                            |
     +---------------+---------------+---------------+---------------+
   28/ Reserved (0)                                                  /
    +/                                                               /
     +---------------+---------------+---------------+---------------+
   48/ Login Parameters in Text Command Format                       /
    +/                                                               /
     +---------------+---------------+---------------+---------------+

2.11.1 Version-major

   Currently 1.

2.11.2 Version-minor

   Currently 0.

2.11.3 CID

   A unique id for this connection within the session

2.11.4 Initiator Task Tag


Satran                Standards-Track, May 2001                    39

                                iSCSI                  November, 2000


   This tag identifies all the commands and responses within the login
   sequence.

2.11.5 InitCmdRN

   Is significant only if TSID is zero and indicates the starting
   Command reference number for this session; it SHOULD be zero for all
   other instances. If it is significant (TSID is 0) and the value is
   zero then this is a single connection session with no support for
   command numbering.

2.11.6 Login Parameters

   The initiator MAY provide some basic parameters in order to enable
   the target to determine if the initiator may in fact use the target's
   resources and the initial text parameters for the security exchange.
   The format of the parameters is as specified for the Text Command.
   Keys and their explanations are listed in Appendixes.

































Satran                Standards-Track, May 2001                    40

                                iSCSI                  November, 2000



2.12 Login Response

   The Login Response indicates the end of the login phase.  Note, if
   security is established, the login response is authenticated.

   Byte /    0       |       1       |       2       |       3       |
      /              |               |               |               |
     |7 6 5 4 3 2 1 0|7 6 5 4 3 2 1 0|7 6 5 4 3 2 1 0|7 6 5 4 3 2 1 0|
     +---------------+---------------+---------------+---------------+
    0|F| 0x43        |1|0| Rsrvd (0) | Version-major | Version-minor |
     +---------------+---------------+---------------+---------------+
    4| Length                                                        |
     +---------------+---------------+---------------+---------------+
    8/ Reserved (0)                                                  /
    +/                                                               /
     +---------------+---------------+---------------+---------------+
   12| ISID                          |TSID                           |
     +---------------+---------------+---------------+---------------+
   16| Initiator Task Tag                                            |
     +---------------+---------------+---------------+---------------+
   20| Reserved (0)                                                  |
     +---------------+---------------+---------------+---------------+
   24| InitStatRN                                                    |
     +---------------+---------------+---------------+---------------+
   28| ExpCmdRN                                                      |
     +---------------+---------------+---------------+---------------+
   32| MaxCmdRN                                                      |
     +---------------+---------------+---------------+---------------+
   36| Status        | Reserved (0)                                  |
     +---------------+---------------+---------------+---------------+
   40/ Reserved (0)                                                  /
    +/                                                               /
     +---------------+---------------+---------------+---------------+
   48/ Login Parameters in Text Command Format                       /
    +/                                                               /
     +---------------+---------------+---------------+---------------+


2.12.1 Version-major minor

   Indicates the version supported. Assuming versions are backward
   compatible, it indicates the highest (compatible) version supported
   by the target.

2.12.2 InitStatRN


Satran                Standards-Track, May 2001                    41

                                iSCSI                  November, 2000


   This is the starting status reference number for this connection.

2.12.3 Status

   The Status returned in a Login Response is one of the following:

      0 accept login   (will now accept SCSI commands)
      1 reject login

   In the case that the Status is "accept login" the initiator may
   proceed to issue SCSI commands.  In the case that the Status is
   "reject login" the initiator should immediately close down its end of
   the TCP connection, thus freeing up the target's port for some other
   connection. The target also has the option of immediately closing
   down its end of the TCP connection.

2.12.4 TSID

   The TSID is an initiator identifying tag set by the target.  A 0 in
   the returned TSID indicates that either the target supports only a
   single connection or that the ISID has already been used as a leading
   ISID. In both cases, the target is rejecting the login.





























Satran                Standards-Track, May 2001                    42

                                iSCSI                  November, 2000



2.13 NOP Command


   Byte /    0       |       1       |       2       |       3       |
      /              |               |               |               |
     |7 6 5 4 3 2 1 0|7 6 5 4 3 2 1 0|7 6 5 4 3 2 1 0|7 6 5 4 3 2 1 0|
     +---------------+---------------+---------------+---------------+
    0|1|  0x06       |1|P| Reserved (0)                              |
     +---------------+---------------+---------------+---------------+
    4| Length                                                        |
     +---------------+---------------+---------------+---------------+
    8/ Reserved (0)                                                  /
    +/                                                               /
     +---------------+---------------+---------------+---------------+
   16| Initiator Task Tag                                            |
     +---------------+---------------+---------------+---------------+
   20| Reserved (0)                                                  |
     +---------------+---------------+---------------+---------------+
   24| CmdRN or (0)                                                  |
     +---------------+---------------+---------------+---------------+
   28| ExpStatRN or (0)                                              |
     +---------------+---------------+---------------+---------------+
   32| ExpDataRN or (0)                                              |
     +---------------+---------------+---------------+---------------+
   36/ Reserved (0)                                                  /
    +/                                                               /
     +---------------+---------------+---------------+---------------+
   48/ Ping Data (optional)                                          /
    +/                                                               /
     +---------------+---------------+---------------+---------------+

   The NOP command with the P bit set acts as a "ping command".
   This form of the NOP Command can be used to verify that a connection
   is still active and all it's components are operational; unlike the
   NOP message, NOP has an Initiator Task Tag and can be delivered in
   order. It may be useful in the case where an initiator has been
   waiting a long time for the response to some command, and the
   initiator suspects that there is some problem with the connection.
   When a target receives the NOP Command with the Ping bit set, it
   should respond with a Ping Response, duplicating as much as possible
   of the data that was provided in the NOP Command.  If the initiator
   does not receive the NOP Response within some time (determined by the
   initiator), or if the data returned by the NOP Response is different
   from the data that was in the NOP Command, the initiator may conclude



Satran                Standards-Track, May 2001                    43

                                iSCSI                  November, 2000


   that there is a problem with the connection. The initiator will then
   close the connection and may try to establish a new connection.

   The NOP command with the P bit not set MAY be used to acknowledge
   data received from a target (data-ack). In this case, the command
   caries the same Initiator Task Tag as the data it acknowledges and
   the CmdRN field MUST be zero.  The field ExpStatRN/ ExpDataRN is then
   understood to be ExpDataRN. Repeated or obsolete data
   acknowledgements MUST be silently discarded by the target.



2.13.1 P - poll bit

   Request a NOP Response

2.13.1.1 Length

   This is the length of the optional Ping Data.

2.13.2 Initiator Task Tag

   An initiator assigned identifier for the operation.

2.13.3 Ping Data

   Binary data that will be reflected in the Ping Response.
























Satran                Standards-Track, May 2001                    44

                                iSCSI                  November, 2000



2.14 NOP Response


   Byte /    0       |       1       |       2       |       3       |
      /              |               |               |               |
     |7 6 5 4 3 2 1 0|7 6 5 4 3 2 1 0|7 6 5 4 3 2 1 0|7 6 5 4 3 2 1 0|
     +---------------+---------------+---------------+---------------+
    0|F| 0x46        |1|0| Reserved (0)                              |
     +---------------+---------------+---------------+---------------+
    4| Length                                                        |
     +---------------+---------------+---------------+---------------+
    8/ Reserved (0)                                                  /
    +/                                                               /
     +---------------+---------------+---------------+---------------+
   16| Initiator Task Tag                                            |
     +---------------+---------------+---------------+---------------+
   20| Reserved (0)                                                  |
     +---------------+---------------+---------------+---------------+
   24| StatRN                                                        |
     +---------------+---------------+---------------+---------------+
   28| ExpCmdRN                                                      |
     +---------------+---------------+---------------+---------------+
   32| MaxCmdRN                                                      |
     +---------------+---------------+---------------+---------------+
   36/ Reserved (0)                                                  /
    +/                                                               /
     +---------------+---------------+---------------+---------------+
   48/ Return Ping Data                                              /
    +/                                                               /
     +---------------+---------------+---------------+---------------+


   When a target receives the NOP Command with the P bit set, it MUST
   respond with a NOP Response, with the same Initiator Task Tag that
   was provided in the Ping Command. It SHOULD also duplicate as much of
   the initiator provided Ping Data as allowed by a configurable target
   parameter.











Satran                Standards-Track, May 2001                    45

                                iSCSI                  November, 2000



2.15 Map Command


   Byte /    0       |       1       |       2       |       3       |
      /              |               |               |               |
     |7 6 5 4 3 2 1 0|7 6 5 4 3 2 1 0|7 6 5 4 3 2 1 0|7 6 5 4 3 2 1 0|
     +---------------+---------------+---------------+---------------+
    0|F| 0x07        |1|0| Function  | Reserved (0)                  |
     +---------------+---------------+---------------+---------------+
    4| Length                                                        |
     +---------------+---------------+---------------+---------------+
    8| Reserved (0)                                                  |
     +                                                               +
   12|                                                               |
     +---------------+---------------+---------------+---------------+
   16| Initiator Task Tag                                            |
     +---------------+---------------+---------------+---------------+
   20| Reserved (0)                                                  |
     +---------------+---------------+---------------+---------------+
   24| CmdRN                                                         |
     +---------------+---------------+---------------+---------------+
   28| ExpStatRN                                                     |
     +---------------+---------------+---------------+---------------+
   32/ Reserved (0)                                                  /
    +/                                                               /
     +---------------+---------------+---------------+---------------+
   48| Descriptor Type               | Descriptor Length             |
     +---------------+---------------+---------------+---------------+
   52/ Descriptor                                                    /
    +/                                                               /
     +---------------+---------------+---------------+---------------+
   ---------------------------------------------------------------------
     +---------------+---------------+---------------+---------------+
     | Descriptor Type               | Descriptor Length             |
     +---------------+---------------+---------------+---------------+
     / Descriptor                                                    /
    +/                                                               /
     +---------------+---------------+---------------+---------------+


        or






Satran                Standards-Track, May 2001                    46

                                iSCSI                  November, 2000




     +---------------+---------------+---------------+---------------+
   48| 8 byte Descriptor                                             |
    +|                                                               |
     +---------------+---------------+---------------+---------------+
   ---------------------------------------------------------------------
     +---------------+---------------+---------------+---------------+
   N | 8 byte Descriptor                                             |
    +|                                                               |
     +---------------+---------------+---------------+---------------+


   The mapping command enables the initiator to map iSCSI specific
   addresses and access control information into formats compliant with
   the SCSI command standards (e.g., [SPC-2]).

2.15.1 Function

   Two functions are required for mapping:

      1    Map - given an address or access control information
      provide the 8 byte SCSI compliant address reference
      0    Unmap - given a SCSI compliant address reference remove
      the mapping associated with it.

   Address/access control descriptors follow the header.  For the map
   function the following descriptor types are defined:

      0    Binary IP Version 4 TCP address (IP+Port) followed by a
      selector string; length should be 6+the selector length+1
      1    Binary IP Version 6 TCP address (IP+Port) followed by a
      selector string; length should be 18+the selector length+1
      2    iSCSI URL (domain name terminated with null followed by a
      selector followed by null)
      3    FC address & port - in case access control is based on
      transport ID
      4    access proxy token

   Details for 3 & 4 have to be coordinated with T10

   For the unmap function the descriptors are standard 8 byte SRAs (SCSI
   Reference Address)





Satran                Standards-Track, May 2001                    47

                                iSCSI                  November, 2000



2.16 Map Response


   Byte /    0       |       1       |       2       |       3       |
      /              |               |               |               |
     |7 6 5 4 3 2 1 0|7 6 5 4 3 2 1 0|7 6 5 4 3 2 1 0|7 6 5 4 3 2 1 0|
     +---------------+---------------+---------------+---------------+
    0|F| 0x47        |1|0| Reserved (0)                              |
     +---------------+---------------+---------------+---------------+
    4| Length                                                        |
     +---------------+---------------+---------------+---------------+
    8| Reserved (0)                                                  |
     +                                                               +
   12|                                                               |
     +---------------+---------------+---------------+---------------+
   16| Initiator Task Tag                                            |
     +---------------+---------------+---------------+---------------+
   20| Entries Mapped                | Entries Available             |
     +---------------+---------------+---------------+---------------+
   24| StatRN                                                        |
     +---------------+---------------+---------------+---------------+
   28| ExpCmdRN                                                      |
     +---------------+---------------+---------------+---------------+
   32| MaxCmdRN                                                      |
     +---------------+---------------+---------------+---------------+
   36| Response      | Reserved (0)                                  |
     +---------------+---------------+---------------+---------------+
   40/ Reserved (0)                                                  /
    +/                                                               /
     +---------------+---------------+---------------+---------------+
   48

2.16.1 Entries Mapped

   The total number of mapped entries.

2.16.2 Entries Available

   The number of still available entries based on 64character mapping
   entry.

2.16.3 Response

   The target provides a Response, which may take on the following
   values:


Satran                Standards-Track, May 2001                    48

                                iSCSI                  November, 2000


      0    Function Complete
      1    Map Function Rejected - Bad Descriptors
      2    Map Function Rejected - too many descriptors
      3    Unmap Function Rejected - Bad Descriptor

   If the Response to a map is function complete the data following the
   header contains the SRAs to be used in third party commands; each SRA
   matches a descriptor in the Map command.

   Note that a map command can only entirely succeed (and then all
   descriptors are mapped or unmapped) or entirely fail.









































Satran                Standards-Track, May 2001                    49

                                iSCSI                  November, 2000


2.17 Logout Command

   The logout command is used by an initiator to "clean-up" the target
   end of a failing connection and enable recovery to start.
   On sessions with a single connection, this might imply opening a
   second connection with the sole purpose of cleaning-up the first.

   Byte /    0       |       1       |       2       |       3       |
      /              |               |               |               |
     |7 6 5 4 3 2 1 0|7 6 5 4 3 2 1 0|7 6 5 4 3 2 1 0|7 6 5 4 3 2 1 0|
     +---------------+---------------+---------------+---------------+
    0|F| 0x08        |1|0|Reserved (0)                               |
     +---------------+---------------+---------------+---------------+
    4| Length                                                        |
     +---------------+---------------+---------------+---------------+
    8| CID                           | Reserved (0)                  |
     +---------------+---------------+---------------+---------------+
   12| Reserved (0)                                                  |
     +---------------+---------------+---------------+---------------+
   16| Initiator Task Tag                                            |
     +---------------+---------------+---------------+---------------+
   20/ Reserved (0)                                                  /
    +/                                                               /
     +---------------+---------------+---------------+---------------+
   48/ Login Parameters in Text Command Format                       /
    +/                                                               /
     +---------------+---------------+---------------+---------------+

2.17.1 CID

   The connection ID of the connection to be cleaned (closed)



















Satran                Standards-Track, May 2001                    50

                                iSCSI                  November, 2000


2.18 Logout Response

   The logout is used by the target to indicate that the cleanup
   operation for the failed connection has completed.



   Byte /    0       |       1       |       2       |       3       |
      /              |               |               |               |
     |7 6 5 4 3 2 1 0|7 6 5 4 3 2 1 0|7 6 5 4 3 2 1 0|7 6 5 4 3 2 1 0|
     +---------------+---------------+---------------+---------------+
    0|1| 0x48        |1|0| Reserved (0)                              |
     +---------------+---------------+---------------+---------------+
    4| Length                                                        |
     +---------------+---------------+---------------+---------------+
    8/ Reserved (0)                                                  /
     /                                                               /
     +---------------+---------------+---------------+---------------+
   16| Initiator Task Tag                                            |
     +---------------+---------------+---------------+---------------+
   20/ Reserved (0)                                                  /
    +/                                                               /
     +---------------+---------------+---------------+---------------+
   36| Status        | Reserved (0)                                  |
     +---------------------------------------------------------------+
   48/ Login Parameters in Text Command Format                       /
    +/                                                               /
     +---------------+---------------+---------------+---------------+

2.18.1 Status

   Logout ending status:

      0 - connection closed successfully
      1 - cleanup failed














Satran                Standards-Track, May 2001                    51

                                iSCSI                  November, 2000




2.19 Ready To Transfer (R2T)

   When an initiator has submitted a SCSI Command with data passing from
   the initiator to the target (WRITE), the target may specify which
   blocks of data it is ready to receive. In general, the target may
   request that the data blocks be delivered in whatever order is
   convenient for the target at that particular instant. This
   information is passed from the target to the initiator in the Ready
   To Transfer (R2T) message.  In order to allow write operations
   without R2T, the initiator and target must have agreed to do so by
   both sending the UseR2T:no key-pair attribute to each other (either
   during Login or through the Text Command/Response mechanism).
   An R2T MUST be answered with one and only one iSCSI Data-out PDU with
   matching Target Task Tag. The Buffer Offset in the Data PDU MUST be
   the same as the one specified by the R2T and the data length of the
   Data PDU must not exceed the Desired Data Length specified in R2T.
   The target may send several R2T PDUs and thus have a number or data
   transfers pending.  The present document does not limit the number of
   outstanding data transfers. However, the target SHOULD NOT issue
   overlapping R2T request (i.e. referring to the same data area).  All
   outstanding R2T should have different Target Transfer Tags.


   Byte /    0       |       1       |       2       |       3       |
      /              |               |               |               |
     |7 6 5 4 3 2 1 0|7 6 5 4 3 2 1 0|7 6 5 4 3 2 1 0|7 6 5 4 3 2 1 0|
     +---------------+---------------+---------------+---------------+
    0|F| 0x50        |1|0| Reserved (0)                              |
     +---------------+---------------+---------------+---------------+
    4| Length                                                        |
     +---------------+---------------+---------------+---------------+
    8| Reserved (0)                                                  |
     +                                                               +
   12|                                                               |
     +---------------+---------------+---------------+---------------+
   16| Initiator Task Tag                                            |
     +---------------+---------------+---------------+---------------+
   20| Target Task Tag                                               |
     +---------------+---------------+---------------+---------------+
   24| Reserved (0)                                                  |
     +---------------+---------------+---------------+---------------+
   28| ExpCmdRN                                                      |
     +---------------+---------------+---------------+---------------+
   32| MaxCmdRN                                                      |
     +---------------+---------------+---------------+---------------+

Satran                Standards-Track, May 2001                    52

                                iSCSI                  November, 2000


   36| Desired Data Length                                           |
     +---------------+---------------+---------------+---------------+
   40| Buffer Offset                                                 |
     +---------------+---------------+---------------+---------------+
   44| Reserved (0)                                                  |
     |                                                               |
     +---------------+---------------+---------------+---------------+
   48

2.19.1 Desired Data Transfer Length and Buffer Offset

   The target specifies how many bytes it wants the initiator to send
   because of this R2T message.  The target may request the data from
   the initiator in several chunks, not necessarily in the original
   order of the data.  The target, therefore, also specifies a Buffer
   Offset indicating the point at which the data transfer should begin,
   relative to the beginning of the total data transfer.


2.19.2 Target Transfer Tag

   The target assigns its own tag to each R2T request that it sends to
   the initiator. This can be used by the target to easily identify data
   it receives.  The Target Transfer Tag is copied in the outgoing data
   PDUs and is provided by the target and used by the target only. There
   is no protocol rule about Target Transfer Tag but it is assumed that
   it will be used to tag the response data to the target (alone or
   combination with the LUN).






















Satran                Standards-Track, May 2001                    53

                                iSCSI                  November, 2000



2.20 Asynchronous Event

   An Asynchronous Event may be sent from the target to the initiator
   without corresponding to a particular command. The target specifies
   the status for the event and sense data.


   Byte /    0       |       1       |       2       |       3       |
      /              |               |               |               |
     |7 6 5 4 3 2 1 0|7 6 5 4 3 2 1 0|7 6 5 4 3 2 1 0|7 6 5 4 3 2 1 0|
     +---------------+---------------+---------------+---------------+
    0|F| 0x51        |1|0| Reserved (0)                              |
     +---------------+---------------+---------------+---------------+
    4| Length                                                        |
     +---------------+---------------+---------------+---------------+
    8| Logical Unit Number (LUN)                                     |
     +                                                               +
   12|                                                               |
     +---------------+---------------+---------------+---------------+
   16/ Reserved (0)                                                  /
    +/                                                               /
     +---------------+---------------+---------------+---------------+
   24| StatRN                                                        |
     +---------------+---------------+---------------+---------------+
   28| ExpCmdRN                                                      |
     +---------------+---------------+---------------+---------------+
   32| MaxCmdRN                                                      |
     +---------------+---------------+---------------+---------------+
   36|SCSI Event Ind |iSCSI Event Ind| Reserved (0)                  |
     +---------------+---------------+---------------+---------------+
   40/ Reserved (0)                                                  /
     /                                                               /
     +---------------+---------------+---------------+---------------+
   48/ Sense Data                                                    /
    +/                                                               /
     +---------------+---------------+---------------+---------------+


2.20.1 iSCSI Event

   Some Asynchronous Events are strictly related to iSCSI while others
   are related to SAM-2.  The codes returned for iSCSI Asynchronous
   Events are:

      1    Target is being reset.


Satran                Standards-Track, May 2001                    54

                                iSCSI                  November, 2000


2.20.2 SCSI Event Indicator

   The following values are defined.  (See [SAM2] for details):

      1    An error condition was encountered after command
      completion.
      2    A newly initialized device is available to this initiator.
      3    All Task Sets are being Reset by another Initiator
      5    Some other type of unit attention condition has occurred.
      6    An asynchronous event has occurred.

   Sense Data accompanying the report identifies the condition.  The
   Length parameter is set to the length of the Sense Data.

   For new device identification an iSCSI target MUST support the Device
   Identification page.

   Please note that StatRN counts this PDU as a acknowledgeable event
   allowing the initiator and target state synchronization.
































Satran                Standards-Track, May 2001                    55

                                iSCSI                  November, 2000



2.21 Third Party Commands

   There are some third-party SCSI commands, such as (EXTENDED) COPY and
   COMPARE that involve more than one target. In it's most general form
   those commands involve the "original target" called the COPY-Manager
   and a (variable) number of other machines called source and
   destination. The whole operation is described by one "master CDB"
   delivered to the Copy manager and a series of descriptor blocks; each
   descriptor block addresses a source and destination target and LU and
   a description of the work to be done in terms of blocks or bytes as
   required by the device types. The relevant SCSI standards do not
   require full support of the (EXTENDED) COPY or COMPARE nor do they
   provide a detailed execution model.  We will assume, in the spirit of
   [SPC-2], that a COPY manager will read data from a source and write
   them to a destination.

   To address them an iSCSI COPY manager will use information provided
   to it through map commands and the SRAs and flags provided in the
   descriptors - allowing for iSCSI and FC sources and destinations.

   Enabling a FC COPY manager to support iSCSI sources and destinations
   is subject to coordination with T10.




























Satran                Standards-Track, May 2001                    56

                                iSCSI                  November, 2000



2.22 Opcode Not Understood


   Byte /    0       |       1       |       2       |       3       |
      /              |               |               |               |
     |7 6 5 4 3 2 1 0|7 6 5 4 3 2 1 0|7 6 5 4 3 2 1 0|7 6 5 4 3 2 1 0|
     +---------------+---------------+---------------+---------------+
    0|F| 0x7f)       |1|0| Reserved (0)                              |
     +---------------+---------------+---------------+---------------+
    4| Length                                                        |
     +---------------+---------------+---------------+---------------+
    8/ Reserved (0)                                                  /
    +/                                                               /
     +---------------+---------------+---------------+---------------+
   48/ Header of Bad Message                                         /
    +/                                                               /
     +---------------+---------------+---------------+---------------+
   96

   It may happen that a target receives a message with an Opcode that it
   doesn't recognize. This may occur because of a new version of the
   protocol that defines a new Opcode, or because of some corruption of
   a message header.  The target returns the header of the message with
   the unrecognized opcode as the data of the response.

























Satran                Standards-Track, May 2001                    57

                                iSCSI                  November, 2000


3. Login phase

   The login phase establishes an iSCSI session between initiator and
   target. It sets the iSCSI protocol parameters, security parameters,
   and authenticates initiator and target to each other.

   The login phase is implemented via login and text commands and
   responses only. The login command is sent from the initiator to
   target in order to start the login phase and the login response is
   sent from the target to the initiator to conclude the login phase.
   Text messages are used to implement negotiation, establish security
   and set operational parameters.

3.1 Login phase start

   The login phase starts with a login request via a login command from
   the initiator to the target. The login request includes:

      -Protocol version supported by the initiator (currently 1.0)
      -Session and connection Ids
      -Security Parameters (if security is requested) and
      -Protocol parameters

   The target can answer in the following ways:

      -Login Response with Login Accept with session ID and iSCSI
      parameters.  In this case, the target does not support any
      security or authentication mechanism and starts with the
      session immediately (enters full feature phase)
      -Login Response with Login Reject.  This is an immediate
      rejection from the target causing the session to terminate.
      Causes for rejection are address rejection, local protection
      etc..
      -A text response with the same Initiator Task ID as the login
      command.  This indicates the start of the authentication
      sequence. The command includes the protocol version supported
      by the target and the security parameters (not iSCSI
      parameters, those will be returned only after security is
      established to protect them) supported by the target.

3.2 Security negotiation

   The negotiation proceeds as follows:





Satran                Standards-Track, May 2001                    58

                                iSCSI                  November, 2000


      -The initiator sends a text command with an ordered list of the
      options it supports for each subject (encryption algorithm,
      authentication algorithm, iSCSI parameters and so on). The
      options are listed from the most preferable (to the initiator)
      to the least.
      -The target MUST reply with the first option in the list it
      supports.  The parameters are encoded in Unicode - UTF8 as
      key:value (e.g., the encryption option of triple-DES will
      appear as encryption:3des-cbc).  The initiator MAY send
      proprietary options as well. The ônoneö option MUST be included
      in the list, indicating no algorithm supported by the target.
      If security is to be established, the initiator MUST NOT send
      parameters other than security parameters in the login command.
      The general parameters should be negotiated only after security
      is established at the desired level.  Any operational
      parameters sent before establishing a secure context MUST be
      reset by both the target and the initiator when establishing
      the security context. For a list of security parameters see
      Appendix A.

3.3 iSCSI Security

   The security exchange sets the security mechanism and authenticates
   the user and the target to each other. The exchange proceeds
   according to the algorithms that were chosen in the negotiation phase
   and is conducted by the text commands key:value parameters.
   The security mechanism includes the following elements:

      -Initial authentication - the host and the target authenticate
      themselves to each other. A negotiable algorithm, e.g.,
      user/password or public key, provides this feature.
      -Message integrity - an integrity and authentication digest is
      attached to each packet and authenticates it. The algorithm is
      negotiable.
      -Encryption - data from host to target and from target to host
      is encrypted. The user MAY choose to encrypt only part of the
      data, e.g., headers only (for complexity reasons). Encryption
      MAY use IPsec. The algorithm and its parameters are negotiable.
      However, encryption is set before login.

   Using IPsec for encryption or authentication may eliminate the need
   for parameter negotiation at the iSCSI level (for example, ISAKMP for
   IPsec). However, there is still a need to negotiate for the algorithm
   itself.

   If security is established in the login phase note that:


Satran                Standards-Track, May 2001                    59

                                iSCSI                  November, 2000



      -After setting message integrity, each iSCSI message MUST
      include the appropriate digest field (i.e., each message after
      the one through which the target choose the algorithm.
      -If encryption is to be set (e.g., IPsec), it should be set
      prior to the login phase.
      -The iSCSI parameter negotiation (non-security parameters)
      SHOULD start only after security is established. This should be
      carried on text commands.











































Satran                Standards-Track, May 2001                    60

                                iSCSI                  November, 2000


4. iSCSI Error Handling and Recovery

4.1 Connection failure

   For any outstanding SCSI command, it is assumed that iSCSI in
   conjunction with SCSI at the initiator is able to keep enough
   information to be able to rebuild the command PDU, that outgoing data
   is available (in host memory) for retransmission while the command is
   outstanding. It is also assumed that at a target iSCSI and
   specialized TCP implementations are able to recover unacknowledged
   data packets from a closing connection or, alternatively the target
   has means to re-read data from a device server.  It is further
   assumed that a target will keep the "status & sense" for a command it
   has executed while the total number of outstanding commands and
   executed commands does not exceed its limit. A target will
   sequentially number the delivered responses and thus enable
   initiators to tell when a response is missing and which response is
   missing.

   Under those conditions, iSCSI will be able to keep a session in
   operation if it is able to keep/establish at least one TCP connection
   between the initiator and target in a timely fashion.  Unfortunately,
   the maximum admissible recovery time is a function of the target and
   for some devices and communications networks recovery may be complex
   and may percolate to upper software layers.  It is assumed that
   targets and/or initiators will recognize a failing connection by
   either transport level means (TCP) or by a gap in the command or
   response stream that is not filled for a long time, or by a failing
   iSCSI ping (the later should be used periodically by highly reliable
   implementations).  Initiators and targets SHOULD use the keep-alive
   option on the TCP connection to enable early link failure detection
   on idle links.

   The iSCSI recovery involves the following steps:

      -abort offending TCP connection(s) (target & initiator) and
      recover at target all unacknowledged read-data
      -issue a Logout command on a remaining connection or create a
      new connection and issue the Logout command
      -wait for the Logout response
      -if needed create one or more new TCP connections (within the
      same session) and associate all outstanding commands from the
      failed connection to the new connection at both initiator and
      target.




Satran                Standards-Track, May 2001                    61

                                iSCSI                  November, 2000


      -the initiator will reissue all outstanding commands with their
      original Initiator Task Tag and their original CmdRN if they
      are not acknowledged yet or a new CmdRN if they were
      acknowledged; the retry (X) flag in the command PDU will be set
      -upon receiving the new/retry commands the target will resume
      command execution; for write commands it means requesting data
      retransmission through R2T, for reads retransmitting recovered
      data and for "terminated" commands retransmitting the status &
      sense while retaining the original StatRN. If data recovery is
      not possible, the target will either provide data from the
      media or redo the operation (if the operation is not idempotent
      the device server may fail the operation).


4.2 Protocol Errors

   The authors recognize that mapping framed messages over a "stream"
   connection (like TCP) makes the proposed mechanisms vulnerable to
   simple software framing errors and introducing framing mechanisms may
   be onerous for performance and bandwidth.  Command reference numbers
   and the above mechanisms for connection drop and reestablishment will
   help handle this type of mapping errors.

4.3 Session Errors

   If all the connections of a session fail and can't be reestablished
   in a short time or if initiators detect protocol errors repeatedly an
   initiator may choose to terminate a session and establish a new
   session. It will terminate all outstanding requests with an iSCSI
   error indication before initiating a new session.  A target that
   detects one of the above errors will take the following actions:

      - Reset the TCP connections (close the session).
      - Abort all Tasks in the task set for the corresponding
      initiator.














Satran                Standards-Track, May 2001                    62

                                iSCSI                  November, 2000


5. Notes to Implementers

   This section notes some of the performance and reliability
   considerations of the iSCSI protocol.  This protocol was designed to
   allow efficient silicon and software implementations. The iSCSI tag
   mechanism was designed to enable RDMA at the iSCSI level or lower.

5.1 Small TCP Segments

   It is recommended that TCP segments be limited in size to no more
   than 8K bytes. One reason we recommend small segments is to allow a
   stronger type of checksum, possibly utilizing CRC, which is practical
   only for smaller segments.

5.2 Multiple Network Adapters

   The iSCSI protocol allows multiple connections, not all of which need
   go over the same network adapter. If multiple network connections are
   to be utilized with hardware support, the iSCSI protocol command-
   data-status allegiance to one TCP connection insure that there is no
   need to replicate information across network adapters or otherwise
   require them to cooperate.

5.3 Autosense

   Autosense refers to the automatic return of sense data to the
   initiator in case a command did not complete successfully. iSCSI
   mandates support for autosense.






















Satran                Standards-Track, May 2001                    63

                                iSCSI                  November, 2000


6. Security Considerations

6.1 Data Integrity

   We assume that basic level end-to-end data integrity can be assured
   by TCP, by using the standard checksum.  For those applications for
   which data integrity is of utmost importance iSCSI will provide an
   integrity option.

6.2 Network operations and the Threat Model

   Historically, native storage systems have not had to consider
   security because their environments offered minimal security risks.
   That is, these environments consisted of storage devices either
   directly attached to hosts or connected via a subnet distinctly
   separate from the communications network. The use of storage
   protocols, such as SCSI, over IP networks requires that security
   concerns be addressed.

6.2.1 Threat Model

   Attacks fall into three main areas; passive, active, and denial of
   service.

6.2.1.1 Passive Attacks

   In general, data transfers will be made through a switched fabric,
   making sniffing difficult. In addition, the nature of the data (block
   transfers), even if sniffed, would not necessarily be readily
   understandable to the attacker.  That being said, a determined
   attacker, by capturing of content and analyzing traffic over time,
   could replicate enough of a drive to make the captured data
   meaningful. Certain storage operations which are mostly
   unidirectional, such as writing to a tape or reading from a CD-ROM,
   are even more susceptible to passive attacks since the listener will
   be able to replicate most if not all of the operation.

   Passive attacks by traffic analysis alone is deemed out of scope
   since it is unlikely that the listener will be able to guess any
   pertinent information without knowing the content of the messages.
   It is also out of scope to detect passive attacks. The protocol must
   be able to prevent passive attacks by masking the contents of
   messages through some form of encryption.

   Finally, it is assumed that a strong authentication mechanism will be
   necessary. Therefore, any long-lived passwords or private keys must
   never be sent in the clear.

Satran                Standards-Track, May 2001                    64

                                iSCSI                  November, 2000



6.2.1.2 Active Attacks

   Whereas passive attacks involve SNIFFING, active attacks will
   generally involve SPOOFING. If an attacker can successfully
   masquerade as a client, he will have total read/write access to those
   storage resources assigned to that client. Spoofing as a server is
   more difficult, since many operations involve client reads of some
   expected or otherwise understandable data.

   Most likely, many of the sessions will be long-lived. This feature
   has a dual effect of making these sessions more vulnerable to attack
   (hijacking TCP connections, cryptographic attacks), while at the same
   time providing mechanisms to detect attacks. An attempt to open a
   session while one is already active can be treated as a possible
   attack. Both the transport and session layer protocols will have
   sequencing that would need to be adhered to by the attacker to avoid
   generating errors that could also be treated as a possible attack.

   Message modification can be a significant threat to an environment
   reliant on the integrity of the data. Message replay, insertion, or
   deletion will generally produce errors (such as data
   overruns/underruns) that can be recovered successfully, they can have
   the effect of reducing performance, and as such can act as a denial
   of service. It is possible that an attacker can modify a message in
   such a way the session becomes uncoordinated, resulting in a tear
   down of the session.

6.2.2 Security Model

6.2.2.1 No Security

   This mode does not authenticate nor does it encrypt data. This mode
   should only be used in environments where there is minimal security
   risk and little chance for configuration errors.

6.2.2.2 End-to-End Authentication

   This mode protects against an unauthorized access to storage
   resources either through an active attack (SPOOFING) or configuration
   errors. Once the client is authenticated, all messages are sent and
   received in the clear.  This mode should only be used when there is
   minimal risk to man-in-the-middle attacks, eavesdropping, message
   insertion, deletion, and modification. For example, this mode can be
   used when IPsec is used in security gateways.

6.2.2.3 iSCSI integrity and authentication

Satran                Standards-Track, May 2001                    65

                                iSCSI                  November, 2000



   The iSCSI protocol provides an authentication mechanism for initiator
   and target. This includes login authentication and authentication
   trailers for headers and data. No encryption is provided at the iSCSI
   protocol level. The implementers may use other protocols (e.g.,
   IPsec, TLS, SSH, SSL and more) for this purpose.

   The implementers MAY use the iSCSI command numbering to protect
   against command replay and/or extend it with a filtering mechanism
   with time and date.

6.2.2.4 Encryption

   This mode provides for the end-to-end encryption (e.g. IPsec). In
   addition to authenticating the client, it provides end-to-end data
   integrity and protects against man-in-the-middle attacks,
   eavesdropping, message insertion, deletion, and modification.

   A connection or multiple connections can be protected end-to-end by
   using IPSec.  In this case, the initiator must use the "Implicit
   Authentication" parameter to indicate that IPSec should be used to
   specify the Access ID and perform authentication.

6.2.3 Other Considerations

   Due to long-lived sessions, is there a need for periodic
   authentication after the session is established? For example, should
   the client be challenged during key-alive exchanges in addition to
   login?

   Due to long-lived sessions with encryption, is there a higher level
   of vulnerability to cryptographic attacks?

6.3 Login Process

   In some environments, a target will not be interested in
   authenticating the initiator. In this case, the target can simply
   ignore some or all of the parameters sent in a Login Command, and the
   target can simply reply with a basic Login Response indicating a
   successful login.  Some targets MAY want to perform some kind of
   authentication. Various authentication schemes can be used, including
   encrypted passwords and trusted certificate authorities.  Once the
   initiator and target are confident of the identity of the attached
   party, the established channel is considered secure.

6.4 Feasibility


Satran                Standards-Track, May 2001                    66

                                iSCSI                  November, 2000


   The encryption algorithms are computationally complex. Therefore, the
   real time constraints on the transmission and reception may render
   difficult the implementation of completely encrypted streams. Working
   with fast networks will force the implementers to use one of the
   following alternatives:

      -Hardware implementation
      -Partial encryption

   The first alternative enables the use of completely encrypted
   streams, that although robust, may be (at least at top speeds)
   expensive.

   The second alternative can be software implemented, but will reduce
   the safety of the system.  In most cases, however, the safety
   tradeoff is acceptable (e.g., encryption of headers only by defining
   an IPsec policy).

   Data integrity/authentication through data and header digests can
   easily be performed.































Satran                Standards-Track, May 2001                    67

                                iSCSI                  November, 2000


7. IANA Considerations

   There will be a well-known port for iSCSI connections.  This well
   known port is registered with IANA.

















































Satran                Standards-Track, May 2001                    68

                                iSCSI                  November, 2000


8. References and Bibliography

      [AC]  A detailed proposal for Access Control, Jim Hafner,
      T10/99-245
      [ALTC]    Internet Draft: Alternative checksums (work in
      progress)
      [CAM]     ANSI X3.232-199X, Common Access Method-3 (Cam-3)
      [CRC]     ISO 3309, High-Level Data Link Control (CRC 32)
      [FIPS-186] Federal Information Processing Standards Publication
      (FIPSPUB) 186, Digital Signature Standard, 18 May 1994.
      [Orm96] Orman, H., "The Oakley Key Determination Protocol",
      version 1, TR97-92, Department of Computer Science Technical
      Report, University of Arizona.
      [PKIX-Part1] Housley, R., et al, "Internet X.509 Public Key
      Infrastructure, Certificate and CRL Profile", Internet Draft,
      draft-ietf-pkix-ipki-part1-11.txt
      [RFC793]  Transmission Control Protocol, RFC 793
      [RFC1122] Requirements for Internet Hosts-Communication Layer,
      RFC1122, R. Braden (editor)
      [RFC-1766] Alvestrand, H., "Tags for the Identification of
      Languages", March 1995.
      [RFC1982] Elz, R., Bush, R., "Serial Number Arithmetic", RFC
      1982, August 1996.
      [RFC2026] Bradner, S., "The Internet Standards Process --
      Revision 3", RFC 2026, October 1996.
      [RFC-2044] Yergeau, F., "UTF-8, a Transformation Format of
      Unicode and ISO 10646", October 1996.
      [RFC-2104] Krawczyk, H., Bellare, M., and Canetti, R., "HMAC:
      Keyed-Hashing for Message Authentication", February 1997
      [RFC-2119] Bradner, S. "Key words for use in RFCs to Indicate
      Requirement Levels", BCP 14, RFC 2119, March 1997.
      [RFC-2144] Adams, C., "The CAST-128 Encryption Algorithm", May
      1997.
      [RFC-2234] D. Crocker, P. Overell Augmented BNF for Syntax
      Specifications: ABNF
      [RFC-2434] T. Narten, and H. Avestrand, "Guidelines for Writing
      an IANA Considerations Section in RFCs.", RFC2434,  October
      1998.
      [RFC-2440] Callas, J., et al, "OpenPGP Message Format",
      November 1998.
      [SAM2]    ANSI X3.270-1998, SCSI-3 Architecture Model (SAM-2)
      [SBC]     ANSI X3.306-199X, SCSI-3 Block Commands (SBC)
      [SCSI2]   ANSI X3.131-1994, SCSI-2
      [Schneier] Schneier, B., "Applied Cryptography Second Edition:
      protocols, algorithms, and source code in C", 2nd edition, John
      Wiley & Sons, New York, NY, 1996.
      [SPC]     ANSI X3.301-199X, SCSI-3 Primary Commands (SPC)

Satran                Standards-Track, May 2001                    69

                                iSCSI                  November, 2000


      [TLS]     The TLS Protocol, RFC 2246, T. Dierks et al.




















































Satran                Standards-Track, May 2001                    70

                                iSCSI                  November, 2000


9. Author's Addresses

        Julian Satran
        Kalman Meth
        IBM, Haifa Research Lab
        MATAM - Advanced Technology Center
        Haifa 31905, Israel
        Phone +972 4 829 6211
        Email: Julian_Satran@vnet.ibm.com meth@il.ibm.com



        Daniel F. Smith
        IBM Almaden Research Center
        650 Harry Road
        San Jose, CA 95120-6099, USA
        Phone: +1 408 927 2072
        Email: dfsmith@almaden.ibm.com


        Costa Sapuntzakis
        Cisco Systems, Inc.
        170 W. Tasman Drive
        San Jose, CA 95134, USA
        Phone: +1 408 525 5497
        Email: csapuntz@cisco.com



        Randy Haagens
        Hewlett-Packard Company
        8000 Foothills Blvd.
        Roseville, CA 95747-5668, USA
        Phone: +1 (916) 785-4578
        E-mail: Randy_Haagens@hp.com

        Matt Wakeley
        Agilent Technologies
        1101 Creekside Ridge Drive
        Suite 100, M/S RH21
        Roseville, CA 95661
        Phone: +1 (916) 788-5670
        E-Mail: matt_wakeley@agilent.com

        Efri Zeidner
        SANGate


Satran                Standards-Track, May 2001                    71

                                iSCSI                  November, 2000


        Israel
        efri@sangate.com



















































Satran                Standards-Track, May 2001                    72

                                iSCSI                  November, 2000




        Paul von Stamwitz
        Adaptec, Inc.
        691 South Milpitas Boulevard
        Milpitas, CA 95035
        Phone: +1(408) 957-5660
        E-mail:  paulv@corp.adaptec.com


        Luciano Dalle Ore
        Quantum Corp.
        Phone: +1(408) 232 6524
        E-mail:  lldalleore@snapserver.com

        Yaron Klein
        SANRAD
        24 Raul Valenberg St.
        Tel-Aviv, 69719 Israel
        Phone: +972-3-7659998
        E-mail:  klein@sanrad.com




   Comments may be sent to Julian Satran
























Satran                Standards-Track, May 2001                    73

                                iSCSI                  November, 2000


Apendix A. iSCSI Security

01 Security keys and values

   The parameters (keys) negotiated for security are:

      - digests (header_digest:, data_digest:)
      - authentication methods (init_auth:, target_auth:)
      - public key algorithm (public_key)

   The following table lists message authentication and checksums for
   the digests.

   +-----------------------------------------------------------+
   | Name          | Description                 | Definition  |
   +-----------------------------------------------------------+
   | hmac-sha1     | HMAC-SHA1 length=20         | RFC-2104    |
   +-----------------------------------------------------------+
   | hmac-sha-96   | first 96 bits of HMAC-SHA 1 | RFC-2104    |
   +-----------------------------------------------------------+
   | hmac-md5      | HMAC-MD5 length 16          | RFC-2104    |
   +-----------------------------------------------------------+
   | hmac-md5-96   | first 96 bits of HMAC-MD5   | RFC-2104    |
   +-----------------------------------------------------------+
   | crc32         | 32 bit CRC for the message  | CCITT       |
   +-----------------------------------------------------------+
   | crc32-2k      | 32 bit CRC per 2k of message| CCITT       |
   +-----------------------------------------------------------+
   | none          | no digest                   | -           |
   +-----------------------------------------------------------+

   Other and proprietary algorithms MAY also be negotiated.
   The none value is the only one that MUST be supported.

   Note that if IPsec is used there is no need for digests.

   CRC32 is effective only for limited data lengths (the probability of
   an error going undetected grows linearly with data length). When
   using CRC32-2K the digest size increases with data length.

   CRC32 and CRC32-2K do not authenticate data (there is no session
   specific parameter in the algorithm).

   The following table details authentication methods.

   +-----------------------------------------------------------+
   | Name          | Description                               |

Satran                Standards-Track, May 2001                    74

                                iSCSI                  November, 2000


   +-----------------------------------------------------------+
   | publickey     | Public key authentication                 |
   +-----------------------------------------------------------+
   | password      | Plain text user-password                  |
   +-----------------------------------------------------------+
   | challenge     | Challenge and response                    |
   +-----------------------------------------------------------+
   | none          | No authentication                         |
   +-----------------------------------------------------------+

   The following table details public key algorithms for authentication.

   +-----------------------------------------------------------+
   | Name          | Description                 | Definition  |
   +-----------------------------------------------------------+
   | ssh-dss       | Simple DSS                  | [FIPS-186]  |
   +-----------------------------------------------------------+
   | x509-v3       | X509                        | [PKIX]      |
   +-----------------------------------------------------------+
   | spki          | SPKI                        | [SPKI]      |
   +-----------------------------------------------------------+
   | pgp           | Open PGP                    | RFC-2440    |
   +-----------------------------------------------------------+
   | none          | No Public Key               | -           |
   +-----------------------------------------------------------+

   Where the public key information is encoded as:

      public_key:<name>,<parameters>

   For example, if ssh-dss is selected:

      public_key:ssh-dss,p,q,g,y

   Here the "p", "q", "g", and "y" parameters (encoded as numbers in
   Unicode UTF8) form the signature key blob.

   Signing and verifying using this key format are done according to the
   Digital Signature Standard [FIPS-186] using the SHA-1 hash. A
   description can also be found in [Schneier].

   The dss signature blob is encoded as a string containing "r" followed
   by "s" (which are 160 bits long integers, without lengths or padding,
   unsigned and in network byte order).




Satran                Standards-Track, May 2001                    75

                                iSCSI                  November, 2000


   The "x509v3" method indicates that the certificates, the public key,
   and the resulting signature are in X.509v3 compatible DER-encoded
   format. The format used in X.509v3 is described in [PKIX] Part1.

   The "spki" method indicates that the certificate blob contains a
   sequence of SPKI certificates. The format of SPKI certificates is
   described in [SPKI].

   The "pgp" method indicates that the certificates, the public key, and
   the signature are in OpenPGP compatible binary format [RFC-2440].


02 Authentication

   The authentication exchange SHOULD authenticate the initiator and
   target to each other.  Authentication is not mandatory and is
   distinct from the data integrity exchange.

   Different levels of authentication can be applied such as initiator
   authentication, target authentication or both.

   The authentication methods to be used are public key, user/password
   or challenge/response.

   If public key is selected then each party MUST use:

      authenticate:<user-id>,<blob>

   where user-id is the SCSI access-id of the host-OS for the initiator
   or the World-Wide-Name for the target and blob is the public-key
   blob.

   For user/password each party must use:

      authenticate:<user-id>,<password>

   where user-id is as above and password is a plain-text password.

03 Salt

   salt:<number> can be used by different authentication schemes to
   prevent replay attacks (a random number or a time stamp or both)

04 Challenge

   challenge:<string> and authenticate:<string> MUST be used for
   challenge answer schemes

Satran                Standards-Track, May 2001                    76

                                iSCSI                  November, 2000



05 Login Phase examples:

   The first example is a "user-password" authentication:

   In this example, the result of the negotiation is to use md5 for
   header digest, crc32-2k for data digest and user/password for
   initiator authentication. No target authentication required.

      I-> Login header_digest:(hmac-md5,hmac-md5-96,crc32,none)
      data_digest:(crc32-2k) init_auth:(public-key,password,none)
      target_auth:(none) public_key:(x509-v3)
      T-> Text header_digest:hmac-md5 data_digest:crc32-2k
      init_auth:password
      I-> Text authenticate:alef,sesam
   If the authentication is successful:
      T->StartSecure:HERE
      ...
      T-> Login ôlogin acceptö
   If the authentication was not successful:
      T-> Login ôlogin rejectö


   Note - the Text command including SecureStart:HERE and each PDU after
   it will have the trailer consisting in a hmac-md5 digest for the
   header and a crc32 for each 2k of data (or fraction thereof).

   The next example is a "public-key" authentication. The initiator
   authenticates itself to the target; no keys are exchanged:

       I-> Login header_digest:(hmac-md5,hmac-md5-
      96,crc32,none)data_digest:(crc32-2k,none)
      init_auth:(publickey,password,none) target_auth:(none)
      public_key:((X509v3,blob),(ssh-dss,blob),none)
      T-> Text header_digest:hmac-md5 data_digest:crc32-2k
      init_auth:publickey public_key:(ssh-dss,blob)
      I-> Text authenticate:user,blob salt:578913456

             NB - where the blob stands for the hash of the packet,
         and the secret i.e., hash(key || packet).  The initiator
         SHOULD add "salt" to the packet, e.g. add the pair
         salt:<random-number> (or timestamp or a mixture) to its
         packet to prevent record and replay.





Satran                Standards-Track, May 2001                    77

                                iSCSI                  November, 2000


         If the user was not confirmed, the target sends a login
         response message with ôlogin rejectö to the initiator. Else,
         it can send a login response with ôlogin acceptö and MAY
         attach a secret:

      T->Text StartSecure:HERE secret:
      I->Text ... parameters ...EndLogin:HERE
      T->Login (accept) ... parameters ...

   The next example is another "public-key" authentication. The
   initiator authenticates itself to the target. The target
   authenticates itself to the initiator and key are exchanged:

       I-> Login header_digest:(hmac-md5,hmac-md5-
      96,crc32,none)data_digest:(crc32-2k,none)
      init_auth:(publickey,password,none) target_auth:
      (none) public_key:((X509v3,blob),(ssh-dss,blob),none)
      T-> Text header_digest:hmac-md5 data_digest:crc32-2k
      init_auth:publickey public_key:(ssh-dss,blob)
      target_auth:(publickey,password,none) public_key:(ssh-
      dss,blob),none
      I-> Text authenticate:user,blob target_auth:publickey
      public_key:ssh_dss,blob salt:20001103172433


         Note: the last packet should have the appropriate trailers.

   If the initiator was not confirmed, the target sends a login response
   message with ôlogin rejectö to the initiator. Else, it can continue
   with the login process:

      T-> Text authenticate:user,blob salt:532678925

   In here, the target authenticates itself to the initiator. If the
   authentication was successful, the initiator responses with an empty
   text command, continuing the login phase. Else, it stops the login
   phase.

      I->Text
      T->Text secret:blob

   Where blob is a key encrypted with the initiatorÆs public key.

      I->Text StartSecure:HERE... parameters ...
      ...
      T->Login "login accept" ... parameters ...


Satran                Standards-Track, May 2001                    78

                                iSCSI                  November, 2000



   In the next example the target authenticates the initiator via
   challenge and response.


      I-> Login header_digest:(hmac-md5,hmac-md5-96,crc32,none)
      data_digest:(crc32-2k) init_auth:(public-
      key,password,challenge,none) target_auth:(none)
      public_key:(x509-v3)
      T-> Text header_digest:hmac-md5 data_digest:crc32-2k
      init_auth:challenge challenge:question
      I-> Text authenticate:answer

   If authentication is successful, i.e., the answer to the question is
   correct, the target may proceeds:

      T->... parameter negotiation

   Or give another challenge:

      T-> Text challenge:question2
      I-> Text authenticate:answer2

   And at the end:

      T-> Login ôlogin acceptö

   If the authentication was not successful:

      T-> Login ôlogin rejectö

   Note - the Text command after authentication and each PDU thereafter
   will have in the trailer an hmac-md5 digest for the header and a
   crc32 for each 2k of data (or fraction of it).















Satran                Standards-Track, May 2001                    79

                                iSCSI                  November, 2000


Apendix B. Examples

06 Read operation example

   |Initiator Function|    Message Type       |  Target Function     |
   +------------------+-----------------------+----------------------+
   |  Command request |SCSI Command (READ)>>> |                      |
   |  (read)          |                       |                      |
   +------------------+-----------------------+----------------------+
   |                  |                       | Prepare Data Transfer|
   +------------------+-----------------------+----------------------+
   |   Receive Data   |   <<< SCSI Data       |   Send Data          |
   +------------------+-----------------------+----------------------+
   |   Receive Data   |   <<< SCSI Data       |   Send Data          |
   +------------------+-----------------------+----------------------+
   |   Receive Data   |   <<< SCSI Data       |   Send Data          |
   +------------------+-----------------------+----------------------+
   |                  |   <<< SCSI Response   |Send Status and Sense |
   +------------------+-----------------------+----------------------+
   | Command Complete |                       |                      |
   +------------------+-----------------------+----------------------+






























Satran                Standards-Track, May 2001                    80

                                iSCSI                  November, 2000



07 Write operation example


   +------------------+-----------------------+---------------------+
   |Initiator Function|    Message Type       |  Target Function    |
   +------------------+-----------------------+---------------------+
   |  Command request |SCSI Command (WRITE)>>>| Receive command     |
   |  (write)         |                       | and queue it        |
   +------------------+-----------------------+---------------------+
   |                  |                       | Process old commands|
   +------------------+-----------------------+---------------------+
   |                  |                       | Ready to process    |
   |                  |   <<< R2T             | WRITE command       |
   +------------------+-----------------------+---------------------+
   |   Send Data      |   SCSI Data >>>       |   Receive Data      |
   +------------------+-----------------------+---------------------+
   |                  |   <<< R2T             |                     |
   +------------------+-----------------------+---------------------+
   |                  |   <<< R2T             |                     |
   +------------------+-----------------------+---------------------+
   |   Send Data      |   SCSI Data >>>       |   Receive Data      |
   +------------------+-----------------------+---------------------+
   |   Send Data      |   SCSI Data >>>       |   Receive Data      |
   +------------------+-----------------------+---------------------+
   |                  |   <<< SCSI Response   |Send Status and Sense|
   +------------------+-----------------------+---------------------+
   | Command Complete |                       |                     |
   +------------------+-----------------------+---------------------+





















Satran                Standards-Track, May 2001                    81

                                iSCSI                  November, 2000


Apendix C. Login/Text keys (not security related)

   ISID and TSID form collectively the SSID (session id). A TSID of zero
   indicates a leading connection. Only a leading connection login can
   carry session specific parameters, e.g. MaxConnections, the maximum
   immediate data length requested, etc..

08 MaxConnections

   MaxConnections:<number-from-1-to-65442>

   Initiator and target negotiate the maximum number of connections
   requested/acceptable.


09 Target

   Target:<domainname>[/modifier]

   Examples:

      Target:disk-array.sj-bldg-h.cisco.com
      Target:disk-array.sj-bldg-h.cisco.com/control7

   This key is provided by the initiator of the TCP connection to the
   remote endpoint. The Target key specifies the domain name of the
   target, since that information is not available from the TCP layer.
   The target is not required to support this key.  The initiator should
   send this key in the first login message. The Target key might be
   used by the target to select a unit within a multi-unit target.

10 Initiator

   Initiator:[domainname[/modifier]] Examples:

      Initiator:sample.foobar.org
      Initiator:cluster.foobar.org/machine1
      Initiator:

   The Initiator key enables the initiator to identify itself to the
   remote endpoint. The domain name should be that of the initiator.  A
   zero-length domain name is interpreted as "other side of TCP
   connection". The target may silently ignore this key if it does not
   support it.

11 UseR2T


Satran                Standards-Track, May 2001                    82

                                iSCSI                  November, 2000


   UseR2T:<yes|no>

   Examples:

      I->UseR2T:no
      T->UseR2T:no

   The UseR2T key is used to turn off the default use of R2T, thus
   allowing an initiator to send data to a target without the target
   having sent an R2T to the initiator.  The default action is that R2T
   is required, unless both the initiator and the target send this key-
   pair attribute specifying UseR2T:no.  Once UseR2T has been set to
   'no', it cannot be set back to 'yes'.  Note than only the first
   outgoing data item (either immediate data or a separate PDU) can be
   sent unsolicited by a R2T.

12 BidiUseR2T

   BidiUseR2T:<yes|no>

   Examples:

      I->BidiUseR2T:no
      T->BidiUseR2T:no

   The BidiUseR2T key is used to turn off the default use of BiDiR2T,
   thus allowing an initiator to send data to a target without the
   target having sent an R2T to the initiator for the output data (write
   part) of a Bi-directional command (having both the R and the W bits
   set).  The default action is that R2T is required, unless both the
   initiator and the target send this key-pair attribute specifying
   BidiUseR2T:no.  Once BidiUseR2T has been set to 'no', it cannot be
   set back to 'yes'.  Note than only the first outgoing data item
   (either immediate data or a separate PDU) can be sent unsolicited by
   a R2T.

13 DataNumber

   DataNumber:<yes|no>

   Example:

   The DataNumber key is used by targets to turn on the use of input
   data packet numbering, thus allowing a target to discard input data
   as soon as acknowledged without loosing recovery capabilities.  By
   default data numbering is off. An initiator MUST support data
   numbering if requested.

Satran                Standards-Track, May 2001                    83

                                iSCSI                  November, 2000



14 ImmediateDataLength

   ImmediateDataLength:<number>

   Initiator and target negotiate the maximum length supported for
   immediate data. Default is 4GB.


15 ITagLength

   ITagLength:<number-from8-to-32>

   Initiator and target negotiate the significant length of the
   initiator tag to be used. Default is 32.


16 PingMaxReplyLength

   PingMaxReplyLength:<number>

   Initiator and target negotiate the maximum length of data contained
   in a ping reply. Default is 4096.

17 StartSecure

   StartSecure:HERE

   Initiator and target indicate the end-of-authentication/integrity
   exchange (start of parameter negotiation if any).

18 EndLogin

   EndLogin:HERE

   Initiator indicates the end of the login phase.

19 TotalText

   TotalText:<number-from-512-to-65442>

   Initiator and target indicate the total text limit for any Text or
   Login command.


20 KeyValueText


Satran                Standards-Track, May 2001                    84

                                iSCSI                  November, 2000


   KeyValueText:<number-from-246-to-8192>

   Initiator and target indicate the total text limit for any key:value
   pair.

21 MaxOutstandingR2T

   MaxOutstandingR2T:<number-from-1-to-65442>

   Initiator and target negotiate the maximum number of outstanding R2Ts
   per task. The default is 256.









































Satran                Standards-Track, May 2001                    85

                                iSCSI                  November, 2000



Full Copyright Statement

   "Copyright (C) The Internet Society (date). All Rights Reserved. This
   document and translations of it may be copied and furnished to
   others, and derivative works that comment on or otherwise explain it
   or assist in its implementation may be prepared, copied, published
   and distributed, in whole or in part, without restriction of any
   kind, provided that the above copyright notice and this paragraph are
   included on all such copies and derivative works. However, this
   document itself may not be modified in any way, such as by removing
   the copyright notice or references to the Internet Society or other
   Internet organizations, except as needed for the purpose of
   developing Internet standards in which case the procedures for
   copyrights defined in the Internet Standards process must be
   followed, or as required to translate it into languages other than
   English.

   The limited permissions granted above are perpetual and will not be
   revoked by the Internet Society or its successors or assigns.

   This document and the information contained herein is provided on an
   "AS IS" basis and THE INTERNET SOCIETY AND THE INTERNET ENGINEERING
   TASK FORCE DISCLAIMS ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING
   BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION
   HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF
   MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE."
















Satran                Standards-Track, May 2001                    86


Html markup produced by rfcmarkup 1.129c, available from https://tools.ietf.org/tools/rfcmarkup/