draft-ietf-ips-iwarp-da-02.txt   draft-ietf-ips-iwarp-da-03.txt 
INTERNET DRAFT Mallikarjun Chadalapaka INTERNET DRAFT Mallikarjun Chadalapaka
draft-ietf-ips-iwarp-da-02.txt HP draft-ietf-ips-iwarp-da-03.txt HP
John Hufferd John Hufferd
IBM IBM
Julian Satran Julian Satran
IBM IBM
Hemal Shah Hemal Shah
Intel Intel
Expires September 2005 Expires December 2005
Datamover Architecture for iSCSI (DA) Datamover Architecture for iSCSI (DA)
1 Status of this Memo Status of this Memo
By submitting this Internet-Draft, each author represents
By submitting this Internet-Draft, we certify that any that any applicable patent or other IPR claims of which he or
applicable patent or other IPR claims of which we are aware she is aware have been or will be disclosed, and any of which
have been disclosed, or will be disclosed, and any of which he or she becomes aware will be disclosed, in accordance with
we become aware will be disclosed, in accordance with RFC Section 6 of BCP 79.
3668.
Internet-Drafts are working documents of the Internet Internet-Drafts are working documents of the Internet
Engineering Task Force (IETF), its areas, and its working Engineering Task Force (IETF), its areas, and its working
groups. Note that other groups may also distribute working groups. Note that other groups may also distribute working
documents as Internet-Drafts. documents as Internet-Drafts.
Internet-Drafts are draft documents valid for a maximum of Internet-Drafts are draft documents valid for a maximum of
six months and may be updated, replaced, or obsoleted by six months and may be updated, replaced, or obsoleted by
other documents at any time. It is inappropriate to use other documents at any time. It is inappropriate to use
Internet-Drafts as reference material or to cite them other Internet-Drafts as reference material or to cite them other
than a "work in progress." than a "work in progress."
The list of current Internet-Drafts can be accessed at The list of current Internet-Drafts can be accessed at
http://www.ietf.org/1id-abstracts.html http://www.ietf.org/1id-abstracts.html
The list of Internet-Draft Shadow Directories can be accessed The list of Internet-Draft Shadow Directories can be accessed
at http://www.ietf.org/shadow.html. at http://www.ietf.org/shadow.html.
2 Abstract Abstract
iSCSI is a SCSI transport protocol that maps the SCSI family iSCSI is a SCSI transport protocol that maps the SCSI family
of application protocols onto TCP/IP. The Datamover of application protocols onto TCP/IP. The Datamover
Architecture for iSCSI (DA) defines an abstract model in Architecture for iSCSI (DA) defines an abstract model in
which the movement of data between iSCSI end nodes is which the movement of data between iSCSI end nodes is
logically separated from the rest of the iSCSI protocol in logically separated from the rest of the iSCSI protocol in
order to allow iSCSI to adapt to innovations available in new order to allow iSCSI to adapt to innovations available in new
IP transports. The new Datamover protocol provides a IP transports. The new Datamover protocol provides a
reliable transport for all iSCSI PDUs, but actually moves the reliable transport for all iSCSI PDUs, but actually moves the
data required for certain iSCSI PDUs without involving the data required for certain iSCSI PDUs without involving the
remote iSCSI layer itself. This document begins with an remote iSCSI layer itself. This document begins with an
skipping to change at page 3, line 7 skipping to change at page 3, line 7
models the interactions within an iSCSI end node between the models the interactions within an iSCSI end node between the
iSCSI layer and the Datamover layer that happen in order to iSCSI layer and the Datamover layer that happen in order to
transparently perform remote data movement within an IP transparently perform remote data movement within an IP
fabric. It is intended that this definition would help map fabric. It is intended that this definition would help map
iSCSI to generic RDMA-capable IP fabrics in the future iSCSI to generic RDMA-capable IP fabrics in the future
comprising TCP, SCTP, and possibly other underlying network comprising TCP, SCTP, and possibly other underlying network
transport layers. transport layers.
Table of Contents Table of Contents
1 Status of this Memo ....................................1 1 Definitions and acronyms ...............................5
2 Abstract ...............................................2 1.1 Definitions ............................................5
3 Definitions and acronyms ...............................6 1.2 Acronyms ...............................................5
3.1 Definitions ............................................6 2 Motivation .............................................7
3.2 Acronyms ...............................................6 2.1 Intent .................................................7
4 Motivation .............................................8 2.2 Interpretation of Requirements .........................8
5 Architectural layering of iSCSI and Datamover layers ..10 3 Architectural layering of iSCSI and Datamover layers ...9
6 Design Overview .......................................12 4 Design Overview .......................................11
7 Architectural Concepts ................................14 5 Architectural Concepts ................................13
7.1 iSCSI PDU types .......................................14 5.1 iSCSI PDU types .......................................13
7.1.1 iSCSI data-type PDUs.................................14 5.1.1 iSCSI data-type PDUs.................................13
7.1.2 iSCSI control-type PDUs..............................15 5.1.2 iSCSI control-type PDUs..............................14
7.2 Data_Descriptor .......................................15 5.2 Data_Descriptor .......................................14
7.3 Connection_Handle .....................................15 5.3 Connection_Handle .....................................14
7.4 Operational Primitive .................................16 5.4 Operational Primitive .................................15
7.5 Transport Connection ..................................17 5.5 Transport Connection ..................................16
8 Datamover layer and Datamover protocol ................18 6 Datamover layer and Datamover protocol ................17
9 Operational Primitives provided by the Datamover layer 20 7 Functional Overview ...................................19
9.1 Send_Control ..........................................20 7.1 Startup ...............................................19
9.2 Put_Data ..............................................21 7.2 Full Feature Phase ....................................19
9.3 Get_Data ..............................................22 7.3 Wrapup ................................................20
9.4 Allocate_Connection_Resources .........................22 8 Operational Primitives provided by the Datamover layer 22
9.5 Deallocate_Connection_Resources .......................23 8.1 Send_Control ..........................................22
9.6 Enable_Datamover ......................................24 8.2 Put_Data ..............................................23
9.7 Connection_Terminate ..................................24 8.3 Get_Data ..............................................24
9.8 Notice_Key_Values .....................................25 8.4 Allocate_Connection_Resources .........................24
9.9 Deallocate_Task_Resources .............................25 8.5 Deallocate_Connection_Resources .......................25
10 Operational Primitives provided by the iSCSI layer ....27 8.6 Enable_Datamover ......................................26
10.1 Control_Notify.......................................27 8.7 Connection_Terminate ..................................26
10.2 Connection_Terminate_Notify..........................28 8.8 Notice_Key_Values .....................................27
10.3 Data_Completion_Notify...............................28 8.9 Deallocate_Task_Resources .............................27
10.4 Data_ACK_Notify......................................29 9 Operational Primitives provided by the iSCSI layer ....29
11 Datamover Interface (DI) ..............................31 9.1 Control_Notify ........................................29
11.1 Overview.............................................31 9.2 Connection_Terminate_Notify ...........................30
11.2 Interactions for handling asynchronous notifications.31 9.3 Data_Completion_Notify ................................30
11.2.1 Connection termination .............................31 9.4 Data_ACK_Notify .......................................31
11.2.2 Data transfer completion ...........................31 10 Datamover Interface (DI) ..............................33
11.2.3 Data acknowledgement ...............................32 10.1 Overview.............................................33
11.3 Interactions for sending an iSCSI PDU................33 10.2 Interactions for handling asynchronous notifications.33
11.3.1 SCSI Command .......................................33 10.2.1 Connection termination .............................33
11.3.2 SCSI Response ......................................34 10.2.2 Data transfer completion ...........................33
11.3.3 Task Management Function Request ...................34 10.2.3 Data acknowledgement ...............................34
11.3.4 Task Management Function Response ..................35 10.3 Interactions for sending an iSCSI PDU................35
11.3.5 SCSI Data-out & SCSI Data-in .......................35 10.3.1 SCSI Command .......................................35
11.3.6 Ready To Transfer (R2T) ............................35 10.3.2 SCSI Response ......................................36
11.3.7 Asynchronous Message ...............................36 10.3.3 Task Management Function Request ...................36
11.3.8 Text Request .......................................36 10.3.4 Task Management Function Response ..................37
11.3.9 Text Response ......................................36 10.3.5 SCSI Data-out & SCSI Data-in .......................37
11.3.10 Login Request ....................................37 10.3.6 Ready To Transfer (R2T) ............................37
11.3.11 Login Response ...................................37 10.3.7 Asynchronous Message ...............................38
11.3.12 Logout Command ...................................38 10.3.8 Text Request .......................................38
11.3.13 Logout Response ..................................38 10.3.9 Text Response ......................................38
11.3.14 SNACK Request ....................................38 10.3.10 Login Request ....................................39
11.3.15 Reject ...........................................39 10.3.11 Login Response ...................................39
11.3.16 NOP-Out ..........................................39 10.3.12 Logout Command ...................................40
11.3.17 NOP-In ...........................................39 10.3.13 Logout Response ..................................40
11.4 Interactions for receiving an iSCSI PDU..............39 10.3.14 SNACK Request ....................................40
11.4.1 SCSI Command .......................................40 10.3.15 Reject ...........................................41
11.4.2 SCSI Response ......................................40 10.3.16 NOP-Out ..........................................41
11.4.3 Task Management Function Request ...................40 10.3.17 NOP-In ...........................................41
11.4.4 Task Management Function Response ..................40 10.4 Interactions for receiving an iSCSI PDU..............41
11.4.5 SCSI Data-out & SCSI Data-in .......................40 10.4.1 General Control-type PDU notification ..............42
11.4.6 Ready To Transfer (R2T) ............................41 10.4.2 SCSI Data Transfer PDUs ............................42
11.4.7 Asynchronous Message ...............................42 10.4.3 Login Request ......................................43
11.4.8 Text Request .......................................42 10.4.4 Login Response .....................................44
11.4.9 Text Response ......................................42 11 Security Considerations ...............................45
11.4.10 Login Request ....................................42 11.1 Architectural Considerations.........................45
11.4.11 Login Response ...................................42 11.2 Wire Protocol Considerations.........................46
11.4.12 Logout Command ...................................43 12 IANA Considerations ...................................47
11.4.13 Logout Response ..................................43 13 References and Bibliography ...........................48
11.4.14 SNACK Request ....................................43 13.1 Normative References.................................48
11.4.15 Reject ...........................................43 13.2 Informative References...............................48
11.4.16 NOP-Out ..........................................43 14 Authors' Addresses ....................................49
11.4.17 NOP-In ...........................................43 15 Acknowledgements ......................................50
12 Security Considerations ...............................44 16 Appendix ..............................................54
13 IANA Considerations ...................................45 16.1 Design considerations for a Datamover protocol.......54
14 References and Bibliography ...........................46 16.2 Examples of Datamover interactions...................54
14.1 Normative References.................................46 17 Full Copyright Statement ..............................64
14.2 Informative References...............................46 18 Intellectual Property Statement .......................65
15 Authors' Addresses ....................................47
16 Acknowledgements ......................................48
17 Appendix ..............................................52
17.1 Design considerations for a Datamover protocol.......52
17.2 Examples of Datamover interactions...................53
18 Full Copyright Statement ..............................62
19 Intellectual Property Statement .......................63
Table of Figures Table of Figures
Figure 1 Datamover Architecture diagram, with the RDMAP Figure 1 Datamover Architecture diagram, with the RDMAP
example.....................................................10 example......................................................9
Figure 2 A successful iSCSI login on initiator..............54 Figure 2 A successful iSCSI login on initiator..............56
Figure 3 A successful iSCSI login on target.................54 Figure 3 A successful iSCSI login on target.................56
Figure 4 A failed iSCSI login on initiator..................55 Figure 4 A failed iSCSI login on initiator..................57
Figure 5 A failed iSCSI login on target.....................55 Figure 5 A failed iSCSI login on target.....................57
Figure 6 iSCSI does not enable the Datamover................56 Figure 6 iSCSI does not enable the Datamover................58
Figure 7 A normal iSCSI connection termination..............57 Figure 7 A normal iSCSI connection termination..............59
Figure 8 An abnormal iSCSI connection termination...........57 Figure 8 An abnormal iSCSI connection termination...........59
Figure 9 A SCSI Write data transfer.........................58 Figure 9 A SCSI Write data transfer.........................60
Figure 10 A SCSI Read data transfer.........................59 Figure 10 A SCSI Read data transfer.........................61
Figure 11 A SCSI Read data acknowledgement..................60 Figure 11 A SCSI Read data acknowledgement..................62
Figure 12 Task resource cleanup on abort...................61 Figure 12 Task resource cleanup on abort...................63
3 Definitions and acronyms 1 Definitions and acronyms
3.1 Definitions 1.1 Definitions
I/O Buffer A buffer that is used in a SCSI Read or Write I/O Buffer A buffer that is used in a SCSI Read or Write
operation so SCSI data may be sent from or received into operation so SCSI data may be sent from or received into
that buffer. that buffer.
Datamover protocol A Datamover protocol is a data transfer Datamover protocol A Datamover protocol is a data transfer
wire protocol for iSCSI that meets the requirements wire protocol for iSCSI that meets the requirements
stated in section 8. stated in section 6.
Datamover layer A Datamover layer is a protocol layer Datamover layer A Datamover layer is a protocol layer
within an end node that implements the Datamover within an end node that implements the Datamover
protocol. protocol.
Datamover-assisted - An iSCSI connection is said to be Datamover-assisted - An iSCSI connection is said to be
"Datamover-assisted" when a Datamover layer is enabled "Datamover-assisted" when a Datamover layer is enabled
for moving control and data information on that for moving control and data information on that iSCSI
connection. connection.
3.2 Acronyms 1.2 Acronyms
Acronym Definition Acronym Definition
------------------------------------------------------------- -------------------------------------------------------------
DA Datamover Architecture DA Datamover Architecture for iSCSI
DDP Direct Data Placement Protocol DDP Direct Data Placement Protocol
DI Datamover Interface DI Datamover Interface
IANA Internet Assigned Numbers Authority IANA Internet Assigned Numbers Authority
IETF Internet Engineering Task Force IETF Internet Engineering Task Force
I/O Input - Output I/O Input - Output
skipping to change at page 8, line 5 skipping to change at page 7, line 5
SN Sequence Number SN Sequence Number
SNACK Selective Negative Acknowledgment - also SNACK Selective Negative Acknowledgment - also
Sequence Number Acknowledgement for data Sequence Number Acknowledgement for data
TCP Transmission Control Protocol TCP Transmission Control Protocol
TTT Target Transfer Tag TTT Target Transfer Tag
4 Motivation 2 Motivation
2.1 Intent
There are new industry and standards initiatives to develop There are new industry and standards initiatives to develop
Remote Direct Memory Access (RDMA) and Remote Direct Data Remote Direct Memory Access (RDMA) and Remote Direct Data
Placement (RDDP) technologies to work over IP fabrics. The Placement (RDDP) technologies to work over IP fabrics. The
principal value proposition of these technologies is that principal value proposition of these technologies is that
they enable one end node to place data in the final intended they enable one end node to place data in the final intended
buffer on the remote end node, thus eliminating the data copy buffer on the remote end node, thus eliminating the data copy
that traditionally happens in the receive path to move the that traditionally happens in the receive path to move the
data to the final buffer. The data copy avoidance in turn data to the final buffer. The data copy avoidance in turn
eliminates unnecessary memory bandwidth consumption, substan- eliminates unnecessary memory bandwidth consumption, substan-
skipping to change at page 8, line 45 skipping to change at page 7, line 47
but would only be privy to the conclusion of the requested but would only be privy to the conclusion of the requested
data transfer Thus, there would be an effective "off- data transfer Thus, there would be an effective "off-
loading" of the work that an iSCSI protocol layer is expected loading" of the work that an iSCSI protocol layer is expected
to perform, compared to today's iSCSI end nodes. For such to perform, compared to today's iSCSI end nodes. For such
RDMA environments, it is highly desirable that there be a RDMA environments, it is highly desirable that there be a
standard architecture to separate the data movement part of standard architecture to separate the data movement part of
the iSCSI protocol definition from the rest of the iSCSI the iSCSI protocol definition from the rest of the iSCSI
functionality. This architecture precisely defines what a functionality. This architecture precisely defines what a
Datamover layer is and also describes the model of Datamover layer is and also describes the model of
interactions between the iSCSI layer and the Datamover layer interactions between the iSCSI layer and the Datamover layer
(section 8). In order to satisfy this need, this document (section 6). In order to satisfy this need, this document
presents a Datamover Architecture (DA) and also summarizes a presents a Datamover Architecture for iSCSI(DA) and also
reasonable model for interactions between the iSCSI layer and summarizes a reasonable model for interactions between the
the Datamover layer for each of the iSCSI PDUs that are iSCSI layer and the Datamover layer for each of the iSCSI
defined in [RFC3720]. Note that while DA is motivated by the PDUs that are defined in [RFC3720]. Note that while DA is
advent of RDMA over TCP/IP technology, the architecture is motivated by the advent of RDMA over TCP/IP technology, the
not dependent on RDMA in its design. DA is intended to be a architecture is not dependent on RDMA in its design. DA is
generic architectural framework for allowing different types intended to be a generic architectural framework for allowing
of Datamovers based on different types of RDMA and transport different types of Datamovers based on different types of
protocols. Adoption of this model will help iSCSI RDMA and transport protocols. Adoption of this model will
proliferate into more environments. help iSCSI proliferate into more environments.
5 Architectural layering of iSCSI and Datamover layers 2.2 Interpretation of Requirements
This draft introduces certain architectural abstractions and
builds an abstract functional interface model between iSCSI
and Datamover protocol layers based on those abstractions.
This architectural style is motivated by the following
desires:
a) Provide guidance to Datamover protocol designers
with respect to the functional boundary between
iSCSI and the Datamover protocols. This guidance is
critical since a significant part of the [RFC3720]
protocol definition is left unchanged by this
Architecture and the iSCSI notions from [RFC3720]
(e.g., tasks, ITTs) are leveraged by the Datamover
protocol.
b) Aid existing iSCSI implementations to rapidly adapt
to this Architecture, largely by leveraging the
architectural abstractions also into implementation
constructs e.g., functions, APIs, modules.
However, note that this Architecture does not intend to
impose any implementation specifics per se. When a DA
architectural concept (e.g., Operational Primitive) is
described as mandatory ("MUST") or recommended ("SHOULD") of
a layer (iSCSI or Datamover) in this document, the intent is
that an implementation respectively MUST or SHOULD produce
the same protocol action as what the model describes.
Specifically, no implementation compliance in terms of names,
modules or API arguments etc. is implied by this Architecture
by such use of [RFC2119] terms, only a functional compliance
is sought.
3 Architectural layering of iSCSI and Datamover layers
Figure 1 illustrates an example of the architectural layering Figure 1 illustrates an example of the architectural layering
of iSCSI and Datamover layers, in conjunction with a TCP/IP of iSCSI and Datamover layers, in conjunction with a TCP/IP
implementation of RDMAP/DDP layers in an iSCSI end node. implementation of RDMAP/DDP layers in an iSCSI end node.
Note that RDMAP/DDP/MPA, and TCP protocol layers are shown Note that RDMAP/DDP/MPA, and TCP protocol layers are shown
here only as an example and in reality, DA is completely here only as an example and in reality, DA is completely
oblivious to protocol layers below the Datamover layer. The oblivious to protocol layers below the Datamover layer. The
RDMAP/DDP/MPA protocol stack provides a generic transport RDMAP/DDP/MPA protocol stack provides a generic transport
service with direct data placement. There is no need to service with direct data placement. There is no need to
tailor the implementation of this protocol stack to the tailor the implementation of this protocol stack to the
skipping to change at page 10, line 30 skipping to change at page 9, line 30
+----------------+ SCSI application +----------------+ +----------------+ SCSI application +----------------+
| SCSI Layer | protocols | SCSI Layer | | SCSI Layer | protocols | SCSI Layer |
+----------------+ +----------------+ +----------------+ +----------------+
^ ^ ^ ^
| | | |
v v v v
+----------------+ iSCSI protocol +----------------+ +----------------+ iSCSI protocol +----------------+
| iSCSI Layer | (excluding data | iSCSI Layer | | iSCSI Layer | (excluding data | iSCSI Layer |
+----------------+ movement) +----------------+ +----------------+ movement) +----------------+
^ ^ ^ ^
-- ---+-- ---- ---- -- DI -- ---- ---- ----+--- ---- -- ---+-- ---- DI (Datamover Interface)--- ----+--- ----
v v v v
+----------------+ a Datamover +----------------+ +----------------+ a Datamover +----------------+
| Datamover Layer| protocol | Datamover Layer| | Datamover Layer| protocol | Datamover Layer|
+----------------+ +----------------+ +----------------+ +----------------+
^ ^ ^ ^
+-------+----------+ +---------+-----------+ +-------+----------+ +---------+-----------+
| v | | v | | v | | v |
|+---------------+ | | +-----------------+ | |+---------------+ | | +-----------------+ |
|| RDMAP/DDP/MPA | | RDMAP/DDP/MPA | | RDMAP/DDP/MPA | | || RDMAP/DDP/MPA | | RDMAP/DDP/MPA | | RDMAP/DDP/MPA | |
|| Layers | | protocols | | Layers | | || Layers | | protocols | | Layers | |
skipping to change at page 11, line 7 skipping to change at page 10, line 7
|+---------------+ | | +----------------+ | |+---------------+ | | +----------------+ |
| ^ | | ^ | | ^ | | ^ |
+-------+----------+ +---------+-----------+ +-------+----------+ +---------+-----------+
+------------------------------------------+ +------------------------------------------+
Figure 1 Datamover Architecture diagram, with the Figure 1 Datamover Architecture diagram, with the
RDMAP example RDMAP example
The scope of this document is limited to: The scope of this document is limited to:
1. Defining the notion of a Datamover layer and a Datamover 1. Defining the notion of a Datamover layer and a Datamover
protocol (section 8), protocol (section 6),
2. Defining the functionality distribution between the 2. Defining the functionality distribution between the
iSCSI layer and the Datamover layer along with the iSCSI layer and the Datamover layer along with the
communication model between the two (Operational communication model between the two (Operational
Primitives), and, Primitives), and,
3. Modeling the interactions between the blocks labeled as 3. Modeling the interactions between the blocks labeled as
"iSCSI Layer" and "Datamover Layer" in Figure 1 i.e. "iSCSI Layer" and "Datamover Layer" in Figure 1 i.e.
defining the interface labeled as "DI" in the figure - defining the interface labeled as "DI" in the figure -
for each defined iSCSI PDU, based on the Operational for each defined iSCSI PDU, based on the Operational
Primitives. Primitives.
6 Design Overview 4 Design Overview
This document discusses and defines a model for interactions This document discusses and defines a model for interactions
between the iSCSI layer and a "Datamover layer" (see section between the iSCSI layer and a "Datamover layer" (see section
8) operating within an iSCSI end node, presumably 6) operating within an iSCSI end node, presumably
communicating with one or more iSCSI end nodes with similar communicating with one or more iSCSI end nodes with similar
layering. The model for interactions for handling different layering. The model for interactions for handling different
iSCSI operations is called the "Datamover Interface" (DI, iSCSI operations is called the "Datamover Interface" (DI,
section 11), while the architecture itself is called section 10), while the architecture itself is called
"Datamover Architecture" (DA). It is likely that the "Datamover Architecture for iSCSI" (DA). It is likely that
architecture will have implications on the Datamover wire the architecture will have implications on the Datamover wire
protocols as DA places certain requirements and functionality protocols as DA places certain requirements and functionality
expectations on the Datamover layer. However, this document expectations on the Datamover layer. However, this document
itself neither defines any new wire protocol for the itself neither defines any new wire protocol for the
Datamover layer, nor any potential modifications to the iSCSI Datamover layer, nor any potential modifications to the iSCSI
wire protocol to employ the Datamover layer. The scope of wire protocol to employ the Datamover layer. The scope of
this document is strictly limited to specifying the this document is strictly limited to specifying the
architectural framework and the minimally required architectural framework and the minimally required
interactions that happen within an iSCSI end node to leverage interactions that happen within an iSCSI end node to leverage
the Datamover layer. the Datamover layer.
The design ideas behind DA can be summarized thus The design ideas behind DA can be summarized thus
1) DA defines an abstract procedural interface definition of 1) DA defines an abstract functional interface model of iSCSI
iSCSI layer's interactions with a Datamover layer below layer's interactions with a Datamover layer below i.e. DA
i.e. DA models the interactions between the logical models the interactions between the logical "bottom"
"bottom" interface of iSCSI and the logical "top" interface interface of iSCSI and the logical "top" interface of a
of a Datamover. Datamover.
2) DA guides the wire protocol for a Datamover layer by 2) DA guides the wire protocol for a Datamover layer by
defining the iSCSI knowledge that the Datamover layer may defining the iSCSI knowledge that the Datamover layer may
utilize in its protocol definition (as an example, this utilize in its protocol definition (as an example, this
draft completely limits the notion of "iSCSI session" to draft completely limits the notion of "iSCSI session" to
the iSCSI layer). the iSCSI layer).
3) DA is designed to allow implementing the Datamover layer 3) DA is designed to allow implementing the Datamover layer
either in hardware or in software. either in hardware or in software.
skipping to change at page 14, line 5 skipping to change at page 13, line 5
several other interactions in a typical implementation in several other interactions in a typical implementation in
order to bootstrap a Datamover layer (or an iSCSI layer) order to bootstrap a Datamover layer (or an iSCSI layer)
into operation, and they are outside the scope of this into operation, and they are outside the scope of this
document. document.
Note that in summary, DA is architected to support many Note that in summary, DA is architected to support many
different Datamover protocols operating under the iSCSI different Datamover protocols operating under the iSCSI
layer. One such example of a Datamover protocol is iSER layer. One such example of a Datamover protocol is iSER
([iSER]). ([iSER]).
7 Architectural Concepts 5 Architectural Concepts
7.1 iSCSI PDU types 5.1 iSCSI PDU types
This section defines the iSCSI PDU classification This section defines the iSCSI PDU classification
terminology, as defined and used in this document. Out of terminology, as defined and used in this document. Out of
the set of legal iSCSI PDUs defined in [RFC3720], as we will the set of legal iSCSI PDUs defined in [RFC3720], as we will
see in section 7.1.1, the iSCSI layer does not request a SCSI see in section 5.1.1, the iSCSI layer does not request a SCSI
Data-Out PDU carrying solicited data for transmission across Data-Out PDU carrying solicited data for transmission across
the Datamover Interface per this architecture. For this the Datamover Interface per this architecture. For this
reason, the SCSI Data-Out PDU carrying solicited data is reason, the SCSI Data-Out PDU carrying solicited data is
excluded in the iSCSI PDU classification we introduce in this excluded in the iSCSI PDU classification we introduce in this
section. The rest of the legal iSCSI PDUs that may be section (for SCSI Data-Out PDUs for unsolicited Data, see
section 5.1.2). The rest of the legal iSCSI PDUs that may be
exchanged across the Datamover Interface are defined to exchanged across the Datamover Interface are defined to
consist of two classes: consist of two classes:
1) iSCSI data-type PDUs 1) iSCSI data-type PDUs
2) iSCSI control-type PDUs 2) iSCSI control-type PDUs
7.1.1 iSCSI data-type PDUs 5.1.1 iSCSI data-type PDUs
An iSCSI data-type PDU is defined as an iSCSI PDU that causes An iSCSI data-type PDU is defined as an iSCSI PDU that causes
data transfer, transparent to the remote iSCSI layer, to take data transfer, transparent to the remote iSCSI layer, to take
place between the peer iSCSI nodes on a full feature phase place between the peer iSCSI nodes on a full feature phase
iSCSI connection. A data-type PDU, when requested for iSCSI connection. A data-type PDU, when requested for
transmission by the sender iSCSI layer, results in the transmission by the sender iSCSI layer, results in the
associated data transfer without the participation of the associated data transfer without the participation of the
remote iSCSI layer, i.e. the PDU itself is not delivered as- remote iSCSI layer, i.e. the PDU itself is not delivered as-
is to the remote iSCSI layer. The following iSCSI PDUs is to the remote iSCSI layer. The following iSCSI PDUs
constitute the set of iSCSI data-type PDUs constitute the set of iSCSI data-type PDUs
skipping to change at page 15, line 5 skipping to change at page 14, line 5
In an iSCSI end node structured as an iSCSI layer and a In an iSCSI end node structured as an iSCSI layer and a
Datamover layer as defined in this document, the solicitation Datamover layer as defined in this document, the solicitation
for Data-out (i.e. R2T PDU) is not delivered to the initiator for Data-out (i.e. R2T PDU) is not delivered to the initiator
iSCSI layer, per the definition of an iSCSI data-type PDU. iSCSI layer, per the definition of an iSCSI data-type PDU.
The data transfer is instead performed via the mechanisms The data transfer is instead performed via the mechanisms
known to the Datamover layer (e.g. RDMA Read). This in turn known to the Datamover layer (e.g. RDMA Read). This in turn
implies that a SCSI Data-Out PDU for solicited data is never implies that a SCSI Data-Out PDU for solicited data is never
requested for transmission across the Datamover Interface at requested for transmission across the Datamover Interface at
the initiator. the initiator.
7.1.2 iSCSI control-type PDUs 5.1.2 iSCSI control-type PDUs
Any iSCSI PDU that is not an iSCSI data-type PDU and also not Any iSCSI PDU that is not an iSCSI data-type PDU and also not
a solicited SCSI Data-out PDU is defined as an iSCSI control- a solicited SCSI Data-out PDU is defined as an iSCSI control-
type PDU. Specifically, it is to be noted that SCSI Data-Out type PDU. Specifically, it is to be noted that SCSI Data-Out
PDUs for unsolicited Data are defined as iSCSI control-type PDUs for unsolicited Data are defined as iSCSI control-type
PDUs. PDUs.
7.2 Data_Descriptor 5.2 Data_Descriptor
A Data_Descriptor is an information element that describes an A Data_Descriptor is an information element that describes an
iSCSI/SCSI data buffer, provided by the iSCSI layer to its iSCSI/SCSI data buffer, provided by the iSCSI layer to its
local Datamover layer or by the Datamover layer to its local local Datamover layer or by the Datamover layer to its local
iSCSI layer for identifying the data associated respectively iSCSI layer for identifying the data associated respectively
with the requested or completed operation. with the requested or completed operation.
In implementation terms, a Data_Descriptor may be a scatter- In implementation terms, a Data_Descriptor may be a scatter-
gather list describing a local buffer, the exact structure of gather list describing a local buffer, the exact structure of
which is subject to the constraints imposed by the operating which is subject to the constraints imposed by the operating
environment on the local iSCSI node. environment on the local iSCSI node.
7.3 Connection_Handle 5.3 Connection_Handle
A Connection_Handle is an information element that identifies A Connection_Handle is an information element that identifies
the particular iSCSI connection for which an inbound or the particular iSCSI connection for which an inbound or
outbound iSCSI PDU is intended. A connection handle is unique outbound iSCSI PDU is intended. A connection handle is unique
for a given pair of an iSCSI layer instance and a Datamover for a given pair of an iSCSI layer instance and a Datamover
layer instance. The Connection_Handle qualifier is used in layer instance. The Connection_Handle qualifier is used in
all invocations of any Operational Primitive for connection all invocations of any Operational Primitive for connection
identification. identification.
Note that the Connection_Handle is conceptually different Note that the Connection_Handle is conceptually different
skipping to change at page 16, line 13 skipping to change at page 15, line 13
possibly multiple iSCSI sessions. possibly multiple iSCSI sessions.
In implementation terms, a Connection_Handle could be an In implementation terms, a Connection_Handle could be an
opaque identifier exchanged between the iSCSI layer and the opaque identifier exchanged between the iSCSI layer and the
Datamover layer at the connection login time. One may also Datamover layer at the connection login time. One may also
consider it to be similar in scope of uniqueness to a socket consider it to be similar in scope of uniqueness to a socket
identifier. The exact structure and modalities of exchange identifier. The exact structure and modalities of exchange
of a Connection_Handle between the two layers is of a Connection_Handle between the two layers is
implementation-specific. implementation-specific.
7.4 Operational Primitive 5.4 Operational Primitive
An Operational Primitive, in this document, is an abstract An Operational Primitive, in this document, is an abstract
functional interface procedure that requests another layer to functional interface procedure that requests another layer to
perform a specific action on the requestor's behalf or perform a specific action on the requestor's behalf or
notifies the other layer of some event. The Datamover notifies the other layer of some event. The Datamover
Interface between an iSCSI layer instance and a Datamover Interface between an iSCSI layer instance and a Datamover
layer instance within an iSCSI end node uses a set of layer instance within an iSCSI end node uses a set of
Operational Primitives to define the functional interface Operational Primitives to define the functional interface
between the two layers. Note that not every invocation of an between the two layers. Note that not every invocation of an
Operational Primitive may elicit a response from the Operational Primitive may elicit a response from the
skipping to change at page 17, line 13 skipping to change at page 16, line 13
realizing the qualifiers (passed synchronously with realizing the qualifiers (passed synchronously with
invocation, or retrieved from task context, or retrieved from invocation, or retrieved from task context, or retrieved from
shared memory etc.) is really upto the implementations. shared memory etc.) is really upto the implementations.
When an Operational Primitive implementation is described as When an Operational Primitive implementation is described as
mandatory ("MUST") or recommended ("SHOULD") of a layer mandatory ("MUST") or recommended ("SHOULD") of a layer
(iSCSI or Datamover) in this document, the intent is that an (iSCSI or Datamover) in this document, the intent is that an
implementation respectively MUST or SHOULD produce the same implementation respectively MUST or SHOULD produce the same
protocol action as what the model describes. protocol action as what the model describes.
7.5 Transport Connection 5.5 Transport Connection
The term "Transport Connection" is used in this document as a The term "Transport Connection" is used in this document as a
generic term to represent the end-to-end logical connection generic term to represent the end-to-end logical connection
as defined by the underlying reliable transport protocol. as defined by the underlying reliable transport protocol.
For this revision of this document, a Transport Connection For this revision of this document, a Transport Connection
means only a TCP connection. means only a TCP connection.
8 Datamover layer and Datamover protocol 6 Datamover layer and Datamover protocol
This section introduces the notion of a "Datamover layer" and This section introduces the notion of a "Datamover layer" and
"Datamover protocol" as meant in this document, and defines "Datamover protocol" as meant in this document, and defines
the requirements on a Datamover protocol. the requirements on a Datamover protocol.
A Datamover layer is the implementation component that A Datamover layer is the implementation component that
realizes a Datamover protocol functionality in an iSCSI- realizes a Datamover protocol functionality in an iSCSI-
capable end node, in communicating with other iSCSI end nodes capable end node, in communicating with other iSCSI end nodes
with similar capabilities. More specifically, a "Datamover with similar capabilities. More specifically, a "Datamover
layer" MUST provide the following functionality and the layer" MUST provide the following functionality and the
skipping to change at page 18, line 32 skipping to change at page 17, line 32
sending/receiving an iSCSI data sequence (in order to sending/receiving an iSCSI data sequence (in order to
complete part of a SCSI command, for a target). complete part of a SCSI command, for a target).
2) transport an iSCSI control-type PDU as-is to the peer 2) transport an iSCSI control-type PDU as-is to the peer
Datamover layer when requested to do so by the local iSCSI Datamover layer when requested to do so by the local iSCSI
layer. layer.
3) provide notification and delivery to the iSCSI layer upon 3) provide notification and delivery to the iSCSI layer upon
arrival of an iSCSI control-type PDU. arrival of an iSCSI control-type PDU.
4) provide an end-to-end data acknowledgement of SCSI read 4) provide an initiator-to-target data acknowledgement of SCSI
data to the target iSCSI layer, when requested. read data back to the target iSCSI layer, when requested.
5) provide an asynchronous notification upon completion of a 5) provide an asynchronous notification upon completion of a
requested data transfer operation that moved data without requested data transfer operation that moved data without
involving the iSCSI layer. involving the iSCSI layer.
6) place the SCSI data into the I/O buffers or pick up the 6) place the SCSI data into the I/O buffers or pick up the
SCSI data for transmission out of the data buffers that the SCSI data for transmission out of the data buffers that the
iSCSI layer had requested to be used for a SCSI I/O. iSCSI layer had requested to be used for a SCSI I/O.
7) guarantee an error-free (i.e. must have at least the same 7) provide an error-free (i.e. must have at least the same
level of assurance of data integrity as the CRC32C iSCSI level of assurance of data integrity as the CRC32C iSCSI
data digest), reliable, in-order delivery transport data digest), reliable, in-order delivery transport
mechanism over IP fabrics in performing the data transfer, mechanism over IP networks in performing the data transfer,
and asynchronously notify the iSCSI layer upon iSCSI and asynchronously notify the iSCSI layer upon iSCSI
connection termination. connection termination.
Note that this architecture expects that each compliant Note that this architecture expects that each compliant
Datamover protocol will define the precise means of Datamover protocol will define the precise means of
satisfying the requirements specified in this section. satisfying the requirements specified in this section.
In order to meet the functional requirements listed in this In order to meet the functional requirements listed in this
section, certain Datamover protocols may require pre-posted section, certain Datamover protocols may require pre-posted
buffers from the local iSCSI protocol layer via mechanisms buffers from the local iSCSI protocol layer via mechanisms
outside the scope of this document and in some outside the scope of this document and in some
implementations, the absence of such buffers may result in a implementations, the absence of such buffers may result in a
connection failure. Datamover protocols may also realize connection failure. Datamover protocols may also realize
these functional requirements via methods not explicitly these functional requirements via methods not explicitly
listed in this document. listed in this document.
9 Operational Primitives provided by the Datamover layer 7 Functional Overview
This section presents an overview of the functional
interactions between the iSCSI layer and the Datamover layer
as intended by this Architecture.
7.1 Startup
The iSCSI Login Phase on an iSCSI connection occurs as
defined in [RFC3720]. The Architecture assumes that at the
end of the Login Phase, both the initiator and target, if
they had so decided, transition the connection to being
Datamover-assisted. The precise means of how an iSCSI
initiator and an iSCSI target agree on having the connection
Datamover-assisted is defined by the Datamover protocol. The
only architectural requirement is that all iSCSI interactions
in the iSCSI Full Feature Phase MUST be Datamover-assisted
subject to the prior agreement, meaning that Datamover
protocol is in the iSCSI-to-iSCSI communication path below
the iSCSI layer on either side as shown in Figure 1. DA
defines the Enable_Datamover Operational Primitive (section
8.6) to bring about this transition to a Datamover-assisted
connection.
The Architecture also assumes that the Datamover layer may
require a certain number of opaque local resources for making
a connection Datamover-assisted. DA thus defines the
Allocate_Connection_Resources Operational Primitive (section
8.4) to model this interaction. This Primitive is intended
to be invoked on each side once the two sides decide (as
previously noted) to have the connection Datamover-assisted.
The expected sequence of Primitive invocations is depicted in
Figure 2 and Figure 3 in section 16.2. Figure 4, Figure 5,
and Figure 6 illustrate how the Primitives may be employed to
deal with various legal login outcomes.
7.2 Full Feature Phase
All iSCSI peer communication in the Full Feature Phase
happens through the Datamover layers if the iSCSI connection
is Datamover-assisted. The Architecture assumes that a
Datamover layer may require a certain number of opaque local
resources for each new iSCSI task. In the normal course of
execution, these task-level resources in the Datamover layer
are assumed to be transparently allocated on each task
initiation and deallocated on the conclusion of each task as
appropriate. In exception scenarios however in scenarios
that do not yield a SCSI Response for each task such as ABORT
TASK operation the Architecture assumes that the Datamover
layer needs to be notified of the individual task
terminations to aid its task-level resource management. DA
thus defines the Deallocate_Task_Resources Operational
Primitive (section 8.9) to model this task-resource
management. In specifying the ITT qualifier for the
Deallocate_Task_Resources Primitive, the Architecture further
assumes that the Datamover layer tracks its opaque task-level
local resources by the iSCSI ITT. DA also defines
Send_Control (section 8.1), Put_Data (section 8.2), Get_Data
(section 8.3), Data_Completion_Notify(section 9.3),
Data_ACK_Notify (section 9.4), and Control_Notify (section
9.1) Operational Primitives to model the various Full Feature
Phase interactions.
Figure 9, Figure 10, and Figure 11 in section 16.2 show some
Full Feature Phase interactions SCSI Write task, SCSI Read
task, and a SCSI Read Data acknowledgement respectively.
Figure 12 in section 16.2 illustrates how an ABORT TASK
operation can be modeled leading to deterministic resource
cleanup on the Datamover layer.
7.3 Wrapup
Once an iSCSI connection becomes Datamover-assisted, the
connection continues in that state till the end of the Full
Feature Phase, i.e. the termination of the connection. The
Architecture assumes that when a connection is normally
logged out, the Datamover layer needs to be notified so that
its connection-level opaque resources (see section 7.1) may
now be freed up. DA thus defines a Connection_Terminate
Operational Primitive (section 8.7) to model this
interaction. The Architecture further assumes that when a
connection termination happens without iSCSI layer's
involvement (e.g., TCP RST), the Datamover layer is capable
of locally cleaning up its task-level and connection-level
resources before notifying the iSCSI layer of the fact. DA
thus defines the Connection_Terminate_Notify Operational
Primitive (section 9.2) to model this interaction.
Figure 7 and Figure 8 in section 16.2 illustrate the
interactions between the iSCSI and Datamover layers in normal
and unexpected connection termination scenarios.
8 Operational Primitives provided by the Datamover layer
While the iSCSI specification itself does not have a notion While the iSCSI specification itself does not have a notion
of Operational Primitives, any iSCSI layer implementing the of Operational Primitives, any iSCSI layer implementing the
iSCSI specification functionally requires the following iSCSI specification functionally requires the following
Operational Primitives from its Datamover layer. Thus, any Operational Primitives from its Datamover layer. Thus, any
Datamover protocol compliant with this architecture MUST Datamover protocol compliant with this architecture MUST
implement the Operational Primitives described in this implement the Operational Primitives described in this
section. These Operational Primitives are invoked by the section. These Operational Primitives are invoked by the
iSCSI layer as appropriate. Unless otherwise stated, all the iSCSI layer as appropriate. Unless otherwise stated, all the
following Operational Primitives may be used both on the following Operational Primitives may be used both on the
skipping to change at page 20, line 38 skipping to change at page 22, line 38
5) Deallocate_Connection_Resources 5) Deallocate_Connection_Resources
6) Enable_Datamover 6) Enable_Datamover
7) Connection_Terminate 7) Connection_Terminate
8) Notice_Key_Values 8) Notice_Key_Values
9) Deallocate_Task_Resources 9) Deallocate_Task_Resources
9.1 Send_Control 8.1 Send_Control
Input qualifiers: Connection_Handle, iSCSI PDU-specific Input qualifiers: Connection_Handle, iSCSI PDU-specific
qualifiers qualifiers
Return Results: Not specified. Return Results: Not specified.
An iSCSI layer requests its local Datamover layer to transmit An iSCSI layer requests its local Datamover layer to transmit
an iSCSI control-type PDU to the peer iSCSI layer operating an iSCSI control-type PDU to the peer iSCSI layer operating
in the remote iSCSI node by this Operational Primitive. The in the remote iSCSI node by this Operational Primitive. The
Datamover layer performs the requested operation, and may add Datamover layer performs the requested operation, and may add
its own protocol headers in doing so. The iSCSI layer MUST its own protocol headers in doing so. The iSCSI layer MUST
NOT invoke the Send_Control Operational Primitive on an iSCSI NOT invoke the Send_Control Operational Primitive on an iSCSI
connection that is not yet Datamover-assisted. connection that is not yet Datamover-assisted.
An initiator iSCSI layer requesting the transfer of a SCSI An initiator iSCSI layer requesting the transfer of a SCSI
command PDU or a target iSCSI layer requesting the transfer command PDU or a target iSCSI layer requesting the transfer
of a SCSI response PDU are examples of invoking the of a SCSI response PDU are examples of invoking the
Send_Control Operational Primitive. Send_Control Operational Primitive. As section 10.3.1
illustrates later on, the iSCSI PDU-specific qualifiers in
this example are: BHS and AHS, DataDescriptorOut,
DataDescriptorIn, ImmediateDataSize, and UnsolicitedDataSize
9.2 Put_Data 8.2 Put_Data
Input qualifiers: Connection_Handle, contents of a SCSI Data- Input qualifiers: Connection_Handle, contents of a SCSI Data-
In PDU header, Data_Descriptor, Notify_Enable In PDU header, Data_Descriptor, Notify_Enable
Return Results: Not specified. Return Results: Not specified.
An iSCSI layer requests its local Datamover layer to transmit An iSCSI layer requests its local Datamover layer to transmit
the data identified by the Data_Descriptor for the SCSI Data- the data identified by the Data_Descriptor for the SCSI Data-
In PDU to the peer iSCSI layer on the remote iSCSI node by In PDU to the peer iSCSI layer on the remote iSCSI node by
this Operational Primitive. The Datamover layer performs the this Operational Primitive. The Datamover layer performs the
operation by using its own protocol means, completely operation by using its own protocol means, completely
transparent to the remote iSCSI layer. The iSCSI layer MUST transparent to the remote iSCSI layer. The iSCSI layer MUST
NOT invoke the Put_Data Operational Primitive on an iSCSI NOT invoke the Put_Data Operational Primitive on an iSCSI
connection that is not yet Datamover-assisted. connection that is not yet Datamover-assisted.
The Notify_Enable qualifier is used to request the local The Notify_Enable qualifier is used to request the local
Datamover layer to generate or to not generate the eventual Datamover layer to generate or to not generate the eventual
local completion notification to the iSCSI layer for this local completion notification to the iSCSI layer for this
Put_Data invocation. For detailed semantics of this Put_Data invocation. For detailed semantics of this
qualifier, see section 10.3. qualifier, see section 9.3.
A Put_Data Primitive may only be invoked by an iSCSI layer on A Put_Data Primitive may only be invoked by an iSCSI layer on
the target to its local Datamover layer. the target to its local Datamover layer.
A target iSCSI layer requesting the transfer of an iSCSI read A target iSCSI layer requesting the transfer of an iSCSI read
data sequence (also known as a read burst) is an example of data sequence (also known as a read burst) is an example of
invoking the Put_Data Operational Primitive. invoking the Put_Data Operational Primitive.
9.3 Get_Data 8.3 Get_Data
Input qualifiers: Connection_Handle, contents of an R2T PDU, Input qualifiers: Connection_Handle, contents of an R2T PDU,
Data_Descriptor, Notify_Enable Data_Descriptor, Notify_Enable
Return Results: Not specified. Return Results: Not specified.
An iSCSI layer requests its local Datamover layer to retrieve An iSCSI layer requests its local Datamover layer to retrieve
certain data identified by the R2T PDU from the peer iSCSI certain data identified by the R2T PDU from the peer iSCSI
layer on the remote iSCSI node into the buffer identified by layer on the remote iSCSI node into the buffer identified by
the Data_Descriptor by invoking this Operational Primitive. the Data_Descriptor by invoking this Operational Primitive.
The Datamover layer performs the operation by using its own The Datamover layer performs the operation by using its own
protocol means, completely transparent to the remote iSCSI protocol means, completely transparent to the remote iSCSI
layer. The iSCSI layer MUST NOT invoke the Get_Data layer. The iSCSI layer MUST NOT invoke the Get_Data
Operational Primitive on an iSCSI connection that is not yet Operational Primitive on an iSCSI connection that is not yet
Datamover-assisted. Datamover-assisted.
The Notify_Enable qualifier is used to request the local The Notify_Enable qualifier is used to request the local
Datamover layer to generate or to not generate the eventual Datamover layer to generate or to not generate the eventual
local completion notification to the iSCSI layer for this local completion notification to the iSCSI layer for this
Get_Data invocation. For detailed semantics of this Get_Data invocation. For detailed semantics of this
qualifier, see section 10.3. qualifier, see section 9.3.
A Get_Data Primitive may only be invoked by an iSCSI layer on A Get_Data Primitive may only be invoked by an iSCSI layer on
the target to its local Datamover layer. the target to its local Datamover layer.
A target iSCSI layer requesting the transfer of an iSCSI A target iSCSI layer requesting the transfer of an iSCSI
write data sequence (also known as a write burst) is an write data sequence (also known as a write burst) is an
example of invoking the Get_Data Operational Primitive. example of invoking the Get_Data Operational Primitive.
9.4 Allocate_Connection_Resources 8.4 Allocate_Connection_Resources
Input qualifiers: Connection_Handle[, Resource_Descriptor ] Input qualifiers: Connection_Handle[, Resource_Descriptor ]
Return Results: Status. Return Results: Status.
By invoking this Operational Primitive, an iSCSI layer By invoking this Operational Primitive, an iSCSI layer
requests its local Datamover layer to perform all the requests its local Datamover layer to perform all the
Datamover-specific resource allocations required for the full Datamover-specific resource allocations required for the full
feature phase of an iSCSI connection. The Connection_Handle feature phase of an iSCSI connection. The Connection_Handle
identifies the connection the iSCSI layer is requesting the identifies the connection the iSCSI layer is requesting the
skipping to change at page 23, line 33 skipping to change at page 25, line 35
Status=failure means that the Allocate_Connection_Resources Status=failure means that the Allocate_Connection_Resources
invocation corresponding to that Connection_Handle failed. invocation corresponding to that Connection_Handle failed.
There MUST NOT be more than one Allocate_Connection_Resources There MUST NOT be more than one Allocate_Connection_Resources
Primitive invocation outstanding for a given Primitive invocation outstanding for a given
Connection_Handle at any time. Connection_Handle at any time.
The iSCSI layer must invoke the Allocate_Connection_Resources The iSCSI layer must invoke the Allocate_Connection_Resources
Primitive before the invocation of the Enable_Datamover Primitive before the invocation of the Enable_Datamover
Primitive. Primitive.
9.5 Deallocate_Connection_Resources 8.5 Deallocate_Connection_Resources
Input qualifiers: Connection_Handle Input qualifiers: Connection_Handle
Return Results: Not specified. Return Results: Not specified.
By invoking this Operational Primitive, an iSCSI layer By invoking this Operational Primitive, an iSCSI layer
requests its local Datamover layer to deallocate all the requests its local Datamover layer to deallocate all the
Datamover-specific resources that may have been allocated Datamover-specific resources that may have been allocated
earlier for the Transport Connection identified by the earlier for the Transport Connection identified by the
Connection_Handle. The iSCSI layer may invoke this Connection_Handle. The iSCSI layer may invoke this
Operational Primitive when the Datamover-specific resources Operational Primitive when the Datamover-specific resources
associated with the Connection_Handle are no longer necessary associated with the Connection_Handle are no longer necessary
(such as the Login failure of the corresponding iSCSI (such as the Login failure of the corresponding iSCSI
connection). connection).
9.6 Enable_Datamover 8.6 Enable_Datamover
Input qualifiers: Connection_Handle, Input qualifiers: Connection_Handle,
Transport_Connection_Descriptor [, Final_Login_Response_PDU] Transport_Connection_Descriptor [, Final_Login_Response_PDU]
Return Results: Not specified. Return Results: Not specified.
By invoking this Operational Primitive, an iSCSI layer By invoking this Operational Primitive, an iSCSI layer
requests its local Datamover layer to assist all further requests its local Datamover layer to assist all further
iSCSI exchanges on the iSCSI connection (i.e. to make the iSCSI exchanges on the iSCSI connection (i.e. to make the
connection Datamover-assisted) identified by the connection Datamover-assisted) identified by the
skipping to change at page 24, line 37 skipping to change at page 26, line 39
byte stream as expected by the initiator iSCSI layer. When byte stream as expected by the initiator iSCSI layer. When
this qualifier is used, the target-Datamover layer MUST this qualifier is used, the target-Datamover layer MUST
transmit this final Login Response before Datamover transmit this final Login Response before Datamover
assistance is enabled for the Transport Connection. assistance is enabled for the Transport Connection.
The iSCSI layer identifies the specific Transport Connection The iSCSI layer identifies the specific Transport Connection
associated with the Connection_Handle to the Datamover layer associated with the Connection_Handle to the Datamover layer
by specifying the Transport_Connection_Descriptor. The exact by specifying the Transport_Connection_Descriptor. The exact
structure of this Descriptor is implementation-dependent. structure of this Descriptor is implementation-dependent.
9.7 Connection_Terminate 8.7 Connection_Terminate
Input qualifiers: Connection_Handle Input qualifiers: Connection_Handle
Return Results: Not specified. Return Results: Not specified.
By invoking this Operational Primitive, an iSCSI layer By invoking this Operational Primitive, an iSCSI layer
requests its local Datamover layer to terminate the Transport requests its local Datamover layer to terminate the Transport
Connection and deallocate all the connection and task Connection and deallocate all the connection and task
resources associated with the Connection_Handle. When this resources associated with the Connection_Handle. When this
Operational Primitive invocation returns to the iSCSI layer, Operational Primitive invocation returns to the iSCSI layer,
the iSCSI layer may assume the full ownership of all the the iSCSI layer may assume the full ownership of all the
iSCSI-level resources, e.g. I/O Buffers, associated with the iSCSI-level resources, e.g. I/O Buffers, associated with the
connection. This Operational Primitive may be invoked only connection. This Operational Primitive may be invoked only
with a valid Connection_Handle and the Transport Connection with a valid Connection_Handle and the Transport Connection
associated with the Connection_Handle must already be associated with the Connection_Handle must already be
Datamover-assisted. Datamover-assisted.
9.8 Notice_Key_Values 8.8 Notice_Key_Values
Input qualifiers: Connection_Handle, Number of keys, a list Input qualifiers: Connection_Handle, Number of keys, a list
of Key-Value pairs of Key-Value pairs
Return Results: Not specified. Return Results: Not specified.
By invoking this Operational Primitive, an iSCSI layer By invoking this Operational Primitive, an iSCSI layer
requests its local Datamover layer to take note of the requests its local Datamover layer to take note of the
negotiated values of the listed keys for the Transport negotiated values of the listed keys for the Transport
Connection. This Operational Primitive may be invoked only Connection. This Operational Primitive may be invoked only
with a valid Connection_Handle and the Key-Value pairs MUST with a valid Connection_Handle and the Key-Value pairs MUST
be the current values that were successfully agreed upon by be the current values that were successfully agreed upon by
the iSCSI peers for the connection. The Datamover layer may the iSCSI peers for the connection. The Datamover layer may
use the values of the keys to aid the Datamover operation as use the values of the keys to aid the Datamover operation as
it deems appropriate. The specific keys to be passed in as it deems appropriate. The specific keys to be passed in as
input qualifiers and the point(s) in time this Operational input qualifiers and the point(s) in time this Operational
Primitive is invoked are implementation-dependent. Primitive is invoked are implementation-dependent.
9.9 Deallocate_Task_Resources 8.9 Deallocate_Task_Resources
Input qualifiers: Connection_Handle, ITT Input qualifiers: Connection_Handle, ITT
Return Results: Not specified. Return Results: Not specified.
By invoking this Operational Primitive, an iSCSI layer By invoking this Operational Primitive, an iSCSI layer
requests its local Datamover Layer to deallocate all requests its local Datamover Layer to deallocate all
Datamover-specific resources that earlier may have been Datamover-specific resources that earlier may have been
allocated for the task identified by the ITT qualifier. The allocated for the task identified by the ITT qualifier. The
iSCSI layer uses this Operational Primitive during exception iSCSI layer uses this Operational Primitive during exception
processing when one or more active tasks are to be terminated processing when one or more active tasks are to be terminated
without corresponding SCSI Response PDUs. This Primitive without corresponding SCSI Response PDUs. This Primitive
MUST be invoked for each active task terminated without a MUST be invoked for each active task terminated without a
SCSI Response PDU. This Primitive MUST NOT be invoked by the SCSI Response PDU. This Primitive MUST NOT be invoked by the
iSCSI layer when a SCSI Response PDU normally concludes a iSCSI layer when a SCSI Response PDU normally concludes a
task. When a SCSI Response PDU normally concludes a task task. When a SCSI Response PDU normally concludes a task
(even if the SCSI Status was not a success), the Datamover (even if the SCSI Status was not a success), the Datamover
layer is assumed to have automatically deallocated all layer is assumed to have automatically deallocated all
Datamover-specific task resources for that task. Datamover-specific task resources for that task. Refer to
section 7.2 for a related discussion on the Architectural
assumptions on the task-level Datamover resource management,
especially with respect to when the resources are assumed to
be allocated.
10 Operational Primitives provided by the iSCSI layer 9 Operational Primitives provided by the iSCSI layer
While the iSCSI specification itself does not have a notion While the iSCSI specification itself does not have a notion
of Operational Primitives, any iSCSI layer implementing the of Operational Primitives, any iSCSI layer implementing the
iSCSI specification would have to provide the following iSCSI specification would have to provide the following
Operational Primitives to its local Datamover layer. Thus, Operational Primitives to its local Datamover layer. Thus,
any iSCSI protocol implementation compliant with this any iSCSI protocol implementation compliant with this
architecture MUST implement the Operational Primitives architecture MUST implement the Operational Primitives
described in this section. These Operational Primitives are described in this section. These Operational Primitives are
invoked by the Datamover layer as appropriate and when the invoked by the Datamover layer as appropriate and when the
iSCSI connection is Datamover-assisted. Unless otherwise iSCSI connection is Datamover-assisted. Unless otherwise
skipping to change at page 27, line 29 skipping to change at page 29, line 29
may be construed as "up calls". may be construed as "up calls".
1) Control_Notify 1) Control_Notify
2) Connection_Terminate_Notify 2) Connection_Terminate_Notify
3) Data_Completion_Notify 3) Data_Completion_Notify
4) Data_ACK_Notify 4) Data_ACK_Notify
10.1 Control_Notify 9.1 Control_Notify
Input qualifiers: Connection_Handle, an iSCSI control-type Input qualifiers: Connection_Handle, an iSCSI control-type
PDU. PDU.
Return Results: Not specified. Return Results: Not specified.
A Datamover layer notifies its local iSCSI layer, via this A Datamover layer notifies its local iSCSI layer, via this
Operational Primitive, of the arrival of an iSCSI control- Operational Primitive, of the arrival of an iSCSI control-
type PDU from the peer Datamover layer on the remote iSCSI type PDU from the peer Datamover layer on the remote iSCSI
node. The iSCSI layer processes the control-type PDU as node. The iSCSI layer processes the control-type PDU as
defined in [RFC3720]. defined in [RFC3720].
A target iSCSI layer being notified of the arrival of a SCSI A target iSCSI layer being notified of the arrival of a SCSI
Command is an example of invoking the Control_Notify Command is an example of invoking the Control_Notify
Operational Primitive. Operational Primitive.
Note that implementations may choose to describe the "iSCSI Note that implementations may choose to describe the "iSCSI
control-type PDU" qualifier in this notification using a control-type PDU" qualifier in this notification using a
Data_Descriptor (section 7.2) and not necessarily one Data_Descriptor (section 5.2) and not necessarily one
contiguous buffer. contiguous buffer.
10.2 Connection_Terminate_Notify 9.2 Connection_Terminate_Notify
Input qualifiers: Connection_Handle Input qualifiers: Connection_Handle
Return Results: Not specified. Return Results: Not specified.
A Datamover layer notifies its local iSCSI layer on an A Datamover layer notifies its local iSCSI layer on an
unsolicited termination or failure of an iSCSI connection unsolicited termination or failure of an iSCSI connection
providing the Connection_Handle associated with the iSCSI providing the Connection_Handle associated with the iSCSI
Connection. The iSCSI Layer MUST consider the Connection. The iSCSI Layer MUST consider the
Connection_Handle to be invalid upon being so notified. The Connection_Handle to be invalid upon being so notified. The
iSCSI layer processes the connection termination as defined iSCSI layer processes the connection termination as defined
in [RFC3720]. The Datamover layer MUST deallocate the in [RFC3720]. The Datamover layer MUST deallocate the
connection and task resources associated with the terminated connection and task resources associated with the terminated
connection before notifying the iSCSI layer of the connection before notifying the iSCSI layer of the
termination via this Operational Primitive. termination via this Operational Primitive.
A target iSCSI layer being notified of the arrival of TCP A target iSCSI layer being notified of the arrival of TCP
RESET is an example of when the Connection_Terminate_Notify RESET is an example of when the Connection_Terminate_Notify
Operational Primitive is invoked. Operational Primitive is invoked.
10.3 Data_Completion_Notify 9.3 Data_Completion_Notify
Input qualifiers: Connection_Handle, ITT, SN Input qualifiers: Connection_Handle, ITT, SN
Return Results: Not specified. Return Results: Not specified.
A Datamover layer notifies its local iSCSI layer on A Datamover layer notifies its local iSCSI layer on
completing the retrieval of the data or upon sending the completing the retrieval of the data or upon sending the
data, as requested in a prior iSCSI data-type PDU, from/to data, as requested in a prior iSCSI data-type PDU, from/to
the peer Datamover layer on the remote iSCSI node via this the peer Datamover layer on the remote iSCSI node via this
Operational Primitive. The iSCSI layer processes the Operational Primitive. The iSCSI layer processes the
skipping to change at page 29, line 27 skipping to change at page 31, line 27
invocation, the Datamover layer MUST invoke the invocation, the Datamover layer MUST invoke the
Data_Completion_Notify Operational Primitive upon completing Data_Completion_Notify Operational Primitive upon completing
that requested data transfer. If the Notify_Enable was that requested data transfer. If the Notify_Enable was
cleared in either a Put_Data or a Get_Data invocation, the cleared in either a Put_Data or a Get_Data invocation, the
Datamover layer MUST NOT invoke the Data_Completion_Notify Datamover layer MUST NOT invoke the Data_Completion_Notify
Operational Primitive upon completing that requested data Operational Primitive upon completing that requested data
transfer. transfer.
A Data_Completion_Notify invocation serves to notify the A Data_Completion_Notify invocation serves to notify the
iSCSI layer of the Put_Data or Get_Data completion iSCSI layer of the Put_Data or Get_Data completion
respectively. As earlier noted in sections 9.2 and 9.3, respectively. As earlier noted in sections 8.2 and 8.3,
specific Datamover protocol definitions may restrict the specific Datamover protocol definitions may restrict the
usage scope of Put_Data and Get_Data, and thus implicitly the usage scope of Put_Data and Get_Data, and thus implicitly the
usage scope of Data_Completion_Notify. usage scope of Data_Completion_Notify.
A target iSCSI layer being notified of the retrieval of a A target iSCSI layer being notified of the retrieval of a
write data sequence is an example of invoking the write data sequence is an example of invoking the
Data_Completion_Notify Operational Primitive. Data_Completion_Notify Operational Primitive.
10.4 Data_ACK_Notify 9.4 Data_ACK_Notify
Input qualifiers: Connection_Handle, ITT, DataSN Input qualifiers: Connection_Handle, ITT, DataSN
Return Results: Not specified. Return Results: Not specified.
A target Datamover layer notifies its local iSCSI layer of A target Datamover layer notifies its local iSCSI layer of
the arrival of a previously requested data acknowledgement the arrival of a previously requested data acknowledgement
from the peer Datamover layer on the remote (initiator) iSCSI from the peer Datamover layer on the remote (initiator) iSCSI
node via this Operational Primitive. The iSCSI layer node via this Operational Primitive. The iSCSI layer
processes the data acknowledgement notification as defined in processes the data acknowledgement notification as defined in
[RFC3720]. [RFC3720].
A target iSCSI layer being notified of the arrival of a data A target iSCSI layer being notified of the arrival of a data
acknowledgement for a certain SCSI Read data PDU is the only acknowledgement for a certain SCSI Read data PDU is the only
example of invoking the Data_ACK_Notify Operational example of invoking the Data_ACK_Notify Operational
Primitive. Primitive.
11 Datamover Interface (DI) 10 Datamover Interface (DI)
11.1 Overview 10.1 Overview
This chapter describes the interactions model between iSCSI This chapter describes the interactions model between iSCSI
and Datamover layers when the iSCSI connection is Datamover- and Datamover layers when the iSCSI connection is Datamover-
assisted so the iSCSI layer may carry out the following - assisted so the iSCSI layer may carry out the following -
- send iSCSI data-type PDUs and exchange iSCSI control-type - send iSCSI data-type PDUs and exchange iSCSI control-type
PDUs, and PDUs, and
- handle asynchronous notifications such as completion of - handle asynchronous notifications such as completion of
data sequence transfer, and connection failure. data sequence transfer, and connection failure.
This chapter relies on the notion of Operational Primitives This chapter relies on the notion of Operational Primitives
(section 7.4) to define DI. (section 5.4) to define DI.
11.2 Interactions for handling asynchronous notifications 10.2 Interactions for handling asynchronous notifications
11.2.1 Connection termination 10.2.1 Connection termination
As stated in section 10.2, the Datamover layer notifies the As stated in section 9.2, the Datamover layer notifies the
iSCSI layer of a failed or terminated connection via the iSCSI layer of a failed or terminated connection via the
Connection_Terminate_Notify Operational Primitive. The iSCSI Connection_Terminate_Notify Operational Primitive. The iSCSI
layer MUST consider the connection as unusable upon the layer MUST consider the connection as unusable upon the
invocation of this Primitive and handle the connection invocation of this Primitive and handle the connection
termination as specified in [RFC3720]. termination as specified in [RFC3720].
11.2.2 Data transfer completion 10.2.2 Data transfer completion
As stated in section 10.3, the Datamover layer notifies the As stated in section 9.3, the Datamover layer notifies the
iSCSI layer of a completed data transfer operation via the iSCSI layer of a completed data transfer operation via the
Data_Completion_Notify Operational Primitive. The iSCSI Data_Completion_Notify Operational Primitive. The iSCSI
layer processes the transfer completion as specified in layer processes the transfer completion as specified in
[RFC3720]. [RFC3720].
11.2.2.1 Completion of a requested SCSI Data transfer 10.2.2.1 Completion of a requested SCSI Data transfer
The Datamover layer, to notify the iSCSI layer of the The Datamover layer, to notify the iSCSI layer of the
completion of a requested iSCSI data-type PDU transfer, uses completion of a requested iSCSI data-type PDU transfer, uses
the Data_Completion_Notify Operational Primitive with the the Data_Completion_Notify Operational Primitive with the
following input qualifiers. following input qualifiers.
a) Connection_Handle a) Connection_Handle
b) ITT: Initiator Task Tag semantics as defined in b) ITT: Initiator Task Tag semantics as defined in
[RFC3720] [RFC3720]
c) SN: DataSN for a SCSI Data-in/Data-out PDU, and R2TSN c) SN: DataSN for a SCSI Data-in/Data-out PDU, and R2TSN
for an iSCSI R2T PDU. The semantics for both types of for an iSCSI R2T PDU. The semantics for both types of
sequence numbers are as defined in [RFC3720]. sequence numbers are as defined in [RFC3720].
The rationale for choosing SN is explained in section 10.3. The rationale for choosing SN is explained in section 9.3.
Every invocation of the Data_Completion_Notify Operational Every invocation of the Data_Completion_Notify Operational
Primitive MUST be preceded by an invocation of the Put_Data Primitive MUST be preceded by an invocation of the Put_Data
or Get_Data Operational Primitive with the Notify_Enable or Get_Data Operational Primitive with the Notify_Enable
qualifier set by the iSCSI layer at an earlier point in time. qualifier set by the iSCSI layer at an earlier point in time.
11.2.3 Data acknowledgement 10.2.3 Data acknowledgement
[RFC3720] allows the iSCSI targets to optionally solicit data [RFC3720] allows the iSCSI targets to optionally solicit data
acknowledgement from the initiator for one or more Data-in acknowledgement from the initiator for one or more Data-in
PDUs, via setting of the A-bit on a Data-in PDU. The PDUs, via setting of the A-bit on a Data-in PDU. The
Data_ACK_Notify Operational Primitive with the following Data_ACK_Notify Operational Primitive with the following
input qualifiers is used by the target Datamover layer to input qualifiers is used by the target Datamover layer to
notify the local iSCSI layer of the arrival of data notify the local iSCSI layer of the arrival of data
acknowledgement of a previously solicited iSCSI read data acknowledgement of a previously solicited iSCSI read data
acknowledgement. This Operational Primitive thus is appli- acknowledgement. This Operational Primitive thus is appli-
cable only to iSCSI targets. cable only to iSCSI targets.
skipping to change at page 33, line 5 skipping to change at page 35, line 5
c) DataSN: of the next SCSI Data-in PDU which immediately c) DataSN: of the next SCSI Data-in PDU which immediately
follows the SCSI Data-in PDU with the A-bit set to which follows the SCSI Data-in PDU with the A-bit set to which
this notification corresponds, with semantics as defined in this notification corresponds, with semantics as defined in
[RFC3720]. [RFC3720].
Every invocation of the Data_ACK_Notify Operational Primitive Every invocation of the Data_ACK_Notify Operational Primitive
MUST be preceded by an invocation of the Put_Data Operational MUST be preceded by an invocation of the Put_Data Operational
Primitive by the iSCSI target layer with the A-bit set to 1 Primitive by the iSCSI target layer with the A-bit set to 1
at an earlier point in time. at an earlier point in time.
11.3 Interactions for sending an iSCSI PDU 10.3 Interactions for sending an iSCSI PDU
This section discusses the interactions model for sending This section discusses the interactions model for sending
each of the iSCSI PDUs defined in [RFC3720]. A each of the iSCSI PDUs defined in [RFC3720]. A
Connection_Handle (see section 7.3) is assumed to qualify Connection_Handle (see section 5.3) is assumed to qualify
each of these interactions so that the Datamover layer can each of these interactions so that the Datamover layer can
route it to the appropriate Transport Connection. The route it to the appropriate Transport Connection. The
qualifying Connection_Handle is not explicitly listed in the qualifying Connection_Handle is not explicitly listed in the
subsequent sections. subsequent sections.
Note that the defined list of input qualifiers represents the Note that the defined list of input qualifiers represents the
semantically required set for the Datamover layer to consider semantically required set for the Datamover layer to consider
in implementing the Primitive in each interaction described in implementing the Primitive in each interaction described
in this section (see section 7.4 for an elaboration). in this section (see section 5.4 for an elaboration).
Implementations may choose to deduce the qualifiers in ways Implementations may choose to deduce the qualifiers in ways
that are optimized for the implementation specifics. Two that are optimized for the implementation specifics. Two
examples of this are: examples of this are:
1. For SCSI Command (section 11.3.1), deducing the 1. For SCSI Command (section 10.3.1), deducing the
ImmediateDataSize input qualifier from the ImmediateDataSize input qualifier from the
DataSegmentLength field of the SCSI Command PDU. DataSegmentLength field of the SCSI Command PDU.
2. For SCSI Data-Out (section 11.3.5.1), deducing the 2. For SCSI Data-Out (section 10.3.5.1), deducing the
DataDescriptorOut input qualifier from the associated DataDescriptorOut input qualifier from the associated
SCSI Command invocation qualifiers (assuming such state SCSI Command invocation qualifiers (assuming such state
is maintained) in conjunction with BHS fields of the is maintained) in conjunction with BHS fields of the
SCSI Data-out PDU. SCSI Data-out PDU.
11.3.1 SCSI Command 10.3.1 SCSI Command
The Send_Control Operational Primitive with the following The Send_Control Operational Primitive with the following
input qualifiers is used for requesting the transmission of a input qualifiers is used for requesting the transmission of a
SCSI Command PDU. SCSI Command PDU.
a) BHS and AHS, if any, of the SCSI Command PDU as defined in a) BHS and AHS, if any, of the SCSI Command PDU as defined in
[RFC3720] [RFC3720]
b) DataDescriptorOut: that defines the I/O Buffer meant for b) DataDescriptorOut: that defines the I/O Buffer meant for
Data-out for the entire command, in the case of a write or Data-out for the entire command, in the case of a write or
skipping to change at page 34, line 12 skipping to change at page 36, line 12
Data-in for the entire command, in the case of a read or Data-in for the entire command, in the case of a read or
bidirectional command bidirectional command
d) ImmediateDataSize: that defines the number of octets of d) ImmediateDataSize: that defines the number of octets of
immediate unsolicited data for a write/bidirectional immediate unsolicited data for a write/bidirectional
command command
e) UnsolicitedDataSize: that defines the number of octets of e) UnsolicitedDataSize: that defines the number of octets of
immediate and non-immediate unsolicited data for a immediate and non-immediate unsolicited data for a
write/bidirectional command. write/bidirectional command.
11.3.2 SCSI Response 10.3.2 SCSI Response
The Send_Control Operational Primitive with the following The Send_Control Operational Primitive with the following
input qualifiers is used for requesting the transmission of a input qualifiers is used for requesting the transmission of a
SCSI Response PDU. SCSI Response PDU.
a) BHS of the SCSI Response PDU as defined in [RFC3720] a) BHS of the SCSI Response PDU as defined in [RFC3720]
b) DataDescriptorStatus: that defines the iSCSI buffer which b) DataDescriptorStatus: that defines the iSCSI buffer which
contains the sense and response information for the command contains the sense and response information for the command
11.3.3 Task Management Function Request 10.3.3 Task Management Function Request
The Send_Control Operational Primitive with the following The Send_Control Operational Primitive with the following
input qualifiers is used for requesting the transmission of a input qualifiers is used for requesting the transmission of a
Task Management Function Request PDU. Task Management Function Request PDU.
a) BHS of the Task Management Function Request PDU as defined a) BHS of the Task Management Function Request PDU as defined
in [RFC3720] in [RFC3720]
b) DataDescriptorOut: that defines the I/O Buffer meant for b) DataDescriptorOut: that defines the I/O Buffer meant for
Data-out for the entire command, in the case of a write or Data-out for the entire command, in the case of a write or
bidirectional command (Only valid if Function="TASK bidirectional command (Only valid if Function="TASK
REASSIGN" [RFC3720] ] REASSIGN" [RFC3720] ]
c) DataDescriptorIn: that defines the I/O Buffer meant for c) DataDescriptorIn: that defines the I/O Buffer meant for
Data-in for the entire command, in the case of a read or Data-in for the entire command, in the case of a read or
bidirectional command (Only valid if Function="TASK bidirectional command (Only valid if Function="TASK
REASSIGN" - [RFC3720] ) REASSIGN" - [RFC3720] )
11.3.4 Task Management Function Response 10.3.4 Task Management Function Response
The Send_Control Operational Primitive with the following The Send_Control Operational Primitive with the following
input qualifier is used for requesting the transmission of a input qualifier is used for requesting the transmission of a
Task Management Function Response PDU. Task Management Function Response PDU.
a) BHS of the Task Management Function Response PDU as defined a) BHS of the Task Management Function Response PDU as defined
in [RFC3720] in [RFC3720]
11.3.5 SCSI Data-out & SCSI Data-in 10.3.5 SCSI Data-out & SCSI Data-in
11.3.5.1 SCSI Data-out 10.3.5.1 SCSI Data-out
The Send_Control Operational Primitive with the following The Send_Control Operational Primitive with the following
input qualifiers is used by the initiator iSCSI layer for input qualifiers is used by the initiator iSCSI layer for
requesting the transmission of a SCSI Data-out PDU carrying requesting the transmission of a SCSI Data-out PDU carrying
the non-immediate unsolicited data. the non-immediate unsolicited data.
a) BHS of the SCSI Data-out PDU as defined in [RFC3720] a) BHS of the SCSI Data-out PDU as defined in [RFC3720]
b) DataDescriptorOut: that defines the I/O Buffer with the b) DataDescriptorOut: that defines the I/O Buffer with the
Data-out to be carried in the iSCSI data segment of the PDU Data-out to be carried in the iSCSI data segment of the PDU
11.3.5.2 SCSI Data-in 10.3.5.2 SCSI Data-in
The Put_Data Operational Primitive with the following input The Put_Data Operational Primitive with the following input
qualifiers is used by the target iSCSI layer for requesting qualifiers is used by the target iSCSI layer for requesting
the transmission of the data carried by a SCSI Data-in PDU. the transmission of the data carried by a SCSI Data-in PDU.
a) BHS of the SCSI Data-in PDU as defined in [RFC3720] a) BHS of the SCSI Data-in PDU as defined in [RFC3720]
b) DataDescriptorIn: that defines the I/O Buffer with the b) DataDescriptorIn: that defines the I/O Buffer with the
Data-in being requested for transmission Data-in being requested for transmission
11.3.6 Ready To Transfer (R2T) 10.3.6 Ready To Transfer (R2T)
The Get_Data Operational Primitive with the following input The Get_Data Operational Primitive with the following input
qualifiers is used by the target iSCSI layer for requesting qualifiers is used by the target iSCSI layer for requesting
the retrieval of the data as specified by the semantic the retrieval of the data as specified by the semantic
content of an R2T PDU. content of an R2T PDU.
a) BHS of the Ready To Transfer PDU as defined in [RFC3720] a) BHS of the Ready To Transfer PDU as defined in [RFC3720]
b) DataDescriptorOut: that defines the I/O Buffer for the b) DataDescriptorOut: that defines the I/O Buffer for the
Data-out being requested for retrieval Data-out being requested for retrieval
11.3.7 Asynchronous Message 10.3.7 Asynchronous Message
The Send_Control Operational Primitive with the following The Send_Control Operational Primitive with the following
input qualifiers is used for requesting the transmission of input qualifiers is used for requesting the transmission of
an Asynchronous Message PDU. an Asynchronous Message PDU.
a) BHS of the Asynchronous Message PDU as defined in [RFC3720] a) BHS of the Asynchronous Message PDU as defined in [RFC3720]
b) DataDescriptorSense: that defines an iSCSI buffer which b) DataDescriptorSense: that defines an iSCSI buffer which
contains the sense and iSCSI Event information. contains the sense and iSCSI Event information.
11.3.8 Text Request 10.3.8 Text Request
The Send_Control Operational Primitive with the following The Send_Control Operational Primitive with the following
input qualifiers is used for requesting the transmission of a input qualifiers is used for requesting the transmission of a
Text Request PDU. Text Request PDU.
a) BHS of the Text Request PDU as defined in [RFC3720] a) BHS of the Text Request PDU as defined in [RFC3720]
b) DataDescriptorTextOut: that defines the iSCSI Text Request b) DataDescriptorTextOut: that defines the iSCSI Text Request
buffer buffer
11.3.9 Text Response 10.3.9 Text Response
The Send_Control Operational Primitive with the following The Send_Control Operational Primitive with the following
input qualifiers is used for requesting the transmission of a input qualifiers is used for requesting the transmission of a
Text Response PDU. Text Response PDU.
a) BHS of the Text Response PDU as defined in [RFC3720] a) BHS of the Text Response PDU as defined in [RFC3720]
b) DataDescriptorTextIn: that defines the iSCSI Text Response b) DataDescriptorTextIn: that defines the iSCSI Text Response
buffer buffer
11.3.10 Login Request 10.3.10 Login Request
The Send_Control Operational Primitive with the following The Send_Control Operational Primitive with the following
input qualifiers is used for requesting the transmission of a input qualifiers is used for requesting the transmission of a
Login Request PDU. Login Request PDU.
a) BHS of the Login Request PDU as defined in [RFC3720] a) BHS of the Login Request PDU as defined in [RFC3720]
b) DataDescriptorLoginRequest: that defines the iSCSI Login b) DataDescriptorLoginRequest: that defines the iSCSI Login
Request buffer Request buffer
skipping to change at page 37, line 31 skipping to change at page 39, line 31
the standard DA Primitives from being used for the iSCSI the standard DA Primitives from being used for the iSCSI
Login phase. When used in conjunction with such Datamover Login phase. When used in conjunction with such Datamover
protocols, an attempt to send a Login Request via the protocols, an attempt to send a Login Request via the
Send_Control Operational Primitive invocation is clearly an Send_Control Operational Primitive invocation is clearly an
error scenario, as the Login Request PDU is being sent while error scenario, as the Login Request PDU is being sent while
the connection is in the iSCSI full feature phase. It is the connection is in the iSCSI full feature phase. It is
outside the scope of this document to specify the resulting outside the scope of this document to specify the resulting
implementation behavior in this case - [RFC3720] already implementation behavior in this case - [RFC3720] already
defines the error handling for this error scenario. defines the error handling for this error scenario.
11.3.11 Login Response 10.3.11 Login Response
The Send_Control Operational Primitive with the following The Send_Control Operational Primitive with the following
input qualifiers is used for requesting the transmission of a input qualifiers is used for requesting the transmission of a
Login Response PDU. Login Response PDU.
a) BHS of the Login Response PDU as defined in [RFC3720] a) BHS of the Login Response PDU as defined in [RFC3720]
b) DataDescriptorLoginResponse: that defines the iSCSI Login b) DataDescriptorLoginResponse: that defines the iSCSI Login
Response buffer Response buffer
Note that specific Datamover protocols may choose to disallow Note that specific Datamover protocols may choose to disallow
the standard DA Primitives from being used for the iSCSI the standard DA Primitives from being used for the iSCSI
Login phase. When used in conjunction with such Datamover Login phase. When used in conjunction with such Datamover
protocols, an attempt to send a Login Response via the protocols, an attempt to send a Login Response via the
Send_Control Operational Primitive invocation is clearly an Send_Control Operational Primitive invocation is clearly an
error scenario, as the Login Response PDU is being sent while error scenario, as the Login Response PDU is being sent while
in the iSCSI full feature phase. It is outside the scope of in the iSCSI full feature phase. It is outside the scope of
this document to specify the resulting implementation this document to specify the resulting implementation
behavior in this case - [RFC3720] already defines the error behavior in this case - [RFC3720] already defines the error
handling for this error scenario. handling for this error scenario.
11.3.12 Logout Command 10.3.12 Logout Command
The Send_Control Operational Primitive with the following The Send_Control Operational Primitive with the following
input qualifier is used for requesting the transmission of a input qualifier is used for requesting the transmission of a
Logout Command PDU. Logout Command PDU.
a) BHS of the Logout Command PDU as defined in [RFC3720] a) BHS of the Logout Command PDU as defined in [RFC3720]
11.3.13 Logout Response 10.3.13 Logout Response
The Send_Control Operational Primitive with the following The Send_Control Operational Primitive with the following
input qualifier is used for requesting the transmission of a input qualifier is used for requesting the transmission of a
Logout Response PDU. Logout Response PDU.
a) BHS of the Logout Response PDU as defined in [RFC3720] a) BHS of the Logout Response PDU as defined in [RFC3720]
11.3.14 SNACK Request 10.3.14 SNACK Request
The Send_Control Operational Primitive with the following The Send_Control Operational Primitive with the following
input qualifier is used for requesting the transmission of a input qualifier is used for requesting the transmission of a
SNACK Request PDU. SNACK Request PDU.
a) BHS of the SNACK Request PDU as defined in [RFC3720] a) BHS of the SNACK Request PDU as defined in [RFC3720]
11.3.15 Reject 10.3.15 Reject
The Send_Control Operational Primitive with the following The Send_Control Operational Primitive with the following
input qualifiers is used for requesting the transmission of a input qualifiers is used for requesting the transmission of a
Reject PDU. Reject PDU.
a) BHS of the Reject PDU as defined in [RFC3720] a) BHS of the Reject PDU as defined in [RFC3720]
b) DataDescriptorReject: that defines the iSCSI Reject buffer b) DataDescriptorReject: that defines the iSCSI Reject buffer
11.3.16 NOP-Out 10.3.16 NOP-Out
The Send_Control Operational Primitive with the following The Send_Control Operational Primitive with the following
input qualifiers is used for requesting the transmission of a input qualifiers is used for requesting the transmission of a
NOP-Out PDU. NOP-Out PDU.
a) BHS of the NOP-Out PDU as defined in [RFC3720] a) BHS of the NOP-Out PDU as defined in [RFC3720]
b) DataDescriptorNOPOut: that defines the iSCSI Ping data b) DataDescriptorNOPOut: that defines the iSCSI Ping data
buffer buffer
11.3.17 NOP-In 10.3.17 NOP-In
The Send_Control Operational Primitive with the following The Send_Control Operational Primitive with the following
input qualifiers is used for requesting the transmission of a input qualifiers is used for requesting the transmission of a
NOP-In PDU. NOP-In PDU.
a) BHS of the NOP-In PDU as defined in [RFC3720] a) BHS of the NOP-In PDU as defined in [RFC3720]
b) DataDescriptorNOPIn: that defines the iSCSI Return Ping b) DataDescriptorNOPIn: that defines the iSCSI Return Ping
data buffer data buffer
11.4 Interactions for receiving an iSCSI PDU 10.4 Interactions for receiving an iSCSI PDU
The only PDUs that are received by an iSCSI layer operating The only PDUs that are received by an iSCSI layer operating
on a Datamover layer are the iSCSI control-type PDUs. The on a Datamover layer are the iSCSI control-type PDUs. The
Datamover layer delivers the iSCSI control-type PDUs as they Datamover layer delivers the iSCSI control-type PDUs as they
arrive, qualifying each with the Connection_Handle (see arrive, qualifying each with the Connection_Handle (see
section 7.3) that identifies the iSCSI connection the PDU is section 5.3) that identifies the iSCSI connection the PDU is
meant for. The subsequent processing of the iSCSI control- meant for. The subsequent processing of the iSCSI control-
type PDUs proceeds as defined in [RFC3720]. type PDUs proceeds as defined in [RFC3720].
11.4.1 SCSI Command 10.4.1 General Control-type PDU notification
The Control_Notify Operational Primitive is used for
notifying the arrival of a SCSI Command PDU.
11.4.2 SCSI Response
The Control_Notify Operational Primitive is used for
notifying the arrival of a SCSI Response PDU.
11.4.3 Task Management Function Request
The Control_Notify Operational Primitive is used for
notifying the arrival of a Task Management Function Request
PDU.
11.4.4 Task Management Function Response This sub-section describes the general mechanics applicable
to several control-type PDUs. The following sub-sections
note additional considerations for control-type PDUs not
covered in this sub-section.
The Control_Notify Operational Primitive is used for The Control_Notify Operational Primitive is used for
notifying the arrival of a Task Management Function Response notifying the arrival of the following iSCSI control-type
PDU. PDUs: SCSI Command, SCSI Response, Task Management Function
Request, Task Management Function Response, Asynchronous
Message, Text Request, Text Response, Logout command, Logout
Response, SNACK, Reject, NOP-Out, NOP-In.
11.4.5 SCSI Data-out & SCSI Data-in 10.4.2 SCSI Data Transfer PDUs
11.4.5.1 SCSI Data-out 10.4.2.1 SCSI Data-out
The Control_Notify Operational Primitive is used for The Control_Notify Operational Primitive is used for
notifying the iSCSI layer of the arrival of a SCSI Data-out notifying the iSCSI layer of the arrival of a SCSI Data-out
PDU carrying the non-immediate unsolicited data. Note PDU carrying the non-immediate unsolicited data. Note
however that the solicited SCSI Data-out arriving on the however that the solicited SCSI Data-out arriving on the
target is not notified to the iSCSI layer using the target is not notified to the iSCSI layer using the
Control_Notify Primitive because the solicited SCSI Data-out Control_Notify Primitive because the solicited SCSI Data-out
was not sent by the initiator iSCSI layer as control-type was not sent by the initiator iSCSI layer as control-type
PDUs. PDUs.
11.4.5.2 SCSI Data-in 10.4.2.2 SCSI Data-in
The arrival of the SCSI Data-in is not notified to the iSCSI The arrival of the SCSI Data-in is not notified to the iSCSI
layer by the Datamover layer at the initiator, because SCSI layer by the Datamover layer at the initiator, because SCSI
Data-in is an iSCSI data-type PDU (see section 7.1). The Data-in is an iSCSI data-type PDU (see section 5.1). The
iSCSI layer at the initiator however may infer the arrival of iSCSI layer at the initiator however may infer the arrival of
the SCSI Data-in when it receives a subsequent notification the SCSI Data-in when it receives a subsequent notification
of the SCSI Response PDU via a Control_Notify invocation. of the SCSI Response PDU via a Control_Notify invocation.
While this document does not contemplate the possibility of a While this document does not contemplate the possibility of a
Data-in PDU being received at the initiator iSCSI layer, Data-in PDU being received at the initiator iSCSI layer,
specific Datamover protocols may define how to deal with an specific Datamover protocols may define how to deal with an
unexpected inbound SCSI Data-in PDU that may result in the unexpected inbound SCSI Data-in PDU that may result in the
initiator iSCSI layer receiving the Data-in PDU. This initiator iSCSI layer receiving the Data-in PDU. This
document leaves the details of handling this error scenario document leaves the details of handling this error scenario
to the specific Datamover protocols, so each may define the to the specific Datamover protocols, so each may define the
appropriate error handling specific to the Datamover appropriate error handling specific to the Datamover
environment. environment.
11.4.6 Ready To Transfer (R2T) 10.4.2.3 Ready To Transfer (R2T)
Because an R2T PDU is an iSCSI data-type PDU (see section Because an R2T PDU is an iSCSI data-type PDU (see section
7.1) that is not delivered as-is to the initiator iSCSI 5.1) that is not delivered as-is to the initiator iSCSI
layer, the arrival of an R2T PDU is not notified to the iSCSI layer, the arrival of an R2T PDU is not notified to the iSCSI
layer by the Datamover layer. When an iSCSI node sends an layer by the Datamover layer. When an iSCSI node sends an
R2T PDU to its local Datamover layer, the local and remote R2T PDU to its local Datamover layer, the local and remote
Datamover layers transparently bring about the data transfer Datamover layers transparently bring about the data transfer
requested by the R2T PDU. requested by the R2T PDU.
While this document does not contemplate the possibility of While this document does not contemplate the possibility of
an R2T PDU being received at the initiator iSCSI layer, an R2T PDU being received at the initiator iSCSI layer,
specific Datamover protocols may define how to deal with an specific Datamover protocols may define how to deal with an
unexpected inbound R2T PDU that may result in the initiator unexpected inbound R2T PDU that may result in the initiator
iSCSI layer receiving the R2T PDU. This document leaves the iSCSI layer receiving the R2T PDU. This document leaves the
details of handling this error scenario to the specific details of handling this error scenario to the specific
Datamover protocols, so each may define the appropriate error Datamover protocols, so each may define the appropriate error
handling specific to the Datamover environment. handling specific to the Datamover environment.
11.4.7 Asynchronous Message 10.4.3 Login Request
The Control_Notify Operational Primitive is used for
notifying the arrival of an Asynchronous Message PDU.
11.4.8 Text Request
The Control_Notify Operational Primitive is used for
notifying the arrival of a Text Request PDU.
11.4.9 Text Response
The Control_Notify Operational Primitive is used for
notifying the arrival of a Text Response PDU.
11.4.10 Login Request
The Control_Notify Operational Primitive is used for The Control_Notify Operational Primitive is used for
notifying the target iSCSI layer of the arrival of a Login notifying the target iSCSI layer of the arrival of a Login
Request PDU. Note that specific Datamover protocols may Request PDU. Note that specific Datamover protocols may
choose to disallow the standard DA Primitives from being used choose to disallow the standard DA Primitives from being used
for the iSCSI Login phase. When used in conjunction with for the iSCSI Login phase. When used in conjunction with
such Datamover protocols, the arrival of a Login Request such Datamover protocols, the arrival of a Login Request
necessitating the Control_Notify Operational Primitive necessitating the Control_Notify Operational Primitive
invocation is clearly an error scenario, as the Login Request invocation is clearly an error scenario, as the Login Request
PDU is arriving in the iSCSI full feature phase. It is PDU is arriving in the iSCSI full feature phase. It is
outside the scope of this document to specify the resulting outside the scope of this document to specify the resulting
implementation behavior in this case - [RFC3720] already implementation behavior in this case - [RFC3720] already
defines the error handling in this error scenario. defines the error handling in this error scenario.
11.4.11 Login Response 10.4.4 Login Response
The Control_Notify Operational Primitive is used for The Control_Notify Operational Primitive is used for
notifying the initiator iSCSI layer of the arrival of a Login notifying the initiator iSCSI layer of the arrival of a Login
Response PDU. Note that specific Datamover protocols may Response PDU. Note that specific Datamover protocols may
choose to disallow the standard DA Primitives from being used choose to disallow the standard DA Primitives from being used
for the iSCSI Login phase. When used in conjunction with for the iSCSI Login phase. When used in conjunction with
such Datamover protocols, the arrival of a Login Response such Datamover protocols, the arrival of a Login Response
necessitating the Control_Notify Operational Primitive necessitating the Control_Notify Operational Primitive
invocation is clearly an error scenario, as the Login invocation is clearly an error scenario, as the Login
Response PDU is arriving in the iSCSI full feature phase. It Response PDU is arriving in the iSCSI full feature phase. It
is outside the scope of this document to specify the is outside the scope of this document to specify the
resulting implementation behavior in this case - [RFC3720] resulting implementation behavior in this case - [RFC3720]
already defines the error handling in this error scenario. already defines the error handling in this error scenario.
11.4.12 Logout Command 11 Security Considerations
The Control_Notify Operational Primitive is used for
notifying the arrival of a Logout Command PDU.
11.4.13 Logout Response
The Control_Notify Operational Primitive is used for
notifying the arrival of a Logout Response PDU.
11.4.14 SNACK Request
The Control_Notify Operational Primitive is used for
notifying the arrival of a SNACK Request PDU.
11.4.15 Reject
The Control_Notify Operational Primitive is used for
notifying the arrival of a Reject PDU.
11.4.16 NOP-Out 11.1 Architectural Considerations
The Control_Notify Operational Primitive is used for DA enables compliant iSCSI implementations to realize a
notifying the arrival of a NOP-Out PDU. control and data separation in the way they interact with
their Datamover protocols. Note however that this separation
does not imply a separation in transport mediums between
control traffic and data traffic - basic iSCSI architecture
with respect to tasks and PDU relationships to tasks remains
unchanged. [RFC3720] defines several MUST requirements on
ordering relationships across control and data for a given
task besides a mandatory deterministic task allegiance model
- DA does not change this basic architecture (DA has a
normative reference on [RFC3720]) nor allow any additional
flexibility in compliance in this area. To summarize,
sending bulk data transfers (prompted by Put_Data and
Get_Data Primitive invocations) on a different transport
medium would be as ill-advised as sending just the Data-
out/Data-in PDUs on a different TCP connection in RFC 3720-
based iSCSI implementations. Consequently, all the iSCSI-
related security text in [RFC3723] is directly applicable to
a DA-enabled iSCSI implementation.
11.4.17 NOP-In Another area with security implications is the Datamover
connection resource management model which DA defines
particularly the Allocate_Connection_Resources Primitive. An
inadvertent realization of this model could leave an iSCSI
implementation exposed to denial of service attacks. As
Figure 2 and Figure 3 in section 16.2 illustrate, the most
effective countermeasure to this potential attack consists of
performing the Datamover resource allocation when the iSCSI
layer is sufficiently far along in the iSCSI Login Phase that
it is reasonably certain that the peer side is not an
attacker. In particular, if the Login Phase includes a
SecurityNegotiation stage, an iSCSI end node MUST defer the
Datamover connection resource allocation (i.e. invoking the
Allocate_Connection_Resources Primitive) to the
LoginOperationalNegotiation stage ([RFC3720]) so that the
resource allocation happens post-authentication. This
considerably minimizes the potential for a denial of service
attack.
The Control_Notify Operational Primitive is used for 11.2 Wire Protocol Considerations
notifying the arrival of a NOP-In PDU.
12 Security Considerations In view of the fact that the DA architecture itself does not
define any new wire protocol nor propose modifications to the
existing protocols, there are no additional wire protocol
security considerations in employing DA itself. However, a
DA-compliant iSCSI implementation MUST comply with all the
iSCSI-related requirements stipulated in [RFC3723] and
[RFC3720]. Note further that in realizing DA, each Datamover
protocol must define and elaborate as appropriate on any
additional security considerations resulting from the use of
that Datamover protocol.
In view of the fact that DA does not define any new wire All Datamover protocol designers are strongly recommended to
protocol nor propose modifications to the existing protocols, refer to [RDDPSEC] for the types of security issues to
there are no additional security considerations in employing consider. While [RDDPSEC] elaborates on the security
DA, in addition to that of using the iSCSI protocol itself. considerations applicable to an RDDP-based Datamover
Any additional security considerations resulting from the use ([iSER]), the document is representative of the type of
of any Datamover protocol must be identified by the specific analysis of resource exhaustion and the application of
Datamover protocol specification as appropriate. countermeasures that needs to be done for any Datamover
protocol.
13 IANA Considerations 12 IANA Considerations
If a well-known port is chosen as the mechanism to identify a If a well-known port is chosen as the mechanism to identify a
Datamover protocol on TCP, the well-known port must be Datamover protocol on TCP, the well-known port must be
registered with IANA. Because the use of the well-known port registered with IANA. Because the use of the well-known port
is specific to the Datamover protocol in such a case, the is specific to the Datamover protocol in such a case, the
resulting IANA considerations from such use must be specified resulting IANA considerations from such use must be specified
by the specific Datamover protocol. DA itself does not have by the specific Datamover protocol. DA itself does not have
any specific IANA considerations. any specific IANA considerations.
14 References and Bibliography 13 References and Bibliography
14.1 Normative References 13.1 Normative References
[RFC3720] J. Satran, K. Meth, C. Sapuntzakis, M. Chadalapaka, [RFC3720] J. Satran, K. Meth, C. Sapuntzakis, M. Chadalapaka,
E. Zeidner, "Internet Small Computer Systems Interface E. Zeidner, "Internet Small Computer Systems Interface
(iSCSI)", RFC 3720, April 2004. (iSCSI)", RFC 3720, April 2004.
14.2 Informative References [RFC3723] B. Aboba, J. Tseng, J. Walker, V. Rangan, F.
Travostino, "Securing Block Storage Protocols over IP",
RFC 3723, April 2004.
13.2 Informative References
[DDP] H. Shah et al., "Direct Data Placement over Reliable [DDP] H. Shah et al., "Direct Data Placement over Reliable
Transports", IETF Internet Draft draft-ietf-rddp-ddp- Transports", IETF Internet Draft draft-ietf-rddp-ddp-
04.txt (work in progress), February 2005. 04.txt (work in progress), February 2005.
[iSER] M. Ko et al., "iSCSI Extensions for RDMA", IETF [iSER] M. Ko et al., "iSCSI Extensions for RDMA", IETF
Internet Draft draft-ietf-ips-iser-00.txt (work in Internet Draft draft-ietf-ips-iser-03.txt (work in
progress), September 2004. progress), April 2005.
[MPA] P. Culley et al., "Marker PDU Aligned Framing for TCP [MPA] P. Culley et al., "Marker PDU Aligned Framing for TCP
Specification", IETF Internet Draft draft-ietf-rddp-mpa- Specification", IETF Internet Draft draft-ietf-rddp-mpa-
02.txt (work in progress), February 2004. 02.txt (work in progress), February 2005.
[RDDPSEC] J. Pinkerton et al., "DDP/RDMAP Security", IETF
Internet Draft draft-ietf-rddp-security-07.txt (work in
progress), April 2005
[RDMAP] R. Recio et al., "An RDMA Protocol Specification", [RDMAP] R. Recio et al., "An RDMA Protocol Specification",
IETF Internet Draft draft-ietf-rddp-rdmap-03.txt (work in IETF Internet Draft draft-ietf-rddp-rdmap-03.txt (work in
progress), February 2005. progress), February 2005.
[RFC2119] Bradner, S. "Key Words for use in RFCs to Indicate [RFC2119] Bradner, S. "Key Words for use in RFCs to Indicate
Requirement Levels", BCP 14, RFC 2119, March 1997. Requirement Levels", BCP 14, RFC 2119, March 1997.
[SAM] ANSI X3.270-1998, SCSI-3 Architecture Model (SAM). [SAM] ANSI X3.270-1998, SCSI-3 Architecture Model (SAM).
[SCTP] R. Stewart et al., "Stream Control Transmission [SCTP] R. Stewart et al., "Stream Control Transmission
Protocol", RFC 2960, October 2000. Protocol", RFC 2960, October 2000.
[SPC3]T10/1416-D, SCSI Primary Commands-3. [SPC3]T10/1416-D, SCSI Primary Commands-3.
15 Authors' Addresses 14 Authors' Addresses
Mallikarjun Chadalapaka Mallikarjun Chadalapaka
Hewlett-Packard Company Hewlett-Packard Company
8000 Foothills Blvd. 8000 Foothills Blvd.
Roseville, CA 95747-5668, USA Roseville, CA 95747-5668, USA
Phone: +1-916-785-5621 Phone: +1-916-785-5621
E-mail: cbm@rose.hp.com E-mail: cbm@rose.hp.com
John L. Hufferd John L. Hufferd
IBM IBM
skipping to change at page 48, line 5 skipping to change at page 50, line 5
Hemal Shah Hemal Shah
Intel Corporation Intel Corporation
MS PTL1 MS PTL1
1501 South Mopac Expressway, #400 1501 South Mopac Expressway, #400
Austin, TX 78746 USA Austin, TX 78746 USA
Phone: +1 (512) 732-3963 Phone: +1 (512) 732-3963
Email: hemal.shah@intel.com Email: hemal.shah@intel.com
Comments may be sent to Mallikarjun Chadalapaka. Comments may be sent to Mallikarjun Chadalapaka.
16 Acknowledgements 15 Acknowledgements
The IPS Working group in the Transport Area of IETF is The IP Storage (ips) Working Group in the Transport Area of
responsible for defining the iSCSI protocol (apart from a IETF has been responsible for defining the iSCSI protocol
host of other relevant IP Storage protocols). The authors (apart from a host of other relevant IP Storage protocols).
are grateful to the entire working group, whose work allowed The authors are grateful to the entire working group, whose
this document to build on the concepts and details of the work allowed this document to build on the concepts and
iSCSI protocol. details of the iSCSI protocol.
In addition, the following individuals had reviewed and In addition, the following individuals had reviewed and
contributed to the improvement of this document. The authors contributed to the improvement of this document. The authors
are grateful for their contribution. are grateful for their contribution.
John Carrier John Carrier
Adaptec, Inc. Adaptec, Inc.
691 S. Milpitas Blvd. 691 S. Milpitas Blvd., Milpitas, CA 95035 USA
Milpitas, CA 95035 USA
Phone: +1 (360) 378-8526 Phone: +1 (360) 378-8526
Email: john_carrier@adaptec.com Email: john_carrier@adaptec.com
Hari Ghadia Hari Ghadia
Adaptec, Inc. Adaptec, Inc.
691 S. Milpitas Blvd., 691 S. Milpitas Blvd., Milpitas, CA 95035 USA
Milpitas, CA 95035 USA
Phone: +1 (408) 957-5608 Phone: +1 (408) 957-5608
Email: hari_ghadia@adaptec.com Email: hari_ghadia@adaptec.com
Hari Mudaliar Hari Mudaliar
Adaptec, Inc. Adaptec, Inc.
691 S. Milpitas Blvd., 691 S. Milpitas Blvd., Milpitas, CA 95035 USA
Milpitas, CA 95035 USA
Phone: +1 (408) 957-6012 Phone: +1 (408) 957-6012
Email: hari_mudaliar@adaptec.com Email: hari_mudaliar@adaptec.com
Patricia Thaler Patricia Thaler
Agilent Technologies, Inc. Agilent Technologies, Inc.
1101 Creekside Ridge Drive, #100 1101 Creekside Ridge Drive, #100, M/S-RG10,
M/S-RG10
Roseville, CA 95678 Roseville, CA 95678
Phone: +1-916-788-5662 Phone: +1-916-788-5662
email: pat_thaler@agilent.com email: pat_thaler@agilent.com
Uri Elzur Uri Elzur
Broadcom Corporation Broadcom Corporation
16215 Alton Parkway 16215 Alton Parkway, Irvine, CA 92619-7013 USA
Irvine, California 92619-7013 USA
Phone: +1 (949) 585-6432 Phone: +1 (949) 585-6432
Email: Uri@Broadcom.com Email: Uri@Broadcom.com
Mike Penna Mike Penna
Broadcom Corporation Broadcom Corporation
16215 Alton Parkway 16215 Alton Parkway,Irvine, CA 92619-7013 USA
Irvine, California 92619-7013 USA
Phone: +1 (949) 926-7149 Phone: +1 (949) 926-7149
Email: MPenna@Broadcom.com Email: MPenna@Broadcom.com
David Black
EMC Corporation
176 South St., Hopkinton, MA 01748, USA
Phone: +1 (508) 293-7953
Email: black_david@emc.com
Ted Compton Ted Compton
EMC Corporation EMC Corporation
Research Triangle Park, NC 27709, USA Research Triangle Park, NC 27709, USA
Phone: 919-248-6075 Phone: +1-919-248-6075
Email: compton_ted@emc.com Email: compton_ted@emc.com
Dwight Barron Dwight Barron
Hewlett-Packard Company Hewlett-Packard Company
20555 SH 249 20555 SH 249, Houston, TX 77070-2698 USA
Houston, TX 77070-2698 USA
Phone: +1 (281) 514-2769 Phone: +1 (281) 514-2769
Email: Dwight.Barron@Hp.com Email: Dwight.Barron@Hp.com
Paul R. Culley Paul R. Culley
Hewlett-Packard Company Hewlett-Packard Company
20555 SH 249 20555 SH 249, Houston, TX 77070-2698 USA
Houston, TX 77070-2698 USA
Phone: +1 (281) 514-5543 Phone: +1 (281) 514-5543
Email: paul.culley@hp.com Email: paul.culley@hp.com
Dave Garcia Dave Garcia
Hewlett-Packard Company Hewlett-Packard Company
19333 Vallco Parkway 19333 Vallco Parkway, Cupertino, Ca. 95014 USA
Cupertino, Ca. 95014 USA
Phone: +1 (408) 285-6116 Phone: +1 (408) 285-6116
Email: dave.garcia@hp.com Email: dave.garcia@hp.com
Randy Haagens Randy Haagens
Hewlett-Packard Company Hewlett-Packard Company
8000 Foothills Blvd, MS 5668 8000 Foothills Blvd, MS 5668, Roseville CA
Roseville CA
Phone: +1-916-785-4578 Phone: +1-916-785-4578
email: randy_haagens@hp.com email: randy_haagens@hp.com
Jeff Hilland Jeff Hilland
Hewlett-Packard Company Hewlett-Packard Company
20555 SH 249 20555 SH 249, Houston, Tx. 77070-2698 USA
Houston, Tx. 77070-2698 USA
Phone: +1 (281) 514-9489 Phone: +1 (281) 514-9489
Email: jeff.hilland@hp.com Email: jeff.hilland@hp.com
Mike Krause Mike Krause
Hewlett-Packard Company, 43LN Hewlett-Packard Company, 43LN
19410 Homestead Road 19410 Homestead Road, Cupertino, CA 95014 USA
Cupertino, CA 95014 USA
Phone: +1 (408) 447-3191 Phone: +1 (408) 447-3191
Email: krause@cup.hp.com Email: krause@cup.hp.com
Jim Wendt Jim Wendt
Hewlett-Packard Company Hewlett-Packard Company
8000 Foothills Blvd, MS 5668 8000 Foothills Blvd, MS 5668, Roseville CA
Roseville CA
Phone: +1-916-785-5198 Phone: +1-916-785-5198
email: jim_wendt@hp.com email: jim_wendt@hp.com
Mike Ko Mike Ko
IBM IBM
650 Harry Rd. 650 Harry Rd, San Jose, CA 95120
San Jose, CA 95120
Phone: +1 (408) 927-2085 Phone: +1 (408) 927-2085
Email: mako@us.ibm.com Email: mako@us.ibm.com
Renato Recio Renato Recio
IBM Corporation IBM Corporation
11501 Burnett Road 11501 Burnett Road, Austin, TX 78758 USA
Austin, TX 78758 USA
Phone: +1 (512) 838-1365 Phone: +1 (512) 838-1365
Email: recio@us.ibm.com Email: recio@us.ibm.com
Howard C. Herbert Howard C. Herbert
Intel Corporation Intel Corporation
MS CH7-404 MS CH7-404,5000 West Chandler Blvd., Chandler, AZ 85226 USA
5000 West Chandler Blvd.
Chandler, AZ 85226 USA
Phone: +1 (480) 554-3116 Phone: +1 (480) 554-3116
Email: howard.c.herbert@intel.com Email: howard.c.herbert@intel.com
Dave Minturn Dave Minturn
Intel Corporation Intel Corporation
MS JF1-210 MS JF1-210, 5200 North East Elam Young Parkway
5200 North East Elam Young Parkway
Hillsboro, OR 97124 USA Hillsboro, OR 97124 USA
Phone: +1 (503) 712-4106 Phone: +1 (503) 712-4106
Email: dave.b.minturn@intel.com Email: dave.b.minturn@intel.com
James Pinkerton James Pinkerton
Microsoft Corporation Microsoft Corporation
One Microsoft Way One Microsoft Way, Redmond, WA 98052 USA
Redmond, WA 98052 USA
Phone: +1 (425) 705-5442 Phone: +1 (425) 705-5442
Email: jpink@microsoft.com Email: jpink@microsoft.com
Tom Talpey Tom Talpey
Network Appliance Network Appliance
375 Totten Pond Road 375 Totten Pond Road, Waltham, MA 02451 USA
Waltham, MA 02451 USA
Phone: +1 (781) 768-5329 Phone: +1 (781) 768-5329
EMail: thomas.talpey@netapp.com EMail: thomas.talpey@netapp.com
17 Appendix 16 Appendix
17.1 Design considerations for a Datamover protocol 16.1 Design considerations for a Datamover protocol
This section discusses the specific considerations for RDMA- This section discusses the specific considerations for RDMA-
based and RDDP-based Datamover protocols, and is only based and RDDP-based Datamover protocols.
informational.
a) Note that the modeling of interactions for SCSI Data-Out a) Note that the modeling of interactions for SCSI Data-Out
(section 11.3.5.1) is only used for unsolicited data (section 10.3.5.1) is only used for unsolicited data
transfer. transfer.
b) The modeling of interactions for SNACK (section 11.3.14, b) The modeling of interactions for SNACK (section 10.3.14,
and section 11.4.14) is not expected to be used given that and section 10.4.1) is not expected to be used given that
one of the design requirements on the Datamover is that it one of the design requirements on the Datamover is that it
"guarantees an error-free, reliable, in-order transport "guarantees an error-free, reliable, in-order transport
mechanism" (section 8). The interactions for sending and mechanism" (section 6). The interactions for sending and
receiving a SNACK are nevertheless modeled in this document receiving a SNACK are nevertheless modeled in this document
because the receiving iSCSI layer can deterministically because the receiving iSCSI layer can deterministically
deal with an inadvertent SNACK. This also shows the DA deal with an inadvertent SNACK. This also shows the DA
designers' intent that DI is not meant to filter certain designers' intent that DI is not meant to filter certain
types of PDUs. types of PDUs.
c) The onus is on a reliable Datamover (per requirements c) The onus is on a reliable Datamover (per requirements
stated in section 8) to realize end-to-end data stated in section 6) to realize end-to-end data
acknowledgements via Datamover-specific means. In view of acknowledgements via Datamover-specific means. In view of
this, even data-ACK-type SNACKs are unnecessary to be used. this, even data-ACK-type SNACKs are unnecessary to be used.
Consequently, an initiator may never request sending a Consequently, an initiator may never request sending a
SNACK Request in this model assuming that the proactive SNACK Request in this model assuming that the proactive
(timeout-driven) SNACK functionality is turned off in the (timeout-driven) SNACK functionality is turned off in the
legacy iSCSI code. legacy iSCSI code.
d) Note that the current DA model for bootstrapping a d) Note that the current DA model for bootstrapping a
Connection_Handle into service i.e. associating a new Connection_Handle into service i.e. associating a new
iSCSI connection with a Connection_Handle clearly implies iSCSI connection with a Connection_Handle clearly implies
that the iSCSI connection must already be in full feature that the iSCSI connection must already be in full feature
phase when the Datamover layer comes into the stack. This phase when the Datamover layer comes into the stack. This
further implies that the iSCSI login phase must be carried further implies that the iSCSI login phase must be carried
out in the traditional "Byte streaming mode" with no out in the traditional "Byte streaming mode" with no
assistance or involvement from the Datamover layer. assistance or involvement from the Datamover layer.
17.2 Examples of Datamover interactions 16.2 Examples of Datamover interactions
The figures described in this section provide some examples The figures described in this section provide some examples
of the usage of Operational Primitives in interactions of the usage of Operational Primitives in interactions
between the iSCSI layer and the Datamover layer. The between the iSCSI layer and the Datamover layer. The
following abbreviations are used in this section. following abbreviations are used in this section.
Avail Available Avail Available
Abted - Aborted Abted - Aborted
skipping to change at page 62, line 5 skipping to change at page 64, line 5
| r| | y| | y| | r| | r| | y| | y| | r|
| | | e| | e| | | | | | e| | e| | |
| | | r| | r| | | | | | r| | r| | |
| | | | | | | | | | | | | | | |
| |Dal_Tk_Res| | | |Dal_Tk_Res | | | |Dal_Tk_Res| | | |Dal_Tk_Res | |
| |--------->| | | |<-----------| | | |--------->| | | |<-----------| |
| | | | | | | | | | | | | | | |
Figure 12 Task resource cleanup on abort Figure 12 Task resource cleanup on abort
18 Full Copyright Statement 17 Full Copyright Statement
Copyright (C) The Internet Society (2004). This document is Copyright (C) The Internet Society (2005). This document is
subject to the rights, licenses and restrictions contained in subject to the rights, licenses and restrictions contained in
BCP 78, and except as set forth therein, the authors retain BCP 78, and except as set forth therein, the authors retain
all their rights. all their rights.
This document and the information contained herein are This document and the information contained herein are
provided on an "AS IS" basis and THE CONTRIBUTOR, THE provided on an "AS IS" basis and THE CONTRIBUTOR, THE
ORGANIZATION HE/SHE REPRESENTS OR IS SPONSORED BY (IF ANY), ORGANIZATION HE/SHE REPRESENTS OR IS SPONSORED BY (IF ANY),
THE INTERNET SOCIETY AND THE INTERNET ENGINEERING TASK FORCE THE INTERNET SOCIETY AND THE INTERNET ENGINEERING TASK FORCE
DISCLAIM ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING BUT DISCLAIM ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING BUT
NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION
HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES
OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE.
19 Intellectual Property Statement 18 Intellectual Property Statement
The IETF takes no position regarding the validity or scope of The IETF takes no position regarding the validity or scope of
any Intellectual Property Rights or other rights that might any Intellectual Property Rights or other rights that might
be claimed to pertain to the implementation or use of the be claimed to pertain to the implementation or use of the
technology described in this document or the extent to which technology described in this document or the extent to which
any license under such rights might or might not be any license under such rights might or might not be
available; nor does it represent that it has made any available; nor does it represent that it has made any
independent effort to identify any such rights. Information independent effort to identify any such rights. Information
on the procedures with respect to rights in RFC documents can on the procedures with respect to rights in RFC documents can
be found in BCP 78 and BCP 79. be found in BCP 78 and BCP 79.
 End of changes. 

This html diff was produced by rfcdiff 1.24, available from http://www.levkowetz.com/ietf/tools/rfcdiff/