[Docs] [txt|pdf] [Tracker] [Email] [Nits]

Versions: 00

INTERNET-DRAFT                            P. Culley
draft-culley-mpa-issueresponses-00.txt      Hewlett-Packard Company


                                          Expires: March 2004


        Comment responses for Marker PDU Aligned Framing for TCP
                             Specification







































Culley                   Expires: March 2004                 [Page 1]


INTERNET-DRAFT           MPA Framing for TCP          23 January 2004



Status of this Memo

   This document is an Internet-Draft and is subject to all
   provisions of Section 10 of RFC2026.

   Internet-Drafts are working documents of the Internet Engineering
   Task Force (IETF), its areas, and its working groups.  Note that
   other groups may also distribute working documents as Internet-
   Drafts.

   Internet-Drafts are draft documents valid for a maximum of six
   months and may be updated, replaced, or obsoleted by other
   documents at any time.  It is inappropriate to use Internet-Drafts
   as reference material or to cite them other than as "work in
   progress."

   The list of current Internet-Drafts can be accessed at
   http://www.ietf.org/1id-abstracts.html. The list of Internet-Draft
   Shadow Directories can be accessed at
   http://www.ietf.org/shadow.html




























Culley                   Expires: March 2004                 [Page 2]


INTERNET-DRAFT           MPA Framing for TCP          23 January 2004



Abstract

   This draft is a response to comments about draft-ietf-rddp-mpa-00
   received on the RDDP reflector, the response is long enough to
   warrant a draft.  Comments on connection startup sequence,
   rejecting a connection, and allowing "Active/Active" connections.



   Table of Contents

Status of this Memo.................................................2
Abstract............................................................3
1       Introduction...............................................4
2       Issue on allowed startup sequence..........................5
2.1     Dual Stack Designs with the current draft..................6
2.2     Dual Stack with no FPDU sending restriction...............11
2.3     Why should we consider changing the draft?................14
3       Active/Active Connections.................................15
3.1     Why should we consider changing the draft?................17
4       Adding a "rejected connection" bit........................18
5       References................................................20
5.1     Normative References......................................20
5.2     Informative References....................................20
6       Author's Addresses........................................20


   Table of Figures

Figure 1 Typical MPA/TCP immediate startup..........................8
Figure 2 MPA/TCP immediate startup with Early FPDU sent by responder11
Figure 3 MPA/TCP immediate startup "Active/Active".................16
















Culley                   Expires: March 2004                 [Page 3]


INTERNET-DRAFT           MPA Framing for TCP          23 January 2004



1  Introduction

   The reader is presumed to be familiar with [MPA], [DDP], and may
   find it useful to have looked at the connection management
   sections of [verbs].  Some portions of the comments also imply
   knowledge of [RDMA].

   This draft is simply a discussion of the design choices that led
   to the current MPA draft and is intended to assist the workgroup
   in making informed decisions on the appropriate direction for
   future MPA drafts.

   It is unlikely that this draft will be updated.  Comments and
   suggested directions are welcome and should be directed to the
   RDDP mailing list (rddp@ietf.org).

































Culley                   Expires: March 2004                 [Page 4]


INTERNET-DRAFT           MPA Framing for TCP          23 January 2004



2  Issue on allowed startup sequence

   An issue was brought regarding a specific MUST in the MPA
   specification (included below):

        4.  MPA "Responder" mode implementations MUST receive and
        validate at least one FPDU before sending any FPDUs or
        markers.

        Note: this requirement is present to allow the Initiator time
        to get its receiver into full operation before an FPDU
        arrives, avoiding potentially difficult requirements on the
        receiver.

   Caitlin wrote:

   "As previously covered, I believe this rule is overly broad. The
   only requirement is that neither side send their MPA Start Frame
   before they are ready to process DDP Segments."

   Caitlin's requirement can be interpreted at least two ways,
   either:

   A) It is ok to send FPDUs as soon as the start frame is sent (but
   potentially before the incoming start frame is received), OR,

   B) It is ok to send as long as the start frame has been sent and
   received.

   Using the first interpretation, Caitlin's rule has the following
   problems:

   1)  The sender must know the receiver's correct marker/CRC
       settings before sending FPDU's or risk having the connection
       torn down.

       Caitlin also feels that this may be a reasonable response, she
       wrote:

        "If a mismatch in capabilities will result in terminating the
        connection, then there is nothing wrong with sending FPDUs
        before this is confirmed. If there is a mismatch, the FPDUs
        will be flushed anyway. Obviously traffic should not be
        placed on the network that has any substantial chance of
        being thrown away. But if the sender has reason to believe
        that it already knows its peers mode (perhaps based on


Culley                   Expires: March 2004                 [Page 5]


INTERNET-DRAFT           MPA Framing for TCP          23 January 2004



        directory information or prior connections) then it should
        not be required to stall waiting for dynamic confirmation."

   2)  The inclusion of the "private data" in the startup frame was
       based on the design requirement for the ULP to send and get a
       reply to some private data before binding the TCP/MPA
       connection to DDP (and choosing a PD).  The rule above allows
       binding to DDP and sending an FPDU before the reply arrives,
       also increasing the opportunity for incorrect ULP
       configuration.  As above, if the ULP already knows the peers
       configuration, then this may not be an issue (and the private
       data was not actually necessary for this ULP).

   3)  The rule requires that the receiver design be prepared to
       receive an FPDU BEFORE it has actually been bound to DDP.
       This can happen with out of order TCP segment arrival, or due
       to internal processing delay.  It is also possible that the
       start frame might even be in the same TCP segment.  This,
       while not any kind of violation of TCP, makes for much more
       Difficult "Dual Stack" implementations.  The effects on dual
       stacks and the transition between them will be discussed
       further below.

   Using the second interpretation of Caitlin's rule (it is ok to
   send FPDUs as long as start frame has been sent and received), the
   problem is only of implementation, again making a "Dual Stack"
   design more difficult.

   In order to evaluate this issue, we present a discussion of what
   we expect to be a common design implementation, a "dual stack".

2.1  Dual Stack Designs with the current draft

   Warning, the following is highly implementation specific, and as
   such must not be regarded as any kind of requirement on
   implementations.

   MPA/DDP implementations are commonly expected to be implemented as
   part of a "Dual stack" architecture.  One "stack" is the
   traditional TCP stack, usually with a sockets interface API and
   the various OS dependent layers and a "standard" NIC.  The second
   stack is the MPA/DDP "stack" with its own API, and potentially
   separate code or hardware to deal with the MPA/DDP data.  Of
   course, implementations may vary, so the following comments are of
   an advisory nature only.



Culley                   Expires: March 2004                 [Page 6]


INTERNET-DRAFT           MPA Framing for TCP          23 January 2004



   The use of the two "stacks" offers advantages:

        TCP connection setup is usually done with the TCP stack. This
        allows use of the usual naming and addressing mechanisms.  It
        also means that any mechanisms used to "harden" the
        connection setup against security threats are also used when
        starting MPA/DDP.

        Some applications may have been originally designed for TCP,
        but are "enhanced" to utilize MPA/DDP after a negotiation
        reveals the capability to do so.  The negotiation process
        takes place in TCP's streaming mode, using the usual TCP
        APIs.

        Even new applications, designed for RDMA or DDP, still need
        to exchange some private data (and the start frames) prior to
        binding DDP. Using the traditional TCP streaming mode for
        this exchange allows this to be done using well understood
        methods and simplifies the full MPA/DDP stack (since full
        streaming mode operations and API's need not also be
        supported by that stack).

   The main disadvantage of using two stacks is the conversion of an
   active TCP connection between them.  This process must be done
   with care to prevent loss of data.

   Following is the proposed connection setup showing interactions at
   each level of the stack.  Note that, while the example shows an
   MPA startup just after the TCP connects, it is also expected that
   some applications will pass some amount of streaming data before
   performing the MPA startup.

















Culley                   Expires: March 2004                 [Page 7]


INTERNET-DRAFT           MPA Framing for TCP          23 January 2004



+----------------+       +----------------+
| MPA INITIATOR  |       | MPA RESPONDER  |
+----------------+       +----------------+
ULP  DDP  MPA  TCP       TCP  MPA  DDP  ULP
 |    |    |    |         |    |    |    |
 |    |    |    |         |<-------------|A
A|------------->|   SYN   |    |    |    |
 |    |    |    |-------->|    |    |    |
 |    |    |    | SYN+ACK |    |    |    |
 |    |    |    |<--------|    |    |    |
 |    |    |    |   ACK   |    |    |    |
 |    |    |    |-------->|    |    |    |
B|<-------------|         |------------->|B
 |    |    |    |         |    |    |    |
 |    |    |    |         |    |    |    |
 |    |    |    |         |    |<--------|C
C|-------->|--->| MPA REQ |    |1   |    |
 |    |   1|2   |-------->|    |    |    |
 |    |    |    |         |--->|-------->|D
 |    |    |    |         |   2|3   |    |
 |    |    |    |         |    |    |    |
 |    |    |    | MPA REP |<---|<---|<---|E
 |    |    |    |<--------|   5|4   |    |
D|<--------|<---|         |    |    |    |
 |    |   4|3   |         |    |    |    |
 |    |    |    |         |    |    |    |
E|--->|    |    |         |    |    |    |
 |    |    |    |         |    |    |    |
F|--->|--->|===>|         |    |    |    |
 |    |   5|6   |========>|    |    |    |
 |    |    |    |         |===>|--->|--->|G
 |    |    |    |         |   6|    |    |
 |    |    |    |         |    |    |    |
 |    |    |    |         |<===|<---|<---|H
 |    |    |    |<========|   8|7   |    |
G|<---|<---|<===|         |    |    |    |
 |    |    |7   |         |    |    |    |
 |    |    |    |         |    |    |    |

               Figure 1 Typical MPA/TCP immediate startup








Culley                   Expires: March 2004                 [Page 8]


INTERNET-DRAFT           MPA Framing for TCP          23 January 2004




Initiator
     ULP A     connect
     ULP B     TCP connection established
     ULP C     MPA initiate negotiation, send Request frame
     ULP D     receive MPA Reply + private data
     ULP E     connect DDP (post untagged buffers)
     ULP F     send DDP message
     ULP G     receive DDP message

Responder
     ULP A     listen
     ULP B     accept (connection established)
     ULP C     MPA listen
     ULP D     receive MPA Request Frame + private data
     ULP E     Accept: Send Response frame + private data,
               connect DDP (post untagged buffers)
     ULP G     receive DDP message
     ULP H     send DDP message

Initiator Using the Legacy stacks
     MPA 1   ULP requests MPA to 'initiate' mode frame exchange
     MPA 2   MPA sends "MPA Request Frame" + private data
     MPA 3   MPA received "MPA Reply Frame" from responder
     MPA 4   MPA sends private data to ULP
          Using the DDP/MPA stack
     MPA 5   ULP/DDP requests MPA to send DDP message
     MPA 6   MPA sends FPDU to responder
     MPA 7   MAP receives FPDU from responder

Responder Using the Legacy stacks
     MPA 1   ULP requests MPA to 'respond' to mode frame exchange
     MPA 2   MPA receives "MPA Request Frame" from initiator
     MPA 3   MPA sends private data to ULP
     MPA 4   MPA receives private data from ULP
          Using the DDP/MPA stack
     MPA 5   MPA sends "MPA Reply Frame" to initiator
     MPA 6   MPA receives FPDU from initiator
     MPA 7   ULP/DDP requests MPA to send DDP message
     MPA 8   MPA sends FPDU to initiator

   The switching between the legacy stack, and the TCP/MPA/DDP stack
   occurs at point (ULP E) for both the initiator and responder.  For
   the initiator, no additional incoming streaming data is expected
   or buffered, so that no buffers need be passed between stacks.  If
   any data did appear, it would be a protocol violation and could be
   dropped.

Culley                   Expires: March 2004                 [Page 9]


INTERNET-DRAFT           MPA Framing for TCP          23 January 2004



   Note that if the last Ack was not sent at the time of the
   transition, or if packet loss occurs, the initiator's RDDP stack
   is responsible for sending or resending that Ack.  Any repeated
   incoming data can be dropped.

   For the Responder, the transition point occurs at the time the
   last streaming mode message (Start frame) is sent.  To allow an
   implementation to avoid a potential race between getting into FPDU
   mode and the arrival of the first FPDU, the transition (as
   described in [verbs]) suggests that the last streaming mode
   message be sent by the RDDP stack.  This allows the stack to do
   any of several strategies to avoid the race.  At the time of the
   transition, again no data is expected inbound or buffered, so that
   no buffers need be passed between stacks.

   As with the initiator, the responder may have to (re)acknowledge
   prior data.  It may also have to retransmit it's last streaming
   message while being prepared to receive an FPDU.

   As can be seen, one advantage of implementing this way is that
   there is no need to pass "undelivered" data between the stacks.



























Culley                   Expires: March 2004                [Page 10]


INTERNET-DRAFT           MPA Framing for TCP          23 January 2004



2.2  Dual Stack with no FPDU sending restriction

   Let us look at using a dual stack design, but allowing FPDUs to be
   sent any time after sending and receiving the start frames (the
   second interpretation of Caitlin's proposed rule).

+----------------+       +----------------+
| MPA INITIATOR  |       | MPA RESPONDER  |
+----------------+       +----------------+
ULP  DDP  MPA  TCP       TCP  MPA  DDP  ULP
 |    |    |    |         |    |    |    |
 |    |    |    |         |<-------------|A
A|------------->|   SYN   |    |    |    |
 |    |    |    |-------->|    |    |    |
 |    |    |    | SYN+ACK |    |    |    |
 |    |    |    |<--------|    |    |    |
 |    |    |    |   ACK   |    |    |    |
 |    |    |    |-------->|    |    |    |
B|<-------------|         |------------->|B
 |    |    |    |         |    |    |    |
 |    |    |    |         |    |    |    |
 |    |    |    |         |    |<--------|C
C|-------->|--->| MPA REQ |    |1   |    |
 |    |   1|2   |-------->|    |    |    |
 |    |    |    |         |--->|-------->|D
 |    |    |    |         |   2|3   |    |
 |    |    |    |         |    |    |    |
 |    |    |    | MPA REP |<---|<---|<---|E
 |    |    |    |<--------|   5|4   |    |
D|<--------|<---|         |    |    |    |
 |    |   4|3   |         |    |    |    |
 |    |    |    |         |<===|<---|<---|H
 |    |    |    |<========|   6|5   |    |
 |    |    |   5|         |    |    |    |
E|--->|    |    |         |    |    |    |
 |    |    |    |         |    |    |    |
F|<---|<---|<===|         |    |    |    |
 |    |    |6   |         |    |    |    |
 |    |    |    |         |    |    |    |
G|--->|--->|===>|         |    |    |    |
 |    |   7|8   |========>|    |    |    |
 |    |    |    |         |===>|--->|--->|G
 |    |    |    |         |   6|    |    |

       Figure 2 MPA/TCP immediate startup with Early FPDU sent by
                               responder


Culley                   Expires: March 2004                [Page 11]


INTERNET-DRAFT           MPA Framing for TCP          23 January 2004




Initiator
     ULP A     connect
     ULP B     TCP connection established
     ULP C     MPA initiate negotiation, send Request frame
     ULP D     receive MPA Reply + private data
     ULP E     connect DDP (post untagged buffers)
     ULP F     receive DDP message
     ULP G     send DDP message

Responder
     ULP A     listen
     ULP B     accept (connection established)
     ULP C     MPA listen
     ULP D     receive MPA Request Frame + private data
     ULP E     Accept: Send Response frame + private data,
               connect DDP (post untagged buffers)
     ULP H     send DDP message
     ULP G     receive DDP message


Initiator Using the Legacy stacks
     MPA 1   ULP requests MPA to 'initiate' mode frame exchange
     MPA 2   MPA sends "MPA Request Frame" + private data
     MPA 3   MPA received "MPA Reply Frame" from responder
     MPA 4   MPA sends private data to ULP
     TCP 5   TCP receives the FPDU but has nowhere to deliver it
          Using the DDP/MPA stack
     MPA 6   MPA receives DDP message from TCP as DDP is bound
             and passes it up to DDP (and ultimately the app)
     MPA 7   ULP/DDP requests MPA to send DDP message
     MPA 8   MPA sends FPDU to responder

Responder Using the Legacy stacks
     MPA 1   ULP enables MPA to Listen for MPA Request Frame
     MPA 2   MPA receives "MPA Request Frame" from initiator
     MPA 3   MPA sends private data to ULP
     MPA 4   MPA receives private data from ULP
          Using the DDP/MPA stack
     MPA 5   MPA sends "MPA Reply Frame" to initiator
     MPA 6   ULP/DDP requests MPA to send DDP message
     MPA 7   MPA sends FPDU to initiator
     MPA 8   MPA receives FPDU from initiator





Culley                   Expires: March 2004                [Page 12]


INTERNET-DRAFT           MPA Framing for TCP          23 January 2004



   Note that between the initiator ULP 'D' and 'E', the incoming
   messages can arrive from TCP in lots of ways.  The example shows
   them arriving separately with no DDP buffer present to place the
   FPDU, because DDP is not yet enabled.  However, several other
   situations could occur, such as;

   *  The two messages can arrive in the same TCP segment.

   *  The two messages can arrive out of order.

   *  The second message could be incomplete.

   *  A message or message fragment can arrive during the stack
      conversion.

   All of these situations require additional design care in dealing
   with the conversion between stacks.

   None of these situations are present if the rules in the current
   draft are adopted.

        Note: It is still possible for an attacker or improperly
        implemented sender to violate the draft rules and send data
        when it is not expected (during the stack conversion).  In
        this case, a receiver implementation SHOULD ignore any
        unexpected data and terminate the connection.  If any such
        data arrives such that it is lost but not easily detected the
        receiver MAY continue to maintain the connection.  This could
        happen if the data arrived in the legacy stack, but was not
        transferred to the MPA/DDP stack at binding time, and normal
        FPDUs or no data at all arrived for the MPA/DDP stack.

        The above note will be added to the Security section of MPA
        if the original draft mechanisms are adopted.



   If we were to further consider using the first interpretation of
   Caitlin's rule (can send FPDUs as soon as we send a start frame)
   we end up with the problems identified at the beginning of the
   draft, as well as the dual stack transition problems just
   described.






Culley                   Expires: March 2004                [Page 13]


INTERNET-DRAFT           MPA Framing for TCP          23 January 2004



2.3  Why should we consider changing the draft?

   Caitlin's suggestions seem to make some sense at first glance, in
   that they can potentially reduce the time for a DDP/MPA connection
   to get into operation.

   Let's look at what kind of applications can actually take
   advantage of the reduced setup time.

   We believe that most ULPs that connect DDP immediately after TCP
   will be using a client/server design.  In this design, the Request
   Frame would start from the client and would carry things like the
   client credentials, Send and RDMA Read Buffer credits and other
   ULP required data as the "private data".  The server would respond
   with its own credits and authorization or denial (and closing of
   the connection) in the Private Data of the Response Frame.  The
   next transaction would be the client beginning the real working
   request in DDP mode.  This fits well with the current draft
   proposal.

   ULPs that might not be as good a fit would be those that wanted
   the server to transmit first after the client started the MPA
   connection; perhaps the initial client request was included in the
   Private Data of the Request Frame.  A ULP that might have been
   designed this way will have to insert another message on the wire
   incurring another round trip of delay before getting the first
   response.  Of course this means that the encoding of the first
   request is different than other requests as it is sent in the
   private data, instead of a DDP message, potentially leading to
   greater ULP protocol complexity.

   Another ULP that would not fit as well would be where the server
   started the MPA connection first; in this case the client might
   want to get to work as soon as it had sent the Response Frame.  We
   don't think this will be common since the server would not have an
   opportunity to authorize the client until after it had committed
   more significant server side resources to the session (goes
   against current DOS avoidance practice).

   The ability to reduce the startup time in the latter two cases
   does require dual stack designs to have the ability to pass
   undelivered data between stacks, some of which is potentially out
   of order or incomplete.  The Stack switch would also have to
   handle potential asynchronous IP packet or TCP segment arrival.
   Since this adds considerable complexity and provides opportunity
   for errors in implementations, the author group felt that the
   gains were not worth the costs.

Culley                   Expires: March 2004                [Page 14]


INTERNET-DRAFT           MPA Framing for TCP          23 January 2004



3  Active/Active Connections

   A second issue brought up for discussion was the possibility for a
   so called "Active-Active" MPA connection startup.

        "All that is required is to define a third value for "instant
        start", which each side sends *and* expects to receive."

   This suggestion has some of the same problems as the prior issue
   and also has the problem that the startup frames and "private
   data" is no longer a request/response type format, but is instead
   blindly sent by both sides prior to the MPA/DDP binding.  Without
   more protocol work to deal with potential rejected configurations,
   there is no way to respond to a bad connection configuration
   except to tear it down.

   An attempt to add "rejected only" responses would complicate the
   protocol and the MPA/DDP engine design, in that a receiver would
   not be sure if the next data was an FPDU or a response frame.

   For the following example, the TCP session is already connected,
   and the MPA connection type is Caitlin's proposed "Active/Active".
   Notice that there is no opportunity for a "private data" response.
   Again we are assuming the second interpretation of Caitlin's
   startup sequence rule (It is ok to send as long as the start frame
   has been sent and received).






















Culley                   Expires: March 2004                [Page 15]


INTERNET-DRAFT           MPA Framing for TCP          23 January 2004



+----------------+                 +----------------+
| MPA INITIATOR  |                 | MPA RESPONDER  |
+----------------+                 +----------------+
ULP  DDP  MPA  TCP                 TCP  MPA  DDP  ULP
 |    |    |    |                   |    |    |    |
C|-------->|--->| MPA REQ   MPA REQ |<---|<--------|C
 |    |   1|2   |-----   -----------|   2|1   |    |
 |    |    |    |     \ /           |    |    |    |
 |    |    |    |      X            |    |    |    |
 |    |    |    |     / \           |    |    |    |
 |    |    |    |<----   \          |    |    |    |
D|<--------|<---|         \         |    |    |    |
 |    |   4|3   |         |         |    |    |    |
 |    |    |    |         |         |    |    |    |
E|--->|    |    |         |         |    |    |    |
 |    |    |    |         |         |    |    |    |
F|--->|--->|===>|          \        |    |    |    |
 |    |   5|6   |===========+======>|(Z) |    |    |
 |    |    |    |            \
 |    |    |    |             ----->|    |    |    |
 |    |    |    |                   |--->|-------->|D
 |    |    |    |                   |   3|4   |    |
 |    |    |    |                   |    |    |    |
 |    |    |    |                   |    |    |<---|E
 |    |    |    |                   |(*) |    |    |
 |    |    |    |                   |===>|--->|--->|F
 |    |    |    |                   |   5|    |    |
 |    |    |    |                   |    |    |    |
 |    |    |    |                   |<===|<---|<---|G
 |    |    |    |<==================|   7|6   |    |
G|<---|<---|<===|                   |    |    |    |
 |    |    |7   |                   |    |    |    |
 |    |    |    |                   |    |    |    |
Note *: The TCP stack must hold the received out of order FPDU and
transfer it to the DDP stack only after the DDP stack is enabled at
point (E) on the responder.

           Figure 3 MPA/TCP immediate startup "Active/Active"

Initiator
     ULP C     MPA initiate negotiation, send Request frame
     ULP D     receive MPA Request + private data
     ULP E     connect DDP (post untagged buffers)
     ULP F     send DDP message
     ULP G     receive DDP message



Culley                   Expires: March 2004                [Page 16]


INTERNET-DRAFT           MPA Framing for TCP          23 January 2004



Responder
     ULP C     MPA initiate negotiation, send Request frame
     ULP D     receive MPA Request Frame + private data
     ULP E     connect DDP (post untagged buffers)
     ULP F     receive DDP message
     ULP G     send DDP message


Initiator Using the Legacy stacks
     MPA 1   ULP requests MPA to 'initiate' mode frame exchange
     MPA 2   MPA sends "MPA Request Frame" + private data
     MPA 3   MPA received "MPA Request Frame" from responder
     MPA 4   MPA sends private data to ULP
          Using the DDP/MPA stack
     MPA 5   ULP/DDP requests MPA to send DDP message
     MPA 6   MPA sends FPDU to responder
     MPA 7   MAP receives FPDU from responder, sends to DDP

Responder Using the Legacy stacks
     MPA 1   ULP requests MPA to 'initiate' mode frame exchange
     MPA 2   MPA sends "MPA Request Frame" + private data
     MPA 3   MPA received "MPA Request Frame" from initiator
     MPA 4   MPA sends private data to ULP
          Using the DDP/MPA stack
     MPA 5   MPA receives FPDU from initiator, sends to DDP
     MPA 6   ULP/DDP requests MPA to send DDP message
     MPA 7   MPA sends FPDU to initiator


   At the stack transition (point E at the responder), the
   responder's TCP stack is holding FPDU data (in this example). So
   it is necessary to pass that data (in whatever fragmented
   condition) to the DDP/MPA stack during the transition.

3.1  Why should we consider changing the draft?

   Even for TCP, active/active startup is seldom used by ULPs.  While
   it is always possible to imagine an application that can use this
   type of startup, it is also usually possible to design around that
   need.

   The author's believe that adding the protocol and implementation
   support for "active/active" startup, while intellectually
   stimulating, is not worth the time, cost, and effort for the
   potential payback.



Culley                   Expires: March 2004                [Page 17]


INTERNET-DRAFT           MPA Framing for TCP          23 January 2004



4  Adding a "rejected connection" bit

   Arkady Kanevsky submitted a comment to the reflector, quoted
   below:

        The MPA draft now defines very nicely an Initiator/Responder
        sequence. But it looks deficient for a broad set of ULPs.
        Most of the client-server ULP uses two type of Responses:
        "accept" and "reject". The MPA draft does not provide a way
        to differentiate between these 2 responses.

        Here is one example of an application use. One of the common
        way the private data is used is to negotiate RDMA Read
        credits for a connection to be used by RDDP. An Initiator
        passes in private data a requested RDMA credits. Responder
        either accepts a connection or rejects a request specifying
        how many RDMA Read credits it can support.

        Currently the Responder has 2 legal choices.
        One it can terminate a connection.
        This does not convey any information to the Initiator.

        The second choice is to generate Responder Frame even though
        RDMA Read credits requested can not be satisfied.
        This will establish connection but it can not be used as
        intended by the Initiator. Initiator can either tear down the
        established stream mode connection or use connection with
        Responder RDMA credits supported.

        This is de-facto ULP protocol since Responder Frame will
        include the RDMA Read credits it can support. On top of that
        both sides have to agree on the action they will take.

        While ULP can use a protocol over private data to
        differentiate between accept and reject, given a commonality
        of this semantic for ULPs we can use one bit from the
        Reserved area of MPA frame for the Responder frame to
        differentiate between accept and reject responses.

        Proposal:

        for Responder MPA frame the 1st Reserved bit to be used as
        accept/reject bit. 0 for accept, 1 for reject.

        The bit will be called Type (T) in figure 7 {of the MPA
        draft}.


Culley                   Expires: March 2004                [Page 18]


INTERNET-DRAFT           MPA Framing for TCP          23 January 2004



   Arkady is mostly correct in his analysis, but he may have missed
   one point; that is that the initiator has the opportunity and
   right to examine the private data in the MPA Response Frame before
   binding DDP to MPA.  It may either complete the connection setup
   by binding DDP (for an "accepted" connection), or tear the
   connection down (for a "rejected" connection).  It is also legal
   for the responder to tear the connection down immediately after
   sending a "rejected" MPA Response Frame.  In either case, no DDP
   connection is actually established until both sides have seen the
   private data in the MPA Response Frame.  In both cases the
   initiator can get a "reason" for a rejected connection.  The
   choice is up to the ULP and can be encoded into the response
   private data just as easily as in a specific bit.

        Note: it is also legal for the responder to tear down the TCP
        connection before sending a response frame, although this
        offers no opportunity to send a "rejected reason".  Again the
        choice is up to the ULP.

   Putting the accept/reject bit in a standard place would be useful
   if MPA or another standard layer needed to examine that bit for
   its own uses.  As currently defined, however, MPA must send
   private data to the ULP for examination, and depends on the ULP
   for the next step (continue or teardown).  So MPA could not do
   anything with that bit other than pass it on to the ULP.

   The author believes that defining an "accept/reject" bit is not
   necessary, that the response private data allows all the room for
   such information necessary (including more detailed reasons if the
   ULP needs them).  Adding such a bit would also require adding
   another input and output command parameter to the MPA ULP
   interface which may be insufficient for many applications anyway.
















Culley                   Expires: March 2004                [Page 19]


INTERNET-DRAFT           MPA Framing for TCP          23 January 2004



5  References

5.1  Normative References

   [RFC2026] Bradner, S., "The Internet Standards Process -- Revision
       3", BCP 9, RFC 2026, October 1996.

   [RFC793] Postel, J., "Transmission Control Protocol - DARPA
       Internet Program Protocol Specification", RFC 793, September
       1981.

5.2  Informative References

   [DDP] H. Shah et al., "Direct Data Placement over Reliable
       Transports", draft-ietf-rddp-ddp-01.txt (Work in progress),
       November 2003

   [RDMA] R. Recio et al., "RDMA Protocol Specification",
       draft-ietf-rddp-rdmap-01.txt, (Work in progress), November
       2003

   [MPA] Culley, P., "Marker PDU Aligned Framing for TCP
       Specification" draft-ietf-rddp-mpa-00.txt, October 2003.

   [Verbs] J. Hilland et al., "RDMA Protocol Verbs Specification"
       draft-hilland-rddp-verbs-v1.0.pdf, April 2003. (see
       http://www.rdmaconsortium.org/)

6  Author's Addresses

   Paul R. Culley
       Hewlett-Packard Company
       20555 SH 249
       Houston, Tx. USA 77070-2698
       Phone:  281-514-5543
       Email:  paul.culley@hp.com












Culley                   Expires: March 2004                [Page 20]


INTERNET-DRAFT           MPA Framing for TCP          23 January 2004



7  Full Copyright Statement


Copyright (C) The Internet Society (2000).  All Rights Reserved.

This document and translations of it may be copied and furnished to
others, and derivative works that comment on or otherwise explain it
or assist in its implementation may be prepared, copied, published
and distributed, in whole or in part, without restriction of any
kind, provided that the above copyright notice and this paragraph
are included on all such copies and derivative works.  However,
this document itself may not be modified in any way, such as by
removing the copyright notice or references to the Internet Society
or other Internet organizations, except as needed for the purpose
of developing Internet standards in which case the procedures
for copyrights defined in the Internet Standards process must be
followed, or as required to translate it into languages other than
English.

The limited permissions granted above are perpetual and will not be
revoked by the Internet Society or its successors or assigns.

This document and the information contained herein is provided on an
"AS IS" basis and THE INTERNET SOCIETY AND THE INTERNET ENGINEERING
TASK FORCE DISCLAIMS ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING
BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION
HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF
MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE.




















Culley                   Expires: March 2004                [Page 21]


Html markup produced by rfcmarkup 1.129d, available from https://tools.ietf.org/tools/rfcmarkup/