| draft-ietf-nfsv4-nfs-ulb-v2-00.txt | draft-ietf-nfsv4-nfs-ulb-v2-01.txt | |||
|---|---|---|---|---|
| Network File System Version 4 C. Lever | Network File System Version 4 C. Lever | |||
| Internet-Draft Oracle | Internet-Draft Oracle | |||
| Intended status: Standards Track November 17, 2019 | Intended status: Standards Track February 3, 2020 | |||
| Expires: May 20, 2020 | Expires: August 6, 2020 | |||
| Network File System (NFS) Upper Layer Binding To RPC-Over-RDMA Version 2 | Network File System (NFS) Upper-Layer Binding To RPC-Over-RDMA Version 2 | |||
| draft-ietf-nfsv4-nfs-ulb-v2-00 | draft-ietf-nfsv4-nfs-ulb-v2-01 | |||
| Abstract | Abstract | |||
| This document specifies Upper Layer Bindings of Network File System | This document specifies Upper-Layer Bindings of Network File System | |||
| (NFS) protocol versions to RPC-over-RDMA version 2. | (NFS) protocol versions to RPC-over-RDMA version 2. | |||
| Status of This Memo | Status of This Memo | |||
| This Internet-Draft is submitted in full conformance with the | This Internet-Draft is submitted in full conformance with the | |||
| provisions of BCP 78 and BCP 79. | provisions of BCP 78 and BCP 79. | |||
| Internet-Drafts are working documents of the Internet Engineering | Internet-Drafts are working documents of the Internet Engineering | |||
| Task Force (IETF). Note that other groups may also distribute | Task Force (IETF). Note that other groups may also distribute | |||
| working documents as Internet-Drafts. The list of current Internet- | working documents as Internet-Drafts. The list of current Internet- | |||
| Drafts is at https://datatracker.ietf.org/drafts/current/. | Drafts is at https://datatracker.ietf.org/drafts/current/. | |||
| Internet-Drafts are draft documents valid for a maximum of six months | Internet-Drafts are draft documents valid for a maximum of six months | |||
| and may be updated, replaced, or obsoleted by other documents at any | and may be updated, replaced, or obsoleted by other documents at any | |||
| time. It is inappropriate to use Internet-Drafts as reference | time. It is inappropriate to use Internet-Drafts as reference | |||
| material or to cite them other than as "work in progress." | material or to cite them other than as "work in progress." | |||
| This Internet-Draft will expire on May 20, 2020. | This Internet-Draft will expire on August 6, 2020. | |||
| Copyright Notice | Copyright Notice | |||
| Copyright (c) 2019 IETF Trust and the persons identified as the | Copyright (c) 2020 IETF Trust and the persons identified as the | |||
| document authors. All rights reserved. | document authors. All rights reserved. | |||
| This document is subject to BCP 78 and the IETF Trust's Legal | This document is subject to BCP 78 and the IETF Trust's Legal | |||
| Provisions Relating to IETF Documents | Provisions Relating to IETF Documents | |||
| (https://trustee.ietf.org/license-info) in effect on the date of | (https://trustee.ietf.org/license-info) in effect on the date of | |||
| publication of this document. Please review these documents | publication of this document. Please review these documents | |||
| carefully, as they describe your rights and restrictions with respect | carefully, as they describe your rights and restrictions with respect | |||
| to this document. Code Components extracted from this document must | to this document. Code Components extracted from this document must | |||
| include Simplified BSD License text as described in Section 4.e of | include Simplified BSD License text as described in Section 4.e of | |||
| the Trust Legal Provisions and are provided without warranty as | the Trust Legal Provisions and are provided without warranty as | |||
| skipping to change at page 2, line 19 ¶ | skipping to change at page 2, line 19 ¶ | |||
| outside the IETF Standards Process, and derivative works of it may | outside the IETF Standards Process, and derivative works of it may | |||
| not be created outside the IETF Standards Process, except to format | not be created outside the IETF Standards Process, except to format | |||
| it for publication as an RFC or to translate it into languages other | it for publication as an RFC or to translate it into languages other | |||
| than English. | than English. | |||
| Table of Contents | Table of Contents | |||
| 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 2 | 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 2 | |||
| 2. Requirements Language . . . . . . . . . . . . . . . . . . . . 3 | 2. Requirements Language . . . . . . . . . . . . . . . . . . . . 3 | |||
| 3. Reply Size Estimation . . . . . . . . . . . . . . . . . . . . 3 | 3. Reply Size Estimation . . . . . . . . . . . . . . . . . . . . 3 | |||
| 4. Upper Layer Binding for NFS Versions 2 and 3 . . . . . . . . 3 | 4. Upper-Layer Binding for NFS Versions 2 and 3 . . . . . . . . 3 | |||
| 4.1. Reply Size Estimation . . . . . . . . . . . . . . . . . . 4 | 4.1. Reply Size Estimation . . . . . . . . . . . . . . . . . . 4 | |||
| 4.2. RPC Binding Considerations . . . . . . . . . . . . . . . 4 | 4.2. RPC Binding Considerations . . . . . . . . . . . . . . . 4 | |||
| 5. Upper Layer Bindings for NFS Version 2 and 3 Auxiliary | 4.3. Transport Considerations . . . . . . . . . . . . . . . . 4 | |||
| Protocols . . . . . . . . . . . . . . . . . . . . . . . . . . 4 | 5. Upper-Layer Bindings for NFS Version 2 and 3 Auxiliary | |||
| 5.1. MOUNT, NLM, and NSM Protocols . . . . . . . . . . . . . . 5 | Protocols . . . . . . . . . . . . . . . . . . . . . . . . . . 5 | |||
| 5.2. NFSACL Protocol . . . . . . . . . . . . . . . . . . . . . 5 | 5.1. MOUNT, NLM, and NSM Protocols . . . . . . . . . . . . . . 6 | |||
| 6. Upper Layer Binding For NFS Version 4 . . . . . . . . . . . . 5 | 5.2. NFSACL Protocol . . . . . . . . . . . . . . . . . . . . . 6 | |||
| 6.1. DDP-Eligibility . . . . . . . . . . . . . . . . . . . . . 5 | 6. Upper-Layer Binding For NFS Version 4 . . . . . . . . . . . . 6 | |||
| 6.2. Reply Size Estimation . . . . . . . . . . . . . . . . . . 6 | 6.1. DDP-Eligibility . . . . . . . . . . . . . . . . . . . . . 7 | |||
| 6.3. RPC Binding Considerations . . . . . . . . . . . . . . . 7 | 6.2. Reply Size Estimation . . . . . . . . . . . . . . . . . . 7 | |||
| 6.4. NFS COMPOUND Requests . . . . . . . . . . . . . . . . . . 7 | 6.3. RPC Binding Considerations . . . . . . . . . . . . . . . 8 | |||
| 6.5. NFS Callback Requests . . . . . . . . . . . . . . . . . . 9 | 6.4. NFS COMPOUND Requests . . . . . . . . . . . . . . . . . . 8 | |||
| 6.6. Session-Related Considerations . . . . . . . . . . . . . 10 | 6.5. NFS Callback Requests . . . . . . . . . . . . . . . . . . 11 | |||
| 6.7. Transport Considerations . . . . . . . . . . . . . . . . 11 | 6.6. Session-Related Considerations . . . . . . . . . . . . . 12 | |||
| 7. Extending NFS Upper Layer Bindings . . . . . . . . . . . . . 13 | 6.7. Transport Considerations . . . . . . . . . . . . . . . . 12 | |||
| 8. Security Considerations . . . . . . . . . . . . . . . . . . . 13 | 7. Extending NFS Upper-Layer Bindings . . . . . . . . . . . . . 14 | |||
| 9. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 13 | 8. Security Considerations . . . . . . . . . . . . . . . . . . . 14 | |||
| 10. References . . . . . . . . . . . . . . . . . . . . . . . . . 13 | 9. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 14 | |||
| 10.1. Normative References . . . . . . . . . . . . . . . . . . 14 | 10. References . . . . . . . . . . . . . . . . . . . . . . . . . 14 | |||
| 10.2. Informative References . . . . . . . . . . . . . . . . . 15 | 10.1. Normative References . . . . . . . . . . . . . . . . . . 15 | |||
| Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . . . 15 | 10.2. Informative References . . . . . . . . . . . . . . . . . 16 | |||
| Author's Address . . . . . . . . . . . . . . . . . . . . . . . . 15 | Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . . . 16 | |||
| Author's Address . . . . . . . . . . . . . . . . . . . . . . . . 16 | ||||
| 1. Introduction | 1. Introduction | |||
| The RPC-over-RDMA version 2 transport may employ direct data | The RPC-over-RDMA version 2 transport may employ direct data | |||
| placement to convey data payloads associated with RPC transactions | placement to convey data payloads associated with RPC transactions | |||
| [I-D.cel-nfsv4-rpcrdma-version-two]. To enable successful | [RPCRDMA2]. RPC client and server implementations using RPC-over- | |||
| interoperation, RPC client and server implementations using RPC-over- | ||||
| RDMA version 2 must agree which XDR data items and RPC procedures are | RDMA version 2 must agree which XDR data items and RPC procedures are | |||
| eligible to use direct data placement (DDP). | eligible to use direct data placement (DDP) to enable successful | |||
| interoperation. | ||||
| An Upper Layer Binding specifies this agreement for one or more | An Upper-Layer Binding specifies this agreement for one or more | |||
| versions of one RPC program. Other operational details, such as RPC | versions of one RPC program. Other operational details, such as RPC | |||
| binding assignments, pairing Write chunks with result data items, and | binding assignments, pairing Write chunks with result data items, and | |||
| reply size estimation, are also specified by this Binding. | reply size estimation, are also specified by this Binding. | |||
| This document contains material required of Upper Layer Bindings, as | This document contains material required of Upper-Layer Bindings, as | |||
| specified in [I-D.cel-nfsv4-rpcrdma-version-two], for the following | specified in [RPCRDMA2], for the following NFS protocol versions: | |||
| NFS protocol versions: | ||||
| o NFS version 2 [RFC1094] | o NFS version 2 [RFC1094] | |||
| o NFS version 3 [RFC1813] | o NFS version 3 [RFC1813] | |||
| o NFS version 4.0 [RFC7530] | o NFS version 4.0 [RFC7530] | |||
| o NFS version 4.1 [RFC5661] | o NFS version 4.1 [RFC5661] | |||
| o NFS version 4.2 [RFC7862] | o NFS version 4.2 [RFC7862] | |||
| Upper Layer Bindings are also provided for auxiliary protocols used | The current document also provides Upper-Layer Bindings for auxiliary | |||
| with NFS versions 2 and 3 (see Section 5). | protocols used with NFS versions 2 and 3 (see Section 5). | |||
| This document assumes the reader is already familiar with concepts | This document assumes the reader is already familiar with concepts | |||
| and terminology defined in [I-D.cel-nfsv4-rpcrdma-version-two] and | and terminology defined in [RPCRDMA2] and the documents it | |||
| the documents it references. | references. | |||
| 2. Requirements Language | 2. Requirements Language | |||
| The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", | The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", | |||
| "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and | "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and | |||
| "OPTIONAL" in this document are to be interpreted as described in BCP | "OPTIONAL" in this document are to be interpreted as described in BCP | |||
| 14 [RFC2119] [RFC8174] when, and only when, they appear in all | 14 [RFC2119] [RFC8174] when, and only when, they appear in all | |||
| capitals, as shown here. | capitals, as shown here. | |||
| 3. Reply Size Estimation | 3. Reply Size Estimation | |||
| During the construction of each RPC Call message, a Requester is | During the construction of each RPC Call message, a Requester is | |||
| responsible for allocating appropriate resources for receiving the | responsible for allocating appropriate resources for receiving the | |||
| corresponding Reply message. If the Requester expects the RPC Reply | corresponding Reply message. If the Requester expects that the RPC | |||
| message will be larger than its inline threshold, it MAY provide | Reply message could be larger than its inline threshold, it MAY | |||
| Write and/or Reply chunks wherein the Responder can place results and | provide Write chunks wherein the Responder can place results and | |||
| the reply's Payload stream. | Reply chunks wherein the Responder can place the reply's Payload | |||
| stream. | ||||
| 4. Upper Layer Binding for NFS Versions 2 and 3 | 4. Upper-Layer Binding for NFS Versions 2 and 3 | |||
| The Upper Layer Binding specification in this section applies to NFS | The Upper-Layer Binding specification in this section applies to NFS | |||
| version 2 [RFC1094] and NFS version 3 [RFC1813]. For brevity, in | version 2 [RFC1094] and NFS version 3 [RFC1813]. For brevity, in | |||
| this document a "Legacy NFS client" refers to an NFS client using | this document, a "Legacy NFS client" refers to an NFS client using | |||
| version 2 or version 3 of the NFS RPC program (100003) to communicate | version 2 or version 3 of the NFS RPC program (100003) to communicate | |||
| with an NFS server. Likewise, a "Legacy NFS server" is an NFS server | with an NFS server. Likewise, a "Legacy NFS server" is an NFS server | |||
| communicating with clients using NFS version 2 or NFS version 3. | communicating with clients using NFS version 2 or NFS version 3. | |||
| The following XDR data items in NFS versions 2 and 3 are DDP- | The following XDR data items in NFS versions 2 and 3 are DDP- | |||
| eligible: | eligible: | |||
| o The opaque file data argument in the NFS WRITE procedure | o The opaque file data argument in the NFS WRITE procedure | |||
| o The pathname argument in the NFS SYMLINK procedure | o The pathname argument in the NFS SYMLINK procedure | |||
| o The opaque file data result in the NFS READ procedure | o The opaque file data result in the NFS READ procedure | |||
| o The pathname result in the NFS READLINK procedure | o The pathname result in the NFS READLINK procedure | |||
| All other argument or result data items in NFS versions 2 and 3 are | All other argument or result data items in NFS versions 2 and 3 are | |||
| not DDP-eligible. | not DDP-eligible. | |||
| A transport error does not give an indication of whether the server | A transport error does not indicate whether the server has processed | |||
| has processed the arguments of the RPC Call, or whether the server | the arguments of the RPC Call, or whether the server has accessed or | |||
| has accessed or modified client memory associated with that RPC. | modified client memory associated with that RPC. | |||
| 4.1. Reply Size Estimation | 4.1. Reply Size Estimation | |||
| A Legacy NFS client determines the maximum reply size for each | A Legacy NFS client determines the maximum reply size for each | |||
| operation using the criteria outlined in Section 3. | operation using the criteria outlined in Section 3. | |||
| 4.2. RPC Binding Considerations | 4.2. RPC Binding Considerations | |||
| Legacy NFS servers traditionally listen for clients on UDP and TCP | Legacy NFS servers traditionally listen for clients on UDP and TCP | |||
| port 2049. Additionally, they register these ports with a local | port 2049. Additionally, they register these ports with a local | |||
| portmapper service [RFC1833]. | portmapper service [RFC1833]. | |||
| A Legacy NFS server supporting RPC-over-RDMA version 2 on such a | A Legacy NFS server supporting RPC-over-RDMA version 2 on such a | |||
| network and registering itself with the RPC portmapper MAY choose an | network and registering itself with the RPC portmapper MAY choose an | |||
| arbitrary port, or MAY use the alternative well-known port number for | arbitrary port, or MAY use the alternative well-known port number for | |||
| its RPC-over-RDMA service (see Section 9). The chosen port MAY be | its RPC-over-RDMA service (see Section 9). The chosen port MAY be | |||
| registered with the RPC portmapper using the netids assigned in | registered with the RPC portmapper using the netids assigned in | |||
| [I-D.cel-nfsv4-rpcrdma-version-two]. | [RPCRDMA2]. | |||
| 5. Upper Layer Bindings for NFS Version 2 and 3 Auxiliary Protocols | 4.3. Transport Considerations | |||
| NFS versions 2 and 3 are typically deployed with several other | Legacy NFS client implementations often rely on a transport-layer | |||
| protocols, sometimes referred to as "NFS auxiliary protocols." These | keep-alive mechanism to detect when a legacy server has become | |||
| are distinct RPC programs that define procedures which are not part | unresponsive. When an NFS server is no longer responsive, client- | |||
| of the NFS RPC program (100003). The Upper Layer Bindings in this | side keep-alive terminates the connection, which in turn triggers | |||
| section apply to: | reconnection and RPC retransmission. | |||
| 4.3.1. Keep-Alive | ||||
| Some RDMA transports (such as the Reliable Connected QP type on | ||||
| InfiniBand) have no keep-alive mechanism. Without a disconnect or | ||||
| new RPC traffic, such connections can remain alive long after an NFS | ||||
| server has become unresponsive or unreachable. Once an NFS client | ||||
| has consumed all available RPC-over-RDMA version 2 credits on that | ||||
| transport connection, it awaits a reply indefinitely before sending | ||||
| another RPC request. | ||||
| Legacy NFS clients SHOULD reserve one RPC-over-RDMA version 2 credit | ||||
| to use for periodic server or connection health assessment. Either | ||||
| peer can use this credit to drive an RPC request on an otherwise idle | ||||
| connection, triggering either an affirmative server response or a | ||||
| connection termination. | ||||
| 4.3.2. Replay Detection | ||||
| Legacy NFS servers can employ request replay detection to reduce the | ||||
| risk of data corruption that could result when a client retransmits a | ||||
| request. A legacy NFS server can choose to send a cached response | ||||
| when a replay is detected, rather than executing the request again. | ||||
| Replay detection is not perfect, but it is usually adequate. | ||||
| For legacy NFS servers, replay detection commonly utilizes heuristic | ||||
| indicators such as the IP address of the NFS client, the source port | ||||
| of the connection, the transaction ID of the request, and the | ||||
| contents of the request's RPC and upper-layer protocol headers. In | ||||
| short, replay detection is typically based on a connection tuple and | ||||
| the request's XID. A legacy NFS client is careful to re-use the same | ||||
| source port, if practical, when reconnecting so that legacy NFS | ||||
| servers are better able to detect retransmissions. | ||||
| However, a legacy NFS client operating over an RDMA transport has no | ||||
| control over connection source ports. It is almost certain that an | ||||
| RPC request that is retransmitted on a new connection can never be | ||||
| detected as a replay if the legacy NFS server includes the connection | ||||
| source port in its replay detection heuristics. | ||||
| Therefore a legacy NFS server using an RDMA transport should never | ||||
| use a legacy NFS client connection's source port as part of RPC | ||||
| request replay detection. | ||||
| 5. Upper-Layer Bindings for NFS Version 2 and 3 Auxiliary Protocols | ||||
| Storage administrators typically deploy NFS versions 2 and 3 with | ||||
| several other protocols, sometimes referred to as "NFS auxiliary | ||||
| protocols." These are distinct RPC programs that define procedures | ||||
| that are not part of the NFS RPC program (100003). The Upper-Layer | ||||
| Bindings in this section apply to: | ||||
| o Versions 2 and 3 of the MOUNT RPC program (100005) [RFC1813] | o Versions 2 and 3 of the MOUNT RPC program (100005) [RFC1813] | |||
| o Versions 1, 3, and 4 of the NLM RPC program (100021) [RFC1813] | o Versions 1, 3, and 4 of the NLM RPC program (100021) [RFC1813] | |||
| o Version 1 of the NSM RPC program (100024), described in Chapter 11 | o Version 1 of the NSM RPC program (100024), described in Chapter 11 | |||
| of [XNFS] | of [XNFS] | |||
| o Version 1 of the NFSACL RPC program (100227), which does not have | o Version 1 of the NFSACL RPC program (100227), which does not have | |||
| a public definition. NFSACL is treated in this document as a de | a public definition. NFSACL is treated in this document as a de | |||
| facto standard, as there are several interoperating | facto standard, as there are several interoperating | |||
| implementations. | implementations. | |||
| 5.1. MOUNT, NLM, and NSM Protocols | 5.1. MOUNT, NLM, and NSM Protocols | |||
| Historically, NFS/RDMA implementations have chosen to convey the | Historically, NFS/RDMA implementations have chosen to convey the | |||
| MOUNT, NLM, and NSM protocols via TCP. To enable interoperation of | MOUNT, NLM, and NSM protocols via TCP. A legacy NFS server | |||
| these protocols when NFS/RDMA is in use, a legacy NFS server MUST | implementation MUST provide support for these protocols via TCP to | |||
| provide support for these protocols via TCP. | enable interoperation of these protocols when NFS/RDMA is in use. | |||
| 5.2. NFSACL Protocol | 5.2. NFSACL Protocol | |||
| Legacy clients and servers that support the NFSACL RPC program | Often legacy clients and servers that support the NFSACL RPC program | |||
| typically convey NFSACL procedures on the same connection as the NFS | convey NFSACL procedures on the same connection as the NFS RPC | |||
| RPC program (100003). This obviates the need for separate rpcbind | program (100003). Utilizing the same connection obviates the need | |||
| queries to discover server support for this RPC program. | for separate rpcbind queries to discover server support for this RPC | |||
| program. | ||||
| ACLs are typically small, but even large ACLs must be encoded and | ACLs are typically small, but even large ACLs must be encoded and | |||
| decoded to some degree. Thus no data item in this Upper Layer | decoded to some degree. Thus no data item in this Upper-Layer | |||
| Protocol is DDP-eligible. | Protocol is DDP-eligible. | |||
| For procedures whose replies do not include an ACL object, the size | For procedures whose replies do not include an ACL object, the size | |||
| of a reply is determined directly from the NFSACL RPC program's XDR | of a reply is determined directly from the NFSACL RPC program's XDR | |||
| definition. Legacy client implementations should choose a maximum | definition. Legacy client implementations should choose a maximum | |||
| size for ACLs based on their own internal limits. | size for ACLs based on internal limits. | |||
| 6. Upper Layer Binding For NFS Version 4 | 6. Upper-Layer Binding For NFS Version 4 | |||
| The Upper Layer Binding specification in this section applies to | The Upper-Layer Binding specification in this section applies to | |||
| versions of the NFS RPC program defined in NFS version 4.0 [RFC7530] | versions of the NFS RPC program defined in NFS version 4.0 [RFC7530] | |||
| NFS version 4.1 [RFC5661] and NFS version 4.2 [RFC7862] | NFS version 4.1 [RFC5661] and NFS version 4.2 [RFC7862]. | |||
| 6.1. DDP-Eligibility | 6.1. DDP-Eligibility | |||
| Only the following XDR data items in the COMPOUND procedure of all | Only the following XDR data items in the COMPOUND procedure of all | |||
| NFS version 4 minor versions are DDP-eligible: | NFS version 4 minor versions are DDP-eligible: | |||
| o The opaque data field in the WRITE4args structure | o The opaque data field in the WRITE4args structure | |||
| o The linkdata field of the NF4LNK arm in the createtype4 union | o The linkdata field of the NF4LNK arm in the createtype4 union | |||
| o The opaque data field in the READ4resok structure | o The opaque data field in the READ4resok structure | |||
| o The linkdata field in the READLINK4resok structure | o The linkdata field in the READLINK4resok structure | |||
| 6.2. Reply Size Estimation | 6.2. Reply Size Estimation | |||
| Within NFS version 4, there are certain variable-length result data | Within NFS version 4, there are certain variable-length result data | |||
| items whose maximum size cannot be estimated by clients reliably | items whose maximum size cannot be estimated by clients reliably | |||
| skipping to change at page 6, line 24 ¶ | skipping to change at page 7, line 32 ¶ | |||
| because there is no protocol-specified size limit on these result | because there is no protocol-specified size limit on these result | |||
| arrays. These include: | arrays. These include: | |||
| o The attrlist4 field | o The attrlist4 field | |||
| o Fields containing ACLs such as fattr4_acl, fattr4_dacl, and | o Fields containing ACLs such as fattr4_acl, fattr4_dacl, and | |||
| fattr4_sacl | fattr4_sacl | |||
| o Fields in the fs_locations4 and fs_locations_info4 data structures | o Fields in the fs_locations4 and fs_locations_info4 data structures | |||
| o Fields opaque to the NFS version 4 protocol which pertain to pNFS | o Fields which pertain to pNFS layout metadata, such as loc_body, | |||
| layout metadata, such as loc_body, loh_body, da_addr_body, | loh_body, da_addr_body, lou_body, lrf_body, fattr_layout_types, | |||
| lou_body, lrf_body, fattr_layout_types, and fs_layout_types | and fs_layout_types | |||
| 6.2.1. Reply Size Estimation for Minor Version 0 | 6.2.1. Reply Size Estimation for Minor Version 0 | |||
| The NFS version 4.0 protocol itself does not impose any bound on the | The NFS version 4.0 protocol itself does not impose any bound on the | |||
| size of NFS calls or responses. | size of NFS calls or responses. | |||
| Some of the data items enumerated in Section 6.2 (in particular, the | Some of the data items enumerated in Section 6.2 (in particular, the | |||
| items related to ACLs and fs_locations) make it difficult to predict | items related to ACLs and fs_locations) make it difficult to predict | |||
| the maximum size of NFS version 4.0 replies that interrogate | the maximum size of NFS version 4.0 replies that interrogate | |||
| variable-length fattr4 attributes. Client implementations might rely | variable-length fattr4 attributes. Client implementations might rely | |||
| on their own internal architectural limits to constrain the reply | upon internal architectural limits to constrain the reply size, but | |||
| size, but such limits are not always guaranteed to be reliable. | such limits are not always guaranteed to be reliable. | |||
| When an especially large fattr4 result is expected, an NFS version | When an NFS version 4.0 client expects an especially sizeable fattr4 | |||
| 4.0 client can provide a Reply chunk to enable a large result to be | result, it can provide a Reply chunk to enable that server to return | |||
| returned via explicit RDMA. An NFS version 4.0 client can use short | that result via explicit RDMA. An NFS version 4.0 client can use | |||
| Reply chunk retry when an NFS COMPOUND containing a GETATTR operation | short Reply chunk retry when an NFS COMPOUND containing a GETATTR | |||
| encounters a transport error. | operation encounters a transport error. | |||
| 6.2.2. Reply Size Estimation for Minor Version 1 and Newer | 6.2.2. Reply Size Estimation for Minor Version 1 and Newer | |||
| In NFS version 4.1 and newer minor versions, the csa_fore_chan_attrs | In NFS version 4.1 and newer minor versions, the csa_fore_chan_attrs | |||
| argument of the CREATE_SESSION operation contains a | argument of the CREATE_SESSION operation contains a | |||
| ca_maxresponsesize field. The value in this field can be taken as | ca_maxresponsesize field. The value in this field can be taken as | |||
| the absolute maximum size of replies generated by an NFS version 4.1 | the absolute maximum size of replies generated by an NFS version 4.1 | |||
| server. | server. | |||
| This value can be used in cases where it is not possible to estimate | A client can use this value in cases where it is not possible to | |||
| a reply size upper bound precisely. In practice, objects such as | estimate a reply size upper bound precisely. In practice, objects | |||
| ACLs, named attributes, layout bodies, and security labels are much | such as ACLs, named attributes, layout bodies, and security labels | |||
| smaller than this maximum. | are much smaller than this maximum. | |||
| 6.3. RPC Binding Considerations | 6.3. RPC Binding Considerations | |||
| NFS version 4 servers are required to listen on TCP port 2049, and | NFS version 4 servers are required to listen on TCP port 2049, and | |||
| they are not required to register with an rpcbind service [RFC7530] | they are not required to register with a rpcbind service [RFC7530]. | |||
| Therefore, an NFS version 4 server supporting RPC-over-RDMA version 2 | Therefore, an NFS version 4 server supporting RPC-over-RDMA version 2 | |||
| MUST use the alternative well-known port number for its RPC-over-RDMA | MUST use the alternative well-known port number for its RPC-over-RDMA | |||
| service (see Section 9 Clients SHOULD connect to this well-known port | service (see Section 9 Clients SHOULD connect to this well-known port | |||
| without consulting the RPC portmapper (as for NFS version 4 on TCP | without consulting the RPC portmapper (as for NFS version 4 on TCP | |||
| transports). | transports). | |||
| 6.4. NFS COMPOUND Requests | 6.4. NFS COMPOUND Requests | |||
| 6.4.1. Multiple DDP-eligible Data Items | 6.4.1. Multiple DDP-eligible Data Items | |||
| An NFS version 4 COMPOUND procedure can contain more than one | An NFS version 4 COMPOUND procedure can contain more than one | |||
| operation that carries a DDP-eligible data item. An NFS version 4 | operation that carries a DDP-eligible data item. An NFS version 4 | |||
| client provides XDR Position values in each Read chunk to | client provides XDR Position values in each Read chunk to | |||
| disambiguate which chunk is associated with which argument data item. | disambiguate which chunk is associated with which argument data item. | |||
| However NFS version 4 server and client implementations must agree in | However, NFS version 4 server and client implementations must agree | |||
| advance on how to pair Write chunks with returned result data items. | in advance on how to pair Write chunks with returned result data | |||
| items. | ||||
| In the following list, a "READ operation" refers to any NFS version 4 | In the following lists, a "READ operation" refers to any NFS version | |||
| operation which has a DDP-eligible result data item. The mechanism | 4 operation that has a DDP-eligible result data item. An NFS version | |||
| specified in Section 4.3.2 of [I-D.cel-nfsv4-rpcrdma-version-two] is | 4 client applies the mechanism specified in Section 4.3.2 of | |||
| applied to this class of operations: | [RPCRDMA2] is applied to this class of operations as follows: | |||
| o If an NFS version 4 client wishes all DDP-eligible items in an NFS | o If an NFS version 4 client wishes all DDP-eligible items in an NFS | |||
| reply to be conveyed inline, it leaves the Write list empty. | reply to be conveyed inline, it leaves the Write list empty. | |||
| An NFS version 4 server applies that mechanism as follows: | ||||
| o The first chunk in the Write list MUST be used by the first READ | o The first chunk in the Write list MUST be used by the first READ | |||
| operation in an NFS version 4 COMPOUND procedure. The next Write | operation in an NFS version 4 COMPOUND procedure. The next Write | |||
| chunk is used by the next READ operation, and so on. | chunk is used by the next READ operation, and so on. | |||
| o If an NFS version 4 client has provided a matching non-empty Write | o If an NFS version 4 client has provided a matching non-empty Write | |||
| chunk, then the corresponding READ operation MUST return its DDP- | chunk, then the corresponding READ operation MUST return its DDP- | |||
| eligible data item using that chunk. | eligible data item using that chunk. | |||
| o If an NFS version 4 client has provided an empty matching Write | o If an NFS version 4 client has provided an empty matching Write | |||
| chunk, then the corresponding READ operation MUST return all of | chunk, then the corresponding READ operation MUST return all of | |||
| skipping to change at page 8, line 22 ¶ | skipping to change at page 9, line 30 ¶ | |||
| return an empty Write chunk in that Write list position. | return an empty Write chunk in that Write list position. | |||
| o If there are more READ operations than Write chunks, then | o If there are more READ operations than Write chunks, then | |||
| remaining NFS Read operations in an NFS version 4 COMPOUND that | remaining NFS Read operations in an NFS version 4 COMPOUND that | |||
| have no matching Write chunk MUST return their results inline. | have no matching Write chunk MUST return their results inline. | |||
| 6.4.2. Chunk List Complexity | 6.4.2. Chunk List Complexity | |||
| The RPC-over-RDMA version 2 protocol does not place any limit on the | The RPC-over-RDMA version 2 protocol does not place any limit on the | |||
| number of chunks or segments that may appear in Read or Write lists. | number of chunks or segments that may appear in Read or Write lists. | |||
| However, for various reasons NFS version 4 server implementations | However, for various reasons, NFS version 4 server implementations | |||
| often have practical limits on the number of chunks or segments they | often have practical limits on the number of chunks or segments they | |||
| are prepared to process in a single RPC transaction conveyed via RPC- | can process in a single RPC transaction conveyed via RPC-over-RDMA | |||
| over-RDMA version 2. | version 2. | |||
| These implementation limits are especially important when Kerberos | These implementation limits are especially important when Kerberos | |||
| integrity or privacy is in use [RFC7861]. GSS services increase the | integrity or privacy is in use [RFC7861]. GSS services increase the | |||
| size of credential material in RPC headers, potentially requiring the | size of credential material in RPC headers, potentially requiring the | |||
| use of Long messages. This can increase the complexity of chunk | use of a Long message, which increases the complexity of chunk lists | |||
| lists independent of the NFS version 4 COMPOUND being conveyed. | independent of the particular NFS version 4 COMPOUND being conveyed. | |||
| In the absence of explicit knowledge of the server's limits, NFS | In the absence of explicit knowledge of the server's limits, NFS | |||
| version 4 clients SHOULD follow the prescriptions listed below when | version 4 clients SHOULD follow the prescriptions listed below when | |||
| constructing RPC-over-RDMA version 2 messages. NFS version 4 servers | constructing RPC-over-RDMA version 2 messages. NFS version 4 servers | |||
| MUST accept and process such requests. | MUST accept and process all such requests. | |||
| o The Read list can contain either a Position-Zero Read chunk, one | o The Read list can contain either a Position-Zero Read chunk, one | |||
| Read chunk with a non-zero Position, or both. | Read chunk with a non-zero Position, or both. | |||
| o The Write list can contain no more than one Write chunk. | o The Write list can contain no more than one Write chunk. | |||
| o Any chunk can contain up to sixteen RDMA segments. | o Any chunk can contain up to sixteen RDMA segments. | |||
| NFS version 4 clients wishing to send more complex chunk lists can | NFS version 4 clients wishing to send more complex chunk lists can | |||
| provide configuration interfaces to bound the complexity of NFS | provide configuration interfaces to bound the complexity of NFS | |||
| version 4 COMPOUNDs, limit the number of elements in scatter-gather | version 4 COMPOUNDs, limit the number of elements in scatter-gather | |||
| operations, and avoid other sources of chunk overruns at the | operations, and avoid other sources of chunk overruns at the | |||
| receiving peer. | receiving peer. | |||
| An NFS version 4 server SHOULD return one of the following responses | If an NFS version 4 server receives an RPC request via RPC-over-RDMA | |||
| to a client that has sent an RPC transaction via RPC-over-RDMA | version 2 that it cannot process due to chunk list complexity limits, | |||
| version 2 which cannot be processed due to chunk list complexity | it SHOULD return one of the following responses to the client: | |||
| limits on the server: | ||||
| o A problem is detected by the transport layer while parsing the | o A problem is detected by the transport layer while parsing the | |||
| transport header in an RPC Call message. The server responds with | transport header in an RPC Call message. The server responds with | |||
| an RDMA2_ERROR message with the err field set to ERR_CHUNK. | an RDMA2_ERROR message with the err field set to ERR_CHUNK. | |||
| o A problem is detected during XDR decoding of the RPC Call message | o A problem is detected during XDR decoding of the RPC Call message | |||
| while the RPC layer reassembles the call's XDR stream. The server | while the RPC layer reassembles the call's XDR stream. The server | |||
| responds with an RPC reply with its "reply_stat" field set to | responds with an RPC reply with its "reply_stat" field set to | |||
| MSG_ACCEPTED and its "accept_stat" field set to GARBAGE_ARGS. | MSG_ACCEPTED and its "accept_stat" field set to GARBAGE_ARGS. | |||
| skipping to change at page 10, line 28 ¶ | skipping to change at page 11, line 34 ¶ | |||
| An NFS version 4.0 client advertises netids and ad hoc port addresses | An NFS version 4.0 client advertises netids and ad hoc port addresses | |||
| for contacting its NFS version 4.0 callback service using the | for contacting its NFS version 4.0 callback service using the | |||
| SETCLIENTID operation. | SETCLIENTID operation. | |||
| 6.5.2. NFS Version 4.1 Callback | 6.5.2. NFS Version 4.1 Callback | |||
| In NFS version 4.1 and newer minor versions, callback operations may | In NFS version 4.1 and newer minor versions, callback operations may | |||
| appear on the same connection as is used for NFS version 4 forward | appear on the same connection as is used for NFS version 4 forward | |||
| channel client requests. NFS version 4 clients and servers MUST use | channel client requests. NFS version 4 clients and servers MUST use | |||
| the approach described in [RFC8167] when backchannel operations are | the approach described in [RFC8167] to convey backchannel operations | |||
| conveyed on RPC-over-RDMA version 2 transports. | on an RPC-over-RDMA version 2 transport. | |||
| The csa_back_chan_attrs argument of the CREATE_SESSION operation | The csa_back_chan_attrs argument of the CREATE_SESSION operation | |||
| contains a ca_maxresponsesize field. The value in this field can be | contains a ca_maxresponsesize field. The value in this field is the | |||
| taken as the absolute maximum size of backchannel replies generated | absolute maximum size of backchannel replies generated by a replying | |||
| by a replying NFS version 4 client. | NFS version 4 client. | |||
| There are no DDP-eligible data items in callback procedures defined | There are no DDP-eligible data items in callback procedures defined | |||
| in NFS version 4.1 or NFS version 4.2. However, some callback | in NFS version 4.1 or NFS version 4.2. However, some callback | |||
| operations, such as messages that convey device ID information, can | operations, such as messages that convey device ID information, can | |||
| be large. Message Continuation or a Long message might be used in | be sizeable. A sender can use Message Continuation or a Long message | |||
| this situation. | in this situation. | |||
| When an NFS version 4.1 client can support Long Calls in its | When an NFS version 4.1 client can support Long Calls in its | |||
| backchannel, it reports a backchannel ca_maxrequestsize that is | backchannel, it reports a backchannel ca_maxrequestsize that is | |||
| larger than the connection's inline thresholds. Otherwise an NFS | larger than the connection's inline thresholds. Otherwise, an NFS | |||
| version 4 server MUST use only Short messages to convey backchannel | version 4 server MUST use only Short messages to convey backchannel | |||
| operations. | operations. | |||
| 6.6. Session-Related Considerations | 6.6. Session-Related Considerations | |||
| The presence of an NFS session (defined in [RFC5661] has no effect on | The presence of an NFS version 4 session (as defined in [RFC5661]) | |||
| the operation of RPC-over-RDMA version 2. None of the operations | does not effect the operation of RPC-over-RDMA version 2. None of | |||
| introduced to support NFS sessions (e.g. the SEQUENCE operation) | the operations introduced to support NFS sessions (e.g., the SEQUENCE | |||
| contain DDP-eligible data items. There is no need to match the | operation) contain DDP-eligible data items. There is no need to | |||
| number of session slots with the number of available RPC-over-RDMA | match the number of session slots with the number of available RPC- | |||
| version 2 credits. | over-RDMA version 2 credits. | |||
| However, there are a few new cases where an RPC transaction can fail. | However, there are a few new cases where an RPC transaction can fail. | |||
| For example, a Requester might receive, in response to an RPC | For example, a Requester might receive, in response to an RPC | |||
| request, an RDMA2_ERROR message with an rdma_err value of ERR_CHUNK. | request, an RDMA2_ERROR message with a rdma_err value of ERR_CHUNK. | |||
| These situations are not different from existing RPC errors which an | These situations are not different from existing RPC errors, which an | |||
| NFS session implementation is already prepared to handle for other | NFS session implementation can already handle for other transport | |||
| transports. And as with other transports during such a failure, | types. Moreover, there might be no SEQUENCE result available to the | |||
| there might be no SEQUENCE result available to the Requester to | Requester to distinguish whether failure occurred before or after the | |||
| distinguish whether failure occurred before or after the requested | Responder executed the requested operations. | |||
| operations were executed on the Responder. | ||||
| When a transport error occurs (e.g. RDMA2_ERROR), the Requester | When a transport error occurs (e.g., an RDMA2_ERROR type message is | |||
| proceeds as usual to match the incoming XID value to a waiting RPC | received), the Requester proceeds, as usual, to match the incoming | |||
| Call. The RPC transaction is terminated, and the result status is | XID value to a waiting RPC Call. The Requester terminates the RPC | |||
| reported to the Upper Layer Protocol. The Requester's session | transaction and reports the result status to the RPC consumer. The | |||
| implementation then determines the session ID and slot for the failed | Requester's session implementation then determines the session ID and | |||
| request, and performs slot recovery to make that slot usable again. | slot for the failed request and performs slot recovery to make that | |||
| If this were not done, that slot could be rendered permanently | slot usable again. Otherwise, that slot could be rendered | |||
| unavailable. | permanently unavailable. | |||
| When an NFS session is not present (for example, when NFS version 4.0 | When an NFS session is not present (for example, when NFS version 4.0 | |||
| is in use), a transport error does not provide an indication of | is in use), a transport error does not indicate whether the server | |||
| whether the server has processed the arguments of the RPC Call, or | has processed the arguments of the RPC Call, or whether the server | |||
| whether the server has accessed or modified client memory associated | has accessed or modified client memory associated with that RPC. | |||
| with that RPC. | ||||
| 6.7. Transport Considerations | 6.7. Transport Considerations | |||
| 6.7.1. Congestion Avoidance | 6.7.1. Congestion Avoidance | |||
| Section 3.1 of [RFC7530] states: | Section 3.1 of [RFC7530] states: | |||
| Where an NFS version 4 implementation supports operation over the | Where an NFS version 4 implementation supports operation over the | |||
| IP network protocol, the supported transport layer between NFS and | IP network protocol, the supported transport layer between NFS and | |||
| IP MUST be an IETF standardized transport protocol that is | IP MUST be an IETF standardized transport protocol that is | |||
| skipping to change at page 12, line 8 ¶ | skipping to change at page 13, line 11 ¶ | |||
| Even if NFS version 4.1 is used over a non-IP network protocol, it | Even if NFS version 4.1 is used over a non-IP network protocol, it | |||
| is RECOMMENDED that the transport support congestion control. | is RECOMMENDED that the transport support congestion control. | |||
| It is permissible for a connectionless transport to be used under | It is permissible for a connectionless transport to be used under | |||
| NFS version 4.1; however, reliable and in-order delivery of data | NFS version 4.1; however, reliable and in-order delivery of data | |||
| combined with congestion control by the connectionless transport | combined with congestion control by the connectionless transport | |||
| is REQUIRED. As a consequence, UDP by itself MUST NOT be used as | is REQUIRED. As a consequence, UDP by itself MUST NOT be used as | |||
| an NFS version 4.1 transport. | an NFS version 4.1 transport. | |||
| RPC-over-RDMA version 2 is constructed on a platform of RDMA Reliable | RPC-over-RDMA version 2 utilizes only RDMA Reliable Connected QP type | |||
| Connected QP type connections [I-D.cel-nfsv4-rpcrdma-version-two] | connections [RPCRDMA2]. RDMA Reliable Connected QPs are reliable, | |||
| [RFC5041]. RDMA Reliable Connected QPs are reliable, connection- | connection-oriented transports that guarantee in-order delivery, | |||
| oriented transports that guarantee in-order delivery, meeting all | meeting all the above requirements. | |||
| above requirements for NFS version 4 transports. | ||||
| 6.7.2. Retransmission and Keep-alive | 6.7.2. Retransmission and Keep-alive | |||
| NFS version 4 client implementations often rely on a transport-layer | NFS version 4 client implementations often rely on a transport-layer | |||
| keep-alive mechanism to detect when an NFS version 4 server has | keep-alive mechanism to detect when an NFS version 4 server has | |||
| become unresponsive. When an NFS server is no longer responsive, | become unresponsive. When an NFS server is no longer responsive, | |||
| client-side keep-alive terminates the connection, which in turn | client-side keep-alive terminates the connection, which in turn | |||
| triggers reconnection and RPC retransmission. | triggers reconnection and RPC retransmission. | |||
| Some RDMA transports (such as the Reliable Connected QP type on | Some RDMA transports (such as the Reliable Connected QP type on | |||
| InfiniBand) have no keep-alive mechanism. Without a disconnect or | InfiniBand) have no keep-alive mechanism. Without a disconnect or | |||
| new RPC traffic, such connections can remain alive long after an NFS | new RPC traffic, such connections can remain alive long after an NFS | |||
| server has become unresponsive. Once an NFS client has consumed all | server has become unresponsive. Once an NFS client has consumed all | |||
| available RPC-over-RDMA version 2 credits on that transport | available RPC-over-RDMA version 2 credits on that transport | |||
| connection, it will forever await a reply before sending another RPC | connection, it indefinitely awaits a reply before sending another RPC | |||
| request. | request. | |||
| NFS version 4 clients SHOULD reserve one RPC-over-RDMA version 2 | NFS version 4 clients SHOULD reserve one RPC-over-RDMA version 2 | |||
| credit to use for periodic server or connection health assessment. | credit to use for periodic server or connection health assessment. | |||
| This credit can be used to drive an RPC request on an otherwise idle | Either peer can use this credit to drive an RPC request on an | |||
| connection, triggering either a quick affirmative server response or | otherwise idle connection, triggering either a quick affirmative | |||
| immediate connection termination. | server response or immediate connection termination. | |||
| In addition to network partition and request loss scenarios, RPC- | In addition to network partition and request loss scenarios, RPC- | |||
| over-RDMA version 2 transport connections can be terminated when a | over-RDMA version 2 transport connections can be terminated when a | |||
| Transport header is malformed, Reply messages are larger than receive | Transport header is malformed, Reply messages exceed receive | |||
| resources, or when too many RPC-over-RDMA messages are sent at once. | resources, or when too many RPC-over-RDMA messages are sent at once. | |||
| In such cases: | In such cases: | |||
| o If there is a transport error indicated (ie, RDMA2_ERROR) before | o If a transport error occurs (e.g., an RDMA2_ERROR type message is | |||
| the disconnect or instead of a disconnect, the Requester MUST | received) before the disconnect or instead of a disconnect, the | |||
| respond to that error as prescribed by the specification of the | Requester MUST respond to that error as prescribed by the | |||
| RPC transport. Then the NFS version 4 rules for handling | specification of the RPC transport. Then the NFS version 4 rules | |||
| retransmission apply. | for handling retransmission apply. | |||
| o If there is a transport disconnect and the Responder has provided | o If there is a transport disconnect and the Responder has provided | |||
| no other response for a request, then only the NFS version 4 rules | no other response for a request, then only the NFS version 4 rules | |||
| for handling retransmission apply. | for handling retransmission apply. | |||
| 7. Extending NFS Upper Layer Bindings | 7. Extending NFS Upper-Layer Bindings | |||
| RPC programs such as NFS are required to have an Upper Layer Binding | RPC programs such as NFS are required to have an Upper-Layer Binding | |||
| specification to interoperate on RPC-over-RDMA version 2 transports | specification to interoperate on RPC-over-RDMA version 2 transports | |||
| [I-D.cel-nfsv4-rpcrdma-version-two]. Via standards action, the Upper | [RPCRDMA2]. Via standards action, the Upper-Layer Binding specified | |||
| Layer Binding specified in this document can be extended to cover | in this document can be extended to cover versions of the NFS version | |||
| versions of the NFS version 4 protocol specified after NFS version 4 | 4 protocol specified after NFS version 4 minor version 2, or to cover | |||
| minor version 2, or to cover separately published extensions to an | separately published extensions to an existing NFS version 4 minor | |||
| existing NFS version 4 minor version, as described in [RFC8178]. | version, as described in [RFC8178]. | |||
| 8. Security Considerations | 8. Security Considerations | |||
| RPC-over-RDMA version 2 supports all RPC security models, including | RPC-over-RDMA version 2 supports all RPC security models, including | |||
| RPCSEC_GSS security and transport-level security [RFC7861]. The | RPCSEC_GSS security and transport-level security [RFC7861]. The | |||
| choice of what Direct Data Placement mechanism to convey RPC argument | choice of what Direct Data Placement mechanism to convey RPC argument | |||
| and results does not affect this, since it changes only the method of | and results does not affect this since it changes only the method of | |||
| data transfer. Because the current document defines only the binding | data transfer. Because the current document defines only the binding | |||
| of the NFS protocols atop [I-D.cel-nfsv4-rpcrdma-version-two], all | of the NFS protocols atop [RPCRDMA2], all relevant security | |||
| relevant security considerations are therefore to be described at | considerations are, therefore, described at that layer. | |||
| that layer. | ||||
| 9. IANA Considerations | 9. IANA Considerations | |||
| The use of direct data placement in NFS introduces a need for an | The use of direct data placement in NFS introduces a need for an | |||
| additional port number assignment for networks that share traditional | additional port number assignment for networks that share traditional | |||
| UDP and TCP port spaces with RDMA services. The iWARP protocol is | UDP and TCP port spaces with RDMA services. The iWARP protocol is | |||
| such an example [RFC5040] [RFC5041]. | such an example [RFC5040] [RFC5041]. | |||
| For this purpose, a set of transport protocol port number assignments | For this purpose, the current document specifies a set of transport | |||
| is specified by this document. IANA has assigned the following ports | protocol port number assignments. IANA has assigned the following | |||
| for NFS/RDMA in the IANA port registry, according to the guidelines | ports for NFS/RDMA in the IANA port registry, according to the | |||
| described in [RFC6335]. | guidelines described in [RFC6335]. | |||
| nfsrdma 20049/tcp Network File System (NFS) over RDMA | nfsrdma 20049/tcp Network File System (NFS) over RDMA | |||
| nfsrdma 20049/udp Network File System (NFS) over RDMA | nfsrdma 20049/udp Network File System (NFS) over RDMA | |||
| nfsrdma 20049/sctp Network File System (NFS) over RDMA | nfsrdma 20049/sctp Network File System (NFS) over RDMA | |||
| This document should be listed as a reference for the nfsrdma port | The current document should be added as a reference for the nfsrdma | |||
| assignments. This document does not alter these assignments. | port assignments. The current document does not alter these | |||
| assignments. | ||||
| 10. References | 10. References | |||
| 10.1. Normative References | 10.1. Normative References | |||
| [I-D.cel-nfsv4-rpcrdma-version-two] | ||||
| Lever, C. and D. Noveck, "RPC-over-RDMA Version 2 | ||||
| Protocol", draft-cel-nfsv4-rpcrdma-version-two-09 (work in | ||||
| progress), May 2019. | ||||
| [RFC1833] Srinivasan, R., "Binding Protocols for ONC RPC Version 2", | [RFC1833] Srinivasan, R., "Binding Protocols for ONC RPC Version 2", | |||
| RFC 1833, DOI 10.17487/RFC1833, August 1995, | RFC 1833, DOI 10.17487/RFC1833, August 1995, | |||
| <https://www.rfc-editor.org/info/rfc1833>. | <https://www.rfc-editor.org/info/rfc1833>. | |||
| [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate | [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate | |||
| Requirement Levels", BCP 14, RFC 2119, | Requirement Levels", BCP 14, RFC 2119, | |||
| DOI 10.17487/RFC2119, March 1997, | DOI 10.17487/RFC2119, March 1997, | |||
| <https://www.rfc-editor.org/info/rfc2119>. | <https://www.rfc-editor.org/info/rfc2119>. | |||
| [RFC5661] Shepler, S., Ed., Eisler, M., Ed., and D. Noveck, Ed., | [RFC5661] Shepler, S., Ed., Eisler, M., Ed., and D. Noveck, Ed., | |||
| skipping to change at page 15, line 5 ¶ | skipping to change at page 15, line 47 ¶ | |||
| November 2016, <https://www.rfc-editor.org/info/rfc7862>. | November 2016, <https://www.rfc-editor.org/info/rfc7862>. | |||
| [RFC8167] Lever, C., "Bidirectional Remote Procedure Call on RPC- | [RFC8167] Lever, C., "Bidirectional Remote Procedure Call on RPC- | |||
| over-RDMA Transports", RFC 8167, DOI 10.17487/RFC8167, | over-RDMA Transports", RFC 8167, DOI 10.17487/RFC8167, | |||
| June 2017, <https://www.rfc-editor.org/info/rfc8167>. | June 2017, <https://www.rfc-editor.org/info/rfc8167>. | |||
| [RFC8174] Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC | [RFC8174] Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC | |||
| 2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174, | 2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174, | |||
| May 2017, <https://www.rfc-editor.org/info/rfc8174>. | May 2017, <https://www.rfc-editor.org/info/rfc8174>. | |||
| [RPCRDMA2] | ||||
| Lever, C. and D. Noveck, "RPC-over-RDMA Version 2 | ||||
| Protocol", draft-ietf-nfsv4-rpcrdma-version-two-01 (work | ||||
| in progress), Jan 2020. | ||||
| 10.2. Informative References | 10.2. Informative References | |||
| [RFC1094] Nowicki, B., "NFS: Network File System Protocol | [RFC1094] Nowicki, B., "NFS: Network File System Protocol | |||
| specification", RFC 1094, DOI 10.17487/RFC1094, March | specification", RFC 1094, DOI 10.17487/RFC1094, March | |||
| 1989, <https://www.rfc-editor.org/info/rfc1094>. | 1989, <https://www.rfc-editor.org/info/rfc1094>. | |||
| [RFC1813] Callaghan, B., Pawlowski, B., and P. Staubach, "NFS | [RFC1813] Callaghan, B., Pawlowski, B., and P. Staubach, "NFS | |||
| Version 3 Protocol Specification", RFC 1813, | Version 3 Protocol Specification", RFC 1813, | |||
| DOI 10.17487/RFC1813, June 1995, | DOI 10.17487/RFC1813, June 1995, | |||
| <https://www.rfc-editor.org/info/rfc1813>. | <https://www.rfc-editor.org/info/rfc1813>. | |||
| skipping to change at page 15, line 36 ¶ | skipping to change at page 16, line 36 ¶ | |||
| [RFC8178] Noveck, D., "Rules for NFSv4 Extensions and Minor | [RFC8178] Noveck, D., "Rules for NFSv4 Extensions and Minor | |||
| Versions", RFC 8178, DOI 10.17487/RFC8178, July 2017, | Versions", RFC 8178, DOI 10.17487/RFC8178, July 2017, | |||
| <https://www.rfc-editor.org/info/rfc8178>. | <https://www.rfc-editor.org/info/rfc8178>. | |||
| [XNFS] The Open Group, "Protocols for Interworking: XNFS, Version | [XNFS] The Open Group, "Protocols for Interworking: XNFS, Version | |||
| 3W", February 1998. | 3W", February 1998. | |||
| Acknowledgments | Acknowledgments | |||
| Thanks to Tom Talpey, who contributed the text of Section 6.4.2. | Thanks to Tom Talpey, who contributed the text of Section 6.4.2. | |||
| Dave Noveck contributed the text of Section 6.6 and Section 7. | Dave Noveck contributed the text of Section 6.6 and Section 7. The | |||
| author also wishes to thank Bill Baker and Greg Marsden for their | ||||
| support of this work. | ||||
| Special thanks go to Transport Area Director Magnus Westerlund, NFSV4 | Special thanks go to Transport Area Director Magnus Westerlund, NFSV4 | |||
| Working Group Chairs Spencer Shepler and Brian Pawlowski, and NFSV4 | Working Group Chairs Spencer Shepler, Brian Pawlowski, and Dave | |||
| Working Group Secretary Thomas Haynes for their support. The author | Noveck, and NFSV4 Working Group Secretary Thomas Haynes for their | |||
| also wishes to thank Bill Baker and Greg Marsden for their support of | support. | |||
| this work. | ||||
| Author's Address | Author's Address | |||
| Charles Lever | Charles Lever | |||
| Oracle Corporation | Oracle Corporation | |||
| United States of America | United States of America | |||
| Email: chuck.lever@oracle.com | Email: chuck.lever@oracle.com | |||
| End of changes. 66 change blocks. | ||||
| 174 lines changed or deleted | 228 lines changed or added | |||
This html diff was produced by rfcdiff 1.47. The latest version is available from http://tools.ietf.org/tools/rfcdiff/ | ||||