--- 1/draft-ietf-nfsv4-mv1-msns-update-00.txt 2018-06-09 06:13:20.734232257 -0700 +++ 2/draft-ietf-nfsv4-mv1-msns-update-01.txt 2018-06-09 06:13:20.890235977 -0700 @@ -1,19 +1,19 @@ NFSv4 D. Noveck, Ed. Internet-Draft NetApp Updates: 5661 (if approved) C. Lever Intended status: Standards Track ORACLE -Expires: July 7, 2018 January 3, 2018 +Expires: December 11, 2018 June 9, 2018 NFSv4.1 Update for Multi-Server Namespace - draft-ietf-nfsv4-mv1-msns-update-00 + draft-ietf-nfsv4-mv1-msns-update-01 Abstract This document presents necessary clarifications and corrections concerning features related to the use of location-related attributes in NFSv4.1. These include migration, which transfers responsibility for a file system from one server to another, and facilities to support trunking by allowing discovery of the set of network addresses to use to access a file system. This document updates RFC5661. @@ -26,21 +26,21 @@ Internet-Drafts are working documents of the Internet Engineering Task Force (IETF). Note that other groups may also distribute working documents as Internet-Drafts. The list of current Internet- Drafts is at https://datatracker.ietf.org/drafts/current/. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." - This Internet-Draft will expire on July 7, 2018. + This Internet-Draft will expire on December 11, 2018. Copyright Notice Copyright (c) 2018 IETF Trust and the persons identified as the document authors. All rights reserved. This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (https://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents @@ -52,83 +52,86 @@ Table of Contents 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 3 2. Requirements Language . . . . . . . . . . . . . . . . . . . . 4 3. Preliminaries . . . . . . . . . . . . . . . . . . . . . . . . 4 3.1. Terminology . . . . . . . . . . . . . . . . . . . . . . . 4 3.2. Summary of Issues . . . . . . . . . . . . . . . . . . . . 6 3.3. Relationship of this Document to RFC5661 . . . . . . . . 8 4. Changes to Section 11 of RFC5661 . . . . . . . . . . . . . . 9 - 4.1. Multi-Server Namespace (as updated) . . . . . . . . . . . 9 - 4.2. Location-related Terminology (to be added) . . . . . . . 9 - 4.3. Location Attributes (as updated) . . . . . . . . . . . . 11 - 4.4. Re-organization of Sections 11.4 and 11.5 of RFC5661 . . 12 - 4.5. Uses of Location Information (as updated) . . . . . . . . 12 + 4.1. Multi-Server Namespace (as updated) . . . . . . . . . . . 10 + 4.2. Location-related Terminology (to be added) . . . . . . . 10 + 4.3. Location Attributes (as updated) . . . . . . . . . . . . 12 + 4.4. Re-organization of Sections 11.4 and 11.5 of RFC5661 . . 13 + 4.5. Uses of Location Information (as updated) . . . . . . . . 13 4.5.1. Combining Multiple Uses in a Single Attribute (to be - added) . . . . . . . . . . . . . . . . . . . . . . . 13 - 4.5.2. Location Attributes and Trunking (to be added) . . . 14 - 4.5.3. File System Replication (as updated) . . . . . . . . 14 - 4.5.4. File System Migration (as updated) . . . . . . . . . 15 - 4.5.5. Referrals (as updated) . . . . . . . . . . . . . . . 16 - 4.5.6. Changes in a Location Attribute (to be added) . . . . 17 - 5. Re-organization of Section 11.7 of RFC5661 . . . . . . . . . 18 - 6. Overview of File Access Transitions (to be added) . . . . . . 19 - 7. Effecting Network Address Transitions (to be added) . . . . . 19 - 8. Effecting File System Transitions (as updated) . . . . . . . 20 + added) . . . . . . . . . . . . . . . . . . . . . . . 14 + 4.5.2. Location Attributes and Trunking (to be added) . . . 15 + 4.5.3. Location Attributes and Connection Type Selection (to + be added) . . . . . . . . . . . . . . . . . . . . . . 15 + 4.5.4. File System Replication (as updated) . . . . . . . . 16 + 4.5.5. File System Migration (as updated) . . . . . . . . . 16 + 4.5.6. Referrals (as updated) . . . . . . . . . . . . . . . 17 + 4.5.7. Changes in a Location Attribute (to be added) . . . . 19 + 5. Re-organization of Section 11.7 of RFC5661 . . . . . . . . . 20 + 6. Overview of File Access Transitions (to be added) . . . . . . 20 + 7. Effecting Network Endpoint Transitions (to be added) . . . . 21 + 8. Effecting File System Transitions (as updated) . . . . . . . 22 8.1. File System Transitions and Simultaneous Access (as - updated) . . . . . . . . . . . . . . . . . . . . . . . . 21 - 8.2. Filehandles and File System Transitions (as updated) . . 21 - 8.3. Fileids and File System Transitions (as updated) . . . . 22 - 8.4. Fsids and File System Transitions (as updated) . . . . . 23 - 8.4.1. File System Splitting (as updated) . . . . . . . . . 23 + updated) . . . . . . . . . . . . . . . . . . . . . . . . 23 + 8.2. Filehandles and File System Transitions (as updated) . . 23 + 8.3. Fileids and File System Transitions (as updated) . . . . 24 + 8.4. Fsids and File System Transitions (as updated) . . . . . 25 + 8.4.1. File System Splitting (as updated) . . . . . . . . . 25 8.5. The Change Attribute and File System Transitions (as - updated) . . . . . . . . . . . . . . . . . . . . . . . . 24 - 8.6. Write Verifiers and File System Transitions (as updated) 24 + updated) . . . . . . . . . . . . . . . . . . . . . . . . 26 + 8.6. Write Verifiers and File System Transitions (as updated) 26 8.7. Readdir Cookies and Verifiers and File System Transitions - (as updated) . . . . . . . . . . . . . . . . . . . . . . 24 - 8.8. File System Data and File System Transitions (as updated) 25 - 8.9. Lock State and File System Transitions (as updated) . . . 26 - 9. Transferring State upon Migration (to be added) . . . . . . . 27 - 9.1. Transparent State Migration and pNFS (to be added) . . . 27 + (as updated) . . . . . . . . . . . . . . . . . . . . . . 26 + 8.8. File System Data and File System Transitions (as updated) 27 + 8.9. Lock State and File System Transitions (as updated) . . . 28 + 9. Transferring State upon Migration (to be added) . . . . . . . 29 + 9.1. Transparent State Migration and pNFS (to be added) . . . 29 10. Client Responsibilities when Access is Transitioned (to be - added) . . . . . . . . . . . . . . . . . . . . . . . . . . . 28 - 10.1. Client Transition Notifications (to be added) . . . . . 29 - 10.2. Performing Migration Discovery (to be added) . . . . . . 31 + added) . . . . . . . . . . . . . . . . . . . . . . . . . . . 30 + + 10.1. Client Transition Notifications (to be added) . . . . . 31 + 10.2. Performing Migration Discovery (to be added) . . . . . . 33 10.3. Overview of Client Response to NFS4ERR_MOVED (to be - added) . . . . . . . . . . . . . . . . . . . . . . . . . 34 + added) . . . . . . . . . . . . . . . . . . . . . . . . . 36 10.4. Obtaining Access to Sessions and State after Migration - (to be added) . . . . . . . . . . . . . . . . . . . . . 36 + (to be added) . . . . . . . . . . . . . . . . . . . . . 37 10.5. Obtaining Access to Sessions and State after Network - Address Transfer (to be added) . . . . . . . . . . . . . 37 - 11. Server Responsibilities Upon Migration (to be added) . . . . 38 + Address Transfer (to be added) . . . . . . . . . . . . . 39 + 11. Server Responsibilities Upon Migration (to be added) . . . . 40 11.1. Server Responsibilities in Effecting Transparent State - Migration (to be added) . . . . . . . . . . . . . . . . 38 + Migration (to be added) . . . . . . . . . . . . . . . . 40 11.2. Server Responsibilities in Effecting Session Transfer - (to be added) . . . . . . . . . . . . . . . . . . . . . 40 - 12. Changes to RFC5661 outside Section 11 . . . . . . . . . . . . 42 - 12.1. (Introduction to) Multi-Server Namespace (as updated) . 43 - 12.2. Server Scope (as updated) . . . . . . . . . . . . . . . 44 - 12.3. Revised Treatment of NFS4ERR_MOVED . . . . . . . . . . . 46 - 12.4. Revised Discussion of Server_owner changes . . . . . . . 46 - 12.5. Revision to Treatment of EXCHANGE_ID . . . . . . . . . . 47 + (to be added) . . . . . . . . . . . . . . . . . . . . . 42 + 12. Changes to RFC5661 outside Section 11 . . . . . . . . . . . . 44 + 12.1. (Introduction to) Multi-Server Namespace (as updated) . 45 + 12.2. Server Scope (as updated) . . . . . . . . . . . . . . . 46 + 12.3. Revised Treatment of NFS4ERR_MOVED . . . . . . . . . . . 47 + 12.4. Revised Discussion of Server_owner changes . . . . . . . 48 + 12.5. Revision to Treatment of EXCHANGE_ID . . . . . . . . . . 49 13. Operation 42: EXCHANGE_ID - Instantiate Client ID (as - updated) . . . . . . . . . . . . . . . . . . . . . . . . . . 48 - 14. Security Considerations . . . . . . . . . . . . . . . . . . . 66 - 15. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 68 - 16. References . . . . . . . . . . . . . . . . . . . . . . . . . 69 - 16.1. Normative References . . . . . . . . . . . . . . . . . . 69 - 16.2. Informative References . . . . . . . . . . . . . . . . . 70 - Appendix A. Classification of Document Sections . . . . . . . . 70 - Appendix B. Updates to RFC5661 . . . . . . . . . . . . . . . . . 71 - Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . . . 74 - Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 75 + updated) . . . . . . . . . . . . . . . . . . . . . . . . . . 50 + 14. Security Considerations . . . . . . . . . . . . . . . . . . . 68 + 15. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 70 + 16. References . . . . . . . . . . . . . . . . . . . . . . . . . 71 + 16.1. Normative References . . . . . . . . . . . . . . . . . . 71 + 16.2. Informative References . . . . . . . . . . . . . . . . . 72 + Appendix A. Classification of Document Sections . . . . . . . . 72 + Appendix B. Updates to RFC5661 . . . . . . . . . . . . . . . . . 74 + Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . . . 76 + Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 77 1. Introduction This document defines the proper handling, within NFSv4.1, of the location-related attributes fs_locations and fs_locations_info and how necessary changes in those attributes are to be dealt with. The necessary corrections and clarifications parallel those done for NFSv4.0 in [RFC7931] and [I-D.cel-nfsv4-mv0-trunking-update]. A large part of the changes to be made are necessary to clarify the @@ -205,34 +208,56 @@ trunking detection to appropriately filter them. Despite the support for trunking detection there was no description of trunking discovery provided in [RFC5661]. Regarding network addresses and the handling of trunking we use the following terminology: o Each NFSv4 server is assumed to have a set of IP addresses to which NFSv4 requests may be sent by clients. These are referred - to as the server's network addresses. + to as the server's network addresses. Access to a specfic server + network address may involve the use of multiple ports, since the + ports to be used for various types of connections might be + required to be different. o Each network address, when combined with a pathname providing the location of a file system root directory relative to the associated server root file handle, defines a file system network access path. + o Server network addresses are used to establish connections to + servers which may be of a number of connection types. Separate + connection types are used to support NFSv4 layered on top of the + RPC stream transport as described in [RFC5531] and on top of RPC- + over-RDMA as described in [RFC8166]. + + o The combination of a server network address and a particular + connection type to be used by a connection is referred to as a + "server endpoint". Although using different connection types may + result in different ports being used, the use of different ports + by multiple connections to the same network address is not the + essence of the distinction between the two endpoints used. + o Two network addresses connected to the same server are said to be server-trunkable. o Two network addresses connected to the same server such that those addresses can be used to support a single common session are referred to as session-trunkable. Note that two addresses may be - server-trunkable without being session-trunkable. + server-trunkable without being session-trunkable and that when two + connections of different connection types are made to the same + network address and are based on a single-location entry they are + always session-trunkable, independent of the connection type, as + specified by [RFC5661], since their derivation from the same + location entry assures that both connections are to the same + server. Discussion of the term "replica" is complicated for a number of reasons: o Even though the term is used in explaining the issues in [RFC5661] that need to be addressed in this document, a full explanation of this term requires explanation of related terms connected to the location attributes which are provided in Section 4.2 of the current document. @@ -395,21 +420,21 @@ Section 11.1. The new material appears in Sections 4.1 through 4.3 below. o A significant reorganization of the material in the existing Sections 11.4 and 11.5 (of [RFC5661]) is necessary. The reasons for the reorganization of these sections into a single section with multiple subsections are discussed in Section 4.4 below. This replacement appears as Section 4.5 below. New material relating to the handling of the location attributes - is contained in Sections 4.5.1 and 4.5.6 below. + is contained in Sections 4.5.1 and 4.5.7 below. o A major replacement for the existing Section 11.7 of [RFC5661] entitled "Effecting File System Transitions", will appear as Sections 6 through 11 of the current document. The reasons for the reorganization of this section into multiple sections are discussed below in Section 5 of the current document. 4.1. Multi-Server Namespace (as updated) NFSv4.1 supports attributes that allow a namespace to extend beyond @@ -436,29 +461,30 @@ instance within a larger multi-server namespace. o The set of all exported file systems for a given server constitutes that server's local namespace. o In some cases, a server will have a namespace, more extensive than its local namespace, by using features associated with attributes that provide location information. These features, which allow construction of a multi-server namespace are all described in individual sections below and include referrals (described in - Section 4.5.5), migration (described in Section 4.5.4), and - replication (described in Section 4.5.3). + Section 4.5.6), migration (described in Section 4.5.5), and + replication (described in Section 4.5.4). o A file system present in a server's pseudo-fs may have multiple file system instances on different servers associated with it. All such instances are considered replicas of one another. o When a file system is present in a server's pseudo-fs, but there is no corresponding local file system, it is said to be "absent". + In such cases, all associated instances will be accessed on other servers. Regarding terminology relating to attributes used in trunking discovery and other multi-server namespace features: o Location attributes include the fs_locations and fs_locations_info attributes. o Location entries are the individual file system locations in the @@ -456,38 +482,45 @@ servers. Regarding terminology relating to attributes used in trunking discovery and other multi-server namespace features: o Location attributes include the fs_locations and fs_locations_info attributes. o Location entries are the individual file system locations in the location attributes. Each such entry specifies a server, in the - form of a host name, and an fs name, which in the location of the - file system within the server's pseudo-fs. The exact form of the - location entry varies with the particular location attribute used - as described in Section 4.3 + form of a host name or IP address, and an fs name, which + designates the location of the file system within the server's + pseudo-fs. A location entry designates a set of server endpoints + to which the client may establish connections. There may be + multiple endpoints because a host name may map to multiple network + addresses and because multiple connection types may be used to + communicate with a single network address. However, all such + endpoints MUST provide a way of connecting to a single server. + The exact form of the location entry varies with the particular + location attribute used as described in Section 4.3. o Location elements are derived from location entries and each - describes a particular network access path. Location elements - need not appear within a location attribute, but the existence of - each location element derives from a corresponding location entry. - When a location entry specifies an IP address there is only a - single corresponding location element. Location entries that - contain a host name, are resolved using DNS, and may result in one - or more location elements. All location elements consist of a - location address which is the IP address of an interface to a - server and an fs name which is the location of the file system - within the server's pseudo-fs. The fs name is empty if the server - has no pseudo-fs and only a single exported file system at the - root filehandle. + describes a particular network access path, consisting of a + network address and a location within the server's pseudo-fs. + Location elements need not appear within a location attribute, but + the existence of each location element derives from a + corresponding location entry. When a location entry specifies an + IP address there is only a single corresponding location element. + Location entries that contain a host name, are resolved using DNS, + and may result in one or more location elements. All location + elements consist of a location address which is the IP address of + an interface to a server and an fs name which is the location of + the file system within the server's pseudo-fs. The fs name is + empty if the server has no pseudo-fs and only a single exported + file system at the root filehandle. o Two location elements are said to be server-trunkable if they specify the same fs name and the location addresses are such that the location addresses are server-trunkable. o Two location elements are said to be session-trunkable if they specify the same fs name and the location addresses are such that the location addresses are session-trunkable. Each set of server-trunkable location elements defines a set of @@ -508,26 +541,33 @@ namespace of one server can be associated with one or more instances of that file system on other servers. These attributes contain location entries specifying a server address target (either as a DNS name representing one or more IP addresses or as a specific IP address) together with the pathname of that file system within the associated single-server namespace. The fs_locations_info RECOMMENDED attribute allows specification of one or more file system instance locations where the data corresponding to a given file system may be found. This attribute - provides to the client, in addition to information about file system - instance locations, significant information about the various file - system instance choices (e.g., priority for use, writability, - currency, etc.). It also includes information to help the client - efficiently effect as seamless a transition as possible among - multiple file system instances, when and if that should be necessary. + provides to the client, in to addition to specification of file + system instance locations, other helpful information such as: + + o Information guiding choices among the various file system + instances provided (e.g., priority for use, writability, currency, + etc.). + + o Information to help the client efficiently effect as seamless a + transition as possible among multiple file system instances, when + and if that should be necessary. + + o Information helping to guide the selection of the appropriate + connection type to be used when establishing a connection. Within the fs_locations_info attribute, each fs_locations_server4 entry corresponds to a location entry with the fls_server field designating the server, with the location pathname within the server's pseudo-fs given by the fl_rootpath field of the encompassing fs_locations_item4. The fs_locations attribute defined in NFSv4.0 is also a part of NFSv4.1. This attribute only allows specification of the file system locations where the data corresponding to a given file system may be @@ -543,63 +583,63 @@ 4.4. Re-organization of Sections 11.4 and 11.5 of RFC5661 Previously, issues related to the fact that multiple location entries directed the client to the same file system instance were dealt with in a separate Section 11.5 of [RFC5661]. Because of the new treatment of trunking, these issues now belong within Section 4.5 below. In this new section of the current document, trunking is dealt with in Section 4.5.2 together with the other uses of location information - described in Sections 4.5.3, 4.5.4, and 4.5.5. + described in Sections 4.5.4, 4.5.5, and 4.5.6. 4.5. Uses of Location Information (as updated) The location attributes (i.e. fs_locations and fs_locations_info), together with the possibility of absent file systems, provide a number of important facilities in providing reliable, manageable, and scalable data access. When a file system is present, these attributes can provide o The locations of alternative replicas, to be used to access the same data in the event of server failures, communications problems, or other difficulties that make continued access to the current replica impossible or otherwise impractical. Provision and use of such alternate replicas is referred to as "replication" - and is discussed in Section 4.5.3 below. + and is discussed in Section 4.5.4 below. o The network address(es) to be used to access the current file system instance or replicas of it. Client use of this information is discussed in Section 4.5.2 below. Under some circumstances, multiple replicas may be used simultaneously to provide higher-performance access to the file system in question, although the lack of state sharing between servers may be an impediment to such use. When a file system is present and becomes absent, clients can be given the opportunity to have continued access to their data, using a different replica. In this case, a continued attempt to use the data in the now-absent file system will result in an NFS4ERR_MOVED error and, at that point, the successor replica or set of possible replica choices can be fetched and used to continue access. Transfer of access to the new replica location is referred to as "migration", and - is discussed in Section 4.5.3 below. + is discussed in Section 4.5.4 below. Where a file system was previously absent, specification of file system location provides a means by which file systems located on one server can be associated with a namespace defined by another server, thus allowing a general multi-server namespace facility. A designation of such a remote instance, in place of a file system never previously present , is called a "pure referral" and is - discussed in Section 4.5.5 below. + discussed in Section 4.5.6 below. Because client support for location-related attributes is OPTIONAL, a server may (but is not required to) take action to hide migration and referral events from such clients, by acting as a proxy, for example. The server can determine the presence of client support from the arguments of the EXCHANGE_ID operation (see Section 13.3 in the current document). 4.5.1. Combining Multiple Uses in a Single Attribute (to be added) @@ -650,29 +690,58 @@ server-trunkable location entries which can provide the addresses specified by the server as desirable to use to access the file system in question. The server can provide location entries that include either names or network addresses. It might use the latter form because of DNS- related security concerns or because the set of addresses to be used might require active management by the server. Locations entries used to discover candidate addresses for use in - trunking are subject to change, as discussed in Section 4.5.6 below. + trunking are subject to change, as discussed in Section 4.5.7 below. The client may respond to such changes by using additional addresses once they are verified or by ceasing to use existing ones. The server can force the client to cease using an address by returning NFS4ERR_MOVED when that address is used to access a file system. This allows a transfer of access similar to migration, although the same file system instance is accessed throughout. -4.5.3. File System Replication (as updated) +4.5.3. Location Attributes and Connection Type Selection (to be added) + + Because of the need to support multiple connections, clients face the + issue of determining the proper connection type to use when + establishing a connection to a given server network address. In some + cases, this issue can be addressed through the use of the connection + "step-up" facility described in Section 18.16 of [RFC5661]. However, + because there are cases is which that fcility is not available, the + client may have to choose a connection type with no possibility of + changing it within the scope of a single connection. + + The two location attributes differ as to the information made + available in this regard. Fs_locations provides no information to + support connection type selection. As a result, clients supporting + multiple connection types need to attempt to establish a connection + on multiple connection types until the one preferred by the client is + successfully established. + + Fs_locations_info provides a flag, FSLI4TF_RDMA flag. indicating + that RPC-over-RDMA support is available using the specfied location + entry. This flag makes it for a convenient for a client wishing to + use RDMA, to establish a TCP connection and then convert to use of + RDMA. After establishing a TCP connection, the step-up facility, can + be used, if available, to convert that connection to RDMA mode. + Otherwise, if RDMA availability is indicated, a new RDMA connection + can be established and it can be bound to the sessiion already + established by the TCP connection, allowing the TCP connection to be + dropped and the session converted to further use in RDMA node. + +4.5.4. File System Replication (as updated) The fs_locations and fs_locations_info attributes provide alternative locations, to be used to access data in place of or in addition to the current file system instance. On first access to a file system, the client should obtain the set of alternate locations by interrogating the fs_locations or fs_locations_info attribute, with the latter being preferred. In the event that server failures, communications problems, or other difficulties make continued access to the current file system @@ -681,21 +750,21 @@ The alternate locations may be physical replicas of the (typically read-only) file system data, or they may provide for the use of various forms of server clustering in which multiple servers provide alternate ways of accessing the same physical file system. How these different modes of file system transition are represented within the fs_locations and fs_locations_info attributes and how the client deals with file system transition issues will be discussed in detail below. -4.5.4. File System Migration (as updated) +4.5.5. File System Migration (as updated) When a file system is present and becomes absent, clients can be given the opportunity to have continued access to their data, at an alternate location, as specified by a location attribute. This migration of access to another replica includes the ability to retain locks across the transition, either by reclaim or by Transparent State Migration. Typically, a client will be accessing the file system in question, get an NFS4ERR_MOVED error, and then use a location attribute to @@ -730,21 +799,21 @@ degree indicated by the fs_locations_info attribute). Where file systems are writable, a change made on the original file system must be visible on all migration targets. Where a file system is not writable but represents a read-only copy (possibly periodically updated) of a writable file system, similar requirements apply to the propagation of updates. Any change visible in the original file system must already be effected on all migration targets, to avoid any possibility that a client, in effecting a transition to the migration target, will see any reversion in file system state. -4.5.5. Referrals (as updated) +4.5.6. Referrals (as updated) Referrals allow the server to associate a file system located on one server with file system located on another server. When this includes the use of pure referrals, servers are provided a way of placing a file system in a location within the namespace essentially without respect to its physical location on a particular server. This allows a single server or a set of servers to present a multi- server namespace that encompasses file systems located on a wider range of servers. Some likely uses of this facility include establishment of site-wide or organization-wide namespaces, with the @@ -769,21 +838,21 @@ read-only locations might not be absolutely up-to-date (as they would have to be in the case of replication and migration). Servers may also specify file system locations that include client-substituted variables so that different clients are referred to different file systems (with different data contents) based on client attributes such as CPU architecture. When the fs_locations_info attribute is such that that there are multiple possible targets listed, the relationships among them may be important to the client in selecting which one to use. The same - rules specified in Section 4.5.4 below regarding multiple migration + rules specified in Section 4.5.5 below regarding multiple migration targets apply to these multiple replicas as well. For example, the client might prefer a writable target on a server that has additional writable replicas to which it subsequently might switch. Note that, as distinguished from the case of replication, there is no need to deal with the case of propagation of updates made by the current client, since the current client has not accessed the file system in question. Use of multi-server namespaces is enabled by NFSv4.1 but is not required. The use of multi-server namespaces and their scope will @@ -798,21 +867,21 @@ namespace. The top-level referral file system or any segment may use replicated referral file systems for higher availability. Generally, multi-server namespaces are for the most part uniform, in that the same data made available to one client at a given location in the namespace is made available to all clients at that location. However, there are facilities provided that allow different clients to be directed different sets of data, to enable adaptation to such client characteristics as CPU architecture. -4.5.6. Changes in a Location Attribute (to be added) +4.5.7. Changes in a Location Attribute (to be added) Although clients will typically fetch a location attribute when first accessing a file system and when NFS4ERR_MOVED is returned, a client can choose to fetch the attribute periodically, in which case, the value fetched may change over time. For clients not prepared to access multiple replicas simultaneously (see Section 8.1 of the current document), the handling of the various cases of change are as follows: @@ -889,35 +958,46 @@ o Those that involve a transition from accessing the current replica to another one in connection with either replication or migration. How these are dealt with is discussed in Section 8 of the current document. o Those in which access to the current file system instance is retained, while the network path used to access that instance is changed. This case is discussed in Section 7 of the current document. -7. Effecting Network Address Transitions (to be added) +7. Effecting Network Endpoint Transitions (to be added) - The addresses used to access a particular file system instance may + The endpoints used to access a particular file system instance may change in a number of ways, as listed below. In each of these cases, the same filehandles, stateids, client IDs and session are used to continue access, with a continuity of lock state. o When use of a particular address is to cease and there is also one currently in use which is server-trunkable with it, requests that would have been issued on the address whose use is to be discontinued can be issued on the remaining address(es). When an address is not a session-trunkable one, the request might need to be modified to reflect the fact that a different session will be used. + o When use of a particular connection is to cease, as indicated by + receiving NFS4ERR_MOVED when using that connection but that + address is still indicated as accessible according to the + appropriate location entries, it is likely that requests can be + issued on a new connection of a different connection type, once + that connection is established. Since any two server endpoints + that share a network address are inherently session-trunkable, the + client can use BIND_CONN_TO_SESSION to access the existing session + using the new connection and proceed to access the file system + using the new connection. + o When there are no potential replacement addresses in use but there are valid addresses session-trunkable with the one whose use is to be discontinued, the client can use BIND_CONN_TO_SESSION to access the existing session using the new address. Although the target session will generally be accessible, there may be cases in which that session in no longer accessible, in which case a new session can be created to provide the client continued access to the existing instance. o When there is no potential replacement address in use and there @@ -1588,21 +1668,21 @@ continually delay the clearing of the LEASE_MOVED indication. To prevent unnecessary lease expiration, it is appropriate for clients to use the discovery of migrations to effect lease renewal immediately, rather than waiting for clearing of the LEASE_MOVED indication when the complete set of migrations is available. 10.3. Overview of Client Response to NFS4ERR_MOVED (to be added) This section outlines a way in which a client that receives NFS4ERR_MOVED can effect transition recovery by using a new server or - network address if one is available. As part of that process, it + server endpoint if one is available. As part of that process, it will determine: o Whether the NFS4ERR_MOVED indicates migration has occurred, or whether it indicates another sort of file system access transition as discussed in Section 7 above. o In the case of migration, whether Transparent State Migration has occurred. o Whether any state has been lost during the process of Transparent @@ -3249,38 +3330,47 @@ Algorithms and Identifiers for RSA Cryptography for use in the Internet X.509 Public Key Infrastructure Certificate and Certificate Revocation List (CRL) Profile", RFC 4055, DOI 10.17487/RFC4055, June 2005, . [RFC5403] Eisler, M., "RPCSEC_GSS Version 2", RFC 5403, DOI 10.17487/RFC5403, February 2009, . + [RFC5531] Thurlow, R., "RPC: Remote Procedure Call Protocol + Specification Version 2", RFC 5531, DOI 10.17487/RFC5531, + May 2009, . + [RFC5661] Shepler, S., Ed., Eisler, M., Ed., and D. Noveck, Ed., "Network File System (NFS) Version 4 Minor Version 1 Protocol", RFC 5661, DOI 10.17487/RFC5661, January 2010, . [RFC7530] Haynes, T., Ed. and D. Noveck, Ed., "Network File System (NFS) Version 4 Protocol", RFC 7530, DOI 10.17487/RFC7530, March 2015, . [RFC7861] Adamson, A. and N. Williams, "Remote Procedure Call (RPC) Security Version 3", RFC 7861, DOI 10.17487/RFC7861, November 2016, . [RFC7931] Noveck, D., Ed., Shivam, P., Lever, C., and B. Baker, "NFSv4.0 Migration: Specification Update", RFC 7931, DOI 10.17487/RFC7931, July 2016, . + [RFC8166] Lever, C., Ed., Simpson, W., and T. Talpey, "Remote Direct + Memory Access Transport for Remote Procedure Call Version + 1", RFC 8166, DOI 10.17487/RFC8166, June 2017, + . + 16.2. Informative References [I-D.cel-nfsv4-mv0-trunking-update] Lever, C. and D. Noveck, "NFS version 4.0 Trunking Update", draft-cel-nfsv4-mv0-trunking-update-00 (work in progress), November 2017. [RFC2104] Krawczyk, H., Bellare, M., and R. Canetti, "HMAC: Keyed- Hashing for Message Authentication", RFC 2104, DOI 10.17487/RFC2104, February 1997, @@ -3293,34 +3383,35 @@ below. In this listing, when we refer to a Section X and there is a Section X.1 within it, the classification of Section X refers to the part of that section exclusive of subsections. In the case when that portion is empty, the section is not counted. o Sections 1 through 4, a total of five sections, are all explanatory. o Section 4.1 is a replacement section. - o Section 4.3 is an additional sections. + o Section 4.3 is an additional section. - o Section 4.3 is a replacement sections. + o Section 4.3 is a replacement section. o Section 4.4 is explanatory. o Section 4.5 is a replacement section. - o Sections 4.5.1 and 4.5.2 are additional sections. + o Sections 4.5.1 through 4.5.3, a total of three sections, are all + additional sections. - o Sections 4.5.3 through 4.5.5, a total of three sections, are all + o Sections 4.5.4 through 4.5.6, a total of three sections, are all replacement sections. - o Section 4.5.6 is an additional section. + o Section 4.5.7 is an additional section. o Section 5 is explanatory. o Sections 6 and 7 are additional sections. o Sections 8 through 8.9, a total of ten sections, are all replacement sections. o Sections 9 through 11.2, a total of eleven sections, are all additional sections. @@ -3340,21 +3431,21 @@ o Section 15 through Acknowledgments, a total of six sections, are all explanatory. To summarize: o There are fifteen explanatory sections. o There are twenty-two replacement sections. - o There are seventeen additional sections. + o There are eightteen additional sections. o There are three editing sections. Appendix B. Updates to RFC5661 In this appendix, we proceed through [RFC5661] identifying sections as unchanged, modified, deleted, or replaced and indicating where additional sections from the current document would appear in an eventual consolidated description of NFSv4.1. In this presentation, when section X is referred to, it denotes that section plus all @@ -3382,30 +3473,29 @@ 4.1 and 4.2 from the current document. o Section 11.1 is replaced by Section 4.3 from the current document. o Sections 11.2, 11.3, 11.3.1, and 11.3.2 are unchanged. o Section 11.4 is replaced by Section 4.5 from the current document. For details regarding subsections see below. - o New sections corresponding to Sections 4.5.1 and 4.5.2 from - the current document appear next. - - o Section 11.4.1 is replaced by Section 4.5.3 + o New sections corresponding to Sections 4.5.1 through 4.5.3 + from the current document appear next. - o Section 11.4.2 is replaced by Section 4.5.4 + o Section 11.4.1 is replaced by Section 4.5.4 - o Section 11.4.3 is replaced by Section 4.5.5 + o Section 11.4.2 is replaced by Section 4.5.5 - o A new section corresponding to Section 4.5.6 from the + o Section 11.4.3 is replaced by Section 4.5.6 + o A new section corresponding to Section 4.5.7 from the current document appears next. o Section 11.5 is to be deleted. o Section 11.6 is unchanged. o New sections corresponding to Sections 6 and 7 from the current document appear next. o Section 11.7 is replaced by Section 8 from the current @@ -3475,21 +3566,21 @@ o Sections outside Section 11. In this table, the counts for top-level sections and TOC entries are for sections including subsections while other counts are for sections exclusive of included subsections. +------------+------+------+--------+------------+--------+ | Status | Top | TOC | in 11 | not in 11 | Total | +------------+------+------+--------+------------+--------+ | Replaced | 0 | 3 | 17 | 7 | 24 | - | Added | 0 | 5 | 22 | 0 | 22 | + | Added | 0 | 6 | 23 | 0 | 23 | | Deleted | 0 | 1 | 4 | 0 | 4 | | Modified | 5 | 4 | 0 | 2 | 2 | | Unchanged | 18 | 212 | 16 | 918 | 934 | | in RFC5661 | 23 | 220 | 37 | 927 | 964 | +------------+------+------+--------+------------+--------+ Acknowledgments The authors wish to acknowledge the important role of Andy Adamson of Netapp in clarifying the need for trunking discovery functionality,