[Docs] [txt|pdf] [Tracker] [Email] [Nits]

Versions: 00 01 02 03 04 05 RFC 4373

Individual Submission to ldup working group                 R. Harrison
Internet Draft                                           J. Sermersheim
Document: draft-rharrison-lburp-00.txt                     Novell, Inc.
Category: Proposed Standard                               October, 1999


                 LDAP Bulk Update/Replication Protocol


Status of this Memo

   This document is an Internet-Draft and is in full conformance with
   all provisions of Section 10 of RFC2026.

   Internet-Drafts are working documents of the Internet Engineering
   Task Force (IETF), its areas, and its working groups. Note that
   other groups may also distribute working documents as Internet-
   Drafts. Internet-Drafts are draft documents valid for a maximum of
   six months and may be updated, replaced, or obsoleted by other
   documents at any time. It is inappropriate to use Internet- Drafts
   as reference material or to cite them other than as "work in
   progress."
   The list of current Internet-Drafts can be accessed at
   http://www.ietf.org/ietf/1id-abstracts.txt
   The list of Internet-Draft Shadow Directories can be accessed at
   http://www.ietf.org/shadow.html.


1. Abstract

   The protocol described in this document allows an LDAP client (a
   genuine client or an LDAP server acting as a client) to perform a
   bulk update to a replica on an LDAP server using standard LDAP
   update operations encapsulated within the LDUP Update Transfer
   Protocols described in [LDUP MODEL] and [LDUP RUP]. The protocol may
   be used to initialize all of the entries in an LDAP replica or to
   incrementally change the existing entries in an LDAP replica. It is
   suitable for client utilities that need to efficiently initialize a
   replica with many entries or efficiently make a substantial set of
   update changes to a replica. It is also suitable as a protocol for
   replication between a single master replica and its slave replicas.


2. Conventions used in this document

   The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
   "SHOULD", "SHOULD NOT", "RECOMMENDED",  "MAY", and "OPTIONAL" in
   this document are to be interpreted as described in RFC-2119
   [ReqsKeywords].

3. Motivation for protocol


            Individual Submission - Expires April 14, 2000
Harrison & Sermersheim                                               1
                LDAP Bulk Update/Replication Protocol   October, 1999


   This protocol arose from the need to allow LDAP clients to
   efficiently present large quantities of updates to an LDAP server
   and have the LDAP server efficiently process them. This protocol
   introduces a minimum of new operational functionality to the LDAP
   protocol since the updates discussed in this draft are done with
   standard LDAP [LDAPv3] update operations. However, this protocol
   greatly facilitates this process by allowing the server to recognize
   the clientÆs intent to perform a potentially large set of update
   operations and then to change its processing strategy to be more
   efficient than it otherwise could be.

   In effect, this protocol gives a hint to the server that the LDAP
   operations bracketed within it can be treated in a special way
   because they are related to each other. The server may then take
   actions that would not otherwise be practical to speed the
   processing of the updates. Examples of such actions include refusing
   to perform operations for other clients during the update sequence
   and grouping update operations.

   Additionally, this protocol deals with a common interoperability
   problem in this space caused by implementations having different
   requirements and abilities for handling linear and circular
   dependencies as entries are created.  A common application of this
   protocol is anticipated to be the initialization or update of an
   LDAP replica from a set of records represented in an LDIF [LDIF]
   file. Due to the abilities of various implementations, these files
   often contain ôout-of-orderö records where an entry is created
   before its parent or it is made a member of a group before the
   memberÆs entry has been created. Some implementations create
   temporary holding objects to deal with these issues, but others do
   not.  This protocol allows the server to reorder update operations
   in a limited way to deal with such cases.

4. Overview of protocol

   This replication update protocol utilizes standard LDAP update
   operations for transmitting replication information between client
   and server.  The client brackets a stream of asynchronous LDAP
   update operations between two LDAP v3 extended requests: the first
   notifies the server that the following stream of LDAP update
   operations are to be treated as a unit of replication information
   and the second marks the end of the replication stream.  This allows
   the LDAP server to make certain assumptions about the content of the
   LDAP requests in the replication stream that can be used to
   facilitate the rapid and efficient processing of the requests.

   Two styles of update--full and incremental--are defined.

   Full update creates a completely new set of entries in the target
   replica; any existing entries in that replica at the start of the
   replication operation are removed before the LDAP requests in the

            Individual Submission - Expires April 14, 2000
Harrison & Sermersheim                                               2
                LDAP Bulk Update/Replication Protocol   October, 1999


   replication stream are processed. The only LDAP operation allowed in
   a full update replication stream is add. After a server receives and
   acknowledges a full replication request, the LDAP server MUST not
   service LDAP requests other than those contained in the replication
   stream for the replica which is being initialized until the
   replication stream is completely processed.

   Incremental update performs a series of incremental changes to the
   replica; existing entries in the replica are only affected if
   explicitly modified in some way by the update operations contained
   in the replication stream. All LDAP v3 update operations defined in
   [LDAPv3]--add, modify, delete, and moddn--are allowed in the
   incremental replication stream.  When a server is processing an
   incremental replication request, the replica which is being modified
   MAY service LDAP requests received on connections other than the
   connection on which the replication stream is being sent.

   The LDAP server processing the replication update operation is
   responsible to send a synchronous extended operation response to the
   client, signaling the beginning of the replication sequence.  It
   sends an asynchronous response signifying the completion status of
   each LDAP operation in the replication stream. After the server has
   completed processing all update operations in the replication stream
   and has responded to each, it sends a final extended operation
   response to the client, which signals the end of the replication
   operation.

   No attempt is made to deal with the issues associated with multiple-
   master replicas or to maintain state information such as
   modification times of entries or attribute values in such a way that
   updates to the same entry on multiple master replicas can be
   correctly ordered.  For this reason convergence of data between all
   replicas can only be assured in a single-master replication
   environment.

5. High-level Description of Protocol Flow

   AUTHOR'S NOTE: This section is taken, almost verbatim, from section
   4 of [LDUP RUP].  It is expected that at some future point, the text
   in section 4 of [LDUP RUP] will be generalized in such a way that
   most or all of the overlapping text can be removed from this
   document.

   The following section provides a high-level overview of the
   replication protocol. Throughout this section, the client or server
   acting as supplier is indicated by the letter "S", and the server
   acting as consumer is indicated by the letter "C". The construct "S
   -> C" indicates that the supplier is sending an LDAPv3 operation to
   the consumer, and "C -> S" indicates that the consumer is sending an
   LDAPv3 operation to the supplier.


            Individual Submission - Expires April 14, 2000
Harrison & Sermersheim                                               3
                LDAP Bulk Update/Replication Protocol   October, 1999




5.1 Supplier-initiated replication protocol

         S -> C: LDAP bind operation (identity and credentials
                used are implementation-defined)

         C -> S: Bind response

         S -> C: StartReplicationRequest LDAPv3 extended
                 operation. The parameters are:

                   1) Root of replicated area (unambiguously
                      identifies the replicated area)

                   2) Supplier's replicaID

                   3) OID of incremental or full update replication
                      protocol as defined in this document.

                   4) The protocol initiation type - Supplier-Initiated

         C -> S: StartReplicationResponse LDAPv3 extended operation.
                 The parameters are:

                   1) A response code (see section 7)

                   2) An optional update vector that is included
                      if and only if the response code is REPL_SUCCESS.

   AUTHOR'S NOTE: The RUP protocol needs to be changed here to
   accommodate the Bulk Update/Replication Protocol.  The client will
   ignore any update vector sent, so I suggest making the update vector
   optional for this protocol as well as in cases of errors.

         S -> C: The supplier may send zero or more asynchronous LDAP
                 v3 update operations.

         C -> S: Upon completion (successful or otherwise) of each LDAP
                 v3 update operation, the consumer sends the
                 appropriate LDAP response for that operation as
                 defined in [LDAPv3].

         S -> C: After all required updates have been sent to the
                 consumer, the supplier sends an EndReplicationRequest
                 LDAPv3 extended operation.

         C -> S: The consumer responds by sending an
                 EndReplicationRequest LDAPv3 extended operation and
                 then closes the connection.



            Individual Submission - Expires April 14, 2000
Harrison & Sermersheim                                               4
                LDAP Bulk Update/Replication Protocol   October, 1999


5.2. Consumer-initiated replication protocol

   AUTHOR'S NOTE: This section has been copied from section 4.2 of
   [LDUP RUP]. References to "this document" in this section thus
   actually refer to [LDUP RUP]. It is anticipated that at some future
   point that this section will simply refer to section 4.2 of [LDUP
   RUP].

      C -> S: LDAP bind operation (identity and credentials
              used are implementation-defined)

      S -> C: Bind response

      C -> S: StartReplicationRequest LDAPv3 extended
              operation. The parameters are:

                1) Root of replicated area (unambiguously
                   identifies the replicated area)
                2) Consumer's replicaID
                3) OID of replication protocol to be used
                   (this document defines IETF-LDUP incremental
                   and IETF-LDUP total update protocols)
                4) The protocol initiation type - Consumer-Initiated
                   in this case

      S -> C: StartReplicationResponse LDAPv3 extended operation. The
              parameters are:

                1) A response code (see section 7)

      S -> C: The supplier server disconnects from the consumer server,
              and then connects to the consumer, beginning a Supplier-
              Initiated protocol session (see section 4.1).

5.3 Client-initiated replication protocol

   There are times when an LDAP client that doesn't have an outstanding
   replication agreement with a consumer server wants to initiate an
   update replication session as a supplier using this protocol.

   In this case, the following changes apply to the Supplier-initiated
   replication protocol:

   1) The client, acting as supplier, sends the special value of [value
   to be determined] as the supplier's replicaID in the
   StartReplicationRequest.

   2) The consumer, recognizing the supplier as a client, omits the
   update vector in the StartReplicationResponse, for every value of
   the response code, including REPL_SUCCESS.


            Individual Submission - Expires April 14, 2000
Harrison & Sermersheim                                               5
                LDAP Bulk Update/Replication Protocol   October, 1999




6. Elements of Protocol

   The LDAP Bulk Update/Replication protocol works within the framework
   of the Replication Update Protocol [LDUP RUP].  Unless otherwise
   stated, those elements of protocol taken from the Replication Update
   Protocol framework are to be used precisely as described there.

6.1 StartReplicationRequest Extended Operation

   The StartReplicationRequest Extended Operation is defined in Section
   5.1 of [LDUP RUP]. Clarification of the use of the replicaID is
   required for this protocol, as is the specification of the
   replicationProtocolOID.  Other parameters of the
   StartReplicationRequest are as defined in Section 5.1 of [LDUP RUP].

6.1.1 replicaID

   In the case where a replication agreement exists between the
   supplier and consumer, the replicaID is as specified in Section 5.1
   of [LDUP RUP]. When a client, acting as supplier, is initiating a
   replication request, it MUST send the special value of [value to be
   determined] as the supplier's replicaID in the
   StartReplicationRequest.

6.1.2 replicationProtocolOID

   The OID for the Full Update Protocol is
   2.16.840.1.113719.1.27.1.4.1.  The OID for the Incremental Update
   Protocol is 2.16.840.1.113719.1.27.1.4.2.

6.2 StartReplicationResponse

   In the case where a replication agreement exists between the
   supplier and consumer, the updateVector is as specified in Section
   5.1 of [LDUP RUP]. When a client, acting as supplier, has initiated
   a replication request, the updateVector SHALL NOT be returned in the
   StartReplicationResponse.

6.3 ReplicationUpdateOperation

   The ReplicationUpdateOperation is not utilized in the LDAP Bulk
   Update/Replication protocol.  Standard LDAP update requests as
   defined in sections 4.6, 4.7, 4.8, and 4.9 of [LDAPv3] are used to
   send all update requests to the consumer server.

6.4 EndReplicationRequest





            Individual Submission - Expires April 14, 2000
Harrison & Sermersheim                                               6
                LDAP Bulk Update/Replication Protocol   October, 1999


   The EndReplicationRequest is sent as specified in section 5.4 of
   [LDUP RUP]. The replicaUpdateVector MUST NOT be sent by the
   supplier, and the returnConsumerUpdateVector MUST be set to FALSE.

6.5 EndReplicationResponse

   The EndReplicat1ionResponse is sent as specified in section 5.4 of
   [LDUP RUP]. No replicaUpdateVector will be returned due to the
   requirement that the supplier not request it.

7. Semantics of Full and Incremental Update Protocols

   Since both the full and incremental update protocols use standard
   LDAP operations to transmit update information, no attempt is made
   by the protocol to include the information needed to support full
   multi-master replication as defined by [LDUP RUP].  Although entries
   and their associated attributes and attribute values can be
   synchronized using this protocol, no CSNs are included in the update
   stream; updates made using this replication protocol are treated the
   same as other LDAP operations wherein they are deemed to occur at
   the present.  Operational attributes such as modifiersTimeStamp MAY
   be included in the replication stream, and if they are, they SHOULD
   be put into the entry.

   The LDAP update operations that form the replication stream may be
   sent asynchronously by the supplier to the consumer. This means that
   the supplier need not wait for a response from one LDAP update
   operation before sending the next. Suppliers MUST NOT send the
   replication stream over multiple connections or do other things that
   could change the order of the operations in the replication
   sequence.

   The LDAP operations in the replication stream, plus the initial
   state of entries in the consumer replica in the case of incremental
   update, collectively represent the desired final state of the
   consumer replica. The consumer server may take any action required
   to efficiently achieve this desired final state of the replica.
   Examples of such actions include reordering LDAP requests to ensure
   that linear dependencies within the replication stream are correctly
   ordered, breaking an LDAP modification request into two or more
   separate modification requests to adequately resolve a circular
   dependency within the replication stream, and aggregating LDAP
   update operations to process them more efficiently.  When taking
   actions of this sort, the server MUST NOT reorder LDAP requests in
   such a way that the desired final state is different from that which
   would have been achieved before the action was taken.


7.1 Semantics of the Full Replication Protocol




            Individual Submission - Expires April 14, 2000
Harrison & Sermersheim                                               7
                LDAP Bulk Update/Replication Protocol   October, 1999


   Full replication creates a completely new set of entries in that
   replica; any existing entries in that replica at the start of the
   replication operation are removed before the LDAP requests in the
   replication stream are processed.

   The only LDAP operation allowed in a full replication stream is add.
   If any other LDAP operation is received as part of the replication
   stream, the server MUST return unwillingToPerform as the response
   for the operation.

   While a server is processing a full replication request, the
   consumer server MUST not service LDAP requests for the replica
   involved in the replication operation other than those contained in
   the replication stream including LDAP requests on other connections.
   This requirement is designed to allow implementers the freedom to
   implement highly-efficient methods of handling the replication
   stream without being constrained by the need to maintain a live,
   working database while doing so.

   If the full update sequence fails for some reason (e.g. a connection
   is abruptly broken in the middle of an otherwise-successful update
   sequence) the server MUST remove any entries created during the
   update sequence, thereby leaving the replica in a known state.


7.2 Semantics of the Incremental Replication Protocol


   Incremental replication performs a series of incremental changes to
   the replica; existing entries in the replica are only affected if
   explicitly modified in some way by the update operations contained
   in the replication stream.

   All LDAP v3 update operations--add, modify, delete, and moddn--are
   allowed in the incremental replication stream. If a non-update
   operation is received as part of the replication stream, the server
   MUST return unwillingToPerform as the response for the operation.

   When a server is processing an incremental replication request, the
   consumer server MAY service LDAP requests for the replica involved
   in the replication operation on connections other than that being
   used to transmit the replication stream.

8. Notes To Implementers

   It is RECOMMENDED that, if possible, the consumer LDAP server refer
   non-replication requests for the replica being updated to another
   server containing that replica during the time that consumer LDAP
   server has the replica off-line for replication.

   Implementations MAY choose to perform the operations in the
   replication stream with special permissions to improve performance.


            Individual Submission - Expires April 14, 2000
Harrison & Sermersheim                                               8
                LDAP Bulk Update/Replication Protocol   October, 1999



9. Security Considerations

   Implementations should ensure that a supplier making a Bulk
   Update/Replication request is bound with appropriate permissions.
   There is a potential for loss of data, especially with the full
   update protocol, which removes all entries in a replica as part of
   its operation. Second, forcing the removal of all entries in a
   replica may consume large amounts of server resources. Third, unlike
   other replication protocols, no existing replication agreement
   between supplier and consumer is required. These risks increase if
   the consumer server also processes the replication stream with
   special permissions to improve performance. For these reasons,
   implementers should carefully consider which identities or
   permissions are required to perform Bulk Update/Replication Protocol
   operations and take steps to ensure that only connections bound with
   appropriate permissions are allowed to perform them.

   The data contained in the replication stream may contain passwords
   and other sensitive data.  Care should be taken to properly
   safeguard this information while in transit between supplier and
   consumer.

   10. References

   [LDAPv3]
        Wahl, M., Howes, T., and S. Kille, "Lightweight Directory
        Access Protocol (v3)", RFC 2251, December 1997.

   [LDIF]
        Gordon Good, "LDAP Data Interchange Format (LDIF)", INTERNET-
        DRAFT: draft-good-ldap-ldif-04.txt, 22 June 1999.

   [LDUP MODEL]
        John Merrells, Ed Reed, and Uppili Srinivasan, "LDAP
        Replication Architecture", INTERNET-DRAFT: draft-ietf-ldup-
        model-01.txt, June 25, 1999.

   [LDUP RUP]
        Ellen Stokes and Gordon Good, "The LDUP Replication Update
        Protocol", UNPUBLISHED INTERNET-DRAFT: draft-ietf-ldup-
        protocol-00.txt.

   [ReqsKeywords]
        Scott Bradner. "Key Words for use in RFCs to Indicate
        Requirement Levels". RFC 2119.

11. Author's Addresses

   Roger Harrison
   Novell, Inc.

            Individual Submission - Expires April 14, 2000
Harrison & Sermersheim                                               9
                LDAP Bulk Update/Replication Protocol   October, 1999


   122 E. 1700 S.
   Provo, UT 84606
   +1 801 861 2642
   roger_harrison@novell.com

   Jim Sermersheim
   Novell, Inc.
   122 E. 1700 S.
   Provo, UT 84606
   +1 801 861 3088
   jimse@novell.com


Full Copyright Statement

   "Copyright (C) The Internet Society (date). All Rights Reserved.
   This document and translations of it may be copied and furnished to
   others, and derivative works that comment on or otherwise explain it
   or assist in its implmentation may be prepared, copied, published
   and distributed, in whole or in part, without restriction of any
   kind, provided that the above copyright notice and this paragraph
   are included on all such copies and derivative works. However, this
   document itself may not be modified in any way, such as by removing
   the copyright notice or references to the Internet Society or other
   Internet organizations, except as needed for the purpose of
   developing Internet standards in which case the procedures for
   copyrights defined in the Internet Standards process must be
   followed, or as required to translate it into languages other than
   English.

   The limited permissions granted above are perpetual and will not be
   revoked by the Internet Society or its successors or assigns.

   This document and the information contained herein is provided on an
   "AS IS" basis and THE INTERNET SOCIETY AND THE INTERNET ENGINEERING
   TASK FORCE DISCLAIMS ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING
   BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION
   HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF
   MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE.














            Individual Submission - Expires April 14, 2000
Harrison & Sermersheim                                              10


Html markup produced by rfcmarkup 1.109, available from https://tools.ietf.org/tools/rfcmarkup/