draft-ietf-regext-rdap-sorting-and-paging-00.txt   draft-ietf-regext-rdap-sorting-and-paging-01.txt 
Registration Protocols Extensions M. Loffredo Registration Protocols Extensions M. Loffredo
Internet-Draft M. Martinelli Internet-Draft M. Martinelli
Intended status: Standards Track IIT-CNR/Registro.it Intended status: Standards Track IIT-CNR/Registro.it
Expires: August 5, 2019 S. Hollenbeck Expires: October 13, 2019 S. Hollenbeck
Verisign Labs Verisign Labs
February 1, 2019 April 11, 2019
Registration Data Access Protocol (RDAP) Query Parameters for Result Registration Data Access Protocol (RDAP) Query Parameters for Result
Sorting and Paging Sorting and Paging
draft-ietf-regext-rdap-sorting-and-paging-00 draft-ietf-regext-rdap-sorting-and-paging-01
Abstract Abstract
The Registration Data Access Protocol (RDAP) does not include core The Registration Data Access Protocol (RDAP) does not include core
functionality for clients to provide sorting and paging parameters functionality for clients to provide sorting and paging parameters
for control of large result sets. This omission can lead to for control of large result sets. This omission can lead to
unpredictable server processing of queries and client processing of unpredictable server processing of queries and client processing of
responses. This unpredictability can be greatly reduced if clients responses. This unpredictability can be greatly reduced if clients
can provide servers with their preferences for managing response can provide servers with their preferences for managing response
values. This document describes RDAP query extensions that allow values. This document describes RDAP query extensions that allow
skipping to change at page 1, line 41 skipping to change at page 1, line 41
Internet-Drafts are working documents of the Internet Engineering Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF). Note that other groups may also distribute Task Force (IETF). Note that other groups may also distribute
working documents as Internet-Drafts. The list of current Internet- working documents as Internet-Drafts. The list of current Internet-
Drafts is at https://datatracker.ietf.org/drafts/current/. Drafts is at https://datatracker.ietf.org/drafts/current/.
Internet-Drafts are draft documents valid for a maximum of six months Internet-Drafts are draft documents valid for a maximum of six months
and may be updated, replaced, or obsoleted by other documents at any and may be updated, replaced, or obsoleted by other documents at any
time. It is inappropriate to use Internet-Drafts as reference time. It is inappropriate to use Internet-Drafts as reference
material or to cite them other than as "work in progress." material or to cite them other than as "work in progress."
This Internet-Draft will expire on August 5, 2019. This Internet-Draft will expire on October 13, 2019.
Copyright Notice Copyright Notice
Copyright (c) 2019 IETF Trust and the persons identified as the Copyright (c) 2019 IETF Trust and the persons identified as the
document authors. All rights reserved. document authors. All rights reserved.
This document is subject to BCP 78 and the IETF Trust's Legal This document is subject to BCP 78 and the IETF Trust's Legal
Provisions Relating to IETF Documents Provisions Relating to IETF Documents
(https://trustee.ietf.org/license-info) in effect on the date of (https://trustee.ietf.org/license-info) in effect on the date of
publication of this document. Please review these documents publication of this document. Please review these documents
skipping to change at page 2, line 22 skipping to change at page 2, line 22
Table of Contents Table of Contents
1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 2 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 2
1.1. Conventions Used in This Document . . . . . . . . . . . . 4 1.1. Conventions Used in This Document . . . . . . . . . . . . 4
2. RDAP Query Parameter Specification . . . . . . . . . . . . . 4 2. RDAP Query Parameter Specification . . . . . . . . . . . . . 4
2.1. Sorting and Paging Metadata . . . . . . . . . . . . . . . 4 2.1. Sorting and Paging Metadata . . . . . . . . . . . . . . . 4
2.2. "count" Parameter . . . . . . . . . . . . . . . . . . . . 6 2.2. "count" Parameter . . . . . . . . . . . . . . . . . . . . 6
2.3. "sort" Parameter . . . . . . . . . . . . . . . . . . . . 7 2.3. "sort" Parameter . . . . . . . . . . . . . . . . . . . . 7
2.3.1. Representing Sorting Links . . . . . . . . . . . . . 11 2.3.1. Representing Sorting Links . . . . . . . . . . . . . 11
2.4. "limit" and "offset" Parameters . . . . . . . . . . . . . 12 2.4. "limit" and "offset" Parameters . . . . . . . . . . . . . 12
2.4.1. Representing Paging Links . . . . . . . . . . . . . . 13 2.4.1. Representing Offset Pagination Links . . . . . . . . 13
3. Negative Answers . . . . . . . . . . . . . . . . . . . . . . 14 2.5. "cursor" Parameter . . . . . . . . . . . . . . . . . . . 14
4. RDAP Conformance . . . . . . . . . . . . . . . . . . . . . . 15 2.5.1. Representing Cursor Pagination Links . . . . . . . . 16
5. Implementation Considerations . . . . . . . . . . . . . . . . 15 3. Negative Answers . . . . . . . . . . . . . . . . . . . . . . 17
5.1. Considerations about Paging Implementation . . . . . . . 16 4. RDAP Conformance . . . . . . . . . . . . . . . . . . . . . . 18
5. Implementation Considerations . . . . . . . . . . . . . . . . 18
6. Implementation Status . . . . . . . . . . . . . . . . . . . . 19 6. Implementation Status . . . . . . . . . . . . . . . . . . . . 19
6.1. IIT-CNR/Registro.it . . . . . . . . . . . . . . . . . . . 19 6.1. IIT-CNR/Registro.it . . . . . . . . . . . . . . . . . . . 20
6.2. Google Registry . . . . . . . . . . . . . . . . . . . . . 19 6.2. Google Registry . . . . . . . . . . . . . . . . . . . . . 20
7. Security Considerations . . . . . . . . . . . . . . . . . . . 20 7. Security Considerations . . . . . . . . . . . . . . . . . . . 20
8. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 20 8. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 21
9. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 20 9. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 21
10. References . . . . . . . . . . . . . . . . . . . . . . . . . 20 10. References . . . . . . . . . . . . . . . . . . . . . . . . . 21
10.1. Normative References . . . . . . . . . . . . . . . . . . 21 10.1. Normative References . . . . . . . . . . . . . . . . . . 21
10.2. Informative References . . . . . . . . . . . . . . . . . 22 10.2. Informative References . . . . . . . . . . . . . . . . . 22
Appendix A. Change Log . . . . . . . . . . . . . . . . . . . . . 24 Appendix A. Change Log . . . . . . . . . . . . . . . . . . . . . 25
Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 24 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 25
1. Introduction 1. Introduction
The availability of functionality for result sorting and paging The availability of functionality for result sorting and paging
provides benefits to both clients and servers in the implementation provides benefits to both clients and servers in the implementation
of RESTful services [REST]. These benefits include: of RESTful services [REST]. These benefits include:
o reducing the server response bandwidth requirements; o reducing the server response bandwidth requirements;
o improving server response time; o improving server response time;
o improving query precision and, consequently, obtaining more o improving query precision and, consequently, obtaining more
skipping to change at page 3, line 49 skipping to change at page 3, line 49
set. Therefore, the client could traverse the whole result set to set. Therefore, the client could traverse the whole result set to
find the relevant objects or, due to truncation, could not find them find the relevant objects or, due to truncation, could not find them
at all. at all.
The protocol described in this specification extends RDAP query The protocol described in this specification extends RDAP query
capabilities to enable result sorting and paging, by adding new query capabilities to enable result sorting and paging, by adding new query
parameters that can be applied to RDAP search path segments. The parameters that can be applied to RDAP search path segments. The
service is implemented using the Hypertext Transfer Protocol (HTTP) service is implemented using the Hypertext Transfer Protocol (HTTP)
[RFC7230] and the conventions described in RFC 7480 [RFC7480]. [RFC7230] and the conventions described in RFC 7480 [RFC7480].
The implementation of these parameters is technically feasible, as The implementation of the new parameters is technically feasible, as
operators for counting, sorting and paging rows are currently operators for counting, sorting and paging rows are currently
supported by the major RDBMSs. supported by the major RDBMSs.
1.1. Conventions Used in This Document 1.1. Conventions Used in This Document
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
"SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
document are to be interpreted as described in [RFC2119]. document are to be interpreted as described in [RFC2119].
2. RDAP Query Parameter Specification 2. RDAP Query Parameter Specification
skipping to change at page 4, line 24 skipping to change at page 4, line 24
defined in RFC 7482 [RFC7482]. They are as follows: defined in RFC 7482 [RFC7482]. They are as follows:
o "count": a boolean value that allows a client to request the total o "count": a boolean value that allows a client to request the total
number of objects found (that due to truncation can be different number of objects found (that due to truncation can be different
from the number of returned objects); from the number of returned objects);
o "sort": a string value that allows a client to request a specific o "sort": a string value that allows a client to request a specific
sort order for the result set; sort order for the result set;
o "limit" and "offset": numeric values that allow a client to o "limit" and "offset": numeric values that allow a client to
request a specific portion of the entire result set. request a specific portion of the entire result set;
o "cursor": a string value representing a pointer to a specific
fixed size portion of the result set. It is alternate to "limit"
and "offset".
Augmented Backus-Naur Form (ABNF) [RFC5234] is used in the following Augmented Backus-Naur Form (ABNF) [RFC5234] is used in the following
sections to describe the formal syntax of these new parameters. sections to describe the formal syntax of these new parameters.
2.1. Sorting and Paging Metadata 2.1. Sorting and Paging Metadata
According to most advanced principles in REST design, collectively According to most advanced principles in REST design, collectively
known as HATEOAS (Hypermedia as the Engine of Application State) known as HATEOAS (Hypermedia as the Engine of Application State)
([HATEOAS]), a client entering a REST application through an initial ([HATEOAS]), a client entering a REST application through an initial
URI should use the server-provided links to dynamically discover URI should use the server-provided links to dynamically discover
available actions and access the resources it needs. In this way, available actions and access the resources it needs. In this way,
the client is not requested to have prior knowledge of the service the client is not requested to have prior knowledge of the service
and, consequently, to hard code the URIs of different resources. and, consequently, to hard code the URIs of different resources.
This would allow the server to make URI changes as the API evolves This would allow the server to make URI changes as the API evolves
without breaking the clients. Definitively, a REST service should be without breaking the clients. Definitively, a REST service should be
as self-descriptive as possible. as self-descriptive as possible.
Therefore, the implementation of the query parameters described in Therefore, servers implementing the query parameters described in
this specification recommends servers to provide additional this specification SHOULD provide additional information in their
information in their responses about both the available sorting responses about both the available sorting criteria and the possible
criteria and the possible pagination. Such information is collected pagination. Such information is collected in two new data structures
in two new data structures named, respectively, "sorting_metadata" named, respectively, "sorting_metadata" and "paging_metadata".
and "paging_metadata".
Obviously, both the new data structures are OPTIONAL because their Obviously, both the new data structures are OPTIONAL because their
presence in the response not only depends on the implementation of presence in the response not only depends on the implementation of
sorting and paging query capabilities but also on some situations sorting and paging query capabilities but also on some situations
related to the results. For example, it is quite natural to expect related to the results. For example, it is quite natural to expect
that the "paging_metadata" section will not be present at the last that the "paging_metadata" element will not be present at the last
result page when the server implements only the forward pagination. result page when the server implements only the forward pagination.
The "sorting_metadata" structure contains the following fields: The "sorting_metadata" structure contains the following properties:
o "currentSort": the value of sort parameter as specified in the o "currentSort": "String" (OPTIONAL) either the value of sort
query string; "parameter" as specified in the query string or the sort applied
by default, if any;
o "availableSorts": an array of objects each one describing an o "availableSorts": "AvailableSort[]" (OPTIONAL) an array of objects
available sorting criterion: each one describing an alternate available sorting criterion.
Members are:
* "property": the name that can be used by the client to request * "property": "String" (REQUIRED) the name that can be used by
the sorting criterion; the client to request the sorting criterion;
* "jsonPath": the JSON Path of the RDAP field corresponding to * "default": "Boolean" (REQUIRED) whether the sorting criterion
the property; is applied by default;
* "default": whether the sorting criterion is applied by default; * "jsonPath": "String" (OPTIONAL) the JSON Path of the RDAP field
* "links": an array of links as described in RFC 8288 [RFC8288] corresponding to the property;
containing the query string that applies the sorting criterion. * "links": "Link[]" (OPTIONAL) an array of links as described in
RFC 8288 [RFC8288] containing the query string that applies the
sorting criterion.
Both "currentSort" and "availableSorts" are OPTIONAL fields of the At least one between "currentSort" and "availableSorts" MUST be
"sorting_metadata" structure. In particular, the "currentSort" field present.
is provided when the query string contains a valid value for sort
parameter, while the "availableSorts" field SHOULD be provided when
the sort parameter is missing in the query string or when it is
present and the server implements more than a sorting criterion for
the RDAP object. At least the "property" field is REQUIRED in each
item of the "availableSorts" array while the other fields are
RECOMMENDED.
The "paging_metadata" structure contains the following fields: The "paging_metadata" structure contains the following fields:
o "totalCount": a numeric value representing the total number of o "totalCount": "Numeric" (OPTIONAL) a numeric value representing
objects found; the total number of objects found. It is provided if the query
string contains the "count" parameter;
o "pageCount": a numeric value representing the number of objects
returned in the current page;
o "offset": a numeric value identifying the start of current page in
the result set;
o "nextOffset": a numeric value identifying the start of the next o "pageCount": "Numeric" (OPTIONAL) a numeric value representing the
page in the result set or null if the result set has been number of objects returned in the current page. It is provided
completely scrolled; when the total number of objects exceeds the page size. This
property is redundant for clients because the page size can be
derived from the length of the search results array but it can be
helpful if the end user interacts with the server through a web
browser;
o "links": an array of links as described in RFC 8288 [RFC8288] o "links": "Link[]" (OPTIONAL) an array of links as described in RFC
containing the reference to next page. 8288 [RFC8288] containing the reference to next page. In this
specification, only the forward pagination is dealt because it is
considered satisfactory in order to traverse the result set.
Only the "pageCount" field is REQUIRED in the "paging_metadata" Examples of additional references are to: the previous page, the
structure. The other fields appear when pagination occurs. In this first page, the last page.
specification, only the forward pagination is dealt because it is
considered satisfactory in order to traverse the result set. If a
server should also implement backward pagination, an appropriate
field (e.g. "prevOffset") identifying the start of the previous page
is RECOMMENDED. Finally, the "totalCount" field is provided if the
query string contains the count parameter.
FOR DISCUSSION: Should the metadata described in this specification At least one between "totalCount" and "links" MUST be present.
be part of a more general "metadata" section including other contents
(e.g rate limits, information about the server, information about the
response, metadata information related to other parameters)?
2.2. "count" Parameter 2.2. "count" Parameter
Currently the RDAP protocol does not allow a client to determine the Currently the RDAP protocol does not allow a client to determine the
total number of the results in a query response when the result set total number of the results in a query response when the result set
is truncated. This is rather inefficient because the user cannot is truncated. This is rather inefficient because the user cannot
evaluate the query precision and, at the same time, cannot receive evaluate the query precision and, at the same time, cannot receive
information that could be relevant. information that could be relevant.
The count parameter provides additional functionality (Figure 1) that The "count" parameter provides additional functionality (Figure 1)
allows a client to request information from the server that specifies that allows a client to request information from the server that
the total number of elements matching a particular search pattern. specifies the total number of elements matching a particular search
pattern.
https://example.com/rdap/domains?name=*nr.com&count=true https://example.com/rdap/domains?name=*nr.com&count=true
Figure 1: Example of RDAP query reporting the count parameter Figure 1: Example of RDAP query reporting the "count" parameter
The ABNF syntax is the following: The ABNF syntax is the following:
count = "count" EQ ( trueValue / falseValue ) count = "count" EQ ( trueValue / falseValue )
trueValue = ("true" / "yes" / "1") trueValue = ("true" / "yes" / "1")
falseValue = ("false" / "no" / "0") falseValue = ("false" / "no" / "0")
EQ = "=" EQ = "="
A trueValue means that the server MUST provide the total number of A trueValue means that the server MUST provide the total number of
the objects in the "totalCount" field of the "paging_metadata" the objects in the "totalCount" field of the "paging_metadata"
section (Figure 2). A falseValue means that the server MUST NOT element (Figure 2). A falseValue means that the server MUST NOT
provide this number. provide this number.
{ {
"rdapConformance": [ "rdapConformance": [
"rdap_level_0", "rdap_level_0",
"paging_level_0" "paging_level_0"
], ],
... ...
"paging_metadata": { "paging_metadata": {
"totalCount": 73 "totalCount": 73
}, },
"domainSearchResults": [ "domainSearchResults": [
... ...
] ]
} }
Figure 2: Example of RDAP response with "paging_metadata" section Figure 2: Example of RDAP response with "paging_metadata" element
containing the "totalCount" field containing the "totalCount" field
2.3. "sort" Parameter 2.3. "sort" Parameter
The RDAP protocol does not provide any capability to specify results The RDAP protocol does not provide any capability to specify results
sort criteria. A server could implement a default sorting scheme sort criteria. A server could implement a default sorting scheme
according to the object class, but this feature is not mandatory and according to the object class, but this feature is not mandatory and
might not meet user requirements. Sorting can be addressed by the might not meet user requirements. Sorting can be addressed by the
client, but this solution is rather inefficient. Sorting and paging client, but this solution is rather inefficient. Sorting and paging
features provided by the RDAP server could help avoid truncation of features provided by the RDAP server could help avoid truncation of
relevant results and allow for scrolling the result set using relevant results and allow for scrolling the result set using
subsequent queries. subsequent queries.
The sort parameter allows the client to ask the server to sort the The "sort" parameter allows the client to ask the server to sort the
results according to the values of one or more properties and results according to the values of one or more properties and
according to the sort direction of each property. The ABNF syntax is according to the sort direction of each property. The ABNF syntax is
the following: the following:
sort = "sort" EQ sortItem *( "," sortItem ) sort = "sort" EQ sortItem *( "," sortItem )
sortItem = property-ref [":" ( "a" / "d" ) ] sortItem = property-ref [":" ( "a" / "d" ) ]
"a" means that the ascending sort MUST be applied, "d" means that the "a" means that the ascending sort MUST be applied, "d" means that the
descending sort MUST be applied. If the sort direction is absent, an descending sort MUST be applied. If the sort direction is absent, an
ascending sort MUST be applied (Figure 3). ascending sort MUST be applied (Figure 3).
In the sort parameter ABNF syntax, property-ref represents a In the "sort" parameter ABNF syntax, property-ref represents a
reference to a property of an RDAP object. Such a reference could be reference to a property of an RDAP object. Such a reference could be
expressed by using a JSON Path. The JSON Path in a JSON document expressed by using a JSON Path. The JSON Path in a JSON document
[RFC8259] is equivalent to the XPath [W3C.CR-xpath-31-20161213] in a [RFC8259] is equivalent to the XPath [W3C.CR-xpath-31-20161213] in a
XML document. For example, the JSON Path to select the value of the XML document. For example, the JSON Path to select the value of the
ASCII name inside an RDAP domain object is "$.ldhName", where $ ASCII name inside an RDAP domain object is "$.ldhName", where $
identifies the root of the document (DOM). Another way to select a identifies the root of the document (DOM). Another way to select a
value inside a JSON document is the JSON Pointer [RFC6901]. While value inside a JSON document is the JSON Pointer [RFC6901]. While
JSON Path or JSON Pointer are both standard ways to select any value JSON Path or JSON Pointer are both standard ways to select any value
inside JSON data, neither is particularly easy to use (e.g. inside JSON data, neither is particularly easy to use (e.g.
"$.events[?(@.eventAction='registration')].eventDate" is the JSON "$.events[?(@.eventAction='registration')].eventDate" is the JSON
Path expression of the registration date in a RDAP domain object). Path expression of the registration date in an RDAP domain object).
Therefore, this specification provides a definition of property-ref Therefore, this specification provides a definition of property-ref
in terms of RDAP properties. However, not all the RDAP properties in terms of RDAP properties. However, not all the RDAP properties
are suitable to be used in sort criteria, such as: are suitable to be used in sort criteria, such as:
o properties providing service information (e.g. links, notices, o properties providing service information (e.g. links, notices,
remarks, etc.); remarks, etc.);
o multivalued properties (e.g. status, roles, variants, etc.); o multivalued properties (e.g. status, roles, variants, etc.);
o properties modeling relationships to other objects (e.g. o properties modeling relationships to other objects (e.g.
entities). entities).
On the contrary, some properties expressed as values of other On the contrary, some properties expressed as values of other
properties (e.g. registration date) could be used in such a context. properties (e.g. registration date) could be used in such a context.
In the following, a list of the proposed properties for sort criteria In the following, a list of properties an RDAP server MAY implement
is presented. The properties are divided in two groups: object is presented. The properties are divided in two groups: object
common properties and object specific properties. common properties and object specific properties.
o Object common properties. Object common properties are derived o Object common properties. Object common properties are derived
from the merge of the "eventAction" and the "eventDate" from the merge of the "eventAction" and the "eventDate"
properties. The following values of the sort parameter are properties. The following values of the "sort" parameter are
defined: defined:
* registrationDate * registrationDate
* reregistrationDate * reregistrationDate
* lastChangedDate * lastChangedDate
* expirationDate * expirationDate
* deletionDate * deletionDate
* reinstantiationDate * reinstantiationDate
* transferDate * transferDate
* lockedDate * lockedDate
skipping to change at page 11, line 17 skipping to change at page 11, line 17
| | email | $.entitySearchResults[*].vcardArray[1][?(@[ | | | email | $.entitySearchResults[*].vcardArray[1][?(@[ |
| | | 0]=="email")][3] | | | | 0]=="email")][3] |
| | country | $.entitySearchResults[*].vcardArray[1][?(@[ | | | country | $.entitySearchResults[*].vcardArray[1][?(@[ |
| | | 0]=="adr")][3][6] | | | | 0]=="adr")][3][6] |
| | city | $.entitySearchResults[*].vcardArray[1][?(@[ | | | city | $.entitySearchResults[*].vcardArray[1][?(@[ |
| | | 0]=="adr")][3][3] | | | | 0]=="adr")][3][3] |
+-------+-------------+---------------------------------------------+ +-------+-------------+---------------------------------------------+
Table 2: Sorting properties - JSON Path Mapping Table 2: Sorting properties - JSON Path Mapping
If the sort parameter reports an allowed sorting property, it MUST be If the "sort" parameter reports an allowed sorting property, it MUST
provided in the "currentSort" field of the "sorting_metadata" be provided in the "currentSort" field of the "sorting_metadata"
structure. structure.
https://example.com/rdap/domains?name=*nr.com&sort=ldhName https://example.com/rdap/domains?name=*nr.com&sort=ldhName
https://example.com/rdap/domains?name=*nr.com&sort=registrationDate:d https://example.com/rdap/domains?name=*nr.com&sort=registrationDate:d
https://example.com/rdap/domains?name=*nr.com&sort=lockedDate,ldhName https://example.com/rdap/domains?name=*nr.com&sort=lockedDate,ldhName
Figure 3: Examples of RDAP query reporting the sort parameter Figure 3: Examples of RDAP query reporting the "sort" parameter
2.3.1. Representing Sorting Links 2.3.1. Representing Sorting Links
An RDAP server MAY use the "links" array of the "sorting_metadata" An RDAP server MAY use the "links" array of the "sorting_metadata"
section to provide ready-made references [RFC8288] to the available section to provide ready-made references [RFC8288] to the available
sort criteria (Figure 4). Each link represents a reference to an sort criteria (Figure 4). Each link represents a reference to an
alternate view of the results. alternate view of the results.
{ {
"rdapConformance": [ "rdapConformance": [
skipping to change at page 13, line 32 skipping to change at page 13, line 32
return, respectively, the first 5 objects, the set of objects return, respectively, the first 5 objects, the set of objects
starting from position 6, and first 5 objects starting from position starting from position 6, and first 5 objects starting from position
11 of the result set (Figure 5). 11 of the result set (Figure 5).
https://example.com/rdap/domains?name=*nr.com&limit=5 https://example.com/rdap/domains?name=*nr.com&limit=5
https://example.com/rdap/domains?name=*nr.com&offset=5 https://example.com/rdap/domains?name=*nr.com&offset=5
https://example.com/rdap/domains?name=*nr.com&limit=5&offset=10 https://example.com/rdap/domains?name=*nr.com&limit=5&offset=10
Figure 5: Examples of RDAP query reporting the limit and offset Figure 5: Examples of RDAP query reporting the "limit" and "offset"
parameters parameters
2.4.1. Representing Paging Links 2.4.1. Representing Offset Pagination Links
An RDAP server MAY use the "links" array of the "paging_metadata" An RDAP server SHOULD use the "links" array of the "paging_metadata"
section to provide a ready-made reference [RFC8288] to the next page element to provide a ready-made reference [RFC8288] to the next page
of the result set (Figure 6). Examples of additional "rel" values of the result set (Figure 6). Examples of additional "rel" values a
are "first", "last", "prev". server MAY implements are "first", "last", "prev".
{ {
"rdapConformance": [ "rdapConformance": [
"rdap_level_0", "rdap_level_0",
"paging_level_0" "paging_level_0"
], ],
... ...
"notices": [ "notices": [
{ {
"title": "Search query limits", "title": "Search query limits",
"type": "result set truncated due to excessive load", "type": "result set truncated due to excessive load",
"description": [ "description": [
"search results for domains are limited to 10" "search results for domains are limited to 10"
] ]
} }
], ],
"paging_metadata": { "paging_metadata": {
"totalCount": 73, "totalCount": 73,
"pageCount": 10, "pageCount": 10,
"offset": 10,
"nextOffset": 20,
"links": [ "links": [
{ {
"value": "https://example.com/rdap/domains?name=*nr.com", "value": "https://example.com/rdap/domains?name=*nr.com",
"rel": "next", "rel": "next",
"href": "https://example.com/rdap/domains?name=*nr.com&limit=10 "href": "https://example.com/rdap/domains?name=*nr.com&limit=10
&offset=10", &offset=10",
"title": "Result Pagination Link", "title": "Result Pagination Link",
"type": "application/rdap+json" "type": "application/rdap+json"
} }
] ]
}, },
"domainSearchResults": [ "domainSearchResults": [
... ...
] ]
} }
Figure 6: Example of a "paging_metadata" instance to implement result Figure 6: Example of a "paging_metadata" instance to implement result
pagination based on offset and limit pagination based on offset and limit
2.5. "cursor" Parameter
The use of limit and offset operators represents the most common way
to implement results pagination. However, when offset has a very
high value, scrolling the result set could take some time. In
addition, offset pagination may return inconsistent pages when data
are frequently updated (i.e. real-time data) but this is not the case
of registration data. An alternative approach to offset pagination
is the keyset pagination, a.k.a. seek-method [SEEK] or cursor
pagination. This method has been taken as the basis for the
implementation of a "cursor" parameter [CURSOR] by some REST API
providers (e.g. [CURSOR-API1],[CURSOR-API2]). The cursor is an
opaque URL-safe string representing a logical pointer to the first
result of the next page (Figure 7).
https://example.com/rdap/domains?name=*nr.com&cursor=wJlCDLIl6KTWypN7T6vc6nWEmEYe99Hjf1XY1xmqV-M=
Figure 7: An example of RDAP query reporting the "cursor" parameter
However cursor pagination raises some drawbacks with respect to
offset pagination:
o it needs at least one key field;
o it does not allow to sort by any field and paginate the results
because sorting has to be made on the key field;
o it does not allow to skip pages because they have to be scrolled
in sequential order starting from the initial page;
o it makes very hard the navigation of the result set in both
directions because all comparison and sort operations have to be
reversed.
Furthermore, in the RDAP context, some additional considerations can
be made:
o an RDAP object is a conceptual aggregation of information
collected from more than one data structure (e.g. table) and this
makes even harder for the developers the implementation of the
seek-method that is already quite difficult. In fact, for
example, the entity object can gather information from different
data structures (registrars, registrants, contacts, resellers, and
so on), each one with its own key field mapping the RDAP entity
handle;
o depending on the number of the page results as well as the number
and the complexity of the properties of each RDAP object in the
response, the time required by offset pagination to skip the
previous pages could be much faster than the processing time
needed to build the current page. In fact, RDAP objects are
usually formed by information belonging to multiple data
structures and containing multivalued properties (i.e. arrays)
and, therefore, data selection is a time consuming process. This
situation occurs even though the data selection process makes use
of indexes;
o depending on the access levels defined by each RDAP operator, the
increase of complexity and the decrease of flexibility of cursor
pagination with respect to the offset pagination could be
considered impractical.
Finally, the cursor pagination seems to be a bit inconsistent with
some capabilities described in this document like sorting and the
implementation of paging links other than the usual "next page" link.
But, while the provisioning of more paging links can be superfluous,
dropping the sorting capability seems quite unreasonable.
Ultimately, both pagination methods have benefits and drawbacks.
Therefore, since the "paging_metadata" element keeps the same
structure and clients merely make use of the paging links as they are
provided, this RDAP specification lets operators to implement a
method according to their needs, the user access levels, the
submitted queries.
2.5.1. Representing Cursor Pagination Links
The same considerations made for offset pagination links are applied
to cursor pagination links (Figure 8).
{
"rdapConformance": [
"rdap_level_0",
"paging_level_0"
],
...
"notices": [
{
"title": "Search query limits",
"type": "result set truncated due to excessive load",
"description": [
"search results for domains are limited to 10"
]
}
],
"paging_metadata": {
"totalCount": 73,
"pageCount": 10,
"links": [
{
"value": "https://example.com/rdap/domains?name=*nr.com",
"rel": "next",
"href": "https://example.com/rdap/domains?name=*nr.com
&cursor=wJlCDLIl6KTWypN7T6vc6nWEmEYe99Hjf1XY1xmqV-M=",
"title": "Result Pagination Link",
"type": "application/rdap+json"
}
]
},
"domainSearchResults": [
...
]
}
Figure 8: Example of a "paging_metadata" instance to implement cursor
pagination
3. Negative Answers 3. Negative Answers
The value constraints for the parameters are defined by their ABNF The value constraints for the parameters are defined by their ABNF
syntax. Therefore, each request providing an invalid value for a syntax. Therefore, each request providing an invalid value for a
parameter SHOULD obtain an HTTP 400 (Bad Request) response code. The parameter SHOULD obtain an HTTP 400 (Bad Request) response code. The
same response SHOULD be returned if the client provides an same response SHOULD be returned if the client provides an
unsupported value for the sort parameter in both single and multi unsupported value for the "sort" parameter in both single and multi
sort. sort.
The server can provide a different response when it supports the The server can provide a different response when it supports the
limit and/or offset parameters and the client submits values that are "limit" and/or "offset" parameters and the client submits values that
out of the valid ranges. The possible cases are: are out of the valid ranges. The possible cases are:
o If the client submits a value for the limit parameter that is o If the client submits a value for the "limit" parameter that is
greater than the number of objects to be processed, it is greater than the number of objects to be processed, it is
RECOMMENDED that server returns a response including only the RECOMMENDED that server returns a response including only the
processed objects. processed objects.
o If the client submits a value for the offset parameter that is o If the client submits a value for the "offset" parameter that is
greater than the number of objects to be processed, it is greater than the number of objects to be processed as well as an
RECOMMENDED that server returns an HTTP 404 (Not Found) response invalid value for the "cursor" parameter, it is RECOMMENDED that
code. server returns an HTTP 404 (Not Found) response code.
Optionally, the response MAY include additional information regarding Optionally, the response MAY include additional information regarding
the negative answer in the HTTP entity body. the negative answer in the HTTP entity body.
4. RDAP Conformance 4. RDAP Conformance
Servers returning the "paging_metadata" section in their responses Servers returning the "paging_metadata" element in their responses
MUST include "paging_level_0" in the rdapConformance array as well as MUST include "paging_level_0" in the rdapConformance array as well as
servers returning the "sorting_metadata" section MUST include servers returning the "sorting_metadata" element MUST include
"sorting_level_0". "sorting_level_0".
5. Implementation Considerations 5. Implementation Considerations
The implementation of the new parameters is technically feasible, as The implementation of the new parameters is technically feasible, as
operators for counting, sorting and paging are currently supported by operators for counting, sorting and paging are currently supported by
the major RDBMSs. the major RDBMSs.
In the following, the match between the new defined parameters and In the following, the match between the new defined parameters and
the SQL operators is shown (Table 3): the SQL operators is shown (Table 3):
skipping to change at page 16, line 31 skipping to change at page 19, line 31
| offset | offset m (in Postgres) | | offset | offset m (in Postgres) |
| | OFFSET m ROWS (in Oracle) | | | OFFSET m ROWS (in Oracle) |
| | | | | |
| limit + offset | limit n offset m (in MySql and Postgres) | | limit + offset | limit n offset m (in MySql and Postgres) |
| | OFFSET m ROWS FETCH NEXT n ROWS ONLY (in Oracle) | | | OFFSET m ROWS FETCH NEXT n ROWS ONLY (in Oracle) |
+----------------+--------------------------------------------------+ +----------------+--------------------------------------------------+
Table 3: New query parameters vs. SQL operators Table 3: New query parameters vs. SQL operators
With regard to Oracle, Table 3 reports only one of the three methods With regard to Oracle, Table 3 reports only one of the three methods
that can be used to implement limit and offset parameters. The that can be used to implement "limit" and "offset" parameters. The
others are described in [ORACLE-ROWNUM] and [ORACLE-ROW-NUMBER]. others are described in [ORACLE-ROWNUM] and [ORACLE-ROW-NUMBER].
In addition, similar operators are completely or partially supported In addition, similar operators are completely or partially supported
by the most known NoSQL databases (MongoDB, CouchDB, HBase, by the most known NoSQL databases (MongoDB, CouchDB, HBase,
Cassandra, Hadoop) so the implementation of the new parameters seems Cassandra, Hadoop) so the implementation of the new parameters seems
to be practicable by servers working without the use of an RDBMS. to be practicable by servers working without the use of an RDBMS.
5.1. Considerations about Paging Implementation
The use of limit and offset operators represents the most common way
to implement results pagination. However, when offset has a high
value, scrolling the result set could take some time. In addition,
offset pagination may return inconsistent pages when data are
frequently updated (i.e. real-time data) but this is not the case of
registration data. An alternative approach to offset pagination is
the keyset pagination, a.k.a. seek-method [SEEK] or cursor based
pagination. This method has been taken as the basis for the
implementation of a cursor parameter [CURSOR] by some REST API
providers (e.g. [CURSOR-API1],[CURSOR-API2]). The cursor parameter
is an opaque URL-safe string representing a logical pointer to the
first result of the next page (Figure 7).
{
"rdapConformance": [
"rdap_level_0",
"paging_level_0"
],
...
"notices": [
{
"title": "Search query limits",
"type": "result set truncated due to excessive load",
"description": [
"search results for domains are limited to 10"
]
}
],
"paging_metadata": {
"totalCount": 73,
"pageCount": 10,
"links": [
{
"value": "https://example.com/rdap/domains?name=*nr.com",
"rel": "next",
"href": "https://example.com/rdap/domains?name=*nr.com&limit=10
&cursor=wJlCDLIl6KTWypN7T6vc6nWEmEYe99Hjf1XY1xmqV-M=",
"title": "Result Pagination Link",
"type": "application/rdap+json"
}
]
},
"domainSearchResults": [
...
]
}
Figure 7: Example of a "paging_metadata" instance to implement keyset
pagination
But keyset pagination raises some drawbacks with respect to offset
pagination:
o it needs at least one key field;
o it does not allow to sort by any field and paginate the results
because sorting has to be made on the key field;
o it does not allow to skip pages because they have to be scrolled
in sequential order starting from the initial page;
o it makes very hard the navigation of the result set in both
directions because all comparison and sort operations have to be
reversed.
Furthermore, in the RDAP context, some additional considerations can
be made:
o an RDAP object is a conceptual aggregation of information
collected from more than one data structure (e.g. table) and this
makes even harder for the developers the implementation of the
seek-method that is already quite difficult. In fact, for
example, the entity object can gather information from different
data structures (registrars, registrants, contacts, resellers, and
so on), each one with its own key field mapping the RDAP entity
handle;
o depending on the number of the page results as well as the number
and the complexity of the properties of each RDAP object in the
response, the time required by offset pagination to skip the
previous pages could be much faster than the processing time
needed to build the current page. In fact, RDAP objects are
usually formed by information belonging to multiple data
structures and containing multivalued properties (e.g. arrays)
and, therefore, data selection is a time consuming process. This
situation occurs even though the data selection process makes use
of indexes;
o depending on the access levels defined by each RDAP operator, the
increase of complexity and the decrease of flexibility of keyset
pagination with respect to the offset pagination could be
considered impractical.
Finally, the keyset pagination is not fully compliant with the
additional RDAP capabilities proposed by this document. In fact, the
presence of a possible cursor parameter does not seem to be
consistent with both the sorting capability and the possibility to
implement additional ready-made links besides the classic "next page"
link. But, while the provisioning of more paging links can be
superfluous, dropping the sorting capability seems quite
unreasonable.
If pagination is implemented by using a cursor, both "offset" and
"nextOffset" fields MUST not be included in the "paging_metadata"
section.
FOR DISCUSSION: Should RDAP specification reports both offset and
cursor parameters and let operators to implement pagination according
to their needs, the user access levels, the submitted queries?
6. Implementation Status 6. Implementation Status
NOTE: Please remove this section and the reference to RFC 7942 prior NOTE: Please remove this section and the reference to RFC 7942 prior
to publication as an RFC. to publication as an RFC.
This section records the status of known implementations of the This section records the status of known implementations of the
protocol defined by this specification at the time of posting of this protocol defined by this specification at the time of posting of this
Internet-Draft, and is based on a proposal described in RFC 7942 Internet-Draft, and is based on a proposal described in RFC 7942
[RFC7942]. The description of implementations in this section is [RFC7942]. The description of implementations in this section is
intended to assist the IETF in its decision processes in progressing intended to assist the IETF in its decision processes in progressing
skipping to change at page 19, line 37 skipping to change at page 20, line 22
It is up to the individual working groups to use this information as It is up to the individual working groups to use this information as
they see fit". they see fit".
6.1. IIT-CNR/Registro.it 6.1. IIT-CNR/Registro.it
Responsible Organization: Institute of Informatics and Telematics Responsible Organization: Institute of Informatics and Telematics
of National Research Council (IIT-CNR)/Registro.it of National Research Council (IIT-CNR)/Registro.it
Location: https://rdap.pubtest.nic.it/ Location: https://rdap.pubtest.nic.it/
Description: This implementation includes support for RDAP queries Description: This implementation includes support for RDAP queries
using data from the public test environment of .it ccTLD. The using data from the public test environment of .it ccTLD. The
RDAP server implements both offset and cursor based pagination RDAP server implements both offset and cursor pagination (the
(the latter only when sort and offset parameters are not present latter only when "sort" and "offset" parameters are not present in
in the query string). the query string).
Level of Maturity: This is a "proof of concept" research Level of Maturity: This is a "proof of concept" research
implementation. implementation.
Coverage: This implementation includes all of the features Coverage: This implementation includes all of the features
described in this specification. described in this specification.
Contact Information: Mario Loffredo, mario.loffredo@iit.cnr.it Contact Information: Mario Loffredo, mario.loffredo@iit.cnr.it
6.2. Google Registry 6.2. Google Registry
Responsible Organization: Google Registry Responsible Organization: Google Registry
Location: https://www.registry.google/rdap/ Location: https://www.registry.google/rdap/
Description: This implementation includes support for RDAP queries Description: This implementation includes support for RDAP queries
for TLDs such as .GOOGLE, .HOW, .SOY, and .xn--q9jyb4c . The RDAP for TLDs such as .GOOGLE, .HOW, .SOY, and .xn--q9jyb4c . The RDAP
server implements cursor based pagination (the number of objects server implements cursor pagination. The link used to request the
per page is fixed so the limit parameter is not available). The next page is included in the notice section of the response.
link used to request the next page is included in the notice
section of the response.
Level of Maturity: Production. Level of Maturity: Production.
Coverage: This implementation includes the cursor parameter Coverage: This implementation includes the "cursor" parameter
described in this specification. described in this specification.
Contact Information: Brian Mountford, mountford@google.com Contact Information: Brian Mountford, mountford@google.com
7. Security Considerations 7. Security Considerations
Security services for the operations specified in this document are Security services for the operations specified in this document are
described in RFC 7481 [RFC7481]. described in RFC 7481 [RFC7481].
Search query typically requires more server resources (such as Search query typically requires more server resources (such as
memory, CPU cycles, and network bandwidth) when compared to lookup memory, CPU cycles, and network bandwidth) when compared to lookup
skipping to change at page 20, line 31 skipping to change at page 21, line 15
mitigated by either restricting search functionality and limiting the mitigated by either restricting search functionality and limiting the
rate of search requests. Servers can also reduce their load by rate of search requests. Servers can also reduce their load by
truncating the results in the response. However, this last security truncating the results in the response. However, this last security
policy can result in a higher inefficiency if the RDAP server does policy can result in a higher inefficiency if the RDAP server does
not provide any functionality to return the truncated results. not provide any functionality to return the truncated results.
The new parameters presented in this document provide the RDAP The new parameters presented in this document provide the RDAP
operators with a way to implement a secure server without penalizing operators with a way to implement a secure server without penalizing
its efficiency. The "count" parameter gives the user a measure to its efficiency. The "count" parameter gives the user a measure to
evaluate the query precision and, at the same time, return a evaluate the query precision and, at the same time, return a
significant information. The sort parameter allows the user to significant information. The "sort" parameter allows the user to
obtain the most relevant information at the beginning of the result obtain the most relevant information at the beginning of the result
set. In both cases, the user doesn't need to submit further set. In both cases, the user doesn't need to submit further
unnecessary search requests. Finally, the limit and offset unnecessary search requests. Finally, the "limit" and "offset"
parameters enable the user to scroll the result set by submitting a parameters enable the user to scroll the result set by submitting a
sequence of sustainable queries according to the server limits. sequence of sustainable queries according to the server limits.
8. IANA Considerations 8. IANA Considerations
This document has no actions for IANA. This document has no actions for IANA.
9. Acknowledgements 9. Acknowledgements
The authors would like to acknowledge Brian Mountford for his The authors would like to acknowledge Brian Mountford for his
skipping to change at page 24, line 45 skipping to change at page 25, line 25
[W3C.CR-xpath-31-20161213] [W3C.CR-xpath-31-20161213]
Robie, J., Dyck, M., and J. Spiegel, "XML Path Language Robie, J., Dyck, M., and J. Spiegel, "XML Path Language
(XPath) 3.1", World Wide Web Consortium CR CR-xpath- (XPath) 3.1", World Wide Web Consortium CR CR-xpath-
31-20161213, December 2016, 31-20161213, December 2016,
<https://www.w3.org/TR/2016/CR-xpath-31-20161213>. <https://www.w3.org/TR/2016/CR-xpath-31-20161213>.
Appendix A. Change Log Appendix A. Change Log
00: Initial working group version ported from draft-loffredo-regext- 00: Initial working group version ported from draft-loffredo-regext-
rdap-sorting-and-paging-05 rdap-sorting-and-paging-05
01: Removed both "offset" and "nextOffset" to keep "paging_metadata"
consistent between the pagination methods. Renamed
"Considerations about Paging Implementation" section in ""cursor"
Parameter". Removed "FOR DISCUSSION" items. Provided a more
detailed description of both "sorting_metadata" and
"paging_metadata" objects.
Authors' Addresses Authors' Addresses
Mario Loffredo Mario Loffredo
IIT-CNR/Registro.it IIT-CNR/Registro.it
Via Moruzzi,1 Via Moruzzi,1
Pisa 56124 Pisa 56124
IT IT
Email: mario.loffredo@iit.cnr.it Email: mario.loffredo@iit.cnr.it
URI: http://www.iit.cnr.it URI: http://www.iit.cnr.it
Maurizio Martinelli Maurizio Martinelli
 End of changes. 53 change blocks. 
223 lines changed or deleted 225 lines changed or added

This html diff was produced by rfcdiff 1.47. The latest version is available from http://tools.ietf.org/tools/rfcdiff/