diff options
Diffstat (limited to 'doc/rfc/rfc8977.txt')
| -rw-r--r-- | doc/rfc/rfc8977.txt | 1243 |
1 files changed, 1243 insertions, 0 deletions
diff --git a/doc/rfc/rfc8977.txt b/doc/rfc/rfc8977.txt new file mode 100644 index 0000000..aeeb750 --- /dev/null +++ b/doc/rfc/rfc8977.txt @@ -0,0 +1,1243 @@ + + + + +Internet Engineering Task Force (IETF) M. Loffredo +Request for Comments: 8977 M. Martinelli +Category: Standards Track IIT-CNR/Registro.it +ISSN: 2070-1721 S. Hollenbeck + Verisign Labs + January 2021 + + + Registration Data Access Protocol (RDAP) Query Parameters for Result + Sorting and Paging + +Abstract + + The Registration Data Access Protocol (RDAP) does not include core + functionality for clients to provide sorting and paging parameters + for control of large result sets. This omission can lead to + unpredictable server processing of queries and client processing of + responses. This unpredictability can be greatly reduced if clients + can provide servers with their preferences for managing large + responses. This document describes RDAP query extensions that allow + clients to specify their preferences for sorting and paging result + sets. + +Status of This Memo + + This is an Internet Standards Track document. + + This document is a product of the Internet Engineering Task Force + (IETF). It represents the consensus of the IETF community. It has + received public review and has been approved for publication by the + Internet Engineering Steering Group (IESG). Further information on + Internet Standards is available in Section 2 of RFC 7841. + + Information about the current status of this document, any errata, + and how to provide feedback on it may be obtained at + https://www.rfc-editor.org/info/rfc8977. + +Copyright Notice + + Copyright (c) 2021 IETF Trust and the persons identified as the + document authors. All rights reserved. + + This document is subject to BCP 78 and the IETF Trust's Legal + Provisions Relating to IETF Documents + (https://trustee.ietf.org/license-info) in effect on the date of + publication of this document. Please review these documents + carefully, as they describe your rights and restrictions with respect + to this document. Code Components extracted from this document must + include Simplified BSD License text as described in Section 4.e of + the Trust Legal Provisions and are provided without warranty as + described in the Simplified BSD License. + +Table of Contents + + 1. Introduction + 1.1. Conventions Used in This Document + 2. RDAP Query Parameter Specification + 2.1. Sorting and Paging Metadata + 2.1.1. RDAP Conformance + 2.2. "count" Parameter + 2.3. "sort" Parameter + 2.3.1. Sorting Properties Declaration + 2.3.2. Representing Sorting Links + 2.4. "cursor" Parameter + 2.4.1. Representing Paging Links + 3. Negative Answers + 4. Implementation Considerations + 5. IANA Considerations + 6. Security Considerations + 7. References + 7.1. Normative References + 7.2. Informative References + Appendix A. JSONPath Operators + Appendix B. Approaches to Result Pagination + B.1. Specific Issues Raised by RDAP + Appendix C. Implementation Notes + C.1. Sorting + C.2. Counting + C.3. Paging + Acknowledgements + Authors' Addresses + +1. Introduction + + The availability of functionality for result sorting and paging + provides benefits to both clients and servers in the implementation + of RESTful services [REST]. These benefits include: + + * reducing the server response bandwidth requirements + + * improving server response time + + * improving query precision and, consequently, obtaining more + relevant results + + * decreasing server query processing load + + * reducing client response processing time + + Approaches to implementing features for result sorting and paging can + be grouped into two main categories: + + 1. Sorting and paging are implemented through the introduction of + additional parameters in the query string (e.g., the Open Data + Protocol (ODATA) [ODATA-PART1]). + + 2. Information related to the number of results and the specific + portion of the result set to be returned, in addition to a set of + ready-made links for the result set scrolling, are inserted in + the HTTP header of the request/response [RFC7231]. + + However, there are some drawbacks associated with the use of the HTTP + header. First, the header properties cannot be set directly from a + web browser. Moreover, in an HTTP session, the information on the + status (i.e., the session identifier) is usually inserted in the + header or a cookie, while the information on the resource + identification or the search type is included in the query string. + Finally, providing custom information through HTTP headers assumes + the client has prior knowledge of the server implementation, which is + widely considered a Representational State Transfer (REST) design + anti-pattern. As a result, this document describes a specification + based on the use of query parameters. + + Currently, RDAP [RFC7482] defines two query types: + + lookup: the server returns only one object + + search: the server returns a collection of objects + + While the lookup query does not raise issues regarding response size + management, the search query can potentially generate a large result + set that is often truncated according to server limits. Besides, it + is not possible to obtain the total number of objects found that + might be returned in a search query response [RFC7483]. Lastly, + there is no way to specify sort criteria to return the most relevant + objects at the beginning of the result set. Therefore, the client + might traverse the whole result set to find the relevant objects or, + due to truncation, might not find them at all. + + The specification described in this document extends RDAP query + capabilities to enable result sorting and paging by adding new query + parameters that can be applied to RDAP search path segments. The + service is implemented using the Hypertext Transfer Protocol (HTTP) + [RFC7230] and the conventions described in [RFC7480]. + + The implementation of the new parameters is technically feasible, as + operators for counting, sorting, and paging rows are currently + supported by the major relational database management systems. + +1.1. Conventions Used in This Document + + The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", + "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and + "OPTIONAL" in this document are to be interpreted as described in + BCP 14 [RFC2119] [RFC8174] when, and only when, they appear in all + capitals, as shown here. + +2. RDAP Query Parameter Specification + + The new query parameters are OPTIONAL extensions of path segments + defined in [RFC7482]. They are as follows: + + "count": a boolean value that allows a client to request the return + of the total number of objects found + + "sort": a string value that allows a client to request a specific + sort order for the result set + + "cursor": a string value representing a pointer to a specific fixed- + size portion of the result set + + Augmented Backus-Naur Form (ABNF) [RFC5234] is used in the following + sections to describe the formal syntax of these new parameters. + +2.1. Sorting and Paging Metadata + + According to most advanced principles in REST design, collectively + known as HATEOAS (Hypermedia as the Engine of Application State) + [HATEOAS], a client entering a REST application through an initial + URI should use server-provided links to dynamically discover + available actions and access the resources it needs. In this way, + the client is neither required to have prior knowledge of the service + nor, consequently, to hard code the URIs of different resources. + This allows the server to make URI changes as the API evolves without + breaking clients. Definitively, a REST service should be as self- + descriptive as possible. + + Therefore, servers implementing the query parameters described in + this specification SHOULD provide additional information in their + responses about both the available sorting criteria and possible + pagination. Such information is collected in two OPTIONAL response + elements named "sorting_metadata" and "paging_metadata". + + The "sorting_metadata" element contains the following properties: + + "currentSort": "String" (OPTIONAL) + Either the value of the "sort" parameter as specified in the query + string or the sort applied by default, if any. + + "availableSorts": "AvailableSort[]" (OPTIONAL) + An array of objects, with each element describing an available + sort criterion. The AvailableSort object includes the following + members: + + "property": "String" (REQUIRED) + The name that can be used by the client to request the sort + criterion. + + "default": "Boolean" (REQUIRED) + Indicator of whether the sort criterion is applied by default. + An RDAP server MUST define only one default sorting property + for each object class. + + "jsonPath": "String" (OPTIONAL) + The JSONPath expression of the RDAP field corresponding to the + property. + + "links": "Link[]" (OPTIONAL) + An array of links as described in [RFC8288] containing the + query string that applies the sort criterion. + + At least one of the "currentSort" and "availableSorts" properties + MUST be present. + + The "paging_metadata" element contains the following fields: + + "totalCount": "Numeric" (OPTIONAL) + A numeric value representing the total number of objects found. + It MUST be provided if and only if the query string contains the + "count" parameter. + + "pageSize": "Numeric" (OPTIONAL) + A numeric value representing the number of objects that should + have been returned in the current page. It MUST be provided if + and only if the total number of objects exceeds the page size. + This property is redundant for RDAP clients because the page size + can be derived from the length of the search results array, but it + can be helpful if the end user interacts with the server through a + web browser. + + "pageNumber": "Numeric" (OPTIONAL) + A numeric value representing the number of the current page in the + result set. It MUST be provided if and only if the total number + of objects found exceeds the page size. + + "links": "Link[]" (OPTIONAL) + An array of links as described in [RFC8288] containing the + reference to the next page. In this specification, only forward + pagination is described because it is all that is necessary to + traverse the result set. + +2.1.1. RDAP Conformance + + Servers returning the "paging_metadata" element in their response + MUST include the string literal "paging" in the rdapConformance + array. Servers returning the "sorting_metadata" element MUST include + the string literal "sorting". + +2.2. "count" Parameter + + Currently, RDAP does not allow a client to determine the total number + of results in a query response when the result set is truncated. + This is inefficient because the user cannot determine if the result + set is complete. + + The "count" parameter provides additional functionality that allows a + client to request information from the server that specifies the + total number of objects matching the search pattern. + + The following is an example of an RDAP query including the "count" + parameter: + + https://example.com/rdap/domains?name=example*.com&count=true + + The ABNF syntax is the following: + + count = "count=" ( trueValue / falseValue ) + trueValue = ("true" / "yes" / "1") + falseValue = ("false" / "no" / "0") + + A trueValue means that the server MUST provide the total number of + objects in the "totalCount" field of the "paging_metadata" element + (Figure 1). A falseValue means that the server MUST NOT provide this + number. + + { + "rdapConformance": [ + "rdap_level_0", + "paging" + ], + ... + "paging_metadata": { + "totalCount": 43 + }, + "domainSearchResults": [ + ... + ] + } + + Figure 1: Example of RDAP Response with "paging_metadata" Element + Containing the "totalCount" Field + +2.3. "sort" Parameter + + RDAP does not provide any capability to specify the result set sort + criteria. A server could implement a default sorting scheme + according to the object class, but this feature is not mandatory and + might not meet user requirements. Sorting can be addressed by the + client, but this solution is rather inefficient. Sorting features + provided by the RDAP server could help avoid truncation of relevant + results. + + The "sort" parameter allows the client to ask the server to sort the + results according to the values of one or more properties and + according to the sort direction of each property. The ABNF syntax is + the following: + + sort = "sort=" sortItem *( "," sortItem ) + sortItem = property-ref [":" ( "a" / "d" ) ] + property-ref = ALPHA *( ALPHA / DIGIT / "_" ) + + "a" means that an ascending sort MUST be applied; "d" means that a + descending sort MUST be applied. If the sort direction is absent, an + ascending sort MUST be applied. + + The following are examples of RDAP queries that include the "sort" + parameter: + + https://example.com/rdap/domains?name=example*.com&sort=name + + https://example.com/rdap/ + domains?name=example*.com&sort=registrationDate:d + + https://example.com/rdap/ + domains?name=example*.com&sort=lockedDate,name + + Except for sorting IP addresses and values denoting dates and times, + servers MUST implement sorting according to the JSON value type of + the RDAP field the sorting property refers to. That is, JSON strings + MUST be sorted lexicographically, and JSON numbers MUST be sorted + numerically. Values denoting dates and times MUST be sorted in + chronological order. If IP addresses are represented as JSON + strings, they MUST be sorted based on their numeric conversion. + + The conversion of an IPv4 address to a number is possible since each + dotted format IPv4 address is a representation of a number written in + a 256-based manner; for example, 192.168.0.1 means 1*256^0 + 0*256^1 + + 168*256^2 + 192*256^3 = 3232235521. Similarly, an IPv6 address can + be converted into a number by applying the base 65536. Therefore, + the numerical representation of the IPv6 address + 2001:0db8:85a3:0:0:8a2e:0370:7334 is + 42540766452641154071740215577757643572. Built-in functions and + libraries for converting IP addresses into numbers are available in + most known programming languages and relational database management + systems. + + If the "sort" parameter presents an allowed sorting property, it MUST + be provided in the "currentSort" field of the "sorting_metadata" + element. + +2.3.1. Sorting Properties Declaration + + In the "sort" parameter ABNF syntax, the element named "property-ref" + represents a reference to a property of an RDAP object. Such a + reference could be expressed by using a JSONPath expression (named + "jsonpath" in the following). + + JSONPath is a syntax, originally based on the XML XPath notation + [W3C.CR-xpath-31-20170321], which represents a path to select an + element (or a set of elements) in a JSON document [RFC8259]. For + example, the jsonpath to select the value of the ASCII name inside an + RDAP domain lookup response is "$.ldhName", where $ identifies the + root of the document object model (DOM). Another way to select a + value inside a JSON document is the JSON Pointer [RFC6901]. + + While JSONPath and JSON Pointer are both commonly adopted notations + to select any value inside JSON data, neither is particularly concise + and easy to use (e.g., "$.domainSearchResults[*].events[?(@.eventActi + on='registration')].eventDate" is the jsonpath of the registration + date in an RDAP domain search response). + + Therefore, this specification defines the "property-ref" element in + terms of names identifying RDAP properties. However, not all the + RDAP properties are suitable to be used in sort criteria. These + properties include: + + * properties providing service information (e.g., links, notices, + and remarks) + + * multivalued properties (e.g., status, roles, and variants) + + * properties representing relationships to other objects (e.g., + entities) + + On the contrary, properties expressed as values of other properties + (e.g., registration date) could be used in such a context. + + A list of properties an RDAP server MAY implement is defined. The + properties are divided into two groups: object-common properties and + object-specific properties. + + * Object-common properties. Object-common properties are derived + from merging the "eventAction" and the "eventDate" properties. + The following values of the "sort" parameter are defined: + + - registrationDate + + - reregistrationDate + + - lastChangedDate + + - expirationDate + + - deletionDate + + - reinstantiationDate + + - transferDate + + - lockedDate + + - unlockedDate + + * Object-specific properties. Note that some of these properties + are also defined as query path segments. These properties + include: + + - Domain: name + + - Nameserver: name, ipv4, ipv6 + + - Entity: fn, handle, org, email, voice, country, cc, city + + The correspondence between these sorting properties and the RDAP + object classes is shown in Table 1. Some of the sorting properties + defined for the RDAP entity class are related to jCard elements + [RFC7095], but because jCard is the JSON format for vCard, the + corresponding definitions are included in the vCard specification + [RFC6350]. + + An RDAP server MUST NOT use the defined sorting properties with a + meaning other than that described in Table 1. + + +============+============+=================+======+=======+======+ + | Object | Sorting | RDAP property | RFC | RFC | RFC | + | class | property | | 7483 | 6350 | 8605 | + +============+============+=================+======+=======+======+ + | Searchable | Common | eventAction | 4.5 | | | + | objects | properties | values suffixed | | | | + | | | by "Date" | | | | + +------------+------------+-----------------+------+-------+------+ + | Domain | name | unicodeName/ | 5.3 | | | + | | | ldhName | | | | + +------------+------------+-----------------+------+-------+------+ + | Nameserver | name | unicodeName/ | 5.2 | | | + | | | ldhName | | | | + +------------+------------+-----------------+------+-------+------+ + | | ipv4 | v4 ipAddress | 5.2 | | | + +------------+------------+-----------------+------+-------+------+ + | | ipv6 | v6 ipAddress | 5.2 | | | + +------------+------------+-----------------+------+-------+------+ + | Entity | handle | handle | 5.1 | | | + +------------+------------+-----------------+------+-------+------+ + | | fn | jCard fn | 5.1 | 6.2.1 | | + +------------+------------+-----------------+------+-------+------+ + | | org | jCard org | 5.1 | 6.6.4 | | + +------------+------------+-----------------+------+-------+------+ + | | voice | jCard tel with | 5.1 | 6.4.1 | | + | | | type="voice" | | | | + +------------+------------+-----------------+------+-------+------+ + | | email | jCard email | 5.1 | 6.4.2 | | + +------------+------------+-----------------+------+-------+------+ + | | country | country name in | 5.1 | 6.3.1 | | + | | | jCard adr | | | | + +------------+------------+-----------------+------+-------+------+ + | | cc | country code in | 5.1 | | 3.1 | + | | | jCard adr | | | | + +------------+------------+-----------------+------+-------+------+ + | | city | locality in | 5.1 | 6.3.1 | | + | | | jCard adr | | | | + +------------+------------+-----------------+------+-------+------+ + + Table 1: Definitions of Sorting Properties + + Regarding the definitions in Table 1, some further considerations are + needed to disambiguate some cases: + + * Since the response to a search on either domains or nameservers + might include both A-labels and U-labels [RFC5890] in general, a + consistent sorting policy MUST treat the unicodeName and ldhName + as two representations of the same value. The unicodeName value + MUST be used while sorting if it is present; when the unicodeName + is unavailable, the value of the ldhName MUST be used instead. + + * The jCard "sort-as" parameter MUST be ignored for the sorting + capability described in this document. + + * Even if a nameserver can have multiple IPv4 and IPv6 addresses, + the most common configuration includes one address for each IP + version. Therefore, this specification makes the assumption that + nameservers have a single IPv4 and/or IPv6 value. When more than + one address per IP version is presented, sorting MUST be applied + to the first value. + + * Multiple events with a given action on an object might be + returned. If this occurs, sorting MUST be applied to the most + recent event. + + * Except for handle values, all the sorting properties defined for + entity objects can be multivalued according to the definition of + vCard as given in [RFC6350]. When more than one value is + presented, sorting MUST be applied to the preferred value + identified by the parameter pref="1". If the "pref" parameter is + missing, sorting MUST be applied to the first value. + + The "jsonPath" field in the "sorting_metadata" element is used to + clarify the RDAP response field the sorting property refers to. The + mapping between the sorting properties and the jsonpaths of the RDAP + response fields is shown below. The JSONPath operators used herein + are described in Appendix A. + + * Searchable objects + + registrationDate + $.domainSearchResults[*].events[?(@.eventAction=="registration" + )].eventDate + + reregistrationDate + $.domainSearchResults[*].events[?(@.eventAction=="reregistratio + n")].eventDate + + lastChangedDate + $.domainSearchResults[*].events[?(@.eventAction=="last + changed")].eventDate + + expirationDate + $.domainSearchResults[*].events[?(@.eventAction=="expiration")] + .eventDate + + deletionDate + $.domainSearchResults[*].events[?(@.eventAction=="deletion")].e + ventDate + + reinstantiationDate + $.domainSearchResults[*].events[?(@.eventAction=="reinstantiati + on")].eventDate + + transferDate + $.domainSearchResults[*].events[?(@.eventAction=="transfer")].e + ventDate + + lockedDate + $.domainSearchResults[*].events[?(@.eventAction=="locked")].eve + ntDate + + unlockedDate + $.domainSearchResults[*].events[?(@.eventAction=="unlocked")].e + ventDate + + * Domain + + name + $.domainSearchResults[*].[unicodeName,ldhName] + + * Nameserver + + name + $.nameserverSearchResults[*].[unicodeName,ldhName] + + ipv4 + $.nameserverSearchResults[*].ipAddresses.v4[0] + + ipv6 + $.nameserverSearchResults[*].ipAddresses.v6[0] + + * Entity + + handle + $.entitySearchResults[*].handle + + fn + $.entitySearchResults[*].vcardArray[1][?(@[0]=="fn")][3] + + org + $.entitySearchResults[*].vcardArray[1][?(@[0]=="org")][3] + + voice + $.entitySearchResults[*].vcardArray[1][?(@[0]=="tel" && + @[1].type=="voice")][3] + + email + $.entitySearchResults[*].vcardArray[1][?(@[0]=="email")][3] + + country + $.entitySearchResults[*].vcardArray[1][?(@[0]=="adr")][3][6] + + cc + $.entitySearchResults[*].vcardArray[1][?(@[0]=="adr")][1].cc + + city + $.entitySearchResults[*].vcardArray[1][?(@[0]=="adr")][3][3] + + Additional notes on the provided jsonpaths: + + * Those related to the event dates are defined only for the "domain" + object. To obtain the equivalent jsonpaths for "entity" and + "nameserver", the path segment "domainSearchResults" must be + replaced with "entitySearchResults" and "nameserverSearchResults", + respectively. + + * Those related to jCard elements are specified without taking into + account the "pref" parameter. Servers that sort those values + identified by the "pref" parameter SHOULD update a jsonpath by + adding an appropriate filter. For example, if the email values + identified by pref="1" are considered for sorting, the jsonpath of + the "email" sorting property should be + $.entitySearchResults[*].vcardArray[1][?(@[0]=="email" && + @[1].pref=="1")][3]. + +2.3.2. Representing Sorting Links + + An RDAP server MAY use the "links" array of the "sorting_metadata" + element to provide ready-made references [RFC8288] to the available + sort criteria (Figure 2). Each link represents a reference to an + alternate view of the results. + + The "value", "rel", and "href" JSON values MUST be specified. All + other JSON values are OPTIONAL. + + { + "rdapConformance": [ + "rdap_level_0", + "sorting" + ], + ... + "sorting_metadata": { + "currentSort": "name", + "availableSorts": [ + { + "property": "registrationDate", + "jsonPath": "$.domainSearchResults[*] + .events[?(@.eventAction==\"registration\")].eventDate", + "default": false, + "links": [ + { + "value": "https://example.com/rdap/domains?name=example*.com + &sort=name", + "rel": "alternate", + "href": "https://example.com/rdap/domains?name=example*.com + &sort=registrationDate", + "title": "Result Ascending Sort Link", + "type": "application/rdap+json" + }, + { + "value": "https://example.com/rdap/domains?name=example*.com + &sort=name", + "rel": "alternate", + "href": "https://example.com/rdap/domains?name=example*.com + &sort=registrationDate:d", + "title": "Result Descending Sort Link", + "type": "application/rdap+json" + } + ] + }, + ... + ] + }, + "domainSearchResults": [ + ... + ] + } + + Figure 2: Example of a "sorting_metadata" Instance to Implement + Result Sorting + +2.4. "cursor" Parameter + + The "cursor" parameter defined in this specification can be used to + encode information about any pagination method. For example, in the + case of a simple implementation of the "cursor" parameter to + represent offset pagination information, the "cursor" value + "b2Zmc2V0PTEwMCxsaW1pdD01MA==" is the base64 encoding of + "offset=100,limit=50". Likewise, in a simple implementation to + represent keyset pagination information, the "cursor" value + "ZXhhbXBsZS1OLmNvbQ==" represents the base64 encoding of + "key=example-N.com" whereby the key value identifies the last row of + the current page. + + Note that this specification uses a base64 encoding for cursor + obfuscation just for example. RDAP servers are NOT RECOMMENDED to + obfuscate the "cursor" value through a mere base64 encoding. + + This solution lets RDAP providers implement a pagination method + according to their needs, a user's access level, and the submitted + query. Besides, servers can change the method over time without + announcing anything to clients. The considerations that have led to + this solution are described in more detail in Appendix B. + + The ABNF syntax of the "cursor" parameter is the following: + + cursor = "cursor=" 1*( ALPHA / DIGIT / "/" / "=" / "-" / "_" ) + + The following is an example of an RDAP query including the "cursor" + parameter: + + https://example.com/rdap/domains?name=example*.com + &cursor=wJlCDLIl6KTWypN7T6vc6nWEmEYe99Hjf1XY1xmqV-M= + +2.4.1. Representing Paging Links + + An RDAP server SHOULD use the "links" array of the "paging_metadata" + element to provide a ready-made reference [RFC8288] to the next page + of the result set (Figure 3). Examples of additional "rel" values a + server MAY implement are "first", "last", and "prev". + + { + "rdapConformance": [ + "rdap_level_0", + "paging" + ], + ... + "notices": [ + { + "title": "Search query limits", + "type": "result set truncated due to excessive load", + "description": [ + "search results for domains are limited to 50" + ] + } + ], + "paging_metadata": { + "totalCount": 73, + "pageSize": 50, + "pageNumber": 1, + "links": [ + { + "value": "https://example.com/rdap/domains?name=example*.com", + "rel": "next", + "href": "https://example.com/rdap/domains?name=example*.com + &cursor=wJlCDLIl6KTWypN7T6vc6nWEmEYe99Hjf1XY1xmqV-M=", + "title": "Result Pagination Link", + "type": "application/rdap+json" + } + ] + }, + "domainSearchResults": [ + ... + ] + } + + Figure 3: Example of a "paging_metadata" Instance to Implement Cursor + Pagination + +3. Negative Answers + + The constraints for the values of parameters are defined by their + ABNF syntax. Therefore, each request that includes an invalid value + for a parameter SHOULD produce an HTTP 400 (Bad Request) response + code. The same response SHOULD be returned in the following cases: + + * if sorting by either single or multiple properties, the client + provides an unsupported value for the "sort" parameter, as well as + a value related to an object property not included in the response + + * if the client submits an invalid value for the "cursor" parameter + + Optionally, the response MAY include additional information regarding + either the supported sorting properties or the correct "cursor" value + in the HTTP entity body (Figure 4). + + { + "errorCode": 400, + "title": "Domain sorting property 'unknown' is not valid", + "description": [ + "Supported domain sorting properties are:" + "'aproperty', 'anotherproperty'" + ] + + } + + Figure 4: Example of RDAP Error Response Due to an Invalid Domain + Sorting Property Included in the Request + +4. Implementation Considerations + + Implementation of the new parameters is technically feasible, as + operators for counting, sorting, and paging are currently supported + by the major relational database management systems. Similar + operators are completely or partially supported by the most well- + known NoSQL databases (e.g., MongoDB, CouchDB, HBase, Cassandra, + Hadoop, etc.). Additional implementation notes are included in + Appendix C. + +5. IANA Considerations + + IANA has registered the following values in the "RDAP Extensions" + registry: + + Extension identifier: paging + Registry operator: Any + Published specification: RFC 8977 + Contact: IETF <iesg@ietf.org> + Intended usage: This extension describes a best practice for result + set paging. + + Extension identifier: sorting + Registry operator: Any + Published specification: RFC 8977 + Contact: IETF <iesg@ietf.org> + Intended usage: This extension describes a best practice for result + set sorting. + +6. Security Considerations + + Security services for the operations specified in this document are + described in [RFC7481]. + + A search query typically requires more server resources (such as + memory, CPU cycles, and network bandwidth) when compared to a lookup + query. This increases the risk of server resource exhaustion and + subsequent denial of service. This risk can be mitigated by either + restricting search functionality or limiting the rate of search + requests. Servers can also reduce their load by truncating the + results in a response. However, this last security policy can result + in a higher inefficiency or risk due to acting on incomplete + information if the RDAP server does not provide any functionality to + return the truncated results. + + The new parameters presented in this document provide RDAP operators + with a way to implement a server that reduces inefficiency risks. + The "count" parameter gives the client the ability to evaluate the + completeness of a response. The "sort" parameter allows the client + to obtain the most relevant information at the beginning of the + result set. This can reduce the number of unnecessary search + requests. Finally, the "cursor" parameter enables the user to scroll + the result set by submitting a sequence of sustainable queries within + server-acceptable limits. + +7. References + +7.1. Normative References + + [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate + Requirement Levels", BCP 14, RFC 2119, + DOI 10.17487/RFC2119, March 1997, + <https://www.rfc-editor.org/info/rfc2119>. + + [RFC5234] Crocker, D., Ed. and P. Overell, "Augmented BNF for Syntax + Specifications: ABNF", STD 68, RFC 5234, + DOI 10.17487/RFC5234, January 2008, + <https://www.rfc-editor.org/info/rfc5234>. + + [RFC5890] Klensin, J., "Internationalized Domain Names for + Applications (IDNA): Definitions and Document Framework", + RFC 5890, DOI 10.17487/RFC5890, August 2010, + <https://www.rfc-editor.org/info/rfc5890>. + + [RFC6350] Perreault, S., "vCard Format Specification", RFC 6350, + DOI 10.17487/RFC6350, August 2011, + <https://www.rfc-editor.org/info/rfc6350>. + + [RFC7095] Kewisch, P., "jCard: The JSON Format for vCard", RFC 7095, + DOI 10.17487/RFC7095, January 2014, + <https://www.rfc-editor.org/info/rfc7095>. + + [RFC7230] Fielding, R., Ed. and J. Reschke, Ed., "Hypertext Transfer + Protocol (HTTP/1.1): Message Syntax and Routing", + RFC 7230, DOI 10.17487/RFC7230, June 2014, + <https://www.rfc-editor.org/info/rfc7230>. + + [RFC7231] Fielding, R., Ed. and J. Reschke, Ed., "Hypertext Transfer + Protocol (HTTP/1.1): Semantics and Content", RFC 7231, + DOI 10.17487/RFC7231, June 2014, + <https://www.rfc-editor.org/info/rfc7231>. + + [RFC7480] Newton, A., Ellacott, B., and N. Kong, "HTTP Usage in the + Registration Data Access Protocol (RDAP)", RFC 7480, + DOI 10.17487/RFC7480, March 2015, + <https://www.rfc-editor.org/info/rfc7480>. + + [RFC7481] Hollenbeck, S. and N. Kong, "Security Services for the + Registration Data Access Protocol (RDAP)", RFC 7481, + DOI 10.17487/RFC7481, March 2015, + <https://www.rfc-editor.org/info/rfc7481>. + + [RFC7482] Newton, A. and S. Hollenbeck, "Registration Data Access + Protocol (RDAP) Query Format", RFC 7482, + DOI 10.17487/RFC7482, March 2015, + <https://www.rfc-editor.org/info/rfc7482>. + + [RFC7483] Newton, A. and S. Hollenbeck, "JSON Responses for the + Registration Data Access Protocol (RDAP)", RFC 7483, + DOI 10.17487/RFC7483, March 2015, + <https://www.rfc-editor.org/info/rfc7483>. + + [RFC8174] Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC + 2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174, + May 2017, <https://www.rfc-editor.org/info/rfc8174>. + + [RFC8259] Bray, T., Ed., "The JavaScript Object Notation (JSON) Data + Interchange Format", STD 90, RFC 8259, + DOI 10.17487/RFC8259, December 2017, + <https://www.rfc-editor.org/info/rfc8259>. + + [RFC8288] Nottingham, M., "Web Linking", RFC 8288, + DOI 10.17487/RFC8288, October 2017, + <https://www.rfc-editor.org/info/rfc8288>. + + [RFC8605] Hollenbeck, S. and R. Carney, "vCard Format Extensions: + ICANN Extensions for the Registration Data Access Protocol + (RDAP)", RFC 8605, DOI 10.17487/RFC8605, May 2019, + <https://www.rfc-editor.org/info/rfc8605>. + +7.2. Informative References + + [CURSOR] Nimesh, R., "Paginating Real-Time Data with Cursor Based + Pagination", July 2014, <https://www.sitepoint.com/ + paginating-real-time-data-cursor-based-pagination/>. + + [CURSOR-API1] + Facebook, "Facebook for Developers -- Using the Graph + API", <https://developers.facebook.com/docs/graph-api/ + using-graph-api>. + + [CURSOR-API2] + Twitter, "Twitter Ads API", + <https://developer.twitter.com/en/docs/twitter-ads-api>. + + [GOESSNER-JSON-PATH] + Goessner, S., "JSONPath - XPath for JSON", February 2007, + <https://goessner.net/articles/JsonPath/>. + + [HATEOAS] Jedrzejewski, B., "HATEOAS - a simple explanation", + February 2018, <https://www.e4developer.com/2018/02/16/ + hateoas-simple-explanation/>. + + [JSONPATH-COMPARISON] + "JSONPath Comparison", + <https://cburgmer.github.io/json-path-comparison/>. + + [JSONPATH-WG] + IETF, "JSON Path (jsonpath)", + <https://datatracker.ietf.org/wg/jsonpath/about/>. + + [ODATA-PART1] + Pizzo, M., Handl, R., and M. Zurmuehl, "OData Version 4.0. + Part 1: Protocol Plus Errata 03", June 2016, + <https://docs.oasis- + open.org/odata/odata/v4.0/errata03/os/complete/part1- + protocol/odata-v4.0-errata03-os-part1-protocol- + complete.pdf>. + + [REST] Fielding, R., "Architectural Styles and the Design of + Network-based Software Architectures", 2000, + <https://www.ics.uci.edu/~fielding/pubs/dissertation/ + fielding_dissertation.pdf>. + + [RFC6901] Bryan, P., Ed., Zyp, K., and M. Nottingham, Ed., + "JavaScript Object Notation (JSON) Pointer", RFC 6901, + DOI 10.17487/RFC6901, April 2013, + <https://www.rfc-editor.org/info/rfc6901>. + + [SEEK] EverSQL, "Faster Pagination in Mysql - Why Order By With + Limit and Offset is Slow?", July 2017, + <https://www.eversql.com/faster-pagination-in-mysql-why- + order-by-with-limit-and-offset-is-slow/>. + + [W3C.CR-xpath-31-20170321] + Robie, J., Dyck, M., and J. Spiegel, "XML Path Language + (XPath) 3.1", World Wide Web Consortium Recommendation + REC-xpath-31-20170321, March 2017, + <https://www.w3.org/TR/2017/REC-xpath-31-20170321/>. + +Appendix A. JSONPath Operators + + The jsonpaths used in this document are provided according to the + Goessner proposal [GOESSNER-JSON-PATH]. + + Such specification requires that implementations support a set of + "basic operators". These operators are used to access the elements + of a JSON structure like objects and arrays, as well as their + subelements (object members and array items, respectively). No + operations are defined for retrieving parent or sibling elements of a + given element. The root element is always referred to as $ + regardless of it being an object or array. + + Additionally, the specification permits implementations to support + arbitrary script expressions. These can be used to index into an + object or array, or to filter elements from an array. While script + expression behavior is implementation-defined, most implementations + support the basic relational and logical operators as well as both + object member and array item access, sufficiently similar for the + purpose of this document. Commonly supported operators/functions + divided into "top-level operators" and "filter operators" are + documented in Tables 2 and 3, respectively. + + For more information on implementation interoperability issues, see + [JSONPATH-COMPARISON]. At the time of writing, work is beginning on + a standardization effort too (see [JSONPATH-WG]). + + +===================+=========================================+ + | Operator | Description | + +===================+=========================================+ + | $ | Root element | + +-------------------+-----------------------------------------+ + | .<name> | Object member access (dot-notation) | + +-------------------+-----------------------------------------+ + | ['<name>'] | Object member access (bracket-notation) | + +-------------------+-----------------------------------------+ + | [<number>] | Array item access | + +-------------------+-----------------------------------------+ + | * | All elements within the specified scope | + +-------------------+-----------------------------------------+ + | [?(<expression>)] | Filter expression | + +-------------------+-----------------------------------------+ + + Table 2: JSONPath Top-Level Operators + + +====================+========================================+ + | Operator | Description | + +====================+========================================+ + | @ | Current element being processed | + +--------------------+----------------------------------------+ + | .<name> | Object member access | + +--------------------+----------------------------------------+ + | .[<name1>,<name2>] | Union of object members | + +--------------------+----------------------------------------+ + | [<number>] | Array item access | + +--------------------+----------------------------------------+ + | == | Left is equal to right | + +--------------------+----------------------------------------+ + | != | Left is not equal to right | + +--------------------+----------------------------------------+ + | < | Left is less than right | + +--------------------+----------------------------------------+ + | <= | Left is less than or equal to right | + +--------------------+----------------------------------------+ + | > | Left is greater than right | + +--------------------+----------------------------------------+ + | >= | Left is greater than or equal to right | + +--------------------+----------------------------------------+ + | && | Logical conjunction | + +--------------------+----------------------------------------+ + | || | Logical disjunction | + +--------------------+----------------------------------------+ + + Table 3: JSONPath Filter Operators + +Appendix B. Approaches to Result Pagination + + An RDAP query could return a response with hundreds, even thousands, + of objects, especially when partial matching is used. For this + reason, the "cursor" parameter addressing result pagination is + defined to make responses easier to handle. + + Presently, the most popular methods to implement pagination in a REST + API include offset pagination and keyset pagination. Neither + pagination method requires the server to handle the result set in a + storage area across multiple requests since a new result set is + generated each time a request is submitted. Therefore, they are + preferred to any other method requiring the management of a REST + session. + + Using limit and offset operators represents the traditionally used + method to implement result pagination. Both of them can be used + individually: + + "limit=N": means that the server returns the first N objects of the + result set + + "offset=N": means that the server skips the first N objects and + returns objects starting from position N+1 + + When limit and offset are used together, they provide the ability to + identify a specific portion of the result set. For example, the pair + "offset=100,limit=50" returns the first 50 objects starting from + position 101 of the result set. + + Though easy to implement, offset pagination also includes drawbacks: + + * When offset has a very high value, scrolling the result set could + take some time. + + * It always requires fetching all rows before dropping as many rows + as specified by offset. + + * It may return inconsistent pages when data are frequently updated + (i.e., real-time data). + + Keyset pagination [SEEK] adds a query condition that enables the + selection of the only data not yet returned. This method has been + taken as the basis for the implementation of a "cursor" parameter + [CURSOR] by some REST API providers [CURSOR-API1] [CURSOR-API2]. The + cursor is a URL-safe string opaque to the client and representing a + logical pointer to the first result of the next page. + + Nevertheless, even keyset pagination can be troublesome: + + * It needs at least one key field. + + * It does not allow sorting simply by any field because the sorting + criterion must contain a key. + + * It works best with full composite values supported by database + management systems (i.e., [x,y]>[a,b]); emulation is possible but + inelegant and less efficient. + + * It does not allow direct navigation to arbitrary pages because the + result set must be scrolled in sequential order starting from the + initial page. + + * Implementing bidirectional navigation is tedious because all + comparison and sort operations have to be reversed. + +B.1. Specific Issues Raised by RDAP + + Some additional considerations can be made in the RDAP context: + + * An RDAP object is a conceptual aggregation of information + generally collected from more than one data structure (e.g., + table), and this makes it even harder to implement keyset + pagination, a task that is already quite difficult. For example, + the entity object can include information from different data + structures (registrars, registrants, contacts, resellers), each + one with its key field mapping the RDAP entity handle. + + * Depending on the number of page results as well as the number and + the complexity of the properties of each RDAP object in the + response, the time required by offset pagination to skip the + previous pages could be much faster than the processing time + needed to build the current page. In fact, RDAP objects are + usually formed by information belonging to multiple data + structures and containing multivalued properties (i.e., arrays); + therefore, data selection might be a time-consuming process. This + situation occurs even though the selection is supported by + indexes. + + * Depending on the access levels defined by each RDAP operator, the + increase in complexity and the decrease in flexibility of keyset + pagination in comparison to offset pagination could be considered + impractical. + + Ultimately, both pagination methods have benefits and drawbacks. + +Appendix C. Implementation Notes + + This section contains an overview of the main choices made during the + implementation of the capabilities defined in this document in the + RDAP public test server of Registro.it at the Institute of + Informatics and Telematics of the National Research Council (IIT- + CNR). The content of this section can represent guidance for + implementers who plan to provide RDAP users with those capabilities. + The RDAP public test server can be accessed at + <https://rdap.pubtest.nic.it/>. Further documentation about the + server features is available at <https://rdap.pubtest.nic.it/doc/ + README.html>. + +C.1. Sorting + + If no sort criterion is specified in the query string, the results + are sorted by a default property: "name" for domains and nameservers, + and "handle" for entities. The server supports multiple property + sorting but the "sorting_metadata" object includes only the links to + alternative result set views sorted by a single property just to show + the list of sorting properties allowed for each searchable object. + The server supports all the object-specific sorting properties + described in the specification except for nameserver sorting based on + unicodeName, that is, the "name" sorting property is mapped onto the + "ldhName" response field. Regarding the object-common properties, + sorting by registrationDate, expirationDate, lastChangedDate, and + transferDate is supported. + +C.2. Counting + + The counting operation is implemented through a separate query. Some + relational database management systems support custom operators to + get the total count together with the rows, but the resulting query + can be considerably more expensive than that performed without the + total count. Therefore, as "totalCount" is an optional response + information, always fetching the total number of rows has been + considered an inefficient solution. Furthermore, to avoid the + processing of unnecessary queries, when the "count" parameter is + included in the submitted query, it is not also repeated in the query + strings of the "links" array provided in both "paging_metadata" and + "sorting_metadata" objects. + +C.3. Paging + + The server implements the cursor pagination through the keyset + pagination when sorting by a unique property is requested or the + default sort is applied. Otherwise, it implements the cursor + pagination through the offset pagination. As most relational + database management systems don't support the comparison of full + composite values natively, the implementation of full keyset + pagination seem to be troublesome so, at least initially, a selective + applicability of keyset pagination is advisable. Moreover, the + "cursor" value encodes not only information about pagination but also + about the search pattern and the other query parameters in order to + check the consistency of the entire query string. If the "cursor" + value is inconsistent with the rest of the query string, the server + returns an error response. + +Acknowledgements + + The authors would like to acknowledge Brian Mountford, Tom Harrison, + Karl Heinz Wolf, Jasdip Singh, Erik Kline, Éric Vyncke, Benjamin + Kaduk, and Roman Danyliw for their contributions to the development + of this document. + +Authors' Addresses + + Mario Loffredo + IIT-CNR/Registro.it + Via Moruzzi,1 + 56124 Pisa + Italy + + Email: mario.loffredo@iit.cnr.it + URI: https://www.iit.cnr.it + + + Maurizio Martinelli + IIT-CNR/Registro.it + Via Moruzzi,1 + 56124 Pisa + Italy + + Email: maurizio.martinelli@iit.cnr.it + URI: https://www.iit.cnr.it + + + Scott Hollenbeck + Verisign Labs + 12061 Bluemont Way + Reston, VA 20190 + United States of America + + Email: shollenbeck@verisign.com + URI: https://www.verisignlabs.com/ |