doc: Add RFC documents

1 files changed, 510 insertions, 0 deletions

diff --git a/doc/rfc/rfc8673.txt b/doc/rfc/rfc8673.txt
new file mode 100644
index 0000000..59a3d82
--- /dev/null
+++ b/doc/rfc/rfc8673.txt
@@ -0,0 +1,510 @@
+
+
+
+
+Internet Engineering Task Force (IETF)                          C. Pratt
+Request for Comments: 8673                                              
+Category: Experimental                                        D. Thakore
+ISSN: 2070-1721                                                CableLabs
+                                                                B. Stark
+                                                                    AT&T
+                                                           November 2019
+
+
+                  HTTP Random Access and Live Content
+
+Abstract
+
+   To accommodate byte-range requests for content that has data appended
+   over time, this document defines semantics that allow an HTTP client
+   and a server to perform byte-range GET and HEAD requests that start
+   at an arbitrary byte offset within the representation and end at an
+   indeterminate offset.
+
+Status of This Memo
+
+   This document is not an Internet Standards Track specification; it is
+   published for examination, experimental implementation, and
+   evaluation.
+
+   This document defines an Experimental Protocol for the Internet
+   community.  This document is a product of the Internet Engineering
+   Task Force (IETF).  It represents the consensus of the IETF
+   community.  It has received public review and has been approved for
+   publication by the Internet Engineering Steering Group (IESG).  Not
+   all documents approved by the IESG are candidates for any level of
+   Internet Standard; see Section 2 of RFC 7841.
+
+   Information about the current status of this document, any errata,
+   and how to provide feedback on it may be obtained at
+   https://www.rfc-editor.org/info/rfc8673.
+
+Copyright Notice
+
+   Copyright (c) 2019 IETF Trust and the persons identified as the
+   document authors.  All rights reserved.
+
+   This document is subject to BCP 78 and the IETF Trust's Legal
+   Provisions Relating to IETF Documents
+   (https://trustee.ietf.org/license-info) in effect on the date of
+   publication of this document.  Please review these documents
+   carefully, as they describe your rights and restrictions with respect
+   to this document.  Code Components extracted from this document must
+   include Simplified BSD License text as described in Section 4.e of
+   the Trust Legal Provisions and are provided without warranty as
+   described in the Simplified BSD License.
+
+Table of Contents
+
+   1.  Introduction
+     1.1.  Notational Conventions
+   2.  Performing Range Requests on Random-Access Aggregating (Live)
+           Content
+     2.1.  Establishing the Randomly Accessible Byte Range
+     2.2.  Byte-Range Requests beyond the Randomly Accessible Byte
+           Range
+   3.  Other Applications of Random-Access Aggregating Content
+     3.1.  Requests Starting at the Aggregation/Live Point
+     3.2.  Shift-Buffer Representations
+   4.  Recommendations for Byte-Range Request last-byte-pos Values
+   5.  IANA Considerations
+   6.  Security Considerations
+   7.  References
+     7.1.  Normative References
+     7.2.  Informative References
+   Acknowledgements
+   Authors' Addresses
+
+1.  Introduction
+
+   Some Hypertext Transfer Protocol (HTTP) clients use byte-range
+   requests (range requests using the "bytes" range unit) to transfer
+   select portions of large representations [RFC7233].  In some cases,
+   large representations require content to be continuously or
+   periodically appended, such as representations consisting of live
+   audio or video sources, blockchain databases, and log files.  Clients
+   cannot access the appended/live content using a range request with
+   the "bytes" range unit using the currently defined byte-range
+   semantics without accepting performance or behavior sacrifices that
+   are not acceptable for many applications.
+
+   For instance, HTTP clients have the ability to access appended
+   content on an indeterminate-length resource by transferring the
+   entire representation from the beginning and continuing to read the
+   appended content as it's made available.  Obviously, this is highly
+   inefficient for cases where the representation is large and only the
+   most recently appended content is needed by the client.
+
+   Alternatively, clients can access appended content by sending
+   periodic, open-ended byte-range requests using the last known end
+   byte position as the range start.  Performing low-frequency periodic
+   byte-range requests in this fashion (polling) introduces latency
+   since the client will necessarily be somewhat behind in transferring
+   the aggregated content, effectively resulting in the same kind of
+   latency issues with the segmented content transfer mechanisms in
+   "HTTP Live Streaming" (HLS) [RFC8216] and "Dynamic Adaptive Streaming
+   over HTTP" [MPEG-DASH].  While performing these range requests at
+   higher frequency can reduce this latency, it also incurs more
+   processing overhead and HTTP exchanges as many of the requests will
+   return no content, since content is usually aggregated in groups of
+   bytes (e.g., a video frame, audio sample, block, or log entry).
+
+   This document describes a usage model for range requests that enables
+   efficient retrieval of representations that are appended to over time
+   by using large values and associated semantics for communicating
+   range end positions.  This model allows representations to be
+   progressively delivered by servers as new content is added.  It also
+   ensures compatibility with servers and intermediaries that don't
+   support this technique.
+
+1.1.  Notational Conventions
+
+   This document cites Augmented Backus-Naur Form (ABNF) productions
+   from [RFC7233], using the notation defined in [RFC5234].
+
+2.  Performing Range Requests on Random-Access Aggregating (Live)
+    Content
+
+   This document recommends a two-step process for accessing resources
+   that have indeterminate-length representations.
+
+   Two steps are necessary because of limitations with the range request
+   header fields and the Content-Range response header fields.  A server
+   cannot know from a range request that a client wishes to receive a
+   response that does not have a definite end.  More critically, the
+   header fields do not allow the server to signal that a resource has
+   indeterminate length without also providing a fixed portion of the
+   resource.
+
+   A client first learns that the resource has a representation of
+   indeterminate length by requesting a range of the resource.  The
+   server responds with the range that is available but indicates that
+   the length of the representation is unknown using the existing
+   Content-Range syntax.  See Section 2.1 for details and examples.
+
+   Once the client knows the resource has indeterminate length, it can
+   request a range with a very large end position from the resource.
+   The client chooses an explicit end value larger than can be
+   transferred in the foreseeable term.  A server that supports range
+   requests of indeterminate length signals its understanding of the
+   client's indeterminate range request by indicating that the range it
+   is providing has a range end that exactly matches the client's
+   requested range end rather than a range that is bounded by what is
+   currently available.  See Section 2.2 for details.
+
+2.1.  Establishing the Randomly Accessible Byte Range
+
+   Determining if a representation is continuously aggregating ("live")
+   and determining the randomly accessible byte range can both be
+   performed using the existing definition for an open-ended byte-range
+   request.  Specifically, Section 2.1 of [RFC7233] defines a byte-range
+   request of the form:
+
+      byte-range-spec = first-byte-pos "-" [ last-byte-pos ]
+
+   which allows a client to send a HEAD request with a first-byte-pos
+   and leave last-byte-pos absent.  A server that receives a satisfiable
+   byte-range request (with first-byte-pos smaller than the current
+   representation length) may respond with a 206 status code (Partial
+   Content) with a Content-Range header field indicating the currently
+   satisfiable byte range.  For example:
+
+   HEAD /resource HTTP/1.1
+   Host: example.com
+   Range: bytes=0-
+
+   returns a response of the form:
+
+   HTTP/1.1 206 Partial Content
+   Content-Range: bytes 0-1234567/*
+
+   from the server indicating that (1) the complete representation
+   length is unknown (via the "*" in place of the complete-length field)
+   and (2) only bytes 0-1234567 were accessible at the time the request
+   was processed by the server.  The client can infer from this response
+   that bytes 0-1234567 of the representation can be requested and
+   transfer can be performed immediately.
+
+2.2.  Byte-Range Requests beyond the Randomly Accessible Byte Range
+
+   Once a client has determined that a representation has an
+   indeterminate length and established the byte range that can be
+   accessed, it may want to perform a request with a start position
+   within the randomly accessible content range and an end position at
+   an indefinite/live point -- a point where the byte-range GET request
+   is fulfilled on-demand as the content is aggregated.
+
+   For example, for a large video asset, a client may wish to start a
+   content transfer from the video "key" frame immediately before the
+   point of aggregation and continue the content transfer indefinitely
+   as content is aggregated, in order to support low-latency startup of
+   a live video stream.
+
+   Unlike a byte-range request header field, a byte-content-range
+   response header field cannot be "open-ended", per Section 4.2 of
+   [RFC7233]:
+
+      byte-content-range  = bytes-unit SP
+                           ( byte-range-resp / unsatisfied-range )
+
+      byte-range-resp     = byte-range "/" ( complete-length / "*" )
+      byte-range          = first-byte-pos "-" last-byte-pos
+      unsatisfied-range   = "*/" complete-length
+
+      complete-length     = 1*DIGIT
+
+   Specifically, last-byte-pos is required in byte-range.  So, in order
+   to preserve interoperability with existing HTTP clients, servers,
+   proxies, and caches, this document proposes a mechanism for a client
+   to indicate support for handling an indeterminate-length byte-range
+   response and a mechanism for a server to indicate if/when it's
+   providing an indeterminate-length response.
+
+   A client can indicate support for handling indeterminate-length byte-
+   range responses by providing a very large value for the last-byte-pos
+   in the byte-range request.  For example, a client can perform a byte-
+   range GET request of the form:
+
+   GET /resource HTTP/1.1
+   Host: example.com
+   Range: bytes=1230000-999999999999
+
+   where the last-byte-pos in the request is much larger than the last-
+   byte-pos returned in response to an open-ended byte-range HEAD
+   request, as described above, and much larger than the expected
+   maximum size of the representation.  See Section 6 for range value
+   considerations.
+
+   In response, a server may indicate that it is supplying a
+   continuously aggregating/live response by supplying the client
+   request's last-byte-pos in the Content-Range response header field.
+
+   For example:
+
+   GET /resource HTTP/1.1
+   Host: example.com
+   Range: bytes=1230000-999999999999
+
+   returns
+
+   HTTP/1.1 206 Partial Content
+   Content-Range: bytes 1230000-999999999999/*
+
+   from the server to indicate that the response will start at byte
+   1230000 and continue indefinitely to include all aggregated content,
+   as it becomes available.
+
+   A server that doesn't support or supply a continuously aggregating/
+   live response will supply the currently satisfiable byte range, as it
+   would with an open-ended byte request.
+
+   For example:
+
+   GET /resource HTTP/1.1
+   Host: example.com
+   Range: bytes=1230000-999999999999
+
+   returns
+
+   HTTP/1.1 206 Partial Content
+   Content-Range: bytes 1230000-1234567/*
+
+   from the server to indicate that the response will start at byte
+   1230000, end at byte 1234567, and not include any aggregated content.
+   This is the response expected from a typical HTTP server -- one that
+   doesn't support byte-range requests on aggregating content.
+
+   A client that doesn't receive a response indicating it is
+   continuously aggregating must use other means to access aggregated
+   content (e.g., periodic byte-range polling).
+
+   A server that does return a continuously aggregating/live response
+   should return data using chunked transfer coding and not provide a
+   Content-Length header field.  A 0-length chunk indicates the end of
+   the transfer, per Section 4.1 of [RFC7230].
+
+3.  Other Applications of Random-Access Aggregating Content
+
+3.1.  Requests Starting at the Aggregation/Live Point
+
+   A client that wishes to only receive newly aggregated portions of a
+   resource (i.e., start at the live point) can use a HEAD request to
+   learn what range the server has currently available and initiate an
+   indeterminate-length transfer.  For example:
+
+   HEAD /resource HTTP/1.1
+   Host: example.com
+   Range: bytes=0-
+
+   with the Content-Range response header field indicating the range (or
+   ranges) available.  For example:
+
+   206 Partial Content
+   Content-Range: bytes 0-1234567/*
+
+   The client can then issue a request for a range starting at the end
+   value (using a very large value for the end of a range) and receive
+   only new content.
+
+   For example:
+
+   GET /resource HTTP/1.1
+   Host: example.com
+   Range: bytes=1234567-999999999999
+
+   with a server returning a Content-Range response indicating that an
+   indeterminate-length response body will be provided:
+
+   206 Partial Content
+   Content-Range: bytes 1234567-999999999999/*
+
+3.2.  Shift-Buffer Representations
+
+   Some representations lend themselves to front-end content removal in
+   addition to aggregation.  While still supporting random access,
+   representations of this type have a portion at the beginning (the "0"
+   end) of the randomly accessible region that becomes inaccessible over
+   time.  Examples of this kind of representation would be an audio-
+   video time-shift buffer or a rolling log file.
+
+   For example, a range request containing:
+
+   HEAD /resource HTTP/1.1
+   Host: example.com
+   Range: bytes=0-
+
+   returns
+
+   206 Partial Content
+   Content-Range: bytes 1000000-1234567/*
+
+   indicating that the first 1000000 bytes were not accessible at the
+   time the HEAD request was processed.  Subsequent HEAD requests could
+   return:
+
+   Content-Range: bytes 1000000-1234567/*
+
+   Content-Range: bytes 1010000-1244567/*
+
+   Content-Range: bytes 1020000-1254567/*
+
+   Note though that the difference between the first-byte-pos and last-
+   byte-pos need not be constant.
+
+   The client could then follow up with a GET range request containing:
+
+   GET /resource HTTP/1.1
+   Host: example.com
+   Range: bytes=1020000-999999999999
+
+   with the server returning
+
+   206 Partial Content
+   Content-Range: bytes 1020000-999999999999/*
+
+   with the response body returning bytes 1020000-1254567 immediately
+   and aggregated/live data being returned as the content is aggregated.
+
+   A server that doesn't support or supply a continuously aggregating/
+   live response will supply the currently satisfiable byte range, as it
+   would with an open-ended byte request.  For example:
+
+   GET /resource HTTP/1.1
+   Host: example.com
+   Range: bytes=0-999999999999
+
+   returns
+
+   HTTP/1.1 206 Partial Content
+   Content-Range: bytes 1020000-1254567/*
+
+   from the server to indicate that the response will start at byte
+   1020000, end at byte 1254567, and not include any aggregated content.
+   This is the response expected from a typical HTTP server -- one that
+   doesn't support byte-range requests on aggregating content.
+
+   Note that responses to GET requests performed on shift-buffer
+   representations using Range headers can be cached by intermediaries,
+   since the Content-Range response header indicates which portion of
+   the representation is being returned in the response body.  However,
+   GET requests without a Range header cannot be cached since the first
+   byte of the response body can vary from request to request.  To
+   ensure GET requests without Range headers on shift-buffer
+   representations are not cached, servers hosting a shift-buffer
+   representation should either not return a 200-level response (e.g.,
+   send a 300-level redirect response with a URI that represents the
+   current start of the shift buffer) or indicate the response is non-
+   cacheable.  See [RFC7234] for details on HTTP cache control.
+
+4.  Recommendations for Byte-Range Request last-byte-pos Values
+
+   While it would be ideal to define a single large last-byte-pos value
+   for byte-range requests, there's no single value that would work for
+   all applications and platforms.  For example, JavaScript numbers
+   cannot represent all integer values above 2^^53, so a JavaScript
+   application may want to use 2^^53-1 for last-byte-pos.  This value,
+   however, would not be sufficient for all applications, such as long-
+   duration high-bitrate streams.  So 2^^53-1 (9007199254740991) is
+   recommended as a last-byte-pos unless an application has a good
+   justification to use a smaller or larger value.  For example, if it
+   is always known that the resource won't exceed a value smaller than
+   the recommended last-byte-pos for an application, a smaller value can
+   be used.  If it's likely that an application will utilize resources
+   larger than the recommended last-byte-pos (such as a continuously
+   aggregating high-bitrate media stream), a larger value should be
+   used.
+
+   Note that, in accordance with the semantics defined above, servers
+   that support random-access live content will need to return the last-
+   byte-pos provided in the byte-range request in some cases -- even if
+   the last-byte-pos cannot be represented as a numerical value
+   internally by the server.  As is the case with any continuously
+   aggregating/live resource, the server should terminate the content
+   transfer when the end of the resource is reached -- whether the end
+   is due to termination of the content source or the content length
+   exceeds the server's maximum representation length.
+
+5.  IANA Considerations
+
+   This document has no IANA actions.
+
+6.  Security Considerations
+
+   As described above, servers need to be prepared to receive last-byte-
+   pos values in range requests that are numerically larger than the
+   server implementation supports and return these values in Content-
+   Range response header fields.  Servers should check the last-byte-pos
+   value before converting and storing them into numeric form to ensure
+   the value doesn't cause an overflow or index incorrect data.  The
+   simplest way to satisfy the live-range semantics defined in this
+   document without potential overflow issues is to store the last-byte-
+   pos as a string value and return it in the byte-range Content-Range
+   response header's last-byte-pos field.
+
+7.  References
+
+7.1.  Normative References
+
+   [RFC5234]  Crocker, D., Ed. and P. Overell, "Augmented BNF for Syntax
+              Specifications: ABNF", STD 68, RFC 5234,
+              DOI 10.17487/RFC5234, January 2008,
+              <https://www.rfc-editor.org/info/rfc5234>.
+
+   [RFC7230]  Fielding, R., Ed. and J. Reschke, Ed., "Hypertext Transfer
+              Protocol (HTTP/1.1): Message Syntax and Routing",
+              RFC 7230, DOI 10.17487/RFC7230, June 2014,
+              <https://www.rfc-editor.org/info/rfc7230>.
+
+   [RFC7233]  Fielding, R., Ed., Lafon, Y., Ed., and J. Reschke, Ed.,
+              "Hypertext Transfer Protocol (HTTP/1.1): Range Requests",
+              RFC 7233, DOI 10.17487/RFC7233, June 2014,
+              <https://www.rfc-editor.org/info/rfc7233>.
+
+   [RFC7234]  Fielding, R., Ed., Nottingham, M., Ed., and J. Reschke,
+              Ed., "Hypertext Transfer Protocol (HTTP/1.1): Caching",
+              RFC 7234, DOI 10.17487/RFC7234, June 2014,
+              <https://www.rfc-editor.org/info/rfc7234>.
+
+7.2.  Informative References
+
+   [MPEG-DASH]
+              ISO, "Information technology -- Dynamic adaptive streaming
+              over HTTP (DASH) -- Part 1: Media presentation description
+              and segment formats", ISO/IEC 23009-1, August 2019,
+              <https://www.iso.org/standard/75485.html>.
+
+   [RFC8216]  Pantos, R., Ed. and W. May, "HTTP Live Streaming",
+              RFC 8216, DOI 10.17487/RFC8216, August 2017,
+              <https://www.rfc-editor.org/info/rfc8216>.
+
+Acknowledgements
+
+   The authors would like to thank Mark Nottingham, Patrick McManus,
+   Julian Reschke, Remy Lebeau, Rodger Combs, Thorsten Lohmar, Martin
+   Thompson, Adrien de Croy, K. Morgan, Roy T. Fielding, and Jeremy
+   Poulter.
+
+Authors' Addresses
+
+   Craig Pratt
+   Portland, OR 97229
+   United States of America
+
+   Email: pratt@acm.org
+
+
+   Darshak Thakore
+   CableLabs
+   858 Coal Creek Circle
+   Louisville, CO 80027
+   United States of America
+
+   Email: d.thakore@cablelabs.com
+
+
+   Barbara Stark
+   AT&T
+   Atlanta, GA
+   United States of America
+
+   Email: barbara.stark@att.com