doc: Add RFC documents

1 files changed, 1179 insertions, 0 deletions

diff --git a/doc/rfc/rfc6249.txt b/doc/rfc/rfc6249.txt
new file mode 100644
index 0000000..fe81441
--- /dev/null
+++ b/doc/rfc/rfc6249.txt
@@ -0,0 +1,1179 @@
+
+
+
+
+
+
+Internet Engineering Task Force (IETF)                          A. Bryan
+Request for Comments: 6249                                      N. McNab
+Category: Standards Track                                   T. Tsujikawa
+ISSN: 2070-1721
+                                                                P. Poeml
+                                                             MirrorBrain
+                                                            H. Nordstrom
+                                                               June 2011
+
+
+                   Metalink/HTTP: Mirrors and Hashes
+
+Abstract
+
+   This document specifies Metalink/HTTP: Mirrors and Cryptographic
+   Hashes in HTTP header fields, a different way to get information that
+   is usually contained in the Metalink XML-based download description
+   format.  Metalink/HTTP describes multiple download locations
+   (mirrors), Peer-to-Peer, cryptographic hashes, digital signatures,
+   and other information using existing standards for HTTP header
+   fields.  Metalink clients can use this information to make file
+   transfers more robust and reliable.  Normative requirements for
+   Metalink/HTTP clients and servers are described here.
+
+Status of This Memo
+
+   This is an Internet Standards Track document.
+
+   This document is a product of the Internet Engineering Task Force
+   (IETF).  It represents the consensus of the IETF community.  It has
+   received public review and has been approved for publication by the
+   Internet Engineering Steering Group (IESG).  Further information on
+   Internet Standards is available in Section 2 of RFC 5741.
+
+   Information about the current status of this document, any errata,
+   and how to provide feedback on it may be obtained at
+   http://www.rfc-editor.org/info/rfc6249.
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+Bryan, et al.                Standards Track                    [Page 1]
+
+RFC 6249            Metalink/HTTP: Mirrors and Hashes          June 2011
+
+
+Copyright Notice
+
+   Copyright (c) 2011 IETF Trust and the persons identified as the
+   document authors.  All rights reserved.
+
+   This document is subject to BCP 78 and the IETF Trust's Legal
+   Provisions Relating to IETF Documents
+   (http://trustee.ietf.org/license-info) in effect on the date of
+   publication of this document.  Please review these documents
+   carefully, as they describe your rights and restrictions with respect
+   to this document.  Code Components extracted from this document must
+   include Simplified BSD License text as described in Section 4.e of
+   the Trust Legal Provisions and are provided without warranty as
+   described in the Simplified BSD License.
+
+Table of Contents
+
+   1. Introduction ....................................................3
+      1.1. Example Metalink Server Response ...........................4
+      1.2. Notational Conventions .....................................4
+      1.3. Terminology ................................................5
+   2. Requirements ....................................................5
+   3. Mirrors / Multiple Download Locations ...........................7
+      3.1. Mirror Priority ............................................8
+      3.2. Mirror Geographical Location ...............................8
+      3.3. Coordinated Mirror Policies ................................8
+      3.4. Mirror Depth ...............................................9
+   4. Peer-to-Peer / Metainfo .........................................9
+      4.1. Metalink/XML Files ........................................10
+   5. Signatures .....................................................10
+      5.1. OpenPGP Signatures ........................................10
+      5.2. S/MIME Signatures .........................................10
+   6. Cryptographic Hashes of Whole Documents ........................11
+   7. Client / Server Multi-Source Download Interaction ..............11
+      7.1. Error Prevention, Detection, and Correction ...............15
+           7.1.1. Error Prevention (Early File Mismatch Detection) ...15
+           7.1.2. Error Correction ...................................16
+   8. IANA Considerations ............................................16
+   9. Security Considerations ........................................17
+      9.1. URIs and IRIs .............................................17
+      9.2. Spoofing ..................................................17
+      9.3. Cryptographic Hashes ......................................17
+      9.4. Signing ...................................................17
+   10. References ....................................................18
+      10.1. Normative References .....................................18
+      10.2. Informative References ...................................19
+   Appendix A. Acknowledgements and Contributors .....................20
+
+
+
+
+Bryan, et al.                Standards Track                    [Page 2]
+
+RFC 6249            Metalink/HTTP: Mirrors and Hashes          June 2011
+
+
+1.  Introduction
+
+   Metalink/HTTP is an alternative and complementary representation of
+   Metalink information, which is usually presented as an XML-based
+   document format [RFC5854].  Metalink/HTTP attempts to provide as much
+   functionality as the Metalink/XML format by using existing standards,
+   such as Web Linking [RFC5988], Instance Digests in HTTP [RFC3230],
+   and Entity Tags (also known as ETags) [RFC2616].  Metalink/HTTP is
+   used to list information about a file to be downloaded.  This can
+   include lists of multiple URIs (mirrors), Peer-to-Peer information,
+   cryptographic hashes, and digital signatures.
+
+   Identical copies of a file are frequently accessible in multiple
+   locations on the Internet over a variety of protocols (such as FTP,
+   HTTP, and Peer-to-Peer).  In some cases, users are shown a list of
+   these multiple download locations (mirrors) and must manually select
+   a single one on the basis of geographical location, priority, or
+   bandwidth.  This distributes the load across multiple servers, and
+   should also increase throughput and resilience.  At times, however,
+   individual servers can be slow, outdated, or unreachable, but this
+   cannot be determined until the download has been initiated.  Users
+   will rarely have sufficient information to choose the most
+   appropriate server and will often choose the first in a list, which
+   might not be optimal for their needs, and will lead to a particular
+   server getting a disproportionate share of load.  The use of
+   suboptimal mirrors can lead to the user canceling and restarting the
+   download to try to manually find a better source.  During downloads,
+   errors in transmission can corrupt the file.  There are no easy ways
+   to repair these files.  For large downloads, this can be extremely
+   troublesome.  Any of the number of problems that can occur during a
+   download lead to frustration on the part of users.
+
+   Some popular sites automate the process of selecting mirrors using
+   DNS load balancing, both to approximately balance load between
+   servers, and to direct clients to nearby servers with the hope that
+   this improves throughput.  Indeed, DNS load balancing can balance
+   long-term server load fairly effectively, but it is less effective at
+   delivering the best throughput to users when the bottleneck is not
+   the server but the network.
+
+   This document describes a mechanism by which the benefit of mirrors
+   can be automatically and more effectively realized.  All the
+   information about a download, including mirrors, cryptographic
+   hashes, digital signatures, and more can be transferred in
+   coordinated HTTP header fields, hereafter referred to as a
+   "Metalink".  This Metalink transfers the knowledge of the download
+   server (and mirror database) to the client.  Clients can fall back to
+   other mirrors if the current one has an issue.  With this knowledge,
+
+
+
+Bryan, et al.                Standards Track                    [Page 3]
+
+RFC 6249            Metalink/HTTP: Mirrors and Hashes          June 2011
+
+
+   the client is enabled to work its way to a successful download even
+   under adverse circumstances.  All this can be done without
+   complicated user interaction, and the download can be much more
+   reliable and efficient.  In contrast, a traditional HTTP redirect to
+   a mirror conveys only minimal information -- one link to one server
+   -- and there is no provision in the HTTP protocol to handle failures.
+   Furthermore, in order to provide better load distribution across
+   servers and potentially faster downloads to users, Metalink/HTTP
+   facilitates multi-source downloads, where portions of a file are
+   downloaded from multiple mirrors (and, optionally, Peer-to-Peer)
+   simultaneously.
+
+   Upon connection to a Metalink/HTTP server, a client will receive
+   information about other sources of the same resource and a
+   cryptographic hash of the whole resource.  The client will then be
+   able to request chunks of the file from the various sources,
+   scheduling appropriately in order to maximize the download rate.
+
+1.1.  Example Metalink Server Response
+
+   This example shows a brief Metalink server response with ETag,
+   mirrors, Peer-to-Peer information, Metalink/XML, OpenPGP signature,
+   and a cryptographic hash of the whole file:
+
+   Etag: "thvDyvhfIqlvFe+A9MYgxAfm1q5="
+   Link: <http://www2.example.com/example.ext>; rel=duplicate
+   Link: <ftp://ftp.example.com/example.ext>; rel=duplicate
+   Link: <http://example.com/example.ext.torrent>; rel=describedby;
+   type="application/x-bittorrent"
+   Link: <http://example.com/example.ext.meta4>; rel=describedby;
+   type="application/metalink4+xml"
+   Link: <http://example.com/example.ext.asc>; rel=describedby;
+   type="application/pgp-signature"
+   Digest: SHA-256=MWVkMWQxYTRiMzk5MDQ0MzI3NGU5NDEyZTk5OWY1ZGFmNzgyZTJlO
+   DYzYjRjYzFhOTlmNTQwYzI2M2QwM2U2MQ==
+
+1.2.  Notational Conventions
+
+   This specification describes conformance of Metalink/HTTP.
+
+   The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
+   "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
+   document are to be interpreted as described in BCP 14, [RFC2119], as
+   scoped to those conformance targets.
+
+
+
+
+
+
+
+Bryan, et al.                Standards Track                    [Page 4]
+
+RFC 6249            Metalink/HTTP: Mirrors and Hashes          June 2011
+
+
+1.3.  Terminology
+
+   The following terms, as used in this document, are defined here:
+
+   o  Metalink server: HTTP server that provides a Metalink in HTTP
+      response header fields.
+
+   o  Metalink : A collection of HTTP response header fields from a
+      Metalink server, which is the reply to a GET or HEAD request from
+      a client and which includes Link header fields listing mirrors and
+      Instance Digests listing a cryptographic hash.
+
+   o  Link header field: HTTP response header field, defined in
+      [RFC5988], that can list mirrors and, potentially, other download
+      methods for obtaining a file, along with digital signatures.
+
+   o  Instance Digest: HTTP response header field, defined in [RFC3230],
+      that contains the cryptographic hash of a file, which is used by
+      the Metalink client to verify the integrity of the file once the
+      download has completed.
+
+   o  Entity Tag or ETag: HTTP response header field, defined in
+      [RFC2616], that, if synchronized between the Metalink server and
+      mirror servers, allows Metalink clients to provide advanced
+      features.
+
+   o  Mirror server: Typically, FTP or HTTP servers that "mirror" the
+      Metalink server, i.e., provide identical copies of (at least some)
+      files that are also on the mirrored server.
+
+   o  Metalink clients: Applications that process Metalinks and use them
+      to provide an improved download experience.  They support HTTP and
+      could also support other download protocols like FTP or various
+      Peer-to-Peer methods.
+
+   o  Metalink/XML: An XML file that can contain similar information to
+      an HTTP response header Metalink, such as mirrors and
+      cryptographic hashes.
+
+2.  Requirements
+
+   In this context, "Metalink" refers to Metalink/HTTP, which consists
+   of mirrors and cryptographic hashes in HTTP header fields as
+   described in this document.  "Metalink/XML" refers to the XML format
+   described in [RFC5854].
+
+
+
+
+
+
+Bryan, et al.                Standards Track                    [Page 5]
+
+RFC 6249            Metalink/HTTP: Mirrors and Hashes          June 2011
+
+
+   Metalink resources include Link header fields [RFC5988] to present a
+   list of mirrors in the response to a client request for the resource.
+   Metalink servers MUST include the cryptographic hash of a resource
+   via Instance Digests in HTTP [RFC3230].  Algorithms used in the
+   Instance Digest field are registered in the IANA registry named
+   "Hypertext Transfer Protocol (HTTP) Digest Algorithm Values" at
+   <http://www.iana.org/>.  This document restricts the use of these
+   algorithms.  SHA-256 and SHA-512 were added to the registry by
+   [RFC5843].  Metalinks contain whole file hashes as described in
+   Section 6, and MUST include SHA-256, as specified in [FIPS-180-3].
+   It MAY also include other hashes.
+
+   Metalink servers are HTTP servers with one or more Metalink
+   resources.  Metalink servers MUST support the Link header fields for
+   listing mirrors and MUST support Instance Digests in HTTP [RFC3230].
+   Metalink servers MUST return the same Link header fields and Instance
+   Digests on HEAD requests.  Metalink servers and their associated
+   preferred mirror servers MUST all share the same ETag policy.
+   Metalink servers and their associated normal mirror servers SHOULD
+   all share the same ETag policy.  (See Section 3.3 for the definition
+   of "preferred" and "normal" mirror servers.)  It is up to the
+   administrator of the Metalink server to communicate the details of
+   the shared ETag policy to the administrators of the mirror servers so
+   that the mirror servers can be configured with the same ETag policy.
+   To have the same ETag policy means that ETags are synchronized across
+   servers for resources that are mirrored; i.e., byte-for-byte
+   identical files will have the same ETag on mirrors that they have on
+   the Metalink server.  For example, it would be better to derive an
+   ETag from a cryptographic hash of the file contents than on server-
+   unique filesystem metadata.  Metalink servers SHOULD offer Metalink/
+   XML documents that contain cryptographic hashes of parts of the file
+   (and other information) if error recovery is desirable.
+
+   Mirror servers are typically FTP or HTTP servers that "mirror"
+   another server.  That is, they provide identical copies of (at least
+   some) files that are also on the mirrored server.  Mirror servers
+   SHOULD support serving partial content.  HTTP mirror servers SHOULD
+   share the same ETag policy as the originating Metalink server.  HTTP
+   mirror servers SHOULD support Instance Digests in HTTP [RFC3230]
+   using the same algorithm as the Metalink server.  Optimally, HTTP
+   mirror servers will share the same ETag policy and support Instance
+   Digests in HTTP.  Mirror servers that share the same ETag policy
+   and/or support Instance Digests in HTTP using the same algorithm as a
+   Metalink server are known as preferred mirror servers.
+
+   Metalink clients use the mirrors provided by a Metalink server in
+   Link header fields [RFC5988] but these clients are restricted to
+   using the mirrors provided by the initial Metalink server they
+
+
+
+Bryan, et al.                Standards Track                    [Page 6]
+
+RFC 6249            Metalink/HTTP: Mirrors and Hashes          June 2011
+
+
+   contacted.  If Metalink clients find Link header fields [RFC5988]
+   from mirrors that in turn list mirrors, or from a Metalink server
+   listing itself as a mirror, they MUST discard such Link header fields
+   [RFC5988] to prevent a possible infinite loop.  Metalink clients MUST
+   support HTTP and SHOULD support FTP [RFC0959].  Metalink clients MAY
+   support BitTorrent [BITTORRENT] or other download methods.  Metalink
+   clients SHOULD switch downloads from one mirror to another if a
+   mirror becomes unreachable.  Metalink clients MAY support multi-
+   source, or parallel, downloads, where portions of a file can be
+   downloaded from multiple mirrors simultaneously (and, optionally,
+   from Peer-to-Peer sources).  Metalink clients MUST support Instance
+   Digests in HTTP [RFC3230] by requesting and verifying cryptographic
+   hashes.  Metalink clients SHOULD support error recovery by using the
+   cryptographic hashes of parts of the file listed in Metalink/XML
+   files.  Metalink clients SHOULD support checking digital signatures.
+
+3.  Mirrors / Multiple Download Locations
+
+   Mirrors are specified with the Link header fields [RFC5988] and a
+   relation type of "duplicate" as defined in Section 8.
+
+   The following list contains OPTIONAL attributes, which are defined
+   elsewhere in this document:
+
+   o  "depth" : mirror depth (see Section 3.4).
+
+   o  "geo" : mirror geographical location (see Section 3.2).
+
+   o  "pref" : a preferred mirror server (see Section 3.3).
+
+   o  "pri" : mirror priority (see Section 3.1).
+
+   This example shows a brief Metalink server response with two mirrors
+   only:
+
+   Link: <http://www2.example.com/example.ext>; rel=duplicate;
+   pri=1; pref
+   Link: <ftp://ftp.example.com/example.ext>; rel=duplicate;
+   pri=2; geo=gb; depth=1
+
+   As some organizations can have many mirrors, it is up to the
+   organization to configure the amount of Link header fields the
+   Metalink server will provide.  Such a decision could be a random
+   selection or a hard-coded limit based on network proximity, file
+   size, server load, or other factors.
+
+
+
+
+
+
+Bryan, et al.                Standards Track                    [Page 7]
+
+RFC 6249            Metalink/HTTP: Mirrors and Hashes          June 2011
+
+
+3.1.  Mirror Priority
+
+   Entries for mirror servers MAY have a "pri" value to designate the
+   priority of a mirror.  Valid ranges for the "pri" attribute are from
+   1 to 999999.  Mirror servers with a lower value of the "pri"
+   attribute have a higher priority, while mirrors with an undefined
+   "pri" attribute are considered to have a value of 999999, which is
+   the lowest priority.  For example, a mirror with "pri=10" has higher
+   priority than a mirror with "pri=20".  Metalink clients SHOULD use
+   mirrors with lower "pri" values first, but depending on other
+   conditions, they MAY decide to use other mirrors.
+
+   This is purely an expression of the server's preferences; it is up to
+   the client what it does with this information, particularly with
+   reference to how many servers to use at any one time.
+
+3.2.  Mirror Geographical Location
+
+   Entries for a mirror server MAY have a "geo" value, which is an
+   [ISO3166-1] alpha-2 two-letter country code for the geographical
+   location of the physical server the URI is used to access.  A client
+   MAY use this information to select a mirror, or set of mirrors, that
+   is geographically near (if the client has access to such
+   information), with the aim of reducing network load at inter-country
+   bottlenecks.
+
+3.3.  Coordinated Mirror Policies
+
+   There are two types of mirror servers: preferred and normal.  Entries
+   for preferred HTTP mirror servers have a "pref" value and entries for
+   normal mirrors don't.  Preferred mirror servers are HTTP mirror
+   servers that MUST share the same ETag policy as the originating
+   Metalink server, or if the ETag is not used MUST provide an Instance
+   Digest using the same algorithm as the Metalink server.  Preferred
+   mirrors make it possible for Metalink clients to detect early on,
+   before data is transferred, if the file requested matches the desired
+   file.  This early file mismatch detection is described in
+   Section 7.1.1.  Normal mirrors do not necessarily share the same ETag
+   policy or support Instance Digests using the same algorithm as the
+   Metalink server.  FTP mirrors are considered "normal", as they do not
+   emit ETags or support Instance Digests.
+
+
+
+
+
+
+
+
+
+
+Bryan, et al.                Standards Track                    [Page 8]
+
+RFC 6249            Metalink/HTTP: Mirrors and Hashes          June 2011
+
+
+3.4.  Mirror Depth
+
+   Some mirrors can mirror single files, whole directories, or multiple
+   directories.
+
+   Entries for mirror servers can have a "depth" value, where "depth=0"
+   is the default.  A value of 0 means that only that file is mirrored
+   and that other URI path segments are not.  A value of 1 means that
+   the file and all other files and URI path segments contained in the
+   rightmost URI path segment are mirrored.  For values of N, N-1 URI
+   path segments closer to the Host are mirrored.  A value of 2 means
+   one URI path segment closer to the Host is mirrored, and all files
+   and URI path segments contained are mirrored.  For each higher value,
+   another URI path segment closer to the Host is mirrored.
+
+   This example shows a mirror with a depth value of 4:
+
+   Link: <http://www2.example.com/dir1/dir2/dir3/dir4/dir5/example.ext>;
+   rel=duplicate; pri=1; pref; depth=4
+
+   In the above example, four URI path segments closer to the Host are
+   mirrored, from /dir2/ and all files and directories included.
+
+4.  Peer-to-Peer / Metainfo
+
+   Entries for metainfo files, which describe ways to download a file
+   over Peer-to-Peer networks or otherwise, are specified with the Link
+   header fields [RFC5988] and a relation type of "describedby" and a
+   type parameter that indicates the MIME type of the metadata available
+   at the URI.  Since metainfo files can sometimes describe multiple
+   files, or the filename MAY not be the same on the Metalink server and
+   in the metainfo file but MAY still have the same content, an OPTIONAL
+   "name" attribute can be used.
+
+   The following list contains an OPTIONAL attribute, which is defined
+   in this document:
+
+   o  "name" : a file described within the metainfo file.
+
+   This example shows a brief Metalink server response with .torrent and
+   .meta4:
+
+   Link: <http://example.com/example.ext.torrent>; rel=describedby;
+   type="application/x-bittorrent"; name="differentname.ext"
+   Link: <http://example.com/example.ext.meta4>; rel=describedby;
+   type="application/metalink4+xml"
+
+
+
+
+
+Bryan, et al.                Standards Track                    [Page 9]
+
+RFC 6249            Metalink/HTTP: Mirrors and Hashes          June 2011
+
+
+   Metalink clients MAY support the use of metainfo files for
+   downloading files.
+
+4.1.  Metalink/XML Files
+
+   Metalink/XML files for a given resource MAY be provided in a Link
+   header field as shown in the example in Section 4.  Metalink/XML
+   files are specified in [RFC5854], and they are particularly useful
+   for providing metadata such as cryptographic hashes of parts of a
+   file, allowing a client to recover from errors (see Section 7.1.2).
+   Metalink servers SHOULD provide Metalink/XML files with partial file
+   hashes in Link header fields, and Metalink clients SHOULD use them
+   for error recovery.
+
+5.  Signatures
+
+5.1.  OpenPGP Signatures
+
+   OpenPGP signatures [RFC3156] of requested files are specified with
+   the Link header fields [RFC5988] and a relation type of "describedby"
+   and a type parameter of "application/pgp-signature".
+
+   This example shows a brief Metalink server response with OpenPGP
+   signature only:
+
+   Link: <http://example.com/example.ext.asc>; rel=describedby;
+   type="application/pgp-signature"
+
+   Metalink clients SHOULD support the use of OpenPGP signatures.
+
+5.2.  S/MIME Signatures
+
+   Secure/Multipurpose Internet Mail Extensions (S/MIME) signatures
+   [RFC5751] of requested files are specified with the Link header
+   fields [RFC5988] and a relation type of "describedby" and a type
+   parameter of "application/pkcs7-mime".
+
+   This example shows a brief Metalink server response with S/MIME
+   signature only:
+
+   Link: <http://example.com/example.ext.p7m>; rel=describedby;
+   type="application/pkcs7-mime"
+
+   Metalink clients SHOULD support the use of S/MIME signatures.
+
+
+
+
+
+
+
+Bryan, et al.                Standards Track                   [Page 10]
+
+RFC 6249            Metalink/HTTP: Mirrors and Hashes          June 2011
+
+
+6.  Cryptographic Hashes of Whole Documents
+
+   If Instance Digests are not provided by the Metalink servers, the
+   Link header fields pertaining to this specification MUST be ignored.
+
+   This example shows a brief Metalink server response with ETag,
+   mirror, and cryptographic hash:
+
+   Etag: "thvDyvhfIqlvFe+A9MYgxAfm1q5="
+   Link: <http://www2.example.com/example.ext>; rel=duplicate
+   Digest: SHA-256=MWVkMWQxYTRiMzk5MDQ0MzI3NGU5NDEyZTk5OWY1ZGFmNzgyZTJlO
+   DYzYjRjYzFhOTlmNTQwYzI2M2QwM2U2MQ==
+
+7.  Client / Server Multi-Source Download Interaction
+
+   Metalink clients begin a download with a standard HTTP [RFC2616] GET
+   request to the Metalink server.  Metalink clients MAY use a range
+   limit if desired.
+
+   GET /distribution/example.ext HTTP/1.1
+   Host: www.example.com
+
+   The Metalink server responds with the data and these header fields:
+
+   HTTP/1.1 200 OK
+   Accept-Ranges: bytes
+   Content-Length: 14867603
+   Content-Type: application/x-cd-image
+   Etag: "thvDyvhfIqlvFe+A9MYgxAfm1q5="
+   Link: <http://www2.example.com/example.ext>; rel=duplicate; pref
+   Link: <ftp://ftp.example.com/example.ext>; rel=duplicate
+   Link: <http://example.com/example.ext.torrent>; rel=describedby;
+   type="application/x-bittorrent"
+   Link: <http://example.com/example.ext.meta4>; rel=describedby;
+   type="application/metalink4+xml"
+   Link: <http://example.com/example.ext.asc>; rel=describedby;
+   type="application/pgp-signature"
+   Digest: SHA-256=MWVkMWQxYTRiMzk5MDQ0MzI3NGU5NDEyZTk5OWY1ZGFmNzgyZTJlO
+   DYzYjRjYzFhOTlmNTQwYzI2M2QwM2U2MQ==
+
+   Alternatively, Metalink clients can begin with a HEAD request to the
+   Metalink server to discover mirrors via Link header fields and then
+   skip to making the following decisions on every available mirror
+   server found via the Link header fields.
+
+   After that, the client follows with a GET request to the desired
+   mirrors.
+
+
+
+
+Bryan, et al.                Standards Track                   [Page 11]
+
+RFC 6249            Metalink/HTTP: Mirrors and Hashes          June 2011
+
+
+   From the Metalink server response, the client learns some or all of
+   the following metadata about the requested object, in addition to
+   starting to receive the object:
+
+   o  Mirror locations, with optional attributes describing the mirror's
+      priority, whether it shares the ETag policy of the originating
+      Metalink server, geographical location, and mirror depth.
+
+   o  Instance Digest, which is the whole file cryptographic hash.
+
+   o  ETag.
+
+   o  Object size from the Content-Length header field.
+
+   o  Metalink/XML, which can include partial file cryptographic hashes
+      to repair a file.
+
+   o  Peer-to-Peer information.
+
+   o  Digital signature.
+
+   Next, the Metalink client requests a range of the object from a
+   preferred mirror server, so it can use If-Match conditions:
+
+   GET /example.ext HTTP/1.1
+   Host: www2.example.com
+   Range: bytes=7433802-
+   If-Match: "thvDyvhfIqlvFe+A9MYgxAfm1q5="
+   Referer: http://www.example.com/distribution/example.ext
+
+   Metalink clients SHOULD use preferred mirrors, if possible, as they
+   allow early file mismatch detection as described in Section 7.1.1.
+   Preferred mirrors have coordinated ETags, as described in
+   Section 3.3, and Metalink clients SHOULD use If-Match conditions
+   based on the ETag to quickly detect out-of-date mirrors by using the
+   ETag from the Metalink server response.  Metalink clients SHOULD use
+   partial file cryptographic hashes as described in Section 7.1.2, if
+   available, to detect if the mirror server returned the correct data.
+
+   Optimally, the mirror server also will include an Instance Digest in
+   the mirror response to the client GET request, which the client can
+   also use to detect a mismatch early.  Metalink clients MUST reject
+   individual downloads from mirrors that support Instance Digests if
+   the Instance Digest from the mirror does not match the Instance
+   Digest as reported by the Metalink server and the same algorithm is
+
+
+
+
+
+
+Bryan, et al.                Standards Track                   [Page 12]
+
+RFC 6249            Metalink/HTTP: Mirrors and Hashes          June 2011
+
+
+   used.  If normal mirrors are used, then a mismatch cannot be detected
+   until the completed object is verified.  Errors in transmission and
+   substitutions of incorrect data on mirrors, whether deliberate or
+   accidental, can be detected with error correction as described in
+   Section 7.1.2.
+
+   Here, the preferred mirror server has the correct file (the If-Match
+   conditions match) and responds with a 206 Partial Content HTTP status
+   code and appropriate "Content-Length", "Content-Range", ETag, and
+   Instance Digest header fields.  In this example, the mirror server
+   responds, with data, to the above request:
+
+   HTTP/1.1 206 Partial Content
+   Accept-Ranges: bytes
+   Content-Length: 7433801
+   Content-Range: bytes 7433802-14867602/14867603
+   Etag: "thvDyvhfIqlvFe+A9MYgxAfm1q5="
+   Digest: SHA-256=MWVkMWQxYTRiMzk5MDQ0MzI3NGU5NDEyZTk5OWY1ZGFmNzgyZTJlO
+   DYzYjRjYzFhOTlmNTQwYzI2M2QwM2U2MQ==
+
+   Metalink clients MAY start a number of parallel range requests (one
+   per selected mirror server other than the first) using mirrors
+   provided by the Link header fields with "duplicate" relation type.
+   Metalink clients MUST limit the number of parallel connections to
+   mirror servers, ideally based on observing how the aggregate
+   throughput changes as connections are opened.  It would be pointless
+   to blindly open connections once the path bottleneck is filled.
+   After establishing a new connection, a Metalink client SHOULD monitor
+   whether the aggregate throughput increases over all connections that
+   are part of the download.  The client SHOULD NOT open additional
+   connections during this period.  If the aggregate throughput has
+   increased, the client MAY open an additional connection and repeat
+   these steps.  Otherwise, the client SHOULD NOT open a new connection
+   until an established one closes.  Metalink clients SHOULD use the
+   location of the original GET request in the "Referer" header field
+   for these range requests.
+
+   The Metalink client can determine the size and number of ranges
+   requested from each server, based upon the type and number of mirrors
+   and performance observed from each mirror.  Note that range requests
+   impose an overhead on servers, and clients need to be aware of that
+   and not abuse them.  When downloading a particular file, Metalink
+   clients MUST NOT make more than one concurrent request to each mirror
+   server from which it downloads.
+
+   Metalink clients SHOULD close all but the fastest connection if any
+   range requests generated after the first request end up with a
+   complete response, instead of a partial response (as some mirrors
+
+
+
+Bryan, et al.                Standards Track                   [Page 13]
+
+RFC 6249            Metalink/HTTP: Mirrors and Hashes          June 2011
+
+
+   might not support HTTP ranges), if the goal is the fastest transfer.
+   Metalink clients MAY monitor mirror conditions and dynamically switch
+   between mirrors to achieve the fastest download possible.  Similarly,
+   Metalink clients SHOULD abort extremely slow or stalled range
+   requests and finish the request on other mirrors.  If all ranges have
+   finished except for the final one, the Metalink client can split the
+   final range into multiple range requests to other mirrors so the
+   transfer finishes faster.
+
+   If the first request was a GET, no Range header field was sent, and
+   the client determines later that it will issue a range request, then
+   the client SHOULD close the first connection when it catches up with
+   the other parallel range requests of the same object.  This means the
+   first connection was sacrificed.  Metalink clients can use a HEAD
+   request first, if possible, so that the client can find out if there
+   are any Link header fields, and then range-based requests are
+   undertaken to the mirror servers without sacrificing a first
+   connection.
+
+   Metalink clients MUST reject individual downloads from mirrors where
+   the file size does not match the file size as reported by the
+   Metalink server.
+
+   If a Metalink client does not support certain download methods (such
+   as FTP or BitTorrent) that a file is available from, and there are no
+   available download methods that the client supports, then the
+   download will have no way to complete.
+
+   Metalink clients MUST verify the cryptographic hash of the file once
+   the download has completed.  If the cryptographic hash offered by the
+   Metalink server with Instance Digests does not match the
+   cryptographic hash of the downloaded file, see Section 7.1.2 for a
+   possible way to repair errors.
+
+   If the download cannot be repaired, it is considered corrupt.  The
+   client can attempt to re-download the file.
+
+   Metalink clients that support verifying digital signatures MUST
+   verify digital signatures of requested files if they are included.
+   Digital signatures MUST validate back to a trust anchor as described
+   in the validation rules in [RFC3156] and [RFC5280].
+
+
+
+
+
+
+
+
+
+
+Bryan, et al.                Standards Track                   [Page 14]
+
+RFC 6249            Metalink/HTTP: Mirrors and Hashes          June 2011
+
+
+7.1.  Error Prevention, Detection, and Correction
+
+   Error prevention, or early file mismatch detection, is possible
+   before file transfers with the use of file sizes, ETags, and Instance
+   Digests provided by Metalink servers.  Error detection requires
+   Instance Digests to detect errors in transfer after the transfers
+   have completed.  Error correction, or download repair, is possible
+   with partial file cryptographic hashes.
+
+   Note that cryptographic hashes obtained from Instance Digests are in
+   base64 encoding, while those from Metalink/XML are in hexadecimal.
+
+7.1.1.  Error Prevention (Early File Mismatch Detection)
+
+   In HTTP terms, the merging of ranges from multiple responses SHOULD
+   be verified with a strong validator, which in this context is either
+   an Instance Digest or a shared ETag from that Metalink server that
+   matches with the Instance Digest or ETag provided by a preferred
+   mirror server.  In most cases, it is sufficient that the Metalink
+   server provides mirrors and Instance Digest information, but
+   operation will be more robust and efficient if the mirror servers do
+   implement a shared ETag policy or Instance Digests as well.  There is
+   no need to specify how the ETag is generated, just that it needs to
+   be shared between the Metalink server and the mirror servers.  The
+   benefit of having mirror servers return an Instance Digest is that
+   the client then can detect mismatches early even if ETags are not
+   used.  Mirrors that support both a shared ETag and Instance Digests
+   do provide value, but just one is sufficient for early detection of
+   mismatches.  If the mirror server provides neither shared ETag nor
+   Instance Digest, then early detection of mismatches is not possible
+   unless file length also differs.  Finally, errors are still
+   detectable after the download has completed, when the cryptographic
+   hash of the merged response is verified.
+
+   ETags cannot be used for verifying the integrity of the received
+   content.  If the ETag given by the mirror server matches the ETag
+   given by the Metalink server, then the Metalink client assumes the
+   responses are valid for that object.
+
+   This guarantees that a mismatch will be detected by using only the
+   shared ETag from a Metalink server and mirror server.  Metalink
+   clients will detect an error if ETags do not match, which will
+   prevent accidental merges of ranges from different versions of files
+   with the same name.
+
+   A shared ETag or Instance Digest cannot strictly protect against
+   malicious attacks or server or network errors replacing content.  An
+   attacker can make a mirror server seemingly respond with the expected
+
+
+
+Bryan, et al.                Standards Track                   [Page 15]
+
+RFC 6249            Metalink/HTTP: Mirrors and Hashes          June 2011
+
+
+   Instance Digest or ETags even if the file contents have been
+   modified.  The same goes for various system failures, which would
+   also cause bad data (i.e., corrupted files) to be returned.  The
+   Metalink client has to rely on the Instance Digest returned by the
+   Metalink server in the first response for the verification of the
+   downloaded object as a whole.  To verify the individual ranges, which
+   might have been requested from different sources, see Section 7.1.2.
+
+7.1.2.  Error Correction
+
+   Partial file cryptographic hashes can be used to detect errors during
+   the download.  Metalink servers SHOULD provide Metalink/XML files
+   with partial file hashes in Link header fields as specified in
+   Section 4.1, and Metalink clients SHOULD use them for error
+   correction.
+
+   An error in transfer or a substitution attack will be detected by a
+   cryptographic hash of the object not matching the Instance Digest
+   from the Metalink server.  If the cryptographic hash of the object
+   does not match the Instance Digest from the Metalink server, then the
+   client SHOULD fetch the Metalink/XML (if available).  This may
+   contain partial file cryptographic hashes, which will allow detection
+   of which mirror server returned incorrect data.  Metalink clients
+   SHOULD use the Metalink/XML data to figure out what ranges of the
+   downloaded data can be recovered and what needs to be fetched again.
+
+   Other methods can be used for error correction.  For example, some
+   other metainfo files also include partial file hashes that can be
+   used to check for errors.
+
+8.  IANA Considerations
+
+   Accordingly, IANA has made the following registration to the "Link
+   Relation Types" registry at <http://www.iana.org/>.
+
+   o  Relation Name: duplicate
+
+   o  Description: Refers to a resource whose available representations
+      are byte-for-byte identical with the corresponding representations
+      of the context IRI.
+
+   o  Reference: This specification.
+
+   o  Notes: This relation is for static resources.  That is, an HTTP
+      GET request on any duplicate will return the same representation.
+      It does not make sense for dynamic or POSTable resources and
+      should not be used for them.
+
+
+
+
+Bryan, et al.                Standards Track                   [Page 16]
+
+RFC 6249            Metalink/HTTP: Mirrors and Hashes          June 2011
+
+
+9.  Security Considerations
+
+9.1.  URIs and IRIs
+
+   Metalink clients handle URIs and Internationalized Resource
+   Identifiers (IRIs).  See Section 7 of [RFC3986] and Section 8 of
+   [RFC3987] for security considerations related to their handling
+   and use.
+
+9.2.  Spoofing
+
+   There is potential for spoofing attacks where the attacker publishes
+   Metalinks with false information.  In that case, this could deceive
+   unaware downloaders into downloading a malicious or worthless file.
+   Metalink clients are advised to prevent loops, possibly from a mirror
+   server to a Metalink server and back again, in Section 2.  As with
+   all downloads, users should only download from trusted sources.
+   Also, malicious publishers could attempt a distributed denial-of-
+   service attack by inserting unrelated URIs into Metalinks.  [RFC4732]
+   contains information on amplification attacks and denial-of-service
+   attacks.
+
+9.3.  Cryptographic Hashes
+
+   Currently, some of the digest values defined in Instance Digests in
+   HTTP [RFC3230] are considered insecure.  These include the whole
+   Message Digest family of algorithms, which are not suitable for
+   cryptographically strong verification.  Malicious people could
+   provide files that appear to be identical to another file because of
+   a collision; i.e., the weak cryptographic hashes of the intended file
+   and a substituted malicious file could match.
+
+9.4.  Signing
+
+   Metalinks SHOULD include digital signatures, as described in
+   Section 5.
+
+   Digital signatures provide authentication and message integrity, and
+   enable non-repudiation with proof of origin.
+
+
+
+
+
+
+
+
+
+
+
+
+Bryan, et al.                Standards Track                   [Page 17]
+
+RFC 6249            Metalink/HTTP: Mirrors and Hashes          June 2011
+
+
+10.  References
+
+10.1.  Normative References
+
+   [BITTORRENT]  Cohen, B., "The BitTorrent Protocol Specification",
+                 BITTORRENT 11031, February 2008,
+                 <http://www.bittorrent.org/beps/bep_0003.html>.
+
+   [FIPS-180-3]  National Institute of Standards and Technology (NIST),
+                 "Secure Hash Standard (SHS)", FIPS PUB 180-3,
+                 October 2008.
+
+   [ISO3166-1]   International Organization for Standardization, "ISO
+                 3166-1:2006.  Codes for the representation of names of
+                 countries and their subdivisions -- Part 1: Country
+                 codes", November 2006.
+
+   [RFC0959]     Postel, J. and J. Reynolds, "File Transfer Protocol",
+                 STD 9, RFC 0959, October 1985.
+
+   [RFC2119]     Bradner, S., "Key words for use in RFCs to Indicate
+                 Requirement Levels", BCP 14, RFC 2119, March 1997.
+
+   [RFC2616]     Fielding, R., Gettys, J., Mogul, J., Frystyk, H.,
+                 Masinter, L., Leach, P., and T. Berners-Lee, "Hypertext
+                 Transfer Protocol -- HTTP/1.1", RFC 2616, June 1999.
+
+   [RFC3156]     Elkins, M., Del Torto, D., Levien, R., and T. Roessler,
+                 "MIME Security with OpenPGP", RFC 3156, August 2001.
+
+   [RFC3230]     Mogul, J. and A. Van Hoff, "Instance Digests in HTTP",
+                 RFC 3230, January 2002.
+
+   [RFC3986]     Berners-Lee, T., Fielding, R., and L. Masinter,
+                 "Uniform Resource Identifier (URI): Generic Syntax",
+                 STD 66, RFC 3986, January 2005.
+
+   [RFC3987]     Duerst, M. and M. Suignard, "Internationalized Resource
+                 Identifiers (IRIs)", RFC 3987, January 2005.
+
+   [RFC5280]     Cooper, D., Santesson, S., Farrell, S., Boeyen, S.,
+                 Housley, R., and W. Polk, "Internet X.509 Public Key
+                 Infrastructure Certificate and Certificate Revocation
+                 List (CRL) Profile", RFC 5280, May 2008.
+
+   [RFC5751]     Ramsdell, B. and S. Turner, "Secure/Multipurpose
+                 Internet Mail Extensions (S/MIME) Version 3.2 Message
+                 Specification", RFC 5751, January 2010.
+
+
+
+Bryan, et al.                Standards Track                   [Page 18]
+
+RFC 6249            Metalink/HTTP: Mirrors and Hashes          June 2011
+
+
+   [RFC5854]     Bryan, A., Tsujikawa, T., McNab, N., and P. Poeml, "The
+                 Metalink Download Description Format", RFC 5854,
+                 June 2010.
+
+   [RFC5988]     Nottingham, M., "Web Linking", RFC 5988, October 2010.
+
+10.2.  Informative References
+
+   [RFC4732]     Handley, M., Ed., Rescorla, E., Ed., and IAB, "Internet
+                 Denial-of-Service Considerations", RFC 4732,
+                 December 2006.
+
+   [RFC5843]     Bryan, A., "Additional Hash Algorithms for HTTP
+                 Instance Digests", RFC 5843, April 2010.
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+Bryan, et al.                Standards Track                   [Page 19]
+
+RFC 6249            Metalink/HTTP: Mirrors and Hashes          June 2011
+
+
+Appendix A.  Acknowledgements and Contributors
+
+   Thanks to the Metalink community, Alexey Melnikov, Julian Reschke,
+   Mark Nottingham, Daniel Stenberg, Matt Domsch, Micah Cowan, David
+   Morris, Yves Lafon, Juergen Schoenwaelder, Ben Campbell, Lars Eggert,
+   Sean Turner, Robert Sparks, and the HTTPBIS Working Group.
+
+   Thanks to Alan Ford and Mark Handley for spurring us on to publish
+   this document.
+
+   This document is dedicated to Zimmy Bryan, Juanita Anthony, and Janie
+   Burnett.
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+Bryan, et al.                Standards Track                   [Page 20]
+
+RFC 6249            Metalink/HTTP: Mirrors and Hashes          June 2011
+
+
+Authors' Addresses
+
+   Anthony Bryan
+   Pompano Beach, FL
+   USA
+
+   EMail: anthonybryan@gmail.com
+   URI:   http://www.metalinker.org
+
+
+   Neil McNab
+
+   EMail: neil@nabber.org
+   URI:   http://www.nabber.org
+
+
+   Tatsuhiro Tsujikawa
+   Shiga
+   Japan
+
+   EMail: tatsuhiro.t@gmail.com
+   URI:   http://aria2.sourceforge.net
+
+
+   Dr. med. Peter Poeml
+   MirrorBrain
+   Venloer Str. 317
+   Koeln  50823
+   DE
+
+   Phone: +49 221 6778 333 8
+   EMail: peter@poeml.de
+   URI:   http://mirrorbrain.org/~poeml/
+
+
+   Henrik Nordstrom
+
+   EMail: henrik@henriknordstrom.net
+   URI:   http://www.henriknordstrom.net/
+
+
+
+
+
+
+
+
+
+
+
+
+Bryan, et al.                Standards Track                   [Page 21]
+