doc: Add RFC documents

1 files changed, 1258 insertions, 0 deletions

diff --git a/doc/rfc/rfc9342.txt b/doc/rfc/rfc9342.txt
new file mode 100644
index 0000000..10085a2
--- /dev/null
+++ b/doc/rfc/rfc9342.txt
@@ -0,0 +1,1258 @@
+
+
+
+
+Internet Engineering Task Force (IETF)                  G. Fioccola, Ed.
+Request for Comments: 9342                           Huawei Technologies
+Obsoletes: 8889                                              M. Cociglio
+Category: Standards Track                                 Telecom Italia
+ISSN: 2070-1721                                                 A. Sapio
+                                                       Intel Corporation
+                                                                R. Sisto
+                                                   Politecnico di Torino
+                                                                 T. Zhou
+                                                     Huawei Technologies
+                                                           December 2022
+
+
+                   Clustered Alternate-Marking Method
+
+Abstract
+
+   This document generalizes and expands the Alternate-Marking
+   methodology to measure any kind of unicast flow whose packets can
+   follow several different paths in the network; this can result in a
+   multipoint-to-multipoint network.  The network clustering approach is
+   presented and, for this reason, the technique described here is
+   called "Clustered Alternate Marking".  This document obsoletes RFC
+   8889.
+
+Status of This Memo
+
+   This is an Internet Standards Track document.
+
+   This document is a product of the Internet Engineering Task Force
+   (IETF).  It represents the consensus of the IETF community.  It has
+   received public review and has been approved for publication by the
+   Internet Engineering Steering Group (IESG).  Further information on
+   Internet Standards is available in Section 2 of RFC 7841.
+
+   Information about the current status of this document, any errata,
+   and how to provide feedback on it may be obtained at
+   https://www.rfc-editor.org/info/rfc9342.
+
+Copyright Notice
+
+   Copyright (c) 2022 IETF Trust and the persons identified as the
+   document authors.  All rights reserved.
+
+   This document is subject to BCP 78 and the IETF Trust's Legal
+   Provisions Relating to IETF Documents
+   (https://trustee.ietf.org/license-info) in effect on the date of
+   publication of this document.  Please review these documents
+   carefully, as they describe your rights and restrictions with respect
+   to this document.  Code Components extracted from this document must
+   include Revised BSD License text as described in Section 4.e of the
+   Trust Legal Provisions and are provided without warranty as described
+   in the Revised BSD License.
+
+Table of Contents
+
+   1.  Introduction
+     1.1.  Summary of Changes from RFC 8889
+     1.2.  Requirements Language
+   2.  Terminology
+     2.1.  Correlation with RFC 5644
+   3.  Flow Classification
+   4.  Extension of the Method to Multipoint Flows
+     4.1.  Monitoring Network
+     4.2.  Network Packet Loss
+   5.  Network Clustering
+     5.1.  Algorithm for Clusters Partition
+   6.  Multipoint Packet-Loss Measurement
+   7.  Multipoint Delay and Delay Variation
+     7.1.  Delay Measurements on a Multipoint-Paths Basis
+       7.1.1.  Single-Marking Measurement
+     7.2.  Delay Measurements on a Single-Packet Basis
+       7.2.1.  Single- and Double-Marking Measurement
+       7.2.2.  Hashing Selection Method
+   8.  Synchronization and Timing
+   9.  Recommendations for Deployment
+   10. A Closed-Loop Performance-Management Approach
+   11. Security Considerations
+   12. IANA Considerations
+   13. References
+     13.1.  Normative References
+     13.2.  Informative References
+   Appendix A.  Example of Monitoring Network and Clusters Partition
+   Acknowledgements
+   Contributors
+   Authors' Addresses
+
+1.  Introduction
+
+   The Alternate-Marking Method, as described in [RFC9341], is
+   applicable to a point-to-point path.  The extension proposed in this
+   document applies to the most general case of a multipoint-to-
+   multipoint path and enables flexible and adaptive performance
+   measurements in a managed network.
+
+   The Alternate-Marking methodology consists of splitting the packet
+   flow into marking blocks, and the monitoring parameters are the
+   packet counters and the timestamps for each marking period.  In some
+   applications of the Alternate-Marking Method, a lot of flows and
+   nodes are to be monitored.  Multipoint Alternate Marking aims to
+   reduce these values and makes the performance monitoring more
+   flexible in case a detailed analysis is not needed.  For instance, by
+   considering n measurement points and m monitored flows, the order of
+   magnitude of the packet counters for each time interval is n*m*2 (1
+   per color).  The number of measurement points and monitored flows may
+   vary and depends on the portion of the network we are monitoring
+   (core network, metro network, access network, etc.) and the
+   granularity (for each service, each customer, etc.).  So if both n
+   and m are high values, the packet counters increase a lot, and
+   Multipoint Alternate Marking offers a tool to control these
+   parameters.
+
+   The approach presented in this document is applied only to unicast
+   flows and not to multicast.  Broadcast, Unknown Unicast, and
+   Multicast (BUM) traffic is not considered here, because traffic
+   replication is not covered by the Multipoint Alternate-Marking
+   Method.  Furthermore, it can be applicable to anycast flows, and
+   Equal-Cost Multipath (ECMP) paths can also be easily monitored with
+   this technique.
+
+   [RFC9341] applies to point-to-point unicast flows and BUM traffic.
+   For BUM traffic, the basic method of [RFC9341] can be easily applied
+   link by link; therefore, the multicast flow tree distribution can be
+   split into separate unicast point-to-point links.
+
+   This document and its Clustered Alternate-Marking Method applies to
+   multipoint-to-multipoint unicast flows, anycast, and ECMP flows.
+   Therefore, the Alternate-Marking Method can be extended to any kind
+   of multipoint-to-multipoint paths, and the network-clustering
+   approach presented in this document is the formalization of how to
+   implement this property and allow a flexible and optimized
+   performance measurement support for network management in every
+   situation.
+
+   Without network clustering, it is possible to apply Alternate Marking
+   only for all the network or per single flow.  Instead, with network
+   clustering, it is possible to partition the network into clusters at
+   different levels in order to provide the needed degree of detail.  In
+   some circumstances, it is possible to monitor a multipoint network by
+   monitoring the network clusters, without examining in depth.  In case
+   of problems (packet loss is measured or the delay is too high), the
+   filtering criteria could be enhanced in order to perform a detailed
+   analysis by using a different combination of clusters up to a per-
+   flow measurement as described in [RFC9341].
+
+   This approach fits very well with the Closed-Loop Network and
+   Software-Defined Network (SDN) paradigm, where the SDN orchestrator
+   and the SDN controllers are the brains of the network and can manage
+   flow control to the switches and routers and, in the same way, can
+   calibrate the performance measurements depending on the desired
+   accuracy.  An SDN controller application can orchestrate how
+   accurately the network performance monitoring is set up by applying
+   the Multipoint Alternate Marking as described in this document.
+
+   It is important to underline that, as an extension of [RFC9341], this
+   is a methodology document, so the mechanism that can be used to
+   transmit the counters and the timestamps is out of scope here.
+
+   This document assumes that the blocks are created according to a
+   fixed timer as per [RFC9341].  Switching after a fixed number of
+   packets is possible, but it is out of scope here.
+
+   Note that the fragmented packets' case can be managed with the
+   Alternate-Marking methodology, and the same guidance provided in
+   Section 6 of [RFC9341] also applies in the case of Multipoint
+   Alternate Marking.
+
+1.1.  Summary of Changes from RFC 8889
+
+   This document defines the Multipoint Alternate-Marking Method,
+   addressing ambiguities and overtaking its experimental phase in the
+   original specification [RFC8889].
+
+   The relevant changes are:
+
+   *  Added the recommendations about the different deployments in case
+      one or two flag bits are available for marking (Section 9).
+
+   *  Changed the structure to improve readability.
+
+   *  Removed the wording about the experimentation of the method and
+      considerations that no longer apply.
+
+   *  Revised the description of detailed aspects of the methodology,
+      e.g., synchronization and timing.
+
+   It is important to note that all the changes are totally backward
+   compatible with [RFC8889], and no new additional technique has been
+   introduced in this document compared to [RFC8889].
+
+1.2.  Requirements Language
+
+   The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
+   "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and
+   "OPTIONAL" in this document are to be interpreted as described in
+   BCP 14 [RFC2119] [RFC8174] when, and only when, they appear in all
+   capitals, as shown here.
+
+2.  Terminology
+
+   The use of the basic terms are identical to those found in Alternate
+   Marking [RFC9341].  It is to be remembered that [RFC9341] is valid
+   for point-to-point unicast flows and BUM traffic.
+
+   The important new terms are explained below:
+
+   Multipoint Alternate Marking:  Extension to [RFC9341], valid for
+      multipoint-to-multipoint unicast flows, anycast, and ECMP flows.
+      It can also be referred to as "Clustered Alternate Marking".
+
+   Flow definition:  The concept of flow is generalized in this
+      document.  The identification fields are selected without any
+      constraints and, in general, the flow can be a multipoint-to-
+      multipoint flow, as a result of aggregate point-to-point flows.
+
+   Monitoring network:  Identified with the nodes of the network that
+      are the measurement points (MPs) and the links that are the
+      connections between MPs.  The monitoring network graph depends on
+      the flow definition, so it can represent a specific flow or the
+      entire network topology as aggregate of all the flows.  Each node
+      of the monitoring network cannot be both a source and a
+      destination of the flow.
+
+   Cluster:  Smallest identifiable non-trivial subnetwork of the entire
+      monitoring network graph that still satisfies the condition that
+      the number of packets that go in is the same as the number that go
+      out.  A cluster partition algorithm, such as that found in
+      Section 5.1, can be applied to split the monitoring network into
+      clusters.
+
+   Multipoint metrics:  Packet loss, delay, and delay variation are
+      extended to the case of multipoint flows.  It is possible to
+      compute these metrics on the basis of multipoint paths in order to
+      associate the measurements to a cluster, a combination of
+      clusters, or the entire monitored network.  For delay and delay
+      variation, it is also possible to define the metrics on a single-
+      packet basis, and it means that the multipoint path is used to
+      easily couple packets between input and output nodes of a
+      multipoint path.
+
+   The next section highlights the correlation with the terms used in
+   [RFC5644].
+
+2.1.  Correlation with RFC 5644
+
+   [RFC5644] is limited to active measurements using a single source
+   packet or stream.  Its scope is also limited to observations of
+   corresponding packets along the path (spatial metric) and at one or
+   more destinations (one-to-group) along the path.
+
+   Instead, the scope of this memo is to define multiparty metrics for
+   passive and hybrid measurements in a group-to-group topology with
+   multiple sources and destinations.
+
+   [RFC5644] introduces metric names that can be reused here but have to
+   be extended and rephrased to be applied to the Alternate-Marking
+   schema:
+
+   a.  the multiparty metrics are not only one-to-group metrics but can
+       also be group-to-group metrics;
+
+   b.  the spatial metrics, used for measuring the performance of
+       segments of a source-to-destination path, are applied here to
+       clusters.
+
+3.  Flow Classification
+
+   A unicast flow is identified by all the packets having a set of
+   common characteristics.  This definition is inspired by [RFC7011].
+
+   As an example, by considering a flow as all the packets sharing the
+   same source IP address or the same destination IP address, it is easy
+   to understand that the resulting pattern will not be a point-to-point
+   connection but a point-to-multipoint or multipoint-to-point
+   connection.
+
+   In general, a flow can be defined by a set of selection rules used to
+   match a subset of the packets processed by the network device.  These
+   rules specify a set of Layer 3 and Layer 4 header fields
+   (identification fields) and the relative values that must be found in
+   matching packets.
+
+   The choice of the identification fields directly affects the type of
+   paths that the flow would follow in the network.  In fact, it is
+   possible to relate a set of identification fields with the pattern of
+   the resulting graphs, as listed in Figure 1.
+
+   A TCP 5-tuple usually identifies flows following either a single path
+   or a point-to-point multipath (in the case of load balancing).  On
+   the contrary, a single source address selects aggregate flows
+   following a point-to-multipoint path, while a multipoint-to-point
+   path can be the result of a matching on a single destination address.
+   In the case where a selection rule and its reverse are used for
+   bidirectional measurements, they can correspond to a point-to-
+   multipoint path in one direction and a multipoint-to-point path in
+   the opposite direction.
+
+   So the flows to be monitored are selected into the monitoring points
+   using packet selection rules, which can also change the pattern of
+   the monitored network.
+
+   Note that, more generally, the flow can be defined at different
+   levels based on the potential encapsulation, and additional
+   conditions that are not in the packet header can also be included as
+   part of matching criteria.
+
+   The Alternate-Marking Method is applicable only to a single path (and
+   partially to a one-to-one multipath), so the extension proposed in
+   this document is suitable also for the most general case of
+   multipoint-to-multipoint, which embraces all the other patterns in
+   Figure 1.
+
+          point-to-point single path
+              +------+      +------+      +------+
+          ---<>  R1  <>----<>  R2  <>----<>  R3  <>---
+              +------+      +------+      +------+
+
+
+          point-to-point multipath
+                           +------+
+                          <>  R2  <>
+                         / +------+ \
+                        /            \
+              +------+ /              \ +------+
+          ---<>  R1  <>                <>  R4  <>---
+              +------+ \              / +------+
+                        \            /
+                         \ +------+ /
+                          <>  R3  <>
+                           +------+
+
+
+          point-to-multipoint
+                                      +------+
+                                     <>  R4  <>---
+                                    / +------+
+                          +------+ /
+                         <>  R2  <>
+                        / +------+ \
+              +------+ /            \ +------+
+          ---<>  R1  <>              <>  R5  <>---
+              +------+ \              +------+
+                        \ +------+
+                         <>  R3  <>
+                          +------+ \
+                                    \ +------+
+                                     <>  R6  <>---
+                                      +------+
+
+
+          multipoint-to-point
+              +------+
+          ---<>  R1  <>
+              +------+ \
+                        \ +------+
+                        <>  R4  <>
+                        / +------+ \
+              +------+ /            \ +------+
+          ---<>  R2  <>              <>  R6  <>---
+              +------+              / +------+
+                          +------+ /
+                         <>  R5  <>
+                        / +------+
+              +------+ /
+          ---<>  R3  <>
+              +------+
+
+
+          multipoint-to-multipoint
+              +------+                +------+
+          ---<>  R1  <>              <>  R6  <>---
+              +------+ \            / +------+
+                        \ +------+ /
+                         <>  R4  <>
+                          +------+ \
+              +------+              \ +------+
+          ---<>  R2  <>             <>  R7  <>---
+              +------+ \            / +------+
+                        \ +------+ /
+                         <>  R5  <>
+                        / +------+ \
+              +------+ /            \ +------+
+          ---<>  R3  <>              <>  R8  <>---
+              +------+                +------+
+
+                       Figure 1: Flow Classification
+
+   The case of unicast flow is considered in Figure 1.  The anycast flow
+   is also covered, since it is only a special case of a unicast flow if
+   routing is stable throughout the measurement period.  Furthermore, an
+   ECMP flow is in scope by definition, since it is a point-to-
+   multipoint unicast flow.
+
+4.  Extension of the Method to Multipoint Flows
+
+   By using the Alternate-Marking Method, only point-to-point paths can
+   be monitored.  To have an IP (TCP/UDP) flow that follows a point-to-
+   point path, in general we have to define, with a specific value, 5
+   identification fields (IP Source, IP Destination, Transport Protocol,
+   Source Port, and Destination Port).
+
+   Multipoint Alternate Marking enables the performance measurement for
+   multipoint flows selected by identification fields without any
+   constraints (even the entire network production traffic).  It is also
+   possible to use multiple marking points for the same monitored flow.
+
+4.1.  Monitoring Network
+
+   The monitoring network is deduced from the production network by
+   identifying the nodes of the graph that are the measurement points
+   and the links that are the connections between measurement points.
+   It can be modeled as a set of nodes and a set of directed arcs that
+   connect pairs of nodes.
+
+   There are some techniques that can help with the building of the
+   monitoring network (as an example, see [RFC9198]).  In general, there
+   are different options: the monitoring network can be obtained by
+   considering all the possible paths for the traffic or periodically
+   checking the traffic (e.g., daily, weekly, and monthly) and updating
+   the graph as appropriate, but this is up to the Network Management
+   System (NMS) configuration.
+
+   So a graph model of the monitoring network can be built according to
+   the Alternate-Marking Method, where the monitored interfaces and
+   links are identified.  Only the measurement points and links where
+   the traffic has flowed have to be represented in the graph.
+
+   A simple example of a monitoring network graph is shown in
+   Appendix A.
+
+   Each monitoring point is characterized by the packet counter that
+   refers only to a marking period of the monitored flow.  Also, it is
+   assumed that there is a monitoring point at all possible egress
+   points of the multipoint monitored network.
+
+   The same is also applicable for the delay, but it will be described
+   in the following sections.
+
+   The rest of the document assumes that the traffic is going from left
+   to right in order to simplify the explanation.  But the analysis done
+   for one direction applies equally to all directions.
+
+4.2.  Network Packet Loss
+
+   Since all the packets of the considered flow leaving the network have
+   previously entered the network, the number of packets counted by all
+   the input nodes is always greater than, or equal to, the number of
+   packets counted by all the output nodes.  It is assumed that routing
+   is stable during the measurement period while packet fragmentation
+   must be handled as described in [RFC9341].
+
+   In the case of no packet loss occurring in the marking period, if all
+   the input and output points of the network domain to be monitored are
+   measurement points, the sum of the number of packets on all the
+   ingress interfaces equals the number on egress interfaces for the
+   monitored flow.  In this circumstance, if no packet loss occurs, the
+   intermediate measurement points only have the task of splitting the
+   measurement.
+
+   It is possible to define the network packet loss of one monitored
+   flow for a single period.  In a packet network, the number of lost
+   packets is the number of packets counted by the input nodes minus the
+   number of packets counted by the output nodes.  This is true for
+   every packet flow in each marking period.
+
+   The monitored network packet loss with n input nodes and m output
+   nodes is given by:
+
+   PL = (PI1 + PI2 +...+ PIn) - (PO1 + PO2 +...+ POm)
+
+   where:
+
+   *  PL is the network packet loss (number of lost packets);
+
+   *  PIi is the number of packets flowed through the i-th input node in
+      this period; and
+
+   *  POj is the number of packets flowed through the j-th output node
+      in this period.
+
+   The equation is applied on a per-time-interval basis and a per-flow
+   basis:
+
+   *  The reference interval is the Alternate-Marking period, as defined
+      in [RFC9341].
+
+   *  The flow definition is generalized here.  Indeed, as described
+      before, a multipoint packet flow is considered, and the
+      identification fields can be selected without any constraints.
+
+5.  Network Clustering
+
+   The previous equation of Section 4.2 can determine the number of
+   packets lost globally in the monitored network, exploiting only the
+   data provided by the counters in the input and output nodes.
+
+   In addition, it is possible to leverage the data provided by the
+   other counters in the network to converge on the smallest
+   identifiable subnetworks where the losses occur.
+
+   As defined in Section 2, a cluster is a non-trivial subnetwork of the
+   entire monitoring network graph that still satisfies the condition
+   that the number of packets that go in is the same as the number that
+   go out, if no packet loss occurs.  According to this definition, a
+   cluster should contain all the arcs emanating from its input nodes
+   and all the arcs terminating at its output nodes.  This ensures that
+   we can count all the packets (and only those) exiting an input node
+   again at the output node, whatever path they follow.
+
+   As for the entire monitoring network graph, the cluster is defined on
+   a per-flow basis.  In a completely monitored network (a network where
+   every network interface is monitored), each network device
+   corresponds to a cluster, and each physical link corresponds to two
+   clusters (one for each device).
+
+   Clusters can have different sizes depending on the flow-filtering
+   criteria adopted.
+
+   Moreover, sometimes clusters can be optionally simplified.  For
+   example, when two monitored interfaces are divided by a single router
+   (one is the input interface, the other is the output interface, and
+   the router has only these two interfaces), instead of counting
+   exactly twice, upon entering and leaving, it is possible to consider
+   a single measurement point.  In this case, we do not care about the
+   internal packet loss of the router.
+
+   It is worth highlighting that it might also be convenient to define
+   clusters based on the topological information so that they are
+   applicable to all the possible flows in the monitored network.
+
+   Note that, in case of translation or encapsulation, the cluster
+   properties must also be invariant.
+
+5.1.  Algorithm for Clusters Partition
+
+   A simple algorithm can be applied in order to split the monitoring
+   network into clusters.  This can be done for each direction
+   separately; indeed, a node cannot be both a source and a destination.
+   The clusters partition is based on the monitoring network graph,
+   which can be valid for a specific flow or can also be general and
+   valid for the entire network topology.
+
+   It is a two-step algorithm:
+
+   *  Group the links where there is the same starting node;
+
+   *  Join the grouped links with at least one ending node in common.
+
+   Considering that the links are unidirectional, the first step implies
+   listing all the links as connections between two nodes and grouping
+   the different links if they have the same starting node.  Note that
+   it is possible to start from any link, and the procedure will work.
+   Following this classification, the second step implies eventually
+   joining the groups classified in the first step by looking at the
+   ending nodes.  If different groups have at least one common ending
+   node, they are put together and belong to the same set.  After the
+   application of the two steps of the algorithm, each one of the
+   composed sets of links, together with the endpoint nodes, constitutes
+   a cluster.
+
+   A simple application of the clusters partition is shown in
+   Appendix A.
+
+   The algorithm, as applied in the example of a point-to-multipoint
+   network, works for the more general case of a multipoint-to-
+   multipoint network in the same way.  It should be highlighted that
+   for a multipoint-to-multipoint network, the multiple sources MUST
+   mark the traffic coherently and MUST be synchronized with all the
+   other nodes according to the timing requirements detailed in
+   Section 8.
+
+   When the clusters partition is done, the calculation of packet loss,
+   delay, and delay variation can be made on a cluster basis.  Note that
+   the packet counters for each marking period permit calculating the
+   packet rate on a cluster basis, so Committed Information Rate (CIR)
+   and Excess Information Rate (EIR) could also be deduced on a cluster
+   basis.
+
+   Obviously, by combining some clusters in a new connected subnetwork,
+   the packet-loss rule is still true.  So it is also possible to
+   consider combinations of clusters if and where it suits.
+
+   In this way, in a very large network, there is no need to configure
+   detailed filter criteria to inspect the traffic.  It is possible to
+   check a multipoint network and, in case of problems, go deep with a
+   step-by-step cluster analysis, but only for the cluster or
+   combination of clusters where the problem happens.
+
+   In summary, once a flow is defined, the algorithm to build the
+   clusters partition is based on topological information; therefore, it
+   considers all the possible links and nodes that could potentially be
+   crossed by the given flow, even if there is no traffic.  So if the
+   flow does not enter or traverse all the nodes, the counters have a
+   non-zero value for the involved nodes and a zero value for the other
+   nodes without traffic; but in the end, all the formulas are still
+   valid.
+
+   The algorithm described above is an iterative clustering algorithm
+   since it executes steps in iterations, but it is also possible to
+   apply a recursive clustering algorithm as detailed in
+   [IEEE-ACM-TON-MPNPM].
+
+   The complete and mathematical analysis of the possible algorithms for
+   the clusters partition, including the considerations in terms of
+   efficiency and a comparison between the different methods, is in the
+   paper [IEEE-ACM-TON-MPNPM].
+
+6.  Multipoint Packet-Loss Measurement
+
+   The network packet loss, defined in Section 4.2, valid for the entire
+   monitored flow, can easily be extended to each multipoint path (e.g.,
+   the whole multipoint network, a cluster, or a combination of
+   clusters).  In this way, it is possible to calculate Multipoint
+   Packet Loss that is representative of a multipoint path.
+
+   The same equation of Section 4.2 can be applied to a generic
+   multipoint path like a cluster or a combination of clusters, where
+   the number of packets are those entering and leaving the multipoint
+   path.
+
+   By applying the algorithm described in Section 5.1, it is possible to
+   split the monitoring network into clusters.  Then, packet loss can be
+   measured on a cluster basis for each single period by considering the
+   counters of the input and output nodes that belong to the specific
+   cluster.  This can be done for every packet flow in each marking
+   period.
+
+7.  Multipoint Delay and Delay Variation
+
+   The same line of reasoning can be applied to delay and delay
+   variation.  The delay measurement methods defined in [RFC9341] can be
+   extended to the case of multipoint flows.  It is important to
+   highlight that both delay and delay-variation measurements make sense
+   in a multipoint path.  The delay variation is calculated by
+   considering the same packets selected for measuring the delay.
+
+   In general, it is possible to perform delay and delay-variation
+   measurements on the basis of multipoint paths or single packets:
+
+   *  Delay measurements on the basis of multipoint paths mean that the
+      delay value is representative of an entire multipoint path (e.g.,
+      the whole multipoint network, a cluster, or a combination of
+      clusters).
+
+   *  Delay measurements on a single-packet basis mean that it is
+      possible to use a multipoint path just to easily couple packets
+      between input and output nodes of a multipoint path, as described
+      in the following sections.
+
+7.1.  Delay Measurements on a Multipoint-Paths Basis
+
+7.1.1.  Single-Marking Measurement
+
+   Mean delay and mean delay-variation measurements can also be
+   generalized to the case of multipoint flows.  It is possible to
+   compute the average one-way delay of packets in one block, a cluster,
+   or the entire monitored network.
+
+   The average latency can be measured as the difference between the
+   weighted averages of the mean timestamps of the sets of output and
+   input nodes.  This means that, in the calculation, it is possible to
+   weigh the timestamps with the number of packets for each endpoint.
+
+   Note that, since the one-way delay value is representative of a
+   multipoint path, it is possible to calculate the two-way delay of a
+   multipoint path by summing the one-way delays of the two directions,
+   similarly to [RFC9341].
+
+7.2.  Delay Measurements on a Single-Packet Basis
+
+7.2.1.  Single- and Double-Marking Measurement
+
+   Delay and delay-variation measurements associated with only one
+   picked packet per period, both single and double marked, cannot be
+   easily performed in a multipoint scenario since there are some
+   limitations:
+
+   *  Single Marking based on the first/last packet of the interval does
+      not work properly.  Indeed, by considering a point-to-multipoint
+      scenario, it is not possible to recognize which path the first
+      packet of each block takes over the multipoint flow in order to
+      correlate it.  This is also true for the general case of the
+      multipoint-to-multipoint scenario.
+
+   *  Double Marking or multiplexed marking works but only through
+      statistical means.  In a point-to-multipoint scenario, by
+      selecting only a single packet with the second marking for each
+      block, it is possible to follow and calculate the delay for that
+      picked packet.  But the measurement can only be done for a single
+      path in each marking period.  To traverse all the paths of the
+      multipoint flow, it can theoretically be done by continuing the
+      measurement for the following marking periods and expect to span
+      all the paths.  In the general case of a multipoint-to-multipoint
+      path, it is also needed to take into account the multiple source
+      nodes that complicate the correlation of the samples.  In this
+      case, it can be possible to select the second marked packet only
+      for a source node at a time for each block and cover the remaining
+      source nodes one by one in the next marking periods.
+
+   Note that, since the one-way delay measurement is done on a single-
+   packet basis, it is always possible to calculate the two-way delay,
+   but it is not immediate since it is necessary to couple the
+   measurement on each single path with the opposite direction.  In this
+   case, the NMS can do the calculation.
+
+   If a delay measurement is performed for more than one picked packet
+   and for all the paths of the multipoint flow in the same marking
+   period, neither the Single- nor the Double-Marking Method are
+   applicable in the multipoint scenario.  The packets follow different
+   paths, and it becomes very difficult to correlate marked packets in a
+   multipoint-to-multipoint path if there are more than one per period.
+
+   A desirable option is to monitor simultaneously all the paths of a
+   multipoint path in the same marking period.  For this purpose,
+   hashing can be used, as reported in the next section.
+
+7.2.2.  Hashing Selection Method
+
+   Sampling and filtering techniques for IP packet selection are
+   introduced in [RFC5474] and [RFC5475].
+
+   The hash-based selection methodologies for delay measurement can work
+   in a multipoint-to-multipoint path and can be used either coupled to
+   mean delay or standalone.
+
+   [IEEE-NETWORK-PNPM] introduces how to use the hash method (see
+   [RFC5474] and [RFC5475]) combined with the Alternate-Marking Method
+   for point-to-point flows.  It is also called "Mixed Hashed Marking"
+   because it refers to the conjunction of the marking method and the
+   hashing technique.  It involves only the Single Marking; indeed, it
+   is supposed that Double Marking is not used with hashing.  The
+   coupling of the Single Marking with the hashing selection allows
+   choosing a simplified hash function since the alternation of blocks
+   gives temporal boundaries for the hashing samples.  The marking
+   batches anchor the samples selected with hashing, and this eases the
+   correlation of the hashing packets along the path.  For example, in
+   case a hashed sample is lost, it is confined to the considered block
+   without affecting the identification of the samples for the following
+   blocks.
+
+   Using the hash-based sampling, the number of samples in each block
+   may vary a lot because it depends on the packet rate that is
+   variable.  A dynamic approach can help to have an almost fixed number
+   of samples for each marking period, and this is a better option for
+   making regular measurements over time.  In the hash-based sampling,
+   Alternate Marking is used to create periods, so that hash-based
+   samples are divided into batches, which allows anchoring the selected
+   samples to their period.  Moreover, in a dynamic hash-based sampling,
+   it can be possible to dynamically adapt the length of the hash value
+   to meet the current packet rate, so that the number of samples is
+   bounded in each marking period.
+
+   In a multipoint environment, the hashing selection may be the
+   solution for performing delay measurements on specific packets and
+   overcoming the Single- and Double-Marking limitations.
+
+8.  Synchronization and Timing
+
+   It is important to consider the timing aspects, since out-of-order
+   packets happen and have to be handled as well, as described in
+   [RFC9341].
+
+   However, in a multisource situation, an additional issue has to be
+   considered.  With multipoint path, the egress nodes will receive
+   alternate marked packets in random order from different ingress
+   nodes, and this must not affect the measurement.
+
+   So, if we analyze a multipoint-to-multipoint path with more than one
+   marking node, it is important to recognize the reference measurement
+   interval.  In general, the measurement interval for describing the
+   results is the interval of the marking node that is more aligned with
+   the start of the measurement, as reported in Figure 2.
+
+   Note that the mark switching approach based on a fixed timer is
+   considered in this document.
+
+           time -> start         stop
+           T(R1)   |-------------|
+           T(R2)     |-------------|
+           T(R3)        |------------|
+
+                       Figure 2: Measurement Interval
+
+   In Figure 2, it is assumed that the node with the earliest clock (R1)
+   identifies the right starting and ending times of the measurement,
+   but it is just an assumption and other possibilities could occur.  So
+   in this case, T(R1) is the measurement interval, and its recognition
+   is essential in order to make comparisons with other active/passive/
+   hybrid packet-loss metrics.
+
+   Regarding the timing constraints of the methodology, [RFC9341]
+   already describes two contributions that are taken into account: the
+   clock error between network devices and the network delay between the
+   measurement points.
+
+   When we expand to a multipoint environment, we have to consider that
+   there are more marking nodes that mark the traffic based on
+   synchronized clock time.  But, due to different synchronization
+   issues that may happen, the marking batches can be of different
+   lengths and with different offsets when they get mixed in a
+   multipoint flow.  According to [RFC9341], the maximum clock skew
+   between the network devices is A.  Therefore, the additional gap that
+   results between the multiple sources can be incorporated into A.
+
+   ...BBBBBBBBB | AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA | BBBBBBBBB...
+                |<======================================>|
+                |                   L                    |
+   ...=========>|<==================><==================>|<==========...
+                |         L/2                L/2         |
+                |<====>|                          |<====>|
+                   d   |                          |   d
+                       |<========================>|
+                       available counting interval
+
+                          Figure 3: Timing Aspects
+
+   Moreover, it is assumed that the multipoint path can be modeled with
+   a normal distribution; otherwise, it is necessary to reformulate
+   based on the type of distribution.  Under this assumption, the
+   definition of the guard band d is still applicable as defined in
+   [RFC9341] and is given by:
+
+   d = A + D_avg + 3*D_stddev,
+
+   where A is the clock accuracy, D_avg is the average value of the
+   network delay, and D_stddev is the standard deviation of the delay.
+
+   As shown in Figure 3 and according to [RFC9341], the condition that
+   must be satisfied to enable the method to function properly is that
+   the available counting interval must be > 0, and that means:
+
+   L - 2d > 0.
+
+   This formula needs to be verified for each measurement point on the
+   multipoint path.
+
+   Note that the timing considerations are valid for both packet loss
+   and delay measurements.
+
+9.  Recommendations for Deployment
+
+   The methodology described in the previous sections can be applied to
+   various performance measurement problems, as also explained in
+   [RFC9341].  [RFC8889] reports experimental examples and
+   [IEEE-NETWORK-PNPM] also includes some information about the
+   deployment experience.
+
+   Different deployments are possible using one flag bit, two flag bits,
+   or the hashing selection:
+
+   One flag:  packet-loss measurement MUST be done as described in
+      Section 6 by applying the network clustering partition described
+      in Section 5.  Delay measurement MUST be done according to the
+      mean delay calculation representative of the multipoint path, as
+      described in Section 7.1.1.  A Single-Marking Method based on the
+      first/last packet of the interval cannot be applied, as mentioned
+      in Section 7.2.1.
+
+   Two flags:  packet-loss measurement MUST be done as described in
+      Section 6 by applying the network clustering partition described
+      in Section 5.  Delay measurement SHOULD be done on a single-packet
+      basis according to the Double-Marking Method (Section 7.2.1).  In
+      this case, the mean delay calculation (Section 7.1.1) MAY also be
+      used as a representative value of a multipoint path.  The choice
+      depends on the kind of information that is needed, as further
+      detailed below.
+
+   One flag with hash-based selection:  packet-loss measurement MUST be
+      done as described in Section 6 by applying the network clustering
+      partition described in Section 5.  Hash-based selection
+      methodologies, introduced in Section 7.2.2, MUST be used for delay
+      measurement.
+
+   Similarly to [RFC9341], there are some operational guidelines to
+   consider when deciding which recommendation to use (i.e., one flag or
+   two flags or one flag with hash-based selection.
+
+   *  The Multipoint Alternate-Marking Method utilizes specific flags in
+      the packet header, so an important factor is the number of flags
+      available for the implementation.  Indeed, if there is only one
+      flag available, there is no other way, while if two flags are
+      available, the option with two flags can be considered in
+      comparison with the option of one flag with hash-based selection.
+
+   *  The duration of the Alternate-Marking period affects the frequency
+      of the measurement, and this is a parameter that can be decided on
+      the basis of the required temporal sampling.  But it cannot be
+      freely chosen, as explained in Section 8.
+
+   *  The Multipoint Alternate-Marking methodologies enable packet loss,
+      delay, and delay variation calculation, but in accordance with the
+      method used (e.g., Single Marking, Double Marking, or hashing
+      selection), there is a different kind of information that can be
+      derived.  For example, to get measurements on a multipoint-paths
+      basis, one flag can be used.  To get measurements on a single-
+      packet basis, two flags are preferred.  For this reason, the type
+      of data needed in the specific scenario is an additional element
+      to take into account.
+
+   *  The Multipoint Alternate-Marking Methods imply different
+      computational load depending on the method employed.  Therefore,
+      the available computational resources on the measurement points
+      can also influence the choice.  As an example, mean delay
+      calculation may require more processing, and it may not be the
+      best option to minimize the computational load.
+
+   The experiment with Multipoint Alternate-Marking methodologies
+   confirmed the benefits of the Alternate-Marking methodology [RFC9341]
+   as its extension to the general case of multipoint-to-multipoint
+   scenarios.
+
+   The Multipoint Alternate-Marking Method MUST only be applied to
+   controlled domains, as per [RFC9341].
+
+10.  A Closed-Loop Performance-Management Approach
+
+   The Multipoint Alternate-Marking framework that is introduced in this
+   document adds flexibility to Performance Management (PM), because it
+   can reduce the order of magnitude of the packet counters.  This
+   allows an SDN orchestrator to supervise, control, and manage PM in
+   large networks.
+
+   The monitoring network can be considered as a whole or split into
+   clusters that are the smallest subnetworks (group-to-group segments),
+   maintaining the packet-loss property for each subnetwork.  The
+   clusters can also be combined in new, connected subnetworks at
+   different levels, depending on the detail we want to achieve.
+
+   An SDN controller or an NMS can calibrate performance measurements,
+   since they are aware of the network topology.  They can start without
+   examining in depth.  In case of necessity (packet loss is measured or
+   the delay is too high), the filtering criteria could be immediately
+   reconfigured in order to perform a partition of the network by using
+   clusters and/or different combinations of clusters.  In this way, the
+   problem can be localized in a specific cluster or a single
+   combination of clusters, and a more detailed analysis can be
+   performed step by step by successive approximation up to a point-to-
+   point flow detailed analysis.  This is the so-called "closed loop".
+
+   This approach can be called "network zooming" and can be performed in
+   two different ways:
+
+   1.  change the traffic filter and select more detailed flows;
+
+   2.  activate new measurement points by defining more specified
+       clusters.
+
+   The network-zooming approach implies that some filters or rules are
+   changed; therefore, there is a transient time to wait once the new
+   network configuration takes effect.  This time can be determined by
+   the network orchestrator/controller, based on the network conditions.
+
+   For example, if the network zooming identifies the performance
+   problem for the traffic coming from a specific source, we need to
+   recognize the marked signal from this specific source node and its
+   relative path.  For this purpose, we can activate all the available
+   measurement points and better specify the flow filter criteria (i.e.,
+   5-tuple).  As an alternative, it can be enough to select packets from
+   the specific source for delay measurements; in this case, it is
+   possible to apply the hashing technique, as mentioned in the previous
+   sections.
+
+   [OPSAWG-IFIT-FRAMEWORK] defines an architecture where the centralized
+   data collector and network management can apply the intelligent and
+   flexible Alternate-Marking algorithm as previously described.
+
+   As for [RFC9341], it is possible to classify the traffic and mark a
+   portion of the total traffic.  For each period, the packet rate and
+   bandwidth are calculated from the number of packets.  In this way,
+   the network orchestrator becomes aware if the traffic rate surpasses
+   limits.  In addition, more precision can be obtained by reducing the
+   marking period; indeed, some implementations use a marking period of
+   1 sec or less.
+
+   In addition, an SDN controller could also collect the measurement
+   history.
+
+   It is important to mention that the Multipoint Alternate-Marking
+   framework also helps Traffic Visualization.  Indeed, this methodology
+   is very useful for identifying which path or cluster is crossed by
+   the flow.
+
+11.  Security Considerations
+
+   This document specifies a method of performing measurements that does
+   not directly affect Internet security or applications that run on the
+   Internet.  However, implementation of this method must be mindful of
+   security and privacy concerns, as explained in [RFC9341].
+
+12.  IANA Considerations
+
+   This document has no IANA actions.
+
+13.  References
+
+13.1.  Normative References
+
+   [RFC2119]  Bradner, S., "Key words for use in RFCs to Indicate
+              Requirement Levels", BCP 14, RFC 2119,
+              DOI 10.17487/RFC2119, March 1997,
+              <https://www.rfc-editor.org/info/rfc2119>.
+
+   [RFC5475]  Zseby, T., Molina, M., Duffield, N., Niccolini, S., and F.
+              Raspall, "Sampling and Filtering Techniques for IP Packet
+              Selection", RFC 5475, DOI 10.17487/RFC5475, March 2009,
+              <https://www.rfc-editor.org/info/rfc5475>.
+
+   [RFC5644]  Stephan, E., Liang, L., and A. Morton, "IP Performance
+              Metrics (IPPM): Spatial and Multicast", RFC 5644,
+              DOI 10.17487/RFC5644, October 2009,
+              <https://www.rfc-editor.org/info/rfc5644>.
+
+   [RFC8174]  Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC
+              2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174,
+              May 2017, <https://www.rfc-editor.org/info/rfc8174>.
+
+   [RFC9341]  Fioccola, G., Ed., Cociglio, M., Mirsky, G., Mizrahi, T.,
+              and T. Zhou, "Alternate-Marking Method", RFC 9341,
+              DOI 10.17487/RFC9341, December 2022,
+              <https://www.rfc-editor.org/info/rfc9341>.
+
+13.2.  Informative References
+
+   [IEEE-ACM-TON-MPNPM]
+              Cociglio, M., Fioccola, G., Marchetto, G., Sapio, A., and
+              R. Sisto, "Multipoint Passive Monitoring in Packet
+              Networks", IEEE/ACM Transactions on Networking, Vol. 27,
+              Issue 6, DOI 10.1109/TNET.2019.2950157, December 2019,
+              <https://doi.org/10.1109/TNET.2019.2950157>.
+
+   [IEEE-NETWORK-PNPM]
+              Mizrahi, T., Navon, G., Fioccola, G., Cociglio, M., Chen,
+              M., and G. Mirsky, "AM-PM: Efficient Network Telemetry
+              using Alternate Marking", IEEE Network, Vol. 33, Issue 4,
+              DOI 10.1109/MNET.2019.1800152, July 2019,
+              <https://doi.org/10.1109/MNET.2019.1800152>.
+
+   [OPSAWG-IFIT-FRAMEWORK]
+              Song, H., Qin, F., Chen, H., Jin, J., and J. Shin, "A
+              Framework for In-situ Flow Information Telemetry", Work in
+              Progress, Internet-Draft, draft-song-opsawg-ifit-
+              framework-19, 24 October 2022,
+              <https://datatracker.ietf.org/doc/html/draft-song-opsawg-
+              ifit-framework-19>.
+
+   [RFC5474]  Duffield, N., Ed., Chiou, D., Claise, B., Greenberg, A.,
+              Grossglauser, M., and J. Rexford, "A Framework for Packet
+              Selection and Reporting", RFC 5474, DOI 10.17487/RFC5474,
+              March 2009, <https://www.rfc-editor.org/info/rfc5474>.
+
+   [RFC7011]  Claise, B., Ed., Trammell, B., Ed., and P. Aitken,
+              "Specification of the IP Flow Information Export (IPFIX)
+              Protocol for the Exchange of Flow Information", STD 77,
+              RFC 7011, DOI 10.17487/RFC7011, September 2013,
+              <https://www.rfc-editor.org/info/rfc7011>.
+
+   [RFC8889]  Fioccola, G., Ed., Cociglio, M., Sapio, A., and R. Sisto,
+              "Multipoint Alternate-Marking Method for Passive and
+              Hybrid Performance Monitoring", RFC 8889,
+              DOI 10.17487/RFC8889, August 2020,
+              <https://www.rfc-editor.org/info/rfc8889>.
+
+   [RFC9198]  Alvarez-Hamelin, J., Morton, A., Fabini, J., Pignataro,
+              C., and R. Geib, "Advanced Unidirectional Route Assessment
+              (AURA)", RFC 9198, DOI 10.17487/RFC9198, May 2022,
+              <https://www.rfc-editor.org/info/rfc9198>.
+
+Appendix A.  Example of Monitoring Network and Clusters Partition
+
+   Figure 4 shows a simple example of a monitoring network graph:
+
+                                                    +------+
+                                                   <>  R6  <>---
+                                                  / +------+
+                           +------+     +------+ /
+                          <>  R2  <>---<>  R4  <>
+                         / +------+ \   +------+ \
+                        /            \            \ +------+
+              +------+ /   +------+   \ +------+   <>  R7  <>---
+          ---<>  R1  <>---<>  R3  <>---<>  R5  <>   +------+
+              +------+ \   +------+ \   +------+ \
+                        \            \            \ +------+
+                         \            \            <>  R8  <>---
+                          \            \            +------+
+                           \            \
+                            \            \ +------+
+                             \            <>  R9  <>---
+                              \            +------+
+                               \
+                                \ +------+
+                                 <>  R10 <>---
+                                  +------+
+
+                     Figure 4: Monitoring Network Graph
+
+   In the monitoring network graph example, it is possible to identify
+   the clusters partition by applying this two-step algorithm described
+   in Section 5.1.
+
+   The first step identifies the following groups:
+
+      Group 1: (R1-R2), (R1-R3), (R1-R10)
+
+      Group 2: (R2-R4), (R2-R5)
+
+      Group 3: (R3-R5), (R3-R9)
+
+      Group 4: (R4-R6), (R4-R7)
+
+      Group 5: (R5-R8)
+
+   Then, the second step builds the clusters partition (in particular,
+   we can underline that Groups 2 and 3 connect together, since R5 is in
+   common):
+
+      Cluster 1: (R1-R2), (R1-R3), (R1-R10)
+
+      Cluster 2: (R2-R4), (R2-R5), (R3-R5), (R3-R9)
+
+      Cluster 3: (R4-R6), (R4-R7)
+
+      Cluster 4: (R5-R8)
+
+   The flow direction considered here is from left to right.  For the
+   opposite direction, the same reasoning can be applied, and in this
+   example, you get the same clusters partition.
+
+   In the end, the following 4 clusters are obtained:
+
+          Cluster 1
+                           +------+
+                          <>  R2  <>---
+                         / +------+
+                        /
+              +------+ /   +------+
+          ---<>  R1  <>---<>  R3  <>---
+              +------+ \   +------+
+                        \
+                         \
+                          \
+                           \
+                            \
+                             \
+                              \
+                               \
+                                \ +------+
+                                 <>  R10 <>---
+                                  +------+
+
+
+          Cluster 2
+              +------+     +------+
+          ---<>  R2  <>---<>  R4  <>---
+              +------+ \   +------+
+                        \
+              +------+   \ +------+
+          ---<>  R3  <>---<>  R5  <>---
+              +------+ \   +------+
+                        \
+                         \
+                          \
+                           \
+                            \ +------+
+                             <>  R9  <>---
+                              +------+
+
+
+          Cluster 3
+                          +------+
+                         <>  R6  <>---
+                        / +------+
+              +------+ /
+          ---<>  R4  <>
+              +------+ \
+                        \ +------+
+                         <>  R7  <>---
+                          +------+
+
+
+          Cluster 4
+              +------+
+          ---<>  R5  <>
+              +------+ \
+                        \ +------+
+                         <>  R8  <>---
+                          +------+
+
+                         Figure 5: Clusters Example
+
+   There are clusters with more than two nodes as well as two-node
+   clusters.  In the two-node clusters, the loss is on the link (Cluster
+   4).  In more-than-two-node clusters, the loss is on the cluster, but
+   we cannot know in which link (Cluster 1, 2, or 3).
+
+Acknowledgements
+
+   The authors would like to thank Martin Duke and Tommy Pauly for their
+   assistance and their detailed and valuable reviews.
+
+Contributors
+
+   Greg Mirsky
+   Ericsson
+   Email: gregimirsky@gmail.com
+
+
+   Tal Mizrahi
+   Huawei Technologies
+   Email: tal.mizrahi.phd@gmail.com
+
+
+   Xiao Min
+   ZTE Corp.
+   Email: xiao.min2@zte.com.cn
+
+
+Authors' Addresses
+
+   Giuseppe Fioccola (editor)
+   Huawei Technologies
+   Riesstrasse, 25
+   80992 Munich
+   Germany
+   Email: giuseppe.fioccola@huawei.com
+
+
+   Mauro Cociglio
+   Telecom Italia
+   Email: mauro.cociglio@outlook.com
+
+
+   Amedeo Sapio
+   Intel Corporation
+   4750 Patrick Henry Dr.
+   Santa Clara, CA 95054
+   United States of America
+   Email: amedeo.sapio@intel.com
+
+
+   Riccardo Sisto
+   Politecnico di Torino
+   Corso Duca degli Abruzzi, 24
+   10129 Torino
+   Italy
+   Email: riccardo.sisto@polito.it
+
+
+   Tianran Zhou
+   Huawei Technologies
+   156 Beiqing Rd.
+   Beijing
+   100095
+   China
+   Email: zhoutianran@huawei.com