summaryrefslogtreecommitdiff
path: root/doc/rfc/rfc7928.txt
diff options
context:
space:
mode:
Diffstat (limited to 'doc/rfc/rfc7928.txt')
-rw-r--r--doc/rfc/rfc7928.txt2075
1 files changed, 2075 insertions, 0 deletions
diff --git a/doc/rfc/rfc7928.txt b/doc/rfc/rfc7928.txt
new file mode 100644
index 0000000..351958e
--- /dev/null
+++ b/doc/rfc/rfc7928.txt
@@ -0,0 +1,2075 @@
+
+
+
+
+
+
+Internet Engineering Task Force (IETF) N. Kuhn, Ed.
+Request for Comments: 7928 CNES, Telecom Bretagne
+Category: Informational P. Natarajan, Ed.
+ISSN: 2070-1721 Cisco Systems
+ N. Khademi, Ed.
+ University of Oslo
+ D. Ros
+ Simula Research Laboratory AS
+ July 2016
+
+
+ Characterization Guidelines for Active Queue Management (AQM)
+
+Abstract
+
+ Unmanaged large buffers in today's networks have given rise to a slew
+ of performance issues. These performance issues can be addressed by
+ some form of Active Queue Management (AQM) mechanism, optionally in
+ combination with a packet-scheduling scheme such as fair queuing.
+ This document describes various criteria for performing
+ characterizations of AQM schemes that can be used in lab testing
+ during development, prior to deployment.
+
+Status of This Memo
+
+ This document is not an Internet Standards Track specification; it is
+ published for informational purposes.
+
+ This document is a product of the Internet Engineering Task Force
+ (IETF). It represents the consensus of the IETF community. It has
+ received public review and has been approved for publication by the
+ Internet Engineering Steering Group (IESG). Not all documents
+ approved by the IESG are a candidate for any level of Internet
+ Standard; see Section 2 of RFC 7841.
+
+ Information about the current status of this document, any errata,
+ and how to provide feedback on it may be obtained at
+ http://www.rfc-editor.org/info/rfc7928.
+
+
+
+
+
+
+
+
+
+
+
+
+
+Kuhn, et al. Informational [Page 1]
+
+RFC 7928 AQM Characterization Guidelines July 2016
+
+
+Copyright Notice
+
+ Copyright (c) 2016 IETF Trust and the persons identified as the
+ document authors. All rights reserved.
+
+ This document is subject to BCP 78 and the IETF Trust's Legal
+ Provisions Relating to IETF Documents
+ (http://trustee.ietf.org/license-info) in effect on the date of
+ publication of this document. Please review these documents
+ carefully, as they describe your rights and restrictions with respect
+ to this document. Code Components extracted from this document must
+ include Simplified BSD License text as described in Section 4.e of
+ the Trust Legal Provisions and are provided without warranty as
+ described in the Simplified BSD License.
+
+Table of Contents
+
+ 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 4
+ 1.1. Reducing the Latency and Maximizing the Goodput . . . . . 5
+ 1.2. Goals of This Document . . . . . . . . . . . . . . . . . 5
+ 1.3. Requirements Language . . . . . . . . . . . . . . . . . . 6
+ 1.4. Glossary . . . . . . . . . . . . . . . . . . . . . . . . 7
+ 2. End-to-End Metrics . . . . . . . . . . . . . . . . . . . . . 7
+ 2.1. Flow Completion Time . . . . . . . . . . . . . . . . . . 8
+ 2.2. Flow Startup Time . . . . . . . . . . . . . . . . . . . . 8
+ 2.3. Packet Loss . . . . . . . . . . . . . . . . . . . . . . . 9
+ 2.4. Packet Loss Synchronization . . . . . . . . . . . . . . . 9
+ 2.5. Goodput . . . . . . . . . . . . . . . . . . . . . . . . . 10
+ 2.6. Latency and Jitter . . . . . . . . . . . . . . . . . . . 11
+ 2.7. Discussion on the Trade-Off between Latency and Goodput . 11
+ 3. Generic Setup for Evaluations . . . . . . . . . . . . . . . . 12
+ 3.1. Topology and Notations . . . . . . . . . . . . . . . . . 12
+ 3.2. Buffer Size . . . . . . . . . . . . . . . . . . . . . . . 14
+ 3.3. Congestion Controls . . . . . . . . . . . . . . . . . . . 14
+ 4. Methodology, Metrics, AQM Comparisons, Packet Sizes,
+ Scheduling, and ECN . . . . . . . . . . . . . . . . . . . . . 14
+ 4.1. Methodology . . . . . . . . . . . . . . . . . . . . . . . 14
+ 4.2. Comments on Metrics Measurement . . . . . . . . . . . . . 15
+ 4.3. Comparing AQM Schemes . . . . . . . . . . . . . . . . . . 15
+ 4.3.1. Performance Comparison . . . . . . . . . . . . . . . 15
+ 4.3.2. Deployment Comparison . . . . . . . . . . . . . . . . 16
+ 4.4. Packet Sizes and Congestion Notification . . . . . . . . 16
+ 4.5. Interaction with ECN . . . . . . . . . . . . . . . . . . 17
+ 4.6. Interaction with Scheduling . . . . . . . . . . . . . . . 17
+ 5. Transport Protocols . . . . . . . . . . . . . . . . . . . . . 18
+ 5.1. TCP-Friendly Sender . . . . . . . . . . . . . . . . . . . 19
+ 5.1.1. TCP-Friendly Sender with the Same Initial Congestion
+ Window . . . . . . . . . . . . . . . . . . . . . . . 19
+
+
+
+Kuhn, et al. Informational [Page 2]
+
+RFC 7928 AQM Characterization Guidelines July 2016
+
+
+ 5.1.2. TCP-Friendly Sender with Different Initial Congestion
+ Windows . . . . . . . . . . . . . . . . . . . . . . . 19
+ 5.2. Aggressive Transport Sender . . . . . . . . . . . . . . . 19
+ 5.3. Unresponsive Transport Sender . . . . . . . . . . . . . . 20
+ 5.4. Less-than-Best-Effort Transport Sender . . . . . . . . . 20
+ 6. Round-Trip Time Fairness . . . . . . . . . . . . . . . . . . 21
+ 6.1. Motivation . . . . . . . . . . . . . . . . . . . . . . . 21
+ 6.2. Recommended Tests . . . . . . . . . . . . . . . . . . . . 21
+ 6.3. Metrics to Evaluate the RTT Fairness . . . . . . . . . . 22
+ 7. Burst Absorption . . . . . . . . . . . . . . . . . . . . . . 22
+ 7.1. Motivation . . . . . . . . . . . . . . . . . . . . . . . 22
+ 7.2. Recommended Tests . . . . . . . . . . . . . . . . . . . . 23
+ 8. Stability . . . . . . . . . . . . . . . . . . . . . . . . . . 24
+ 8.1. Motivation . . . . . . . . . . . . . . . . . . . . . . . 24
+ 8.2. Recommended Tests . . . . . . . . . . . . . . . . . . . . 24
+ 8.2.1. Definition of the Congestion Level . . . . . . . . . 25
+ 8.2.2. Mild Congestion . . . . . . . . . . . . . . . . . . . 25
+ 8.2.3. Medium Congestion . . . . . . . . . . . . . . . . . . 25
+ 8.2.4. Heavy Congestion . . . . . . . . . . . . . . . . . . 25
+ 8.2.5. Varying the Congestion Level . . . . . . . . . . . . 26
+ 8.2.6. Varying Available Capacity . . . . . . . . . . . . . 26
+ 8.3. Parameter Sensitivity and Stability Analysis . . . . . . 27
+ 9. Various Traffic Profiles . . . . . . . . . . . . . . . . . . 27
+ 9.1. Traffic Mix . . . . . . . . . . . . . . . . . . . . . . . 28
+ 9.2. Bidirectional Traffic . . . . . . . . . . . . . . . . . . 28
+ 10. Example of a Multi-AQM Scenario . . . . . . . . . . . . . . . 29
+ 10.1. Motivation . . . . . . . . . . . . . . . . . . . . . . . 29
+ 10.2. Details on the Evaluation Scenario . . . . . . . . . . . 29
+ 11. Implementation Cost . . . . . . . . . . . . . . . . . . . . . 30
+ 11.1. Motivation . . . . . . . . . . . . . . . . . . . . . . . 30
+ 11.2. Recommended Discussion . . . . . . . . . . . . . . . . . 30
+ 12. Operator Control and Auto-Tuning . . . . . . . . . . . . . . 30
+ 12.1. Motivation . . . . . . . . . . . . . . . . . . . . . . . 30
+ 12.2. Recommended Discussion . . . . . . . . . . . . . . . . . 31
+ 13. Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
+ 14. Security Considerations . . . . . . . . . . . . . . . . . . . 32
+ 15. References . . . . . . . . . . . . . . . . . . . . . . . . . 32
+ 15.1. Normative References . . . . . . . . . . . . . . . . . . 32
+ 15.2. Informative References . . . . . . . . . . . . . . . . . 33
+ Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . . 36
+ Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 37
+
+
+
+
+
+
+
+
+
+
+Kuhn, et al. Informational [Page 3]
+
+RFC 7928 AQM Characterization Guidelines July 2016
+
+
+1. Introduction
+
+ Active Queue Management (AQM) addresses the concerns arising from
+ using unnecessarily large and unmanaged buffers to improve network
+ and application performance, such as those presented in Section 1.2
+ of the AQM recommendations document [RFC7567]. Several AQM
+ algorithms have been proposed in the past years, most notably Random
+ Early Detection (RED) [FLOY1993], BLUE [FENG2002], Proportional
+ Integral controller (PI) [HOLLO2001], and more recently, Controlled
+ Delay (CoDel) [CODEL] and Proportional Integral controller Enhanced
+ (PIE) [AQMPIE]. In general, these algorithms actively interact with
+ the Transmission Control Protocol (TCP) and any other transport
+ protocol that deploys a congestion control scheme to manage the
+ amount of data they keep in the network. The available buffer space
+ in the routers and switches should be large enough to accommodate the
+ short-term buffering requirements. AQM schemes aim at reducing
+ buffer occupancy, and therefore the end-to-end delay. Some of these
+ algorithms, notably RED, have also been widely implemented in some
+ network devices. However, the potential benefits of the RED scheme
+ have not been realized since RED is reported to be usually turned
+ off.
+
+ A buffer is a physical volume of memory in which a queue or set of
+ queues are stored. When speaking of a specific queue in this
+ document, "buffer occupancy" refers to the amount of data (measured
+ in bytes or packets) that are in the queue, and the "maximum buffer
+ size" refers to the maximum buffer occupancy. In switches and
+ routers, a global memory space is often shared between the available
+ interfaces, and thus, the maximum buffer size for any given interface
+ may vary over time.
+
+ Bufferbloat [BB2011] is the consequence of deploying large, unmanaged
+ buffers on the Internet -- the buffering has often been measured to
+ be ten times or a hundred times larger than needed. Large buffer
+ sizes in combination with TCP and/or unresponsive flows increases
+ end-to-end delay. This results in poor performance for latency-
+ sensitive applications such as real-time multimedia (e.g., voice,
+ video, gaming, etc.). The degree to which this affects modern
+ networking equipment, especially consumer-grade equipment, produces
+ problems even with commonly used web services. Active queue
+ management is thus essential to control queuing delay and decrease
+ network latency.
+
+ The Active Queue Management and Packet Scheduling Working Group (AQM
+ WG) was chartered to address the problems with large unmanaged
+ buffers in the Internet. Specifically, the AQM WG is tasked with
+ standardizing AQM schemes that not only address concerns with such
+ buffers, but are also robust under a wide variety of operating
+
+
+
+Kuhn, et al. Informational [Page 4]
+
+RFC 7928 AQM Characterization Guidelines July 2016
+
+
+ conditions. This document provides characterization guidelines that
+ can be used to assess the applicability, performance, and
+ deployability of an AQM, whether it is a candidate for
+ standardization at IETF or not.
+
+ The AQM algorithm implemented in a router can be separated from the
+ scheduling of packets sent out by the router as discussed in the AQM
+ recommendations document [RFC7567]. The rest of this memo refers to
+ the AQM as a dropping/marking policy as a separate feature to any
+ interface-scheduling scheme. This document may be complemented with
+ another one on guidelines for assessing the combination of packet
+ scheduling and AQM. We note that such a document will inherit all
+ the guidelines from this document, plus any additional scenarios
+ relevant for packet scheduling such as flow-starvation evaluation or
+ the impact of the number of hash buckets.
+
+1.1. Reducing the Latency and Maximizing the Goodput
+
+ The trade-off between reducing the latency and maximizing the goodput
+ is intrinsically linked to each AQM scheme and is key to evaluating
+ its performance. To ensure the safety deployment of an AQM, its
+ behavior should be assessed in a variety of scenarios. Whenever
+ possible, solutions ought to aim at both maximizing goodput and
+ minimizing latency.
+
+1.2. Goals of This Document
+
+ This document recommends a generic list of scenarios against which an
+ AQM proposal should be evaluated, considering both potential
+ performance gain and safety of deployment. The guidelines help to
+ quantify performance of AQM schemes in terms of latency reduction,
+ goodput maximization, and the trade-off between these two. The
+ document presents central aspects of an AQM algorithm that should be
+ considered, whatever the context, such as burst absorption capacity,
+ RTT fairness, or resilience to fluctuating network conditions. The
+ guidelines also discuss methods to understand the various aspects
+ associated with safely deploying and operating the AQM scheme. Thus,
+ one of the key objectives behind formulating the guidelines is to
+ help ascertain whether a specific AQM is not only better than drop-
+ tail (i.e., without AQM and with a BDP-sized buffer), but also safe
+ to deploy: the guidelines can be used to compare several AQM
+ proposals with each other, but should be used to compare a proposal
+ with drop-tail.
+
+ This memo details generic characterization scenarios against which
+ any AQM proposal should be evaluated, irrespective of whether or not
+ an AQM is standardized by the IETF. This document recommends the
+ relevant scenarios and metrics to be considered. This document
+
+
+
+Kuhn, et al. Informational [Page 5]
+
+RFC 7928 AQM Characterization Guidelines July 2016
+
+
+ presents central aspects of an AQM algorithm that should be
+ considered whatever the context, such as burst absorption capacity,
+ RTT fairness, or resilience to fluctuating network conditions.
+
+ These guidelines do not define and are not bound to a particular
+ deployment scenario or evaluation toolset. Instead, the guidelines
+ can be used to assert the potential gain of introducing an AQM for
+ the particular environment, which is of interest to the testers.
+ These guidelines do not cover every possible aspect of a particular
+ algorithm. These guidelines do not present context-dependent
+ scenarios (such as IEEE 802.11 WLANs, data centers, or rural
+ broadband networks). To keep the guidelines generic, a number of
+ potential router components and algorithms (such as Diffserv) are
+ omitted.
+
+ The goals of this document can thus be summarized as follows:
+
+ o The present characterization guidelines provide a non-exhaustive
+ list of scenarios to help ascertain whether an AQM is not only
+ better than drop-tail (with a BDP-sized buffer), but also safe to
+ deploy; the guidelines can also be used to compare several AQM
+ proposals with each other.
+
+ o The present characterization guidelines (1) are not bound to a
+ particular evaluation toolset and (2) can be used for various
+ deployment contexts; testers are free to select a toolset that is
+ best suited for the environment in which their proposal will be
+ deployed.
+
+ o The present characterization guidelines are intended to provide
+ guidance for better selecting an AQM for a specific environment;
+ it is not required that an AQM proposal is evaluated following
+ these guidelines for its standardization.
+
+1.3. Requirements Language
+
+ The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
+ "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
+ document are to be interpreted as described in RFC 2119 [RFC2119].
+
+
+
+
+
+
+
+
+
+
+
+
+Kuhn, et al. Informational [Page 6]
+
+RFC 7928 AQM Characterization Guidelines July 2016
+
+
+1.4. Glossary
+
+ o Application-limited traffic: A type of traffic that does not have
+ an unlimited amount of data to transmit.
+
+ o AQM: The Active Queue Management (AQM) algorithm implemented in a
+ router can be separated from the scheduling of packets sent by the
+ router. The rest of this memo refers to the AQM as a dropping/
+ marking policy as a separate feature to any interface scheduling
+ scheme [RFC7567].
+
+ o BDP: Bandwidth Delay Product.
+
+ o Buffer: A physical volume of memory in which a queue or set of
+ queues are stored.
+
+ o Buffer Occupancy: The amount of data stored in a buffer, measured
+ in bytes or packets.
+
+ o Buffer Size: The maximum buffer occupancy, that is the maximum
+ amount of data that may be stored in a buffer, measured in bytes
+ or packets.
+
+ o Initial Window 10 (IW10): TCP initial congestion window set to 10
+ packets.
+
+ o Latency: One-way delay of packets across Internet paths. This
+ definition suits transport layer definition of the latency, which
+ should not be confused with an application-layer view of the
+ latency.
+
+ o Goodput: Goodput is defined as the number of bits per unit of time
+ forwarded to the correct destination, minus any bits lost or
+ retransmitted [RFC2647]. The goodput should be determined for
+ each flow and not for aggregates of flows.
+
+ o SQRT: The square root function.
+
+ o ROUND: The round function.
+
+2. End-to-End Metrics
+
+ End-to-end delay is the result of propagation delay, serialization
+ delay, service delay in a switch, medium-access delay, and queuing
+ delay, summed over the network elements along the path. AQM schemes
+ may reduce the queuing delay by providing signals to the sender on
+ the emergence of congestion, but any impact on the goodput must be
+ carefully considered. This section presents the metrics that could
+
+
+
+Kuhn, et al. Informational [Page 7]
+
+RFC 7928 AQM Characterization Guidelines July 2016
+
+
+ be used to better quantify (1) the reduction of latency, (2)
+ maximization of goodput, and (3) the trade-off between these two.
+ This section provides normative requirements for metrics that can be
+ used to assess the performance of an AQM scheme.
+
+ Some metrics listed in this section are not suited to every type of
+ traffic detailed in the rest of this document. It is therefore not
+ necessary to measure all of the following metrics: the chosen metric
+ may not be relevant to the context of the evaluation scenario (e.g.,
+ latency vs. goodput trade-off in application-limited traffic
+ scenarios). Guidance is provided for each metric.
+
+2.1. Flow Completion Time
+
+ The flow completion time is an important performance metric for the
+ end-user when the flow size is finite. The definition of the flow
+ size may be a source of contradictions, thus, this metric can
+ consider a flow as a single file. Considering the fact that an AQM
+ scheme may drop/mark packets, the flow completion time is directly
+ linked to the dropping/marking policy of the AQM scheme. This metric
+ helps to better assess the performance of an AQM depending on the
+ flow size. The Flow Completion Time (FCT) is related to the flow
+ size (Fs) and the goodput for the flow (G) as follows:
+
+ FCT [s] = Fs [byte] / ( G [bit/s] / 8 [bit/byte] )
+
+ Where flow size is the size of the transport-layer payload in bits
+ and goodput is the transport-layer payload transfer time (described
+ in Section 2.5).
+
+ If this metric is used to evaluate the performance of web transfers,
+ it is suggested to rather consider the time needed to download all
+ the objects that compose the web page, as this makes more sense in
+ terms of user experience, rather than assessing the time needed to
+ download each object.
+
+2.2. Flow Startup Time
+
+ The flow startup time is the time between when the request was sent
+ from the client and when the server starts to transmit data. The
+ amount of packets dropped by an AQM may seriously affect the waiting
+ period during which the data transfer has not started. This metric
+ would specifically focus on the operations such as DNS lookups, TCP
+ opens, and Secure Socket Layer (SSL) handshakes.
+
+
+
+
+
+
+
+Kuhn, et al. Informational [Page 8]
+
+RFC 7928 AQM Characterization Guidelines July 2016
+
+
+2.3. Packet Loss
+
+ Packet loss can occur en route, this can impact the end-to-end
+ performance measured at the receiver end.
+
+ The tester should evaluate the loss experienced at the receiver end
+ using one of two metrics:
+
+ o The packet loss ratio: This metric is to be frequently measured
+ during the experiment. The long-term loss ratio is of interest
+ for steady-state scenarios only;
+
+ o The interval between consecutive losses: The time between two
+ losses is to be measured.
+
+ The packet loss ratio can be assessed by simply evaluating the loss
+ ratio as a function of the number of lost packets and the total
+ number of packets sent. This might not be easily done in laboratory
+ testing, for which these guidelines advise the tester:
+
+ o To check that for every packet, a corresponding packet was
+ received within a reasonable time, as presented in the document
+ that proposes a metric for one-way packet loss across Internet
+ paths [RFC7680].
+
+ o To keep a count of all packets sent, and a count of the non-
+ duplicate packets received, as discussed in [RFC2544], which
+ presents a benchmarking methodology.
+
+ The interval between consecutive losses, which is also called a
+ "gap", is a metric of interest for Voice over IP (VoIP) traffic
+ [RFC3611].
+
+2.4. Packet Loss Synchronization
+
+ One goal of an AQM algorithm is to help to avoid global
+ synchronization of flows sharing a bottleneck buffer on which the AQM
+ operates ([RFC2309] and [RFC7567]). The "degree" of packet-loss
+ synchronization between flows should be assessed, with and without
+ the AQM under consideration.
+
+ Loss synchronization among flows may be quantified by several
+ slightly different metrics that capture different aspects of the same
+ issue [HASS2008]. However, in real-world measurements the choice of
+ metric could be imposed by practical considerations -- e.g., whether
+ fine-grained information on packet losses at the bottleneck is
+ available or not. For the purpose of AQM characterization, a good
+ candidate metric is the global synchronization ratio, measuring the
+
+
+
+Kuhn, et al. Informational [Page 9]
+
+RFC 7928 AQM Characterization Guidelines July 2016
+
+
+ proportion of flows losing packets during a loss event. This metric
+ can be used in real-world experiments to characterize synchronization
+ along arbitrary Internet paths [JAY2006].
+
+ If an AQM scheme is evaluated using real-life network environments,
+ it is worth pointing out that some network events, such as failed
+ link restoration may cause synchronized losses between active flows,
+ and thus confuse the meaning of this metric.
+
+2.5. Goodput
+
+ The goodput has been defined as the number of bits per the unit of
+ time forwarded to the correct destination interface, minus any bits
+ lost or retransmitted, such as proposed in Section 3.17 of [RFC2647],
+ which describes the benchmarking terminology for firewall
+ performances. This definition requires that the test setup needs to
+ be qualified to assure that it is not generating losses on its own.
+
+ Measuring the end-to-end goodput provides an appreciation of how well
+ an AQM scheme improves transport and application performance. The
+ measured end-to-end goodput is linked to the dropping/marking policy
+ of the AQM scheme -- e.g., the fewer the number of packet drops, the
+ fewer packets need retransmission, minimizing the impact of AQM on
+ transport and application performance. Additionally, an AQM scheme
+ may resort to Explicit Congestion Notification (ECN) marking as an
+ initial means to control delay. Again, marking packets instead of
+ dropping them reduces the number of packet retransmissions and
+ increases goodput. End-to-end goodput values help to evaluate the
+ AQM scheme's effectiveness in minimizing packet drops that impact
+ application performance and to estimate how well the AQM scheme works
+ with ECN.
+
+ The measurement of the goodput allows the tester to evaluate to what
+ extent an AQM is able to maintain a high bottleneck utilization.
+ This metric should also be obtained frequently during an experiment,
+ as the long-term goodput is relevant for steady-state scenarios only
+ and may not necessarily reflect how the introduction of an AQM
+ actually impacts the link utilization during a certain period of
+ time. Fluctuations in the values obtained from these measurements
+ may depend on other factors than the introduction of an AQM, such as
+ link-layer losses due to external noise or corruption, fluctuating
+ bandwidths (IEEE 802.11 WLANs), heavy congestion levels, or the
+ transport layer's rate reduction by the congestion control mechanism.
+
+
+
+
+
+
+
+
+Kuhn, et al. Informational [Page 10]
+
+RFC 7928 AQM Characterization Guidelines July 2016
+
+
+2.6. Latency and Jitter
+
+ The latency, or the one-way delay metric, is discussed in [RFC7679].
+ There is a consensus on an adequate metric for the jitter that
+ represents the one-way delay variations for packets from the same
+ flow: the Packet Delay Variation (PDV) serves well in all use cases
+ [RFC5481].
+
+ The end-to-end latency includes components other than just the
+ queuing delay, such as the signal-processing delay, transmission
+ delay, and processing delay. Moreover, the jitter is caused by
+ variations in queuing and processing delay (e.g., scheduling
+ effects). The introduction of an AQM scheme would impact end-to-end
+ latency and jitter, and therefore these metrics should be considered
+ in the end-to-end evaluation of performance.
+
+2.7. Discussion on the Trade-Off between Latency and Goodput
+
+ The metrics presented in this section may be considered in order to
+ discuss and quantify the trade-off between latency and goodput.
+
+ With regards to the goodput, and in addition to the long-term
+ stationary goodput value, it is recommended to take measurements at
+ every multiple of the minimum RTT (minRTT) between A and B. It is
+ suggested to take measurements at least every K * minRTT (to smooth
+ out the fluctuations), with K=10. Higher values for K can be
+ considered whenever it is more appropriate for the presentation of
+ the results, since the value for K may depend on the network's path
+ characteristics. The measurement period must be disclosed for each
+ experiment, and when results/values are compared across different AQM
+ schemes, the comparisons should use exactly the same measurement
+ periods. With regards to latency, it is recommended to take the
+ samples on a per-packet basis whenever possible, depending on the
+ features provided by the hardware and software and the impact of
+ sampling itself on the hardware performance.
+
+ From each of these sets of measurements, the cumulative density
+ function (CDF) of the considered metrics should be computed. If the
+ considered scenario introduces dynamically varying parameters,
+ temporal evolution of the metrics could also be generated. For each
+ scenario, the following graph may be generated: the x-axis shows a
+ queuing delay (that is, the average per-packet delay in excess of
+ minimum RTT), the y-axis the goodput. Ellipses are computed as
+ detailed in [WINS2014]: "We take each individual [...] run [...] as
+ one point, and then compute the 1-epsilon elliptic contour of the
+ maximum-likelihood 2D Gaussian distribution that explains the points.
+ [...] we plot the median per-sender throughput and queueing delay as
+ a circle. [...] The orientation of an ellipse represents the
+
+
+
+Kuhn, et al. Informational [Page 11]
+
+RFC 7928 AQM Characterization Guidelines July 2016
+
+
+ covariance between the throughput and delay measured for the
+ protocol." This graph provides part of a better understanding of (1)
+ the delay/goodput trade-off for a given congestion control mechanism
+ (Section 5), and (2) how the goodput and average queue delay vary as
+ a function of the traffic load (Section 8.2).
+
+3. Generic Setup for Evaluations
+
+ This section presents the topology that can be used for each of the
+ following scenarios, the corresponding notations, and discusses
+ various assumptions that have been made in the document.
+
+3.1. Topology and Notations
+
+ +--------------+ +--------------+
+ |sender A_i | |receive B_i |
+ |--------------| |--------------|
+ | SEN.Flow1.1 +---------+ +-----------+ REC.Flow1.1 |
+ | + | | | | + |
+ | | | | | | | |
+ | + | | | | + |
+ | SEN.Flow1.X +-----+ | | +--------+ REC.Flow1.X |
+ +--------------+ | | | | +--------------+
+ + +-+---+---+ +--+--+---+ +
+ | |Router L | |Router R | |
+ | |---------| |---------| |
+ | | AQM | | | |
+ | | BuffSize| | BuffSize| |
+ | | (Bsize) +-----+ (Bsize) | |
+ | +-----+--++ ++-+------+ |
+ + | | | | +
+ +--------------+ | | | | +--------------+
+ |sender A_n | | | | | |receive B_n |
+ |--------------| | | | | |--------------|
+ | SEN.FlowN.1 +---------+ | | +-----------+ REC.FlowN.1 |
+ | + | | | | + |
+ | | | | | | | |
+ | + | | | | + |
+ | SEN.FlowN.Y +------------+ +-------------+ REC.FlowN.Y |
+ +--------------+ +--------------+
+
+ Figure 1: Topology and Notations
+
+
+
+
+
+
+
+
+
+Kuhn, et al. Informational [Page 12]
+
+RFC 7928 AQM Characterization Guidelines July 2016
+
+
+ Figure 1 is a generic topology where:
+
+ o The traffic profile is a set of flows with similar characteristics
+ -- RTT, congestion control scheme, transport protocol, etc.;
+
+ o Senders with different traffic characteristics (i.e., traffic
+ profiles) can be introduced;
+
+ o The timing of each flow could be different (i.e., when does each
+ flow start and stop?);
+
+ o Each traffic profile can comprise various number of flows;
+
+ o Each link is characterized by a couple (one-way delay, capacity);
+
+ o Sender A_i is instantiated for each traffic profile. A
+ corresponding receiver B_i is instantiated for receiving the flows
+ in the profile;
+
+ o Flows share a bottleneck (the link between routers L and R);
+
+ o The tester should consider both scenarios of asymmetric and
+ symmetric bottleneck links in terms of bandwidth. In the case of
+ an asymmetric link, the capacity from senders to receivers is
+ higher than the one from receivers to senders; the symmetric link
+ scenario provides a basic understanding of the operation of the
+ AQM mechanism, whereas the asymmetric link scenario evaluates an
+ AQM mechanism in a more realistic setup;
+
+ o In asymmetric link scenarios, the tester should study the
+ bidirectional traffic between A and B (downlink and uplink) with
+ the AQM mechanism deployed in one direction only. The tester may
+ additionally consider a scenario with the AQM mechanism being
+ deployed in both directions. In each scenario, the tester should
+ investigate the impact of the drop policy of the AQM on TCP ACK
+ packets and its impact on the performance (Section 9.2).
+
+ Although this topology may not perfectly reflect actual topologies,
+ the simple topology is commonly used in the world of simulations and
+ small testbeds. It can be considered as adequate to evaluate AQM
+ proposals [TCPEVAL]. Testers ought to pay attention to the topology
+ used to evaluate an AQM scheme when comparing it with a newly
+ proposed AQM scheme.
+
+
+
+
+
+
+
+
+Kuhn, et al. Informational [Page 13]
+
+RFC 7928 AQM Characterization Guidelines July 2016
+
+
+3.2. Buffer Size
+
+ The size of the buffers should be carefully chosen, and may be set to
+ the bandwidth-delay product; the bandwidth being the bottleneck
+ capacity and the delay being the largest RTT in the considered
+ network. The size of the buffer can impact the AQM performance and
+ is a dimensioning parameter that will be considered when comparing
+ AQM proposals.
+
+ If a specific buffer size is required, the tester must justify and
+ detail the way the maximum queue size is set. Indeed, the maximum
+ size of the buffer may affect the AQM's performance and its choice
+ should be elaborated for a fair comparison between AQM proposals.
+ While comparing AQM schemes, the buffer size should remain the same
+ across the tests.
+
+3.3. Congestion Controls
+
+ This document considers running three different congestion control
+ algorithms between A and B:
+
+ o Standard TCP congestion control: The base-line congestion control
+ is TCP NewReno with selective acknowledgment (SACK) [RFC5681].
+
+ o Aggressive congestion controls: A base-line congestion control for
+ this category is CUBIC [CUBIC].
+
+ o Less-than-Best-Effort (LBE) congestion controls: Per [RFC6297], an
+ LBE service "results in smaller bandwidth and/or delay impact on
+ standard TCP than standard TCP itself, when sharing a bottleneck
+ with it." A base-line congestion control for this category is Low
+ Extra Delay Background Transport (LEDBAT) [RFC6817].
+
+ Other transport congestion controls can OPTIONALLY be evaluated in
+ addition. Recent transport layer protocols are not mentioned in the
+ following sections, for the sake of simplicity.
+
+4. Methodology, Metrics, AQM Comparisons, Packet Sizes, Scheduling, and
+ ECN
+
+4.1. Methodology
+
+ A description of each test setup should be detailed to allow this
+ test to be compared with other tests. This also allows others to
+ replicate the tests if needed. This test setup should detail
+ software and hardware versions. The tester could make its data
+ available.
+
+
+
+
+Kuhn, et al. Informational [Page 14]
+
+RFC 7928 AQM Characterization Guidelines July 2016
+
+
+ The proposals should be evaluated on real-life systems, or they may
+ be evaluated with event-driven simulations (such as ns-2, ns-3,
+ OMNET, etc.). The proposed scenarios are not bound to a particular
+ evaluation toolset.
+
+ The tester is encouraged to make the detailed test setup and the
+ results publicly available.
+
+4.2. Comments on Metrics Measurement
+
+ This document presents the end-to-end metrics that ought to be used
+ to evaluate the trade-off between latency and goodput as described in
+ Section 2. In addition to the end-to-end metrics, the queue-level
+ metrics (normally collected at the device operating the AQM) provide
+ a better understanding of the AQM behavior under study and the impact
+ of its internal parameters. Whenever it is possible (e.g., depending
+ on the features provided by the hardware/software), these guidelines
+ advise considering queue-level metrics, such as link utilization,
+ queuing delay, queue size, or packet drop/mark statistics in addition
+ to the AQM-specific parameters. However, the evaluation must be
+ primarily based on externally observed end-to-end metrics.
+
+ These guidelines do not aim to detail the way these metrics can be
+ measured, since that is expected to depend on the evaluation toolset.
+
+4.3. Comparing AQM Schemes
+
+ This document recognizes that these guidelines may be used for
+ comparing AQM schemes.
+
+ AQM schemes need to be compared against both performance and
+ deployment categories. In addition, this section details how best to
+ achieve a fair comparison of AQM schemes by avoiding certain
+ pitfalls.
+
+4.3.1. Performance Comparison
+
+ AQM schemes should be compared against the generic scenarios that are
+ summarized in Section 13. AQM schemes may be compared for specific
+ network environments such as data centers, home networks, etc. If an
+ AQM scheme has parameter(s) that were externally tuned for
+ optimization or other purposes, these values must be disclosed.
+
+ AQM schemes belong to different varieties such as queue-length based
+ schemes (for example, RED) or queuing-delay based scheme (for
+ example, CoDel, PIE). AQM schemes expose different control knobs
+ associated with different semantics. For example, while both PIE and
+ CoDel are queuing-delay based schemes and each expose a knob to
+
+
+
+Kuhn, et al. Informational [Page 15]
+
+RFC 7928 AQM Characterization Guidelines July 2016
+
+
+ control the queuing delay -- PIE's "queuing delay reference" vs.
+ CoDel's "queuing delay target", the two tuning parameters of the two
+ schemes have different semantics, resulting in different control
+ points. Such differences in AQM schemes can be easily overlooked
+ while making comparisons.
+
+ This document recommends the following procedures for a fair
+ performance comparison between the AQM schemes:
+
+ 1. Similar control parameters and implications: Testers should be
+ aware of the control parameters of the different schemes that
+ control similar behavior. Testers should also be aware of the
+ input value ranges and corresponding implications. For example,
+ consider two different schemes -- (A) queue-length based AQM
+ scheme, and (B) queuing-delay based scheme. A and B are likely
+ to have different kinds of control inputs to control the target
+ delay -- the target queue length in A vs. target queuing delay in
+ B, for example. Setting parameter values such as 100 MB for A
+ vs. 10 ms for B will have different implications depending on
+ evaluation context. Such context-dependent implications must be
+ considered before drawing conclusions on performance comparisons.
+ Also, it would be preferable if an AQM proposal listed such
+ parameters and discussed how each relates to network
+ characteristics such as capacity, average RTT, etc.
+
+ 2. Compare over a range of input configurations: There could be
+ situations when the set of control parameters that affect a
+ specific behavior have different semantics between the two AQM
+ schemes. As mentioned above, PIE has tuning parameters to
+ control queue delay that have different semantics from those used
+ in CoDel. In such situations, these schemes need to be compared
+ over a range of input configurations. For example, compare PIE
+ vs. CoDel over the range of target delay input configurations.
+
+4.3.2. Deployment Comparison
+
+ AQM schemes must be compared against deployment criteria such as the
+ parameter sensitivity (Section 8.3), auto-tuning (Section 12), or
+ implementation cost (Section 11).
+
+4.4. Packet Sizes and Congestion Notification
+
+ An AQM scheme may be considering packet sizes while generating
+ congestion signals [RFC7141]. For example, control packets such as
+ DNS requests/responses, TCP SYNs/ACKs are small, but their loss can
+ severely impact application performance. An AQM scheme may therefore
+ be biased towards small packets by dropping them with lower
+ probability compared to larger packets. However, such an AQM scheme
+
+
+
+Kuhn, et al. Informational [Page 16]
+
+RFC 7928 AQM Characterization Guidelines July 2016
+
+
+ is unfair to data senders generating larger packets. Data senders,
+ malicious or otherwise, are motivated to take advantage of such an
+ AQM scheme by transmitting smaller packets, and this could result in
+ unsafe deployments and unhealthy transport and/or application
+ designs.
+
+ An AQM scheme should adhere to the recommendations outlined in the
+ Best Current Practice for dropping and marking packets [BCP41], and
+ should not provide undue advantage to flows with smaller packets,
+ such as discussed in Section 4.4 of the AQM recommendation document
+ [RFC7567]. In order to evaluate if an AQM scheme is biased towards
+ flows with smaller size packets, traffic can be generated, as defined
+ in Section 8.2.2, where half of the flows have smaller packets (e.g.,
+ 500-byte packets) than the other half of the flow (e.g., 1500-byte
+ packets). In this case, the metrics reported could be the same as in
+ Section 6.3, where Category I is the set of flows with smaller
+ packets and Category II the one with larger packets. The
+ bidirectional scenario could also be considered (Section 9.2).
+
+4.5. Interaction with ECN
+
+ ECN [RFC3168] is an alternative that allows AQM schemes to signal to
+ receivers about network congestion that does not use packet drops.
+ There are benefits to providing ECN support for an AQM scheme
+ [WELZ2015].
+
+ If the tested AQM scheme can support ECN, the testers must discuss
+ and describe the support of ECN, such as discussed in the AQM
+ recommendation document [RFC7567]. Also, the AQM's ECN support can
+ be studied and verified by replicating tests in Section 6.2 with ECN
+ turned ON at the TCP senders. The results can be used not only to
+ evaluate the performance of the tested AQM with and without ECN
+ markings, but also to quantify the interest of enabling ECN.
+
+4.6. Interaction with Scheduling
+
+ A network device may use per-flow or per-class queuing with a
+ scheduling algorithm to either prioritize certain applications or
+ classes of traffic, limit the rate of transmission, or to provide
+ isolation between different traffic flows within a common class, such
+ as discussed in Section 2.1 of the AQM recommendation document
+ [RFC7567].
+
+ The scheduling and the AQM conjointly impact the end-to-end
+ performance. Therefore, the AQM proposal must discuss the
+ feasibility of adding scheduling combined with the AQM algorithm. It
+ can be explained whether the dropping policy is applied when packets
+ are being enqueued or dequeued.
+
+
+
+Kuhn, et al. Informational [Page 17]
+
+RFC 7928 AQM Characterization Guidelines July 2016
+
+
+ These guidelines do not propose guidelines to assess the performance
+ of scheduling algorithms. Indeed, as opposed to characterizing AQM
+ schemes that is related to their capacity to control the queuing
+ delay in a queue, characterizing scheduling schemes is related to the
+ scheduling itself and its interaction with the AQM scheme. As one
+ example, the scheduler may create sub-queues and the AQM scheme may
+ be applied on each of the sub-queues, and/or the AQM could be applied
+ on the whole queue. Also, schedulers might, such as FQ-CoDel
+ [HOEI2015] or FavorQueue [ANEL2014], introduce flow prioritization.
+ In these cases, specific scenarios should be proposed to ascertain
+ that these scheduler schemes not only help in tackling the
+ bufferbloat, but also are robust under a wide variety of operating
+ conditions. This is out of the scope of this document, which focuses
+ on dropping and/or marking AQM schemes.
+
+5. Transport Protocols
+
+ Network and end-devices need to be configured with a reasonable
+ amount of buffer space to absorb transient bursts. In some
+ situations, network providers tend to configure devices with large
+ buffers to avoid packet drops triggered by a full buffer and to
+ maximize the link utilization for standard loss-based TCP traffic.
+
+ AQM algorithms are often evaluated by considering the Transmission
+ Control Protocol (TCP) [RFC793] with a limited number of
+ applications. TCP is a widely deployed transport. It fills up
+ available buffers until a sender transferring a bulk flow with TCP
+ receives a signal (packet drop) that reduces the sending rate. The
+ larger the buffer, the higher the buffer occupancy, and therefore the
+ queuing delay. An efficient AQM scheme sends out early congestion
+ signals to TCP to bring the queuing delay under control.
+
+ Not all endpoints (or applications) using TCP use the same flavor of
+ TCP. A variety of senders generate different classes of traffic,
+ which may not react to congestion signals (aka non-responsive flows
+ in Section 3 of the AQM recommendation document [RFC7567]) or may not
+ reduce their sending rate as expected (aka Transport Flows that are
+ less responsive than TCP, such as proposed in Section 3 of the AQM
+ recommendation document [RFC7567], also called "aggressive flows").
+ In these cases, AQM schemes seek to control the queuing delay.
+
+ This section provides guidelines to assess the performance of an AQM
+ proposal for various traffic profiles -- different types of senders
+ (with different TCP congestion control variants, unresponsive, and
+ aggressive).
+
+
+
+
+
+
+Kuhn, et al. Informational [Page 18]
+
+RFC 7928 AQM Characterization Guidelines July 2016
+
+
+5.1. TCP-Friendly Sender
+
+5.1.1. TCP-Friendly Sender with the Same Initial Congestion Window
+
+ This scenario helps to evaluate how an AQM scheme reacts to a TCP-
+ friendly transport sender. A single, long-lived, non-application-
+ limited, TCP NewReno flow, with an Initial congestion Window (IW) set
+ to 3 packets, transfers data between sender A and receiver B. Other
+ TCP-friendly congestion control schemes such as TCP-Friendly Rate
+ Control [RFC5348], etc., may also be considered.
+
+ For each TCP-friendly transport considered, the graph described in
+ Section 2.7 could be generated.
+
+5.1.2. TCP-Friendly Sender with Different Initial Congestion Windows
+
+ This scenario can be used to evaluate how an AQM scheme adapts to a
+ traffic mix consisting of TCP flows with different values of the IW.
+
+ For this scenario, two types of flows must be generated between
+ sender A and receiver B:
+
+ o A single, long-lived non-application-limited TCP NewReno flow;
+
+ o A single, application-limited TCP NewReno flow, with an IW set to
+ 3 or 10 packets. The size of the data transferred must be
+ strictly higher than 10 packets and should be lower than 100
+ packets.
+
+ The transmission of the non-application-limited flow must start first
+ and the transmission of the application-limited flow starts after the
+ non-application-limited flow has reached steady state. The steady
+ state can be assumed when the goodput is stable.
+
+ For each of these scenarios, the graph described in Section 2.7 could
+ be generated for each class of traffic (application-limited and non-
+ application-limited). The completion time of the application-limited
+ TCP flow could be measured.
+
+5.2. Aggressive Transport Sender
+
+ This scenario helps testers to evaluate how an AQM scheme reacts to a
+ transport sender that is more aggressive than a single TCP-friendly
+ sender. We define 'aggressiveness' as a higher-than-standard
+ increase factor upon a successful transmission and/or a lower-than-
+ standard decrease factor upon a unsuccessful transmission (e.g., in
+ case of congestion controls with the Additive Increase Multiplicative
+ Decrease (AIMD) principle, a larger AI and/or MD factors). A single
+
+
+
+Kuhn, et al. Informational [Page 19]
+
+RFC 7928 AQM Characterization Guidelines July 2016
+
+
+ long-lived, non-application-limited, CUBIC flow transfers data
+ between sender A and receiver B. Other aggressive congestion control
+ schemes may also be considered.
+
+ For each flavor of aggressive transports, the graph described in
+ Section 2.7 could be generated.
+
+5.3. Unresponsive Transport Sender
+
+ This scenario helps testers evaluate how an AQM scheme reacts to a
+ transport sender that is less responsive than TCP. Note that faulty
+ transport implementations on an end host and/or faulty network
+ elements en route that "hide" congestion signals in packet headers
+ may also lead to a similar situation, such that the AQM scheme needs
+ to adapt to unresponsive traffic (see Section 3 of the AQM
+ recommendation document [RFC7567]). To this end, these guidelines
+ propose the two following scenarios:
+
+ o The first scenario can be used to evaluate queue build up. It
+ considers unresponsive flow(s) whose sending rate is greater than
+ the bottleneck link capacity between routers L and R. This
+ scenario consists of a long-lived non-application-limited UDP flow
+ that transmits data between sender A and receiver B. The graph
+ described in Section 2.7 could be generated.
+
+ o The second scenario can be used to evaluate if the AQM scheme is
+ able to keep the responsive fraction under control. This scenario
+ considers a mixture of TCP-friendly and unresponsive traffic. It
+ consists of a long-lived UDP flow from unresponsive application
+ and a single long-lived, non-application-limited (unlimited data
+ available to the transport sender from the application layer), TCP
+ New Reno flow that transmit data between sender A and receiver B.
+ As opposed to the first scenario, the rate of the UDP traffic
+ should not be greater than the bottleneck capacity, and should be
+ higher than half of the bottleneck capacity. For each type of
+ traffic, the graph described in Section 2.7 could be generated.
+
+5.4. Less-than-Best-Effort Transport Sender
+
+ This scenario helps to evaluate how an AQM scheme reacts to LBE
+ congestion control that "results in smaller bandwidth and/or delay
+ impact on standard TCP than standard TCP itself, when sharing a
+ bottleneck with it" [RFC6297]. There are potential fateful
+ interactions when AQM and LBE techniques are combined [GONG2014];
+ this scenario helps to evaluate whether the coexistence of the
+ proposed AQM and LBE techniques may be possible.
+
+
+
+
+
+Kuhn, et al. Informational [Page 20]
+
+RFC 7928 AQM Characterization Guidelines July 2016
+
+
+ A single long-lived non-application-limited TCP NewReno flow
+ transfers data between sender A and receiver B. Other TCP-friendly
+ congestion control schemes may also be considered. Single long-lived
+ non-application-limited LEDBAT [RFC6817] flows transfer data between
+ sender A and receiver B. We recommend setting the target delay and
+ gain values of LEDBAT to 5 ms and 10, respectively [TRAN2014]. Other
+ LBE congestion control schemes may also be considered and are listed
+ in the IETF survey of LBE protocols [RFC6297].
+
+ For each of the TCP-friendly and LBE transports, the graph described
+ in Section 2.7 could be generated.
+
+6. Round-Trip Time Fairness
+
+6.1. Motivation
+
+ An AQM scheme's congestion signals (via drops or ECN marks) must
+ reach the transport sender so that a responsive sender can initiate
+ its congestion control mechanism and adjust the sending rate. This
+ procedure is thus dependent on the end-to-end path RTT. When the RTT
+ varies, the onset of congestion control is impacted, and in turn
+ impacts the ability of an AQM scheme to control the queue. It is
+ therefore important to assess the AQM schemes for a set of RTTs
+ between A and B (e.g., from 5 to 200 ms).
+
+ The asymmetry in terms of difference in intrinsic RTT between various
+ paths sharing the same bottleneck should be considered, so that the
+ fairness between the flows can be discussed. In this scenario, a
+ flow traversing on a shorter RTT path may react faster to congestion
+ and recover faster from it compared to another flow on a longer RTT
+ path. The introduction of AQM schemes may potentially improve the
+ RTT fairness.
+
+ Introducing an AQM scheme may cause unfairness between the flows,
+ even if the RTTs are identical. This potential unfairness should be
+ investigated as well.
+
+6.2. Recommended Tests
+
+ The recommended topology is detailed in Figure 1.
+
+ To evaluate the RTT fairness, for each run, two flows are divided
+ into two categories. Category I whose RTT between sender A and
+ receiver B should be 100 ms. Category II, in which the RTT between
+ sender A and receiver B should be in the range [5 ms, 560 ms]
+ inclusive. The maximum value for the RTT represents the RTT of a
+ satellite link [RFC2488].
+
+
+
+
+Kuhn, et al. Informational [Page 21]
+
+RFC 7928 AQM Characterization Guidelines July 2016
+
+
+ A set of evaluated flows must use the same congestion control
+ algorithm: all the generated flows could be single long-lived non-
+ application-limited TCP NewReno flows.
+
+6.3. Metrics to Evaluate the RTT Fairness
+
+ The outputs that must be measured are: (1) the cumulative average
+ goodput of the flow from Category I, goodput_Cat_I (see Section 2.5
+ for the estimation of the goodput); (2) the cumulative average
+ goodput of the flow from Category II, goodput_Cat_II (see Section 2.5
+ for the estimation of the goodput); (3) the ratio goodput_Cat_II/
+ goodput_Cat_I; and (4) the average packet drop rate for each category
+ (Section 2.3).
+
+7. Burst Absorption
+
+ "AQM mechanisms might need to control the overall queue sizes to
+ ensure that arriving bursts can be accommodated without dropping
+ packets" [RFC7567].
+
+7.1. Motivation
+
+ An AQM scheme can face bursts of packet arrivals due to various
+ reasons. Dropping one or more packets from a burst can result in
+ performance penalties for the corresponding flows, since dropped
+ packets have to be retransmitted. Performance penalties can result
+ in failing to meet Service Level Agreements (SLAs) and can be a
+ disincentive to AQM adoption.
+
+ The ability to accommodate bursts translates to larger queue length
+ and hence more queuing delay. On the one hand, it is important that
+ an AQM scheme quickly brings bursty traffic under control. On the
+ other hand, a peak in the packet drop rates to bring a packet burst
+ quickly under control could result in multiple drops per flow and
+ severely impact transport and application performance. Therefore, an
+ AQM scheme ought to bring bursts under control by balancing both
+ aspects -- (1) queuing delay spikes are minimized and (2) performance
+ penalties for ongoing flows in terms of packet drops are minimized.
+
+ An AQM scheme that maintains short queues allows some remaining space
+ in the buffer for bursts of arriving packets. The tolerance to
+ bursts of packets depends upon the number of packets in the queue,
+ which is directly linked to the AQM algorithm. Moreover, an AQM
+ scheme may implement a feature controlling the maximum size of
+ accepted bursts that can depend on the buffer occupancy or the
+ currently estimated queuing delay. The impact of the buffer size on
+ the burst allowance may be evaluated.
+
+
+
+
+Kuhn, et al. Informational [Page 22]
+
+RFC 7928 AQM Characterization Guidelines July 2016
+
+
+7.2. Recommended Tests
+
+ For this scenario, the tester must evaluate how the AQM performs with
+ a traffic mix. The traffic mix could be composed of (from sender A
+ to receiver B):
+
+ o Burst of packets at the beginning of a transmission, such as web
+ traffic with IW10;
+
+ o Applications that send large bursts of data, such as bursty video
+ frames;
+
+ o Background traffic, such as Constant Bit Rate (CBR) UDP traffic
+ and/or A single non-application-limited bulk TCP flow as
+ background traffic.
+
+ Figure 2 presents the various cases for the traffic that must be
+ generated between sender A and receiver B.
+
+ +-------------------------------------------------+
+ |Case| Traffic Type |
+ | +-----+------------+----+--------------------+
+ | |Video|Web (IW 10)| CBR| Bulk TCP Traffic |
+ +----|-----|------------|----|--------------------|
+ |I | 0 | 1 | 1 | 0 |
+ +----|-----|------------|----|--------------------|
+ |II | 0 | 1 | 1 | 1 |
+ |----|-----|------------|----|--------------------|
+ |III | 1 | 1 | 1 | 0 |
+ +----|-----|------------|----|--------------------|
+ |IV | 1 | 1 | 1 | 1 |
+ +----+-----+------------+----+--------------------+
+
+ Figure 2: Bursty Traffic Scenarios
+
+ A new web page download could start after the previous web page
+ download is finished. Each web page could be composed of at least 50
+ objects and the size of each object should be at least 1 KB. Six TCP
+ parallel connections should be generated to download the objects,
+ each parallel connection having an initial congestion window set to
+ 10 packets.
+
+ For each of these scenarios, the graph described in Section 2.7 could
+ be generated for each application. Metrics such as end-to-end
+ latency, jitter, and flow completion time may be generated. For the
+ cases of frame generation of bursty video traffic as well as the
+ choice of web traffic pattern, these details and their presentation
+ are left to the testers.
+
+
+
+Kuhn, et al. Informational [Page 23]
+
+RFC 7928 AQM Characterization Guidelines July 2016
+
+
+8. Stability
+
+8.1. Motivation
+
+ The safety of an AQM scheme is directly related to its stability
+ under varying operating conditions such as varying traffic profiles
+ and fluctuating network conditions. Since operating conditions can
+ vary often, the AQM needs to remain stable under these conditions
+ without the need for additional external tuning.
+
+ Network devices can experience varying operating conditions depending
+ on factors such as time of the day, deployment scenario, etc. For
+ example:
+
+ o Traffic and congestion levels are higher during peak hours than
+ off-peak hours.
+
+ o In the presence of a scheduler, the draining rate of a queue can
+ vary depending on the occupancy of other queues: a low load on a
+ high-priority queue implies a higher draining rate for the lower-
+ priority queues.
+
+ o The capacity available can vary over time (e.g., a lossy channel,
+ a link supporting traffic in a higher Diffserv class).
+
+ Whether or not the target context is a stable environment, the
+ ability of an AQM scheme to maintain its control over the queuing
+ delay and buffer occupancy can be challenged. This document proposes
+ guidelines to assess the behavior of AQM schemes under varying
+ congestion levels and varying draining rates.
+
+8.2. Recommended Tests
+
+ Note that the traffic profiles explained below comprises non-
+ application-limited TCP flows. For each of the below scenarios, the
+ graphs described in Section 2.7 should be generated, and the goodput
+ of the various flows should be cumulated. For Section 8.2.5 and
+ Section 8.2.6, they should incorporate the results in a per-phase
+ basis as well.
+
+ Wherever the notion of time has been explicitly mentioned in this
+ subsection, time 0 starts from the moment all TCP flows have already
+ reached their congestion avoidance phase.
+
+
+
+
+
+
+
+
+Kuhn, et al. Informational [Page 24]
+
+RFC 7928 AQM Characterization Guidelines July 2016
+
+
+8.2.1. Definition of the Congestion Level
+
+ In these guidelines, the congestion levels are represented by the
+ projected packet drop rate, which is determined when there is no AQM
+ scheme (i.e., a drop-tail queue). When the bottleneck is shared
+ among non-application-limited TCP flows, l_r (the loss rate
+ projection) can be expressed as a function of N, the number of bulk
+ TCP flows, and S, the sum of the bandwidth-delay product and the
+ maximum buffer size, both expressed in packets, based on Eq. 3 of
+ [MORR2000]:
+
+ l_r = 0.76 * N^2 / S^2
+
+ N = S * SQRT(1/0.76) * SQRT(l_r)
+
+ These guidelines use the loss rate to define the different congestion
+ levels, but they do not stipulate that in other circumstances,
+ measuring the congestion level gives you an accurate estimation of
+ the loss rate or vice versa.
+
+8.2.2. Mild Congestion
+
+ This scenario can be used to evaluate how an AQM scheme reacts to a
+ light load of incoming traffic resulting in mild congestion -- packet
+ drop rates around 0.1%. The number of bulk flows required to achieve
+ this congestion level, N_mild, is then:
+
+ N_mild = ROUND (0.036*S)
+
+8.2.3. Medium Congestion
+
+ This scenario can be used to evaluate how an AQM scheme reacts to
+ incoming traffic resulting in medium congestion -- packet drop rates
+ around 0.5%. The number of bulk flows required to achieve this
+ congestion level, N_med, is then:
+
+ N_med = ROUND (0.081*S)
+
+8.2.4. Heavy Congestion
+
+ This scenario can be used to evaluate how an AQM scheme reacts to
+ incoming traffic resulting in heavy congestion -- packet drop rates
+ around 1%. The number of bulk flows required to achieve this
+ congestion level, N_heavy, is then:
+
+ N_heavy = ROUND (0.114*S)
+
+
+
+
+
+Kuhn, et al. Informational [Page 25]
+
+RFC 7928 AQM Characterization Guidelines July 2016
+
+
+8.2.5. Varying the Congestion Level
+
+ This scenario can be used to evaluate how an AQM scheme reacts to
+ incoming traffic resulting in various levels of congestion during the
+ experiment. In this scenario, the congestion level varies within a
+ large timescale. The following phases may be considered: phase I --
+ mild congestion during 0-20 s; phase II -- medium congestion during
+ 20-40 s; phase III -- heavy congestion during 40-60 s; phase I again,
+ and so on.
+
+8.2.6. Varying Available Capacity
+
+ This scenario can be used to help characterize how the AQM behaves
+ and adapts to bandwidth changes. The experiments are not meant to
+ reflect the exact conditions of Wi-Fi environments since it is hard
+ to design repetitive experiments or accurate simulations for such
+ scenarios.
+
+ To emulate varying draining rates, the bottleneck capacity between
+ nodes 'Router L' and 'Router R' varies over the course of the
+ experiment as follows:
+
+ o Experiment 1: The capacity varies between two values within a
+ large timescale. As an example, the following phases may be
+ considered: phase I -- 100 Mbps during 0-20 s; phase II -- 10 Mbps
+ during 20-40 s; phase I again, and so on.
+
+ o Experiment 2: The capacity varies between two values within a
+ short timescale. As an example, the following phases may be
+ considered: phase I -- 100 Mbps during 0-100 ms; phase II -- 10
+ Mbps during 100-200 ms; phase I again, and so on.
+
+ The tester may choose a phase time-interval value different than what
+ is stated above, if the network's path conditions (such as bandwidth-
+ delay product) necessitate. In this case, the choice of such a time-
+ interval value should be stated and elaborated.
+
+ The tester may additionally evaluate the two mentioned scenarios
+ (short-term and long-term capacity variations), during and/or
+ including the TCP slow-start phase.
+
+ More realistic fluctuating capacity patterns may be considered. The
+ tester may choose to incorporate realistic scenarios with regards to
+ common fluctuation of bandwidth in state-of-the-art technologies.
+
+ The scenario consists of TCP NewReno flows between sender A and
+ receiver B. To better assess the impact of draining rates on the AQM
+ behavior, the tester must compare its performance with those of drop-
+
+
+
+Kuhn, et al. Informational [Page 26]
+
+RFC 7928 AQM Characterization Guidelines July 2016
+
+
+ tail and should provide a reference document for their proposal
+ discussing performance and deployment compared to those of drop-tail.
+ Burst traffic, such as presented in Section 7.2, could also be
+ considered to assess the impact of varying available capacity on the
+ burst absorption of the AQM.
+
+8.3. Parameter Sensitivity and Stability Analysis
+
+ The control law used by an AQM is the primary means by which the
+ queuing delay is controlled. Hence, understanding the control law is
+ critical to understanding the behavior of the AQM scheme. The
+ control law could include several input parameters whose values
+ affect the AQM scheme's output behavior and its stability.
+ Additionally, AQM schemes may auto-tune parameter values in order to
+ maintain stability under different network conditions (such as
+ different congestion levels, draining rates, or network
+ environments). The stability of these auto-tuning techniques is also
+ important to understand.
+
+ Transports operating under the control of AQM experience the effect
+ of multiple control loops that react over different timescales. It
+ is therefore important that proposed AQM schemes are seen to be
+ stable when they are deployed at multiple points of potential
+ congestion along an Internet path. The pattern of congestion signals
+ (loss or ECN-marking) arising from AQM methods also needs to not
+ adversely interact with the dynamics of the transport protocols that
+ they control.
+
+ AQM proposals should provide background material showing theoretical
+ analysis of the AQM control law and the input parameter space within
+ which the control law operates, or they should use another way to
+ discuss the stability of the control law. For parameters that are
+ auto-tuned, the material should include stability analysis of the
+ auto-tuning mechanism(s) as well. Such analysis helps to understand
+ an AQM control law better and the network conditions/deployments
+ under which the AQM is stable.
+
+9. Various Traffic Profiles
+
+ This section provides guidelines to assess the performance of an AQM
+ proposal for various traffic profiles such as traffic with different
+ applications or bidirectional traffic.
+
+
+
+
+
+
+
+
+
+Kuhn, et al. Informational [Page 27]
+
+RFC 7928 AQM Characterization Guidelines July 2016
+
+
+9.1. Traffic Mix
+
+ This scenario can be used to evaluate how an AQM scheme reacts to a
+ traffic mix consisting of different applications such as:
+
+ o Bulk TCP transfer
+
+ o Web traffic
+
+ o VoIP
+
+ o Constant Bit Rate (CBR) UDP traffic
+
+ o Adaptive video streaming (either unidirectional or bidirectional)
+
+ Various traffic mixes can be considered. These guidelines recommend
+ examining at least the following example: 1 bidirectional VoIP; 6 web
+ page downloads (such as those detailed in Section 7.2); 1 CBR; 1
+ Adaptive Video; 5 bulk TCP. Any other combinations could be
+ considered and should be carefully documented.
+
+ For each scenario, the graph described in Section 2.7 could be
+ generated for each class of traffic. Metrics such as end-to-end
+ latency, jitter, and flow completion time may be reported.
+
+9.2. Bidirectional Traffic
+
+ Control packets such as DNS requests/responses, TCP SYNs/ACKs are
+ small, but their loss can severely impact the application
+ performance. The scenario proposed in this section will help in
+ assessing whether the introduction of an AQM scheme increases the
+ loss probability of these important packets.
+
+ For this scenario, traffic must be generated in both downlink and
+ uplink, as defined in Section 3.1. The amount of asymmetry between
+ the uplink and the downlink depends on the context. These guidelines
+ recommend considering a mild congestion level and the traffic
+ presented in Section 8.2.2 in both directions. In this case, the
+ metrics reported must be the same as in Section 8.2 for each
+ direction.
+
+ The traffic mix presented in Section 9.1 may also be generated in
+ both directions.
+
+
+
+
+
+
+
+
+Kuhn, et al. Informational [Page 28]
+
+RFC 7928 AQM Characterization Guidelines July 2016
+
+
+10. Example of a Multi-AQM Scenario
+
+10.1. Motivation
+
+ Transports operating under the control of AQM experience the effect
+ of multiple control loops that react over different timescales. It
+ is therefore important that proposed AQM schemes are seen to be
+ stable when they are deployed at multiple points of potential
+ congestion along an Internet path. The pattern of congestion signals
+ (loss or ECN-marking) arising from AQM methods also need to not
+ adversely interact with the dynamics of the transport protocols that
+ they control.
+
+10.2. Details on the Evaluation Scenario
+
+ +---------+ +-----------+
+ |senders A|---+ +---|receivers A|
+ +---------+ | | +-----------+
+ +-----+---+ +---------+ +--+-----+
+ |Router L |--|Router M |--|Router R|
+ |AQM A | |AQM M | |No AQM |
+ +---------+ +--+------+ +--+-----+
+ +---------+ | | +-----------+
+ |senders B|-------------+ +---|receivers B|
+ +---------+ +-----------+
+
+ Figure 3: Topology for the Multi-AQM Scenario
+
+ Figure 3 describes topology options for evaluating multi-AQM
+ scenarios. The AQM schemes are applied in sequence and impact the
+ induced latency reduction, the induced goodput maximization, and the
+ trade-off between these two. Note that AQM schemes A and B
+ introduced in Routers L and M could be (I) same scheme with identical
+ parameter values, (ii) same scheme with different parameter values,
+ or (iii) two different schemes. To best understand the interactions
+ and implications, the mild congestion scenario as described in
+ Section 8.2.2 is recommended such that the number of flows is equally
+ shared among senders A and B. Other relevant combinations of
+ congestion levels could also be considered. We recommend measuring
+ the metrics presented in Section 8.2.
+
+
+
+
+
+
+
+
+
+
+
+Kuhn, et al. Informational [Page 29]
+
+RFC 7928 AQM Characterization Guidelines July 2016
+
+
+11. Implementation Cost
+
+11.1. Motivation
+
+ Successful deployment of AQM is directly related to its cost of
+ implementation. Network devices may need hardware or software
+ implementations of the AQM mechanism. Depending on a device's
+ capabilities and limitations, the device may or may not be able to
+ implement some or all parts of their AQM logic.
+
+ AQM proposals should provide pseudocode for the complete AQM scheme,
+ highlighting generic implementation-specific aspects of the scheme
+ such as "drop-tail" vs. "drop-head", inputs (e.g., current queuing
+ delay, and queue length), computations involved, need for timers,
+ etc. This helps to identify costs associated with implementing the
+ AQM scheme on a particular hardware or software device. This also
+ facilitates discussions around which kind of devices can easily
+ support the AQM and which cannot.
+
+11.2. Recommended Discussion
+
+ AQM proposals should highlight parts of their AQM logic that are
+ device dependent and discuss if and how AQM behavior could be
+ impacted by the device. For example, a queuing-delay-based AQM
+ scheme requires current queuing delay as input from the device. If
+ the device already maintains this value, then it can be trivial to
+ implement the AQM logic on the device. If the device provides
+ indirect means to estimate the queuing delay (for example, timestamps
+ and dequeuing rate), then the AQM behavior is sensitive to the
+ precision of the queuing delay estimations are for that device.
+ Highlighting the sensitivity of an AQM scheme to queuing delay
+ estimations helps implementers to identify appropriate means of
+ implementing the mechanism on a device.
+
+12. Operator Control and Auto-Tuning
+
+12.1. Motivation
+
+ One of the biggest hurdles of RED deployment was/is its parameter
+ sensitivity to operating conditions -- how difficult it is to tune
+ RED parameters for a deployment to achieve acceptable benefit from
+ using RED. Fluctuating congestion levels and network conditions add
+ to the complexity. Incorrect parameter values lead to poor
+ performance.
+
+ Any AQM scheme is likely to have parameters whose values affect the
+ control law and behavior of an AQM. Exposing all these parameters as
+ control parameters to a network operator (or user) can easily result
+
+
+
+Kuhn, et al. Informational [Page 30]
+
+RFC 7928 AQM Characterization Guidelines July 2016
+
+
+ in an unsafe AQM deployment. Unexpected AQM behavior ensues when
+ parameter values are set improperly. A minimal number of control
+ parameters minimizes the number of ways a user can break a system
+ where an AQM scheme is deployed at. Fewer control parameters make
+ the AQM scheme more user-friendly and easier to deploy and debug.
+
+ "AQM algorithms SHOULD NOT require tuning of initial or configuration
+ parameters in common use cases." such as stated in Section 4 of the
+ AQM recommendation document [RFC7567]. A scheme ought to expose only
+ those parameters that control the macroscopic AQM behavior such as
+ queue delay threshold, queue length threshold, etc.
+
+ Additionally, the safety of an AQM scheme is directly related to its
+ stability under varying operating conditions such as varying traffic
+ profiles and fluctuating network conditions, as described in
+ Section 8. Operating conditions vary often and hence the AQM needs
+ to remain stable under these conditions without the need for
+ additional external tuning. If AQM parameters require tuning under
+ these conditions, then the AQM must self-adapt necessary parameter
+ values by employing auto-tuning techniques.
+
+12.2. Recommended Discussion
+
+ In order to understand an AQM's deployment considerations and
+ performance under a specific environment, AQM proposals should
+ describe the parameters that control the macroscopic AQM behavior,
+ and identify any parameters that require tuning to operational
+ conditions. It could be interesting to also discuss that, even if an
+ AQM scheme may not adequately auto-tune its parameters, the resulting
+ performance may not be optimal, but close to something reasonable.
+
+ If there are any fixed parameters within the AQM, their setting
+ should be discussed and justified to help understand whether a fixed
+ parameter value is applicable for a particular environment.
+
+ If an AQM scheme is evaluated with parameter(s) that were externally
+ tuned for optimization or other purposes, these values must be
+ disclosed.
+
+13. Summary
+
+ Figure 4 lists the scenarios for an extended characterization of an
+ AQM scheme. This table comes along with a set of requirements to
+ present more clearly the weight and importance of each scenario. The
+ requirements listed here are informational and their relevance may
+ depend on the deployment scenario.
+
+
+
+
+
+Kuhn, et al. Informational [Page 31]
+
+RFC 7928 AQM Characterization Guidelines July 2016
+
+
+ +------------------------------------------------------------------+
+ |Scenario |Sec. |Informational requirement |
+ +------------------------------------------------------------------+
+ +------------------------------------------------------------------+
+ |Interaction with ECN | 4.5 |must be discussed if supported |
+ +------------------------------------------------------------------+
+ |Interaction with Scheduling| 4.6 |should be discussed |
+ +------------------------------------------------------------------+
+ |Transport Protocols | 5 | |
+ | TCP-friendly sender | 5.1 |scenario must be considered |
+ | Aggressive sender | 5.2 |scenario must be considered |
+ | Unresponsive sender | 5.3 |scenario must be considered |
+ | LBE sender | 5.4 |scenario may be considered |
+ +------------------------------------------------------------------+
+ |Round-Trip Time Fairness | 6.2 |scenario must be considered |
+ +------------------------------------------------------------------+
+ |Burst Absorption | 7.2 |scenario must be considered |
+ +------------------------------------------------------------------+
+ |Stability | 8 | |
+ | Varying congestion levels | 8.2.5|scenario must be considered |
+ | Varying available capacity| 8.2.6|scenario must be considered |
+ | Parameters and stability | 8.3 |this should be discussed |
+ +------------------------------------------------------------------+
+ |Various Traffic Profiles | 9 | |
+ | Traffic mix | 9.1 |scenario is recommended |
+ | Bidirectional traffic | 9.2 |scenario may be considered |
+ +------------------------------------------------------------------+
+ |Multi-AQM | 10.2 |scenario may be considered |
+ +------------------------------------------------------------------+
+
+ Figure 4: Summary of the Scenarios and their Requirements
+
+14. Security Considerations
+
+ Some security considerations for AQM are identified in [RFC7567].
+ This document, by itself, presents no new privacy or security issues.
+
+15. References
+
+15.1. Normative References
+
+ [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate
+ Requirement Levels", RFC 2119, 1997.
+
+ [RFC2544] Bradner, S. and J. McQuaid, "Benchmarking Methodology for
+ Network Interconnect Devices", RFC 2544,
+ DOI 10.17487/RFC2544, March 1999,
+ <http://www.rfc-editor.org/info/rfc2544>.
+
+
+
+Kuhn, et al. Informational [Page 32]
+
+RFC 7928 AQM Characterization Guidelines July 2016
+
+
+ [RFC2647] Newman, D., "Benchmarking Terminology for Firewall
+ Performance", RFC 2647, DOI 10.17487/RFC2647, August 1999,
+ <http://www.rfc-editor.org/info/rfc2647>.
+
+ [RFC5481] Morton, A. and B. Claise, "Packet Delay Variation
+ Applicability Statement", RFC 5481, DOI 10.17487/RFC5481,
+ March 2009, <http://www.rfc-editor.org/info/rfc5481>.
+
+ [RFC7567] Baker, F., Ed. and G. Fairhurst, Ed., "IETF
+ Recommendations Regarding Active Queue Management",
+ BCP 197, RFC 7567, DOI 10.17487/RFC7567, July 2015,
+ <http://www.rfc-editor.org/info/rfc7567>.
+
+ [RFC7679] Almes, G., Kalidindi, S., Zekauskas, M., and A. Morton,
+ Ed., "A One-Way Delay Metric for IP Performance Metrics
+ (IPPM)", STD 81, RFC 7679, DOI 10.17487/RFC7679, January
+ 2016, <http://www.rfc-editor.org/info/rfc7679>.
+
+ [RFC7680] Almes, G., Kalidindi, S., Zekauskas, M., and A. Morton,
+ Ed., "A One-Way Loss Metric for IP Performance Metrics
+ (IPPM)", STD 82, RFC 7680, DOI 10.17487/RFC7680, January
+ 2016, <http://www.rfc-editor.org/info/rfc7680>.
+
+15.2. Informative References
+
+ [ANEL2014] Anelli, P., Diana, R., and E. Lochin, "FavorQueue: a
+ Parameterless Active Queue Management to Improve TCP
+ Traffic Performance", Computer Networks Vol. 60,
+ DOI 10.1016/j.bjp.2013.11.008, 2014.
+
+ [AQMPIE] Pan, R., Natarajan, P., Baker, F., and G. White, "PIE: A
+ Lightweight Control Scheme To Address the Bufferbloat
+ Problem", Work in Progress, draft-ietf-aqm-pie-08, June
+ 2016.
+
+ [BB2011] Cerf, V., Jacobson, V., Weaver, N., and J. Gettys,
+ "BufferBloat: what's wrong with the internet?", ACM
+ Queue Vol. 55, DOI 10.1145/2076450.2076464, 2012.
+
+ [BCP41] Floyd, S., "Congestion Control Principles", BCP 41,
+ RFC 2914, September 2000.
+
+ Briscoe, B. and J. Manner, "Byte and Packet Congestion
+ Notification", BCP 41, RFC 7141, February 2014.
+
+ <http://www.rfc-editor.org/info/bcp41>
+
+
+
+
+
+Kuhn, et al. Informational [Page 33]
+
+RFC 7928 AQM Characterization Guidelines July 2016
+
+
+ [CODEL] Nichols, K., Jacobson, V., McGregor, A., and J. Iyengar,
+ "Controlled Delay Active Queue Management", Work in
+ Progress, draft-ietf-aqm-codel-04, June 2016.
+
+ [CUBIC] Rhee, I., Xu, L., Ha, S., Zimmermann, A., Eggert, L., and
+ R. Scheffenegger, "CUBIC for Fast Long-Distance Networks",
+ Work in Progress, draft-ietf-tcpm-cubic-01, January 2016.
+
+ [FENG2002] Feng, W., Shin, K., Kandlur, D., and D. Saha, "The BLUE
+ active queue management algorithms", IEEE Transactions on
+ Networking Vol.10 Issue 4, DOI 10.1109/TNET.2002.801399,
+ 2002, <http://ieeexplore.ieee.org/xpl/
+ articleDetails.jsp?arnumber=1026008>.
+
+ [FLOY1993] Floyd, S. and V. Jacobson, "Random Early Detection (RED)
+ Gateways for Congestion Avoidance", IEEE Transactions on
+ Networking Vol. 1 Issue 4, DOI 10.1109/90.251892, 1993,
+ <http://ieeexplore.ieee.org/xpl/
+ articleDetails.jsp?arnumber=251892>.
+
+ [GONG2014] Gong, Y., Rossi, D., Testa, C., Valenti, S., and D. Taht,
+ "Fighting the bufferbloat: on the coexistence of AQM and
+ low priority congestion control", Computer Networks,
+ Elsevier, 2014, pp.115-128 Vol. 60,
+ DOI 10.1109/INFCOMW.2013.6562885, 2014.
+
+ [HASS2008] Hassayoun, S. and D. Ros, "Loss Synchronization and Router
+ Buffer Sizing with High-Speed Versions of TCP",
+ IEEE INFOCOM Workshops, DOI 10.1109/INFOCOM.2008.4544632,
+ 2008, <http://ieeexplore.ieee.org/xpl/
+ articleDetails.jsp?arnumber=4544632>.
+
+ [HOEI2015] Hoeiland-Joergensen, T., McKenney, P.,
+ dave.taht@gmail.com, d., Gettys, J., and E. Dumazet, "The
+ FlowQueue-CoDel Packet Scheduler and Active Queue
+ Management Algorithm", Work in Progress, draft-ietf-aqm-
+ fq-codel-06, March 2016.
+
+ [HOLLO2001]
+ Hollot, C., Misra, V., Towsley, V., and W. Gong, "On
+ Designing Improved Controller for AQM Routers Supporting
+ TCP Flows", IEEE INFOCOM, DOI 10.1109/INFCOM.2001.916670,
+ 2001, <http://ieeexplore.ieee.org/xpl/
+ articleDetails.jsp?arnumber=916670>.
+
+
+
+
+
+
+
+Kuhn, et al. Informational [Page 34]
+
+RFC 7928 AQM Characterization Guidelines July 2016
+
+
+ [JAY2006] Jay, P., Fu, Q., and G. Armitage, "A preliminary analysis
+ of loss synchronisation between concurrent TCP flows",
+ Australian Telecommunication Networks and Application
+ Conference (ATNAC), 2006.
+
+ [MORR2000] Morris, R., "Scalable TCP congestion control",
+ IEEE INFOCOM, DOI 10.1109/INFCOM.2000.832487, 2000,
+ <http://ieeexplore.ieee.org/xpl/
+ articleDetails.jsp?arnumber=832487>.
+
+ [RFC793] Postel, J., "Transmission Control Protocol", STD 7,
+ RFC 793, DOI 10.17487/RFC0793, September 1981,
+ <http://www.rfc-editor.org/info/rfc793>.
+
+ [RFC2309] Braden, B., Clark, D., Crowcroft, J., Davie, B., Deering,
+ S., Estrin, D., Floyd, S., Jacobson, V., Minshall, G.,
+ Partridge, C., Peterson, L., Ramakrishnan, K., Shenker,
+ S., Wroclawski, J., and L. Zhang, "Recommendations on
+ Queue Management and Congestion Avoidance in the
+ Internet", RFC 2309, DOI 10.17487/RFC2309, April 1998,
+ <http://www.rfc-editor.org/info/rfc2309>.
+
+ [RFC2488] Allman, M., Glover, D., and L. Sanchez, "Enhancing TCP
+ Over Satellite Channels using Standard Mechanisms",
+ BCP 28, RFC 2488, DOI 10.17487/RFC2488, January 1999,
+ <http://www.rfc-editor.org/info/rfc2488>.
+
+ [RFC3168] Ramakrishnan, K., Floyd, S., and D. Black, "The Addition
+ of Explicit Congestion Notification (ECN) to IP",
+ RFC 3168, DOI 10.17487/RFC3168, September 2001,
+ <http://www.rfc-editor.org/info/rfc3168>.
+
+ [RFC3611] Friedman, T., Ed., Caceres, R., Ed., and A. Clark, Ed.,
+ "RTP Control Protocol Extended Reports (RTCP XR)",
+ RFC 3611, DOI 10.17487/RFC3611, November 2003,
+ <http://www.rfc-editor.org/info/rfc3611>.
+
+ [RFC5348] Floyd, S., Handley, M., Padhye, J., and J. Widmer, "TCP
+ Friendly Rate Control (TFRC): Protocol Specification",
+ RFC 5348, DOI 10.17487/RFC5348, September 2008,
+ <http://www.rfc-editor.org/info/rfc5348>.
+
+ [RFC5681] Allman, M., Paxson, V., and E. Blanton, "TCP Congestion
+ Control", RFC 5681, DOI 10.17487/RFC5681, September 2009,
+ <http://www.rfc-editor.org/info/rfc5681>.
+
+
+
+
+
+
+Kuhn, et al. Informational [Page 35]
+
+RFC 7928 AQM Characterization Guidelines July 2016
+
+
+ [RFC6297] Welzl, M. and D. Ros, "A Survey of Lower-than-Best-Effort
+ Transport Protocols", RFC 6297, DOI 10.17487/RFC6297, June
+ 2011, <http://www.rfc-editor.org/info/rfc6297>.
+
+ [RFC6817] Shalunov, S., Hazel, G., Iyengar, J., and M. Kuehlewind,
+ "Low Extra Delay Background Transport (LEDBAT)", RFC 6817,
+ DOI 10.17487/RFC6817, December 2012,
+ <http://www.rfc-editor.org/info/rfc6817>.
+
+ [RFC7141] Briscoe, B. and J. Manner, "Byte and Packet Congestion
+ Notification", BCP 41, RFC 7141, DOI 10.17487/RFC7141,
+ February 2014, <http://www.rfc-editor.org/info/rfc7141>.
+
+ [TCPEVAL] Hayes, D., Ros, D., Andrew, L., and S. Floyd, "Common TCP
+ Evaluation Suite", Work in Progress, draft-irtf-iccrg-
+ tcpeval-01, July 2014.
+
+ [TRAN2014] Trang, S., Kuhn, N., Lochin, E., Baudoin, C., Dubois, E.,
+ and P. Gelard, "On The Existence Of Optimal LEDBAT
+ Parameters", IEEE ICC 2014 - Communication
+ QoS, Reliability and Modeling Symposium,
+ DOI 10.1109/ICC.2014.6883487, 2014,
+ <http://ieeexplore.ieee.org/xpl/
+ articleDetails.jsp?arnumber=6883487>.
+
+ [WELZ2015] Welzl, M. and G. Fairhurst, "The Benefits to Applications
+ of using Explicit Congestion Notification (ECN)", Work in
+ Progress, draft-welzl-ecn-benefits-02, March 2015.
+
+ [WINS2014] Winstein, K., "Transport Architectures for an Evolving
+ Internet", PhD thesis, Massachusetts Institute of
+ Technology, June 2014.
+
+Acknowledgements
+
+ This work has been partially supported by the European Community
+ under its Seventh Framework Programme through the Reducing Internet
+ Transport Latency (RITE) project (ICT-317700).
+
+ Many thanks to S. Akhtar, A.B. Bagayoko, F. Baker, R. Bless, D.
+ Collier-Brown, G. Fairhurst, J. Gettys, P. Goltsman, T. Hoiland-
+ Jorgensen, K. Kilkki, C. Kulatunga, W. Lautenschlager, A.C. Morton,
+ R. Pan, G. Skinner, D. Taht, and M. Welzl for detailed and wise
+ feedback on this document.
+
+
+
+
+
+
+
+Kuhn, et al. Informational [Page 36]
+
+RFC 7928 AQM Characterization Guidelines July 2016
+
+
+Authors' Addresses
+
+ Nicolas Kuhn (editor)
+ CNES, Telecom Bretagne
+ 18 avenue Edouard Belin
+ Toulouse 31400
+ France
+
+ Phone: +33 5 61 27 32 13
+ Email: nicolas.kuhn@cnes.fr
+
+
+ Preethi Natarajan (editor)
+ Cisco Systems
+ 510 McCarthy Blvd
+ Milpitas, California
+ United States of America
+
+ Email: prenatar@cisco.com
+
+
+ Naeem Khademi (editor)
+ University of Oslo
+ Department of Informatics, PO Box 1080 Blindern
+ N-0316 Oslo
+ Norway
+
+ Phone: +47 2285 24 93
+ Email: naeemk@ifi.uio.no
+
+
+ David Ros
+ Simula Research Laboratory AS
+ P.O. Box 134
+ Lysaker, 1325
+ Norway
+
+ Phone: +33 299 25 21 21
+ Email: dros@simula.no
+
+
+
+
+
+
+
+
+
+
+
+
+Kuhn, et al. Informational [Page 37]
+