summaryrefslogtreecommitdiff
path: root/doc/rfc/rfc8869.txt
diff options
context:
space:
mode:
authorThomas Voss <mail@thomasvoss.com> 2024-11-27 20:54:24 +0100
committerThomas Voss <mail@thomasvoss.com> 2024-11-27 20:54:24 +0100
commit4bfd864f10b68b71482b35c818559068ef8d5797 (patch)
treee3989f47a7994642eb325063d46e8f08ffa681dc /doc/rfc/rfc8869.txt
parentea76e11061bda059ae9f9ad130a9895cc85607db (diff)
doc: Add RFC documents
Diffstat (limited to 'doc/rfc/rfc8869.txt')
-rw-r--r--doc/rfc/rfc8869.txt1057
1 files changed, 1057 insertions, 0 deletions
diff --git a/doc/rfc/rfc8869.txt b/doc/rfc/rfc8869.txt
new file mode 100644
index 0000000..feeccb0
--- /dev/null
+++ b/doc/rfc/rfc8869.txt
@@ -0,0 +1,1057 @@
+
+
+
+
+Internet Engineering Task Force (IETF) Z. Sarker
+Request for Comments: 8869 Ericsson AB
+Category: Informational X. Zhu
+ISSN: 2070-1721 J. Fu
+ Cisco Systems
+ January 2021
+
+
+ Evaluation Test Cases for Interactive Real-Time Media over Wireless
+ Networks
+
+Abstract
+
+ The Real-time Transport Protocol (RTP) is a common transport choice
+ for interactive multimedia communication applications. The
+ performance of these applications typically depends on a well-
+ functioning congestion control algorithm. To ensure a seamless and
+ robust user experience, a well-designed RTP-based congestion control
+ algorithm should work well across all access network types. This
+ document describes test cases for evaluating performances of
+ candidate congestion control algorithms over cellular and Wi-Fi
+ networks.
+
+Status of This Memo
+
+ This document is not an Internet Standards Track specification; it is
+ published for informational purposes.
+
+ This document is a product of the Internet Engineering Task Force
+ (IETF). It represents the consensus of the IETF community. It has
+ received public review and has been approved for publication by the
+ Internet Engineering Steering Group (IESG). Not all documents
+ approved by the IESG are candidates for any level of Internet
+ Standard; see Section 2 of RFC 7841.
+
+ Information about the current status of this document, any errata,
+ and how to provide feedback on it may be obtained at
+ https://www.rfc-editor.org/info/rfc8869.
+
+Copyright Notice
+
+ Copyright (c) 2021 IETF Trust and the persons identified as the
+ document authors. All rights reserved.
+
+ This document is subject to BCP 78 and the IETF Trust's Legal
+ Provisions Relating to IETF Documents
+ (https://trustee.ietf.org/license-info) in effect on the date of
+ publication of this document. Please review these documents
+ carefully, as they describe your rights and restrictions with respect
+ to this document. Code Components extracted from this document must
+ include Simplified BSD License text as described in Section 4.e of
+ the Trust Legal Provisions and are provided without warranty as
+ described in the Simplified BSD License.
+
+Table of Contents
+
+ 1. Introduction
+ 2. Cellular Network Specific Test Cases
+ 2.1. Varying Network Load
+ 2.1.1. Network Connection
+ 2.1.2. Simulation Setup
+ 2.1.3. Expected Behavior
+ 2.2. Bad Radio Coverage
+ 2.2.1. Network Connection
+ 2.2.2. Simulation Setup
+ 2.2.3. Expected Behavior
+ 2.3. Desired Evaluation Metrics for Cellular Test Cases
+ 3. Wi-Fi Networks Specific Test Cases
+ 3.1. Bottleneck in Wired Network
+ 3.1.1. Network Topology
+ 3.1.2. Test/Simulation Setup
+ 3.1.3. Typical Test Scenarios
+ 3.1.4. Expected Behavior
+ 3.2. Bottleneck in Wi-Fi Network
+ 3.2.1. Network Topology
+ 3.2.2. Test/Simulation Setup
+ 3.2.3. Typical Test Scenarios
+ 3.2.4. Expected Behavior
+ 3.3. Other Potential Test Cases
+ 3.3.1. EDCA/WMM usage
+ 3.3.2. Effect of Heterogeneous Link Rates
+ 4. IANA Considerations
+ 5. Security Considerations
+ 6. References
+ 6.1. Normative References
+ 6.2. Informative References
+ Contributors
+ Acknowledgments
+ Authors' Addresses
+
+1. Introduction
+
+ Wireless networks (both cellular and Wi-Fi [IEEE802.11]) are an
+ integral and increasingly more significant part of the Internet.
+ Typical application scenarios for interactive multimedia
+ communication over wireless include video conferencing calls in a bus
+ or train as well as live media streaming at home. It is well known
+ that the characteristics and technical challenges for supporting
+ multimedia services over wireless are very different from those of
+ providing the same service over a wired network. Although the basic
+ test cases as defined in [RFC8867] have covered many common effects
+ of network impairments for evaluating RTP-based congestion control
+ schemes, they remain to be tested over characteristics and dynamics
+ unique to a given wireless environment. For example, in cellular
+ networks, the base station maintains individual queues per radio
+ bearer per user hence it leads to a different nature of interactions
+ between traffic flows of different users. This contrasts with a
+ typical wired network setting where traffic flows from all users
+ share the same queue at the bottleneck. Furthermore, user mobility
+ patterns in a cellular network differ from those in a Wi-Fi network.
+ Therefore, it is important to evaluate the performance of proposed
+ candidate RTP-based congestion control solutions over cellular mobile
+ networks and over Wi-Fi networks respectively.
+
+ [RFC8868] provides guidelines for evaluating candidate algorithms and
+ recognizes the importance of testing over wireless access networks.
+ However, it does not describe any specific test cases for performance
+ evaluation of candidate algorithms. This document describes test
+ cases specifically targeting cellular and Wi-Fi networks.
+
+2. Cellular Network Specific Test Cases
+
+ A cellular environment is more complicated than its wireline
+ counterpart since it seeks to provide services in the context of
+ variable available bandwidth, location dependencies, and user
+ mobilities at different speeds. In a cellular network, the user may
+ reach the cell edge, which may lead to a significant number of
+ retransmissions to deliver the data from the base station to the
+ destination and vice versa. These radio links will often act as a
+ bottleneck for the rest of the network and will eventually lead to
+ excessive delays or packet drops. An efficient retransmission or
+ link adaptation mechanism can reduce the packet loss probability, but
+ some packet losses and delay variations will remain. Moreover, with
+ increased cell load or handover to a congested cell, congestion in
+ the transport network will become even worse. Besides, there exist
+ certain characteristics that distinguish the cellular network from
+ other wireless access networks such as Wi-Fi. In a cellular network:
+
+ * The bottleneck is often a shared link with relatively few users.
+
+ - The cost per bit over the shared link varies over time and is
+ different for different users.
+
+ - Leftover/unused resources can be consumed by other greedy
+ users.
+
+ * Queues are always per radio bearer, hence each user can have many
+ such queues.
+
+ * Users can experience both inter- and intra-Radio Access Technology
+ (RAT) handovers (see [HO-def-3GPP] for the definition of
+ "handover").
+
+ * Handover between cells or change of serving cells (as described in
+ [HO-LTE-3GPP] and [HO-UMTS-3GPP]) might cause user plane
+ interruptions, which can lead to bursts of packet losses, delay,
+ and/or jitter. The exact behavior depends on the type of radio
+ bearer. Typically, the default best-effort bearers do not
+ generate packet loss, instead, packets are queued up and
+ transmitted once the handover is completed.
+
+ * The network part decides how much the user can transmit.
+
+ * The cellular network has variable link capacity per user.
+
+ - It can vary as fast as a period of milliseconds.
+
+ - It depends on many factors (such as distance, speed,
+ interference, different flows).
+
+ - It uses complex and smart link adaptation, which makes the link
+ behavior ever more dynamic.
+
+ - The scheduling priority depends on the estimated throughput.
+
+ * Both Quality of Service (QoS) and non-QoS radio bearers can be
+ used.
+
+ Hence, a real-time communication application operating over a
+ cellular network needs to cope with a shared bottleneck link and
+ variable link capacity, events like handover, non-congestion-related
+ loss, and abrupt changes in bandwidth (both short term and long term)
+ due to handover, network load, and bad radio coverage. Even though
+ 3GPP has defined QoS bearers [QoS-3GPP] to ensure high-quality user
+ experience, it is still preferable for real-time applications to
+ behave in an adaptive manner.
+
+ Different mobile operators deploy their own cellular networks with
+ their own set of network functionalities and policies. Usually, a
+ mobile operator network includes a range of radio access technologies
+ such as 3G and 4G/LTE. Looking at the specifications of such radio
+ technologies, it is evident that only the more recent radio
+ technologies can support the high bandwidth requirements from real-
+ time interactive video applications. Future real-time interactive
+ applications will impose even greater demand on cellular network
+ performance, which makes 4G (and beyond) radio technologies more
+ suitable for such genre of application.
+
+ The key factors in defining test cases for cellular networks are:
+
+ * Shared and varying link capacity
+
+ * Mobility
+
+ * Handover
+
+ However, these factors are typically highly correlated in a cellular
+ network. Therefore, instead of devising separate test cases for
+ individual important events, we have divided the test cases into two
+ categories. It should be noted that the goal of the following test
+ cases is to evaluate the performance of candidate algorithms over the
+ radio interface of the cellular network. Hence, it is assumed that
+ the radio interface is the bottleneck link between the communicating
+ peers and that the core network does not introduce any extra
+ congestion along the path. Consequently, this document has left out
+ of scope the combination of multiple access technologies involving
+ both cellular and Wi-Fi users. In this latter case, the shared
+ bottleneck is likely at the wired backhaul link. These test cases
+ further assume a typical real-time telephony scenario where one real-
+ time session consists of one voice stream and one video stream.
+
+ Even though it is possible to carry out tests over operational
+ cellular networks (e.g., LTE/5G), and actually such tests are already
+ available today, these tests cannot in general be carried out in a
+ deterministic fashion to ensure repeatability. The main reason is
+ that these networks are controlled by cellular operators, and there
+ exists various amounts of competing traffic in the same cell(s). In
+ practice, it is only in underground mines that one can carry out near
+ deterministic testing. Even there, it is not guaranteed either as
+ workers in the mines may carry with them their personal mobile
+ phones. Furthermore, the underground mining setting may not reflect
+ typical usage patterns in an urban setting. We, therefore, recommend
+ that a cellular network simulator be used for the test cases defined
+ in this document, for example -- the LTE simulator in [NS-3].
+
+2.1. Varying Network Load
+
+ The goal of this test is to evaluate the performance of the candidate
+ congestion control algorithm under varying network load. The network
+ load variation is created by adding and removing network users,
+ a.k.a. User Equipment (UE), during the simulation. In this test
+ case, each user/UE in the media session is an endpoint following RTP-
+ based congestion control. User arrivals follow a Poisson
+ distribution proportional to the length of the call, to keep the
+ number of users per cell fairly constant during the evaluation
+ period. At the beginning of the simulation, there should be enough
+ time to warm up the network. This is to avoid running the evaluation
+ in an empty network where network nodes have empty buffers and low
+ interference at the beginning of the simulation. This network
+ initialization period should be excluded from the evaluation period.
+ Typically, the evaluation period starts 30 seconds after test
+ initialization.
+
+ This test case also includes user mobility and some competing
+ traffic. The latter includes both the same types of flows (with same
+ adaptation algorithms) and different types of flows (with different
+ services and congestion control schemes).
+
+2.1.1. Network Connection
+
+ Each mobile user is connected to a fixed user. The connection
+ between the mobile user and fixed user consists of a cellular radio
+ access, an Evolved Packet Core (EPC), and an Internet connection.
+ The mobile user is connected to the EPC using cellular radio access
+ technology, which is further connected to the Internet. At the other
+ end, the fixed user is connected to the Internet via a wired
+ connection with sufficiently high bandwidth, for instance, 10 Gbps,
+ so that the system bottleneck is on the cellular radio access
+ interface. The wired connection in this setup does not introduce any
+ network impairments to the test; it only adds 10 ms of one-way
+ propagation delay.
+
+ The path from the fixed user to the mobile users is defined as
+ "downlink", and the path from the mobile users to the fixed user is
+ defined as "uplink". We assume that only uplink or downlink is
+ congested for mobile users. Hence, we recommend that the uplink and
+ downlink simulations are run separately.
+
+ uplink
+ ++))) +-------------------------->
+ ++-+ ((o))
+ | | / \ +-------+ +------+ +---+
+ +--+ / \----+ +-----+ +----+ |
+ / \ +-------+ +------+ +---+
+ UE BS EPC Internet fixed
+ <--------------------------+
+ downlink
+
+ Figure 1: Simulation Topology
+
+2.1.2. Simulation Setup
+
+ The values enclosed within "[ ]" for the following simulation
+ attributes follow the same notion as in [RFC8867]. The desired
+ simulation setup is as follows:
+
+ Radio environment:
+
+ Deployment and propagation model: 3GPP case 1 (see
+ [HO-deploy-3GPP])
+
+ Antenna: Multiple-Input and Multiple-Output (MIMO), 2D or 3D
+ antenna pattern
+
+ Mobility: [3 km/h, 30 km/h]
+
+ Transmission bandwidth: 10 MHz
+
+ Number of cells: multi-cell deployment (3 cells per Base Station
+ (BS) * 7 BS) = 21 cells
+
+ Cell radius: 166.666 meters
+
+ Scheduler: Proportional fair with no priority
+
+ Bearer: Default bearer for all traffic
+
+ Active Queue Management (AQM) settings: AQM [on, off]
+
+ End-to-end Round Trip Time (RTT): [40 ms, 150 ms]
+
+ User arrival model: Poisson arrival model
+
+ User intensity:
+
+ Downlink user intensity: {0.7, 1.4, 2.1, 2.8, 3.5, 4.2, 4.9, 5.6,
+ 6.3, 7.0, 7.7, 8.4, 9,1, 9.8, 10.5}
+
+ Uplink user intensity: {0.7, 1.4, 2.1, 2.8, 3.5, 4.2, 4.9, 5.6,
+ 6.3, 7.0}
+
+ Simulation duration: 91 s
+
+ Evaluation period: 30 s - 60 s
+
+ Media traffic:
+
+ Media type: Video
+
+ Media direction: [uplink, downlink]
+
+ Number of media sources per user: One (1)
+
+ Media duration per user: 30 s
+
+ Media source: same as defined in Section 4.3 of [RFC8867]
+
+ Media type: Audio
+
+ Media direction: [uplink, downlink]
+
+ Number of media sources per user: One (1)
+
+ Media duration per user: 30 s
+
+ Media codec: Constant Bit Rate (CBR)
+
+ Media bitrate: 20 Kbps
+
+ Adaptation: off
+
+ Other traffic models:
+
+ Downlink simulation: Maximum of 4 Mbps/cell (web browsing or FTP
+ traffic following default TCP congestion control [RFC5681])
+
+ Uplink simulation: Maximum of 2 Mbps/cell (web browsing or FTP
+ traffic following default TCP congestion control [RFC5681])
+
+2.1.3. Expected Behavior
+
+ The investigated congestion control algorithms should result in
+ maximum possible network utilization and stability in terms of rate
+ variations, lowest possible end-to-end frame latency, network
+ latency, and Packet Loss Rate (PLR) at different cell load levels.
+
+2.2. Bad Radio Coverage
+
+ The goal of this test is to evaluate the performance of the candidate
+ congestion control algorithm when users visit part of the network
+ with bad radio coverage. The scenario is created by using a larger
+ cell radius than that in the previous test case. In this test case,
+ each user/UE in the media session is an endpoint following RTP-based
+ congestion control. User arrivals follow a Poisson distribution
+ proportional to the length of the call, to keep the number of users
+ per cell fairly constant during the evaluation period. At the
+ beginning of the simulation, there should be enough time to warm up
+ the network. This is to avoid running the evaluation in an empty
+ network where network nodes have empty buffers and low interference
+ at the beginning of the simulation. This network initialization
+ period should be excluded from the evaluation period. Typically, the
+ evaluation period starts 30 seconds after test initialization.
+
+ This test case also includes user mobility and some competing
+ traffic. The latter includes the same kind of flows (with same
+ adaptation algorithms).
+
+2.2.1. Network Connection
+
+ Same as defined in Section 2.1.1.
+
+2.2.2. Simulation Setup
+
+ The desired simulation setup is the same as the Varying Network Load
+ test case defined in Section 2.1 except for the following changes:
+
+ Radio environment: Same as defined in Section 2.1.2 except for the
+ following:
+
+ Deployment and propagation model: 3GPP case 3 (see
+ [HO-deploy-3GPP])
+
+ Cell radius: 577.3333 meters
+
+ Mobility: 3 km/h
+
+ User intensity: {0.7, 1.4, 2.1, 2.8, 3.5, 4.2, 4.9, 5.6, 6.3, 7.0}
+
+ Media traffic model: Same as defined in Section 2.1.2
+
+ Other traffic models:
+
+ Downlink simulation: Maximum of 2 Mbps/cell (web browsing or FTP
+ traffic following default TCP congestion control [RFC5681])
+
+ Uplink simulation: Maximum of 1 Mbps/cell (web browsing or FTP
+ traffic following default TCP congestion control [RFC5681])
+
+2.2.3. Expected Behavior
+
+ The investigated congestion control algorithms should result in
+ maximum possible network utilization and stability in terms of rate
+ variations, lowest possible end-to-end frame latency, network
+ latency, and Packet Loss Rate (PLR) at different cell load levels.
+
+2.3. Desired Evaluation Metrics for Cellular Test Cases
+
+ The evaluation criteria document [RFC8868] defines the metrics to be
+ used to evaluate candidate algorithms. Considering the nature and
+ distinction of cellular networks, we recommend that at least the
+ following metrics be used to evaluate the performance of the
+ candidate algorithms:
+
+ * Average cell throughput (for all cells), shows cell utilization.
+
+ * Application sending and receiving bitrate, goodput.
+
+ * Packet Loss Rate (PLR).
+
+ * End-to-end media frame delay. For video, this means the delay
+ from capture to display.
+
+ * Transport delay.
+
+ * Algorithm stability in terms of rate variation.
+
+3. Wi-Fi Networks Specific Test Cases
+
+ Given the prevalence of Internet access links over Wi-Fi, it is
+ important to evaluate candidate RTP-based congestion control
+ solutions over test cases that include Wi-Fi access links. Such
+ evaluations should highlight the inherently different characteristics
+ of Wi-Fi networks in contrast to their wired counterparts:
+
+ * The wireless radio channel is subject to interference from nearby
+ transmitters, multipath fading, and shadowing. These effects lead
+ to fluctuations in the link throughput and sometimes an error-
+ prone communication environment.
+
+ * Available network bandwidth is not only shared over the air
+ between concurrent users but also between uplink and downlink
+ traffic due to the half-duplex nature of the wireless transmission
+ medium.
+
+ * Packet transmissions over Wi-Fi are susceptible to contentions and
+ collisions over the air. Consequently, traffic load beyond a
+ certain utilization level over a Wi-Fi network can introduce
+ frequent collisions over the air and significant network overhead,
+ as well as packet drops due to buffer overflow at the
+ transmitters. This, in turn, leads to excessive delay,
+ retransmissions, packet losses, and lower effective bandwidth for
+ applications. Note further that the collision-induced delay and
+ loss patterns are qualitatively different from those caused by
+ congestion over a wired connection.
+
+ * The IEEE 802.11 standard (i.e., Wi-Fi) supports multi-rate
+ transmission capabilities by dynamically choosing the most
+ appropriate modulation and coding scheme (MCS) for the given
+ received signal strength. A different choice in the MCS Index
+ leads to different physical-layer (PHY-layer) link rates and
+ consequently different application-layer throughput.
+
+ * The presence of legacy devices (e.g., ones operating only in IEEE
+ 802.11b) at a much lower PHY-layer link rate can significantly
+ slow down the rest of a modern Wi-Fi network. As discussed in
+ [Heusse2003], the main reason for such anomaly is that it takes
+ much longer to transmit the same packet over a slower link than
+ over a faster link, thereby consuming a substantial portion of air
+ time.
+
+ * Handover from one Wi-Fi Access Point (AP) to another may lead to
+ excessive packet delays and losses during the process.
+
+ * IEEE 802.11e has introduced the Enhanced Distributed Channel
+ Access (EDCA) mechanism to allow different traffic categories to
+ contend for channel access using different random back-off
+ parameters. This mechanism is a mandatory requirement for the Wi-
+ Fi Multimedia (WMM) certification in Wi-Fi Alliance. It allows
+ for prioritization of real-time application traffic such as voice
+ and video over non-urgent data transmissions (e.g., file
+ transfer).
+
+ In summary, the presence of Wi-Fi access links in different network
+ topologies can exert different impacts on the network performance in
+ terms of application-layer effective throughput, packet loss rate,
+ and packet delivery delay. These, in turn, will influence the
+ behavior of end-to-end real-time multimedia congestion control.
+
+ Unless otherwise mentioned, the test cases in this section choose the
+ PHY- and MAC-layer parameters based on the IEEE 802.11n standard.
+ Statistics collected from enterprise Wi-Fi networks show that the two
+ dominant physical modes are 802.11n and 802.11ac, accounting for 41%
+ and 58% of connected devices, respectively. As Wi-Fi standards
+ evolve over time -- for instance, with the introduction of the
+ emerging Wi-Fi 6 (based on IEEE 802.11ax) products -- the PHY- and
+ MAC-layer test case specifications need to be updated accordingly to
+ reflect such changes.
+
+ Typically, a Wi-Fi access network connects to a wired infrastructure.
+ Either the wired or the Wi-Fi segment of the network can be the
+ bottleneck. The following sections describe basic test cases for
+ both scenarios separately. The same set of performance metrics as in
+ [RFC8867]) should be collected for each test case.
+
+ We recommend carrying out the test cases as defined in this document
+ using a simulator, such as [NS-2] or [NS-3]. When feasible, it is
+ encouraged to perform testbed-based evaluations using Wi-Fi access
+ points and endpoints running up-to-date IEEE 802.11 protocols, such
+ as 802.11ac and the emerging Wi-Fi 6, so as to verify the viability
+ of the candidate schemes.
+
+3.1. Bottleneck in Wired Network
+
+ The test scenarios below are intended to mimic the setup of video
+ conferencing over Wi-Fi connections from the home. Typically, the
+ Wi-Fi home network is not congested, and the bottleneck is present
+ over the wired home access link. Although it is expected that test
+ evaluation results from this section are similar to those in
+ [RFC8867], it is still worthwhile to run through these tests as
+ sanity checks.
+
+3.1.1. Network Topology
+
+ Figure 2 shows the network topology of Wi-Fi test cases. The test
+ contains multiple mobile nodes (MNs) connected to a common Wi-Fi AP
+ and their corresponding wired clients on fixed nodes (FNs). Each
+ connection carries either an RTP-based media flow or a TCP traffic
+ flow. Directions of the flows can be uplink (i.e., from mobile nodes
+ to fixed nodes), downlink (i.e., from fixed nodes to mobile nodes),
+ or bidirectional. The total number of uplink/downlink/bidirectional
+ flows for RTP-based media traffic and TCP traffic are denoted as N
+ and M, respectively.
+
+ Uplink
+ +----------------->+
+ +------+ +------+
+ | MN_1 |)))) /=====| FN_1 |
+ +------+ )) // +------+
+ . )) // .
+ . )) // .
+ . )) // .
+ +------+ +----+ +-----+ +------+
+ | MN_N | ))))))) | | | |========| FN_N |
+ +------+ | | | | +------+
+ | AP |=========| FN0 |
+ +----------+ | | | | +----------+
+ | MN_tcp_1 | )))) | | | |======| FN_tcp_1 |
+ +----------+ +----+ +-----+ +----------+
+ . )) \\ .
+ . )) \\ .
+ . )) \\ .
+ +----------+ )) \\ +----------+
+ | MN_tcp_M |))) \=====| FN_tcp_M |
+ +----------+ +----------+
+ +<-----------------+
+ Downlink
+
+ Figure 2: Network Topology for Wi-Fi Test Cases
+
+3.1.2. Test/Simulation Setup
+
+ Test duration: 120 s
+
+ Wi-Fi network characteristics:
+
+ Radio propagation model: Log-distance path loss propagation model
+ (see [NS3WiFi])
+
+ PHY- and MAC-layer configuration: IEEE 802.11n
+
+ MCS Index at 11: Raw data rate at 52 Mbps, 16-QAM (Quadrature
+ amplitude modulation) and 1/2 coding rate
+
+ Wired path characteristics:
+
+ Path capacity: 1 Mbps
+
+ One-way propagation delay: 50 ms
+
+ Maximum end-to-end jitter: 30 ms
+
+ Bottleneck queue type: Drop tail
+
+ Bottleneck queue size: 300 ms
+
+ Path loss ratio: 0%
+
+ Application characteristics:
+
+ Media traffic:
+
+ Media type: Video
+
+ Media direction: See Section 3.1.3
+
+ Number of media sources (N): See Section 3.1.3
+
+ Media timeline:
+
+ Start time: 0 s
+
+ End time: 119 s
+
+ Competing traffic:
+
+ Type of sources: Long-lived TCP or CBR over UDP
+
+ Traffic direction: See Section 3.1.3
+
+ Number of sources (M): See Section 3.1.3
+
+ Congestion control: Default TCP congestion control [RFC5681]
+ or CBR traffic over UDP
+
+ Traffic timeline: See Section 3.1.3
+
+3.1.3. Typical Test Scenarios
+
+ Single uplink RTP-based media flow: N=1 with uplink direction and
+ M=0.
+
+ One pair of bidirectional RTP-based media flows: N=2 (i.e., one
+ uplink flow and one downlink flow); M=0.
+
+ One pair of bidirectional RTP-based media flows: N=2; one uplink on-
+ off CBR flow over UDP: M=1 (uplink). The CBR flow has ON time at
+ t=0s-60s and OFF time at t=60s-119s.
+
+ One pair of bidirectional RTP-based media flows: N=2; one uplink
+ off-on CBR flow over UDP: M=1 (uplink). The CBR flow has OFF time
+ at t=0s-60s and ON time at t=60s-119s.
+
+ One RTP-based media flow competing against one long-lived TCP flow
+ in the uplink direction: N=1 (uplink) and M=1 (uplink). The TCP
+ flow has start time at t=0s and end time at t=119s.
+
+3.1.4. Expected Behavior
+
+ Single uplink RTP-based media flow: The candidate algorithm is
+ expected to detect the path capacity constraint, to converge to
+ the bottleneck link capacity, and to adapt the flow to avoid
+ unwanted oscillations when the sending bit rate is approaching the
+ bottleneck link capacity. No excessive oscillations in the media
+ rate should be present.
+
+ Bidirectional RTP-based media flows: The candidate algorithm is
+ expected to converge to the bottleneck capacity of the wired path
+ in both directions despite the presence of measurement noise over
+ the Wi-Fi connection. In the presence of background TCP or CBR
+ over UDP traffic, the rate of RTP-based media flows should adapt
+ promptly to the arrival and departure of background traffic flows.
+
+ One RTP-based media flow competing with long-lived TCP flow in the
+ uplink direction: The candidate algorithm is expected to avoid
+ congestion collapse and to stabilize at a fair share of the
+ bottleneck link capacity.
+
+3.2. Bottleneck in Wi-Fi Network
+
+ The test cases in this section assume that the wired segment along
+ the media path is well-provisioned, whereas the bottleneck exists
+ over the Wi-Fi access network. This is to mimic the application
+ scenarios typically encountered by users in an enterprise environment
+ or at a coffee house.
+
+3.2.1. Network Topology
+
+ Same as defined in Section 3.1.1.
+
+3.2.2. Test/Simulation Setup
+
+ Test duration: 120 s
+
+ Wi-Fi network characteristics:
+
+ Radio propagation model: Log-distance path loss propagation model
+ (see [NS3WiFi])
+
+ PHY- and MAC-layer configuration: IEEE 802.11n
+
+ MCS Index at 11: Raw data rate at 52 Mbps, 16-QAM (Quadrature
+ amplitude modulation) and 1/2 coding rate
+
+ Wired path characteristics:
+
+ Path capacity: 100 Mbps
+
+ One-Way propagation delay: 50 ms
+
+ Maximum end-to-end jitter: 30 ms
+
+ Bottleneck queue type: Drop tail
+
+ Bottleneck queue size: 300 ms
+
+ Path loss ratio: 0%
+
+ Application characteristics
+
+ Media traffic:
+
+ Media type: Video
+
+ Media direction: See Section 3.2.3
+
+ Number of media sources (N): See Section 3.2.3
+
+ Media timeline:
+
+ Start time: 0 s
+
+ End time: 119 s
+
+ Competing traffic:
+
+ Type of sources: long-lived TCP or CBR over UDP
+
+ Number of sources (M): See Section 3.2.3
+
+ Traffic direction: See Section 3.2.3
+
+ Congestion control: Default TCP congestion control [RFC5681]
+ or CBR traffic over UDP
+
+ Traffic timeline: See Section 3.2.3
+
+3.2.3. Typical Test Scenarios
+
+ This section describes a few test scenarios that are deemed as
+ important for understanding the behavior of a candidate RTP-based
+ congestion control scheme over a Wi-Fi network.
+
+ Multiple RTP-based media flows sharing the wireless downlink: N=16
+ (all downlink); M=0. This test case is for studying the impact of
+ contention on the multiple concurrent media flows. For an 802.11n
+ network, given the MCS Index of 11 and the corresponding link rate
+ of 52 Mbps, the total application-layer throughput (assuming
+ reasonable distance, low interference, and infrequent contentions
+ caused by competing streams) is around 20 Mbps. A total of N=16
+ RTP-based media flows (with a maximum rate of 1.5 Mbps each) are
+ expected to saturate the wireless interface in this experiment.
+ Evaluation of a given candidate scheme should focus on whether the
+ downlink media flows can stabilize at a fair share of the total
+ application-layer throughput.
+
+ Multiple RTP-based media flows sharing the wireless uplink: N=16
+ (all uplink); M=0. When multiple clients attempt to transmit
+ media packets uplink over the Wi-Fi network, they introduce more
+ frequent contentions and potential collisions. Per-flow
+ throughput is expected to be lower than that in the previous
+ downlink-only scenario. Evaluation of a given candidate scheme
+ should focus on whether the uplink flows can stabilize at a fair
+ share of the total application-layer throughput.
+
+ Multiple bidirectional RTP-based media flows: N=16 (8 uplink and 8
+ downlink); M=0. The goal of this test is to evaluate the
+ performance of the candidate scheme in terms of bandwidth fairness
+ between uplink and downlink flows.
+
+ Multiple bidirectional RTP-based media flows with on-off CBR
+ traffic over UDP: N=16 (8 uplink and 8 downlink); M=5 (uplink). The
+ goal of this test is to evaluate the adaptation behavior of the
+ candidate scheme when its available bandwidth changes due to the
+ departure of background traffic. The background traffic consists
+ of several (e.g., M=5) CBR flows transported over UDP. These
+ background flows are ON at time t=0-60s and OFF at time t=61-120s.
+
+ Multiple bidirectional RTP-based media flows with off-on CBR
+ traffic over UDP: N=16 (8 uplink and 8 downlink); M=5 (uplink). The
+ goal of this test is to evaluate the adaptation behavior of the
+ candidate scheme when its available bandwidth changes due to the
+ arrival of background traffic. The background traffic consists of
+ several (e.g., M=5) parallel CBR flows transported over UDP.
+ These background flows are OFF at time t=0-60s and ON at times
+ t=61-120s.
+
+ Multiple bidirectional RTP-based media flows in the presence of
+ background TCP traffic: N=16 (8 uplink and 8 downlink); M=5
+ (uplink). The goal of this test is to evaluate how RTP-based
+ media flows compete against TCP over a congested Wi-Fi network for
+ a given candidate scheme. TCP flows have start time at t=40s and
+ end time at t=80s.
+
+ Varying number of RTP-based media flows: A series of tests can be
+ carried out for the above test cases with different values of N,
+ e.g., N=[4, 8, 12, 16, 20]. The goal of this test is to evaluate
+ how a candidate scheme responds to varying traffic load/demand
+ over a congested Wi-Fi network. The start times of the media
+ flows are randomly distributed within a window of t=0-10s; their
+ end times are randomly distributed within a window of t=110-120s.
+
+3.2.4. Expected Behavior
+
+ Multiple downlink RTP-based media flows: Each media flow is expected
+ to get its fair share of the total bottleneck link bandwidth.
+ Overall bandwidth usage should not be significantly lower than
+ that experienced by the same number of concurrent downlink TCP
+ flows. In other words, the behavior of multiple concurrent TCP
+ flows will be used as a performance benchmark for this test
+ scenario. The end-to-end delay and packet loss ratio experienced
+ by each flow should be within an acceptable range for real-time
+ multimedia applications.
+
+ Multiple uplink RTP-based media flows: Overall bandwidth usage by
+ all media flows should not be significantly lower than that
+ experienced by the same number of concurrent uplink TCP flows. In
+ other words, the behavior of multiple concurrent TCP flows will be
+ used as a performance benchmark for this test scenario.
+
+ Multiple bidirectional RTP-based media flows with dynamic
+ background traffic carrying CBR flows over UDP: The media flows are
+ expected to adapt in a timely fashion to the changes in available
+ bandwidth introduced by the arrival/departure of background
+ traffic.
+
+ Multiple bidirectional RTP-based media flows with dynamic
+ background traffic over TCP: During the presence of TCP background
+ flows, the overall bandwidth usage by all media flows should not
+ be significantly lower than those achieved by the same number of
+ bidirectional TCP flows. In other words, the behavior of multiple
+ concurrent TCP flows will be used as a performance benchmark for
+ this test scenario. All downlink media flows are expected to
+ obtain similar bandwidth as each other. The throughput of each
+ media flow is expected to decrease upon the arrival of TCP
+ background traffic and, conversely, increase upon their departure.
+ Both reactions should occur in a timely fashion, for example,
+ within 10s of seconds.
+
+ Varying number of bidirectional RTP-based media flows: The test
+ results for varying values of N -- while keeping all other
+ parameters constant -- is expected to show steady and stable per-
+ flow throughput for each value of N. The average throughput of
+ all media flows is expected to stay constant around the maximum
+ rate when N is small, then gradually decrease with increasing
+ value of N till it reaches the minimum allowed rate, beyond which
+ the offered load to the Wi-Fi network exceeds its capacity (i.e.,
+ with a very large value of N).
+
+3.3. Other Potential Test Cases
+
+3.3.1. EDCA/WMM usage
+
+ The EDCA/WMM mechanism defines prioritized QoS for four traffic
+ classes (or Access Categories). RTP-based real-time media flows
+ should achieve better performance in terms of lower delay and fewer
+ packet losses with EDCA/WMM enabled when competing against non-
+ interactive background traffic such as file transfers. When most of
+ the traffic over Wi-Fi is dominated by media, however, turning on WMM
+ may degrade performance since all media flows now attempt to access
+ the wireless transmission medium more aggressively, thereby causing
+ more frequent collisions and collision-induced losses. This is a
+ topic worthy of further investigation.
+
+3.3.2. Effect of Heterogeneous Link Rates
+
+ As discussed in [Heusse2003], the presence of clients operating over
+ slow PHY-layer link rates (e.g., a legacy 802.11b device) connected
+ to a modern network may adversely impact the overall performance of
+ the network. Additional test cases can be devised to evaluate the
+ effect of clients with heterogeneous link rates on the performance of
+ the candidate congestion control algorithm. Such test cases, for
+ instance, can specify that the PHY-layer link rates for all clients
+ span over a wide range (e.g., 2 Mbps to 54 Mbps) for investigating
+ its effect on the congestion control behavior of the real-time
+ interactive applications.
+
+4. IANA Considerations
+
+ This document has no IANA actions.
+
+5. Security Considerations
+
+ The security considerations in [RFC8868] and the relevant congestion
+ control algorithms apply. The principles for congestion control are
+ described in [RFC2914], and in particular, any new method must
+ implement safeguards to avoid congestion collapse of the Internet.
+
+ Given the difficulty of deterministic wireless testing, it is
+ recommended and expected that the tests described in this document
+ would be done via simulations. However, in the case where these test
+ cases are carried out in a testbed setting, the evaluation should
+ take place in a controlled lab environment. In the testbed, the
+ applications, simulators, and network nodes ought to be well-behaved
+ and should not impact the desired results. It is important to take
+ appropriate caution to avoid leaking nonresponsive traffic with
+ unproven congestion avoidance behavior onto the open Internet.
+
+6. References
+
+6.1. Normative References
+
+ [HO-deploy-3GPP]
+ 3GPP, "Physical layer aspects for evolved Universal
+ Terrestrial Radio Access (UTRA)", TS 25.814, October 2006,
+ <http://www.3gpp.org/ftp/specs/
+ archive/25_series/25.814/25814-710.zip>.
+
+ [IEEE802.11]
+ IEEE, "Standard for Information technology--
+ Telecommunications and information exchange between
+ systems Local and metropolitan area networks--Specific
+ requirements Part 11: Wireless LAN Medium Access Control
+ (MAC) and Physical Layer (PHY) Specifications",
+ IEEE 802.11-2012,
+ <https://ieeexplore.ieee.org/document/7786995>.
+
+ [NS3WiFi] "ns3::YansWifiChannel Class Reference",
+ <https://www.nsnam.org/doxygen/
+ classns3_1_1_yans_wifi_channel.html>.
+
+ [RFC5681] Allman, M., Paxson, V., and E. Blanton, "TCP Congestion
+ Control", RFC 5681, DOI 10.17487/RFC5681, September 2009,
+ <https://www.rfc-editor.org/info/rfc5681>.
+
+ [RFC8867] Sarker, Z., Singh, V., Zhu, X., and M. Ramalho, "Test
+ Cases for Evaluating Congestion Control for Interactive
+ Real-Time Media", RFC 8867, DOI 10.17487/RFC8867, January
+ 2021, <https://www.rfc-editor.org/info/rfc8867>.
+
+ [RFC8868] Singh, V., Ott, J., and S. Holmer, "Evaluating Congestion
+ Control for Interactive Real-Time Media", RFC 8868,
+ DOI 10.17487/RFC8868, January 2021,
+ <https://www.rfc-editor.org/info/rfc8868>.
+
+6.2. Informative References
+
+ [Heusse2003]
+ Heusse, M., Rousseau, F., Berger-Sabbatel, G., and A.
+ Duda, "Performance anomaly of 802.11b", IEEE INFOCOM 2003,
+ Twenty-second Annual Joint Conference of the IEEE Computer
+ and Communications Societies,
+ DOI 10.1109/INFCOM.2003.1208921, March 2003,
+ <https://ieeexplore.ieee.org/document/1208921>.
+
+ [HO-def-3GPP]
+ 3GPP, "Vocabulary for 3GPP Specifications", 3GPP
+ TS 21.905, December 2009, <http://www.3gpp.org/ftp/specs/
+ archive/21_series/21.905/21905-940.zip>.
+
+ [HO-LTE-3GPP]
+ 3GPP, "Evolved Universal Terrestrial Radio Access
+ (E-UTRA); Radio Resource Control (RRC); Protocol
+ specification", 3GPP TS 36.331, December 2011,
+ <http://www.3gpp.org/ftp/specs/
+ archive/36_series/36.331/36331-990.zip>.
+
+ [HO-UMTS-3GPP]
+ 3GPP, "Radio Resource Control (RRC); Protocol
+ specification", 3GPP TS 25.331, December 2011,
+ <http://www.3gpp.org/ftp/specs/
+ archive/25_series/25.331/25331-990.zip>.
+
+ [NS-2] "ns-2", December 2014,
+ <http://nsnam.sourceforge.net/wiki/index.php/Main_Page>.
+
+ [NS-3] "ns-3 Network Simulator", <https://www.nsnam.org/>.
+
+ [QoS-3GPP] 3GPP, "Policy and charging control architecture", 3GPP
+ TS 23.203, June 2011, <http://www.3gpp.org/ftp/specs/
+ archive/23_series/23.203/23203-990.zip>.
+
+ [RFC2914] Floyd, S., "Congestion Control Principles", BCP 41,
+ RFC 2914, DOI 10.17487/RFC2914, September 2000,
+ <https://www.rfc-editor.org/info/rfc2914>.
+
+Contributors
+
+ The following individuals contributed to the design, implementation,
+ and verification of the proposed test cases during earlier stages of
+ this work. They have helped to validate and substantially improve
+ this specification.
+
+ Ingemar Johansson <ingemar.s.johansson@ericsson.com> of Ericsson AB
+ contributed to the description and validation of cellular test cases
+ during the earlier stage of this document.
+
+ Wei-Tian Tan <dtan2@cisco.com> of Cisco Systems designed and set up a
+ Wi-Fi testbed for evaluating parallel video conferencing streams,
+ based upon which proposed Wi-Fi test cases are described. He also
+ recommended additional test cases to consider, such as the impact of
+ EDCA/WMM usage.
+
+ Michael A. Ramalho <mar42@cornell.edu> of AcousticComms Consulting
+ (previously at Cisco Systems) applied lessons from Cisco's internal
+ experimentation to the draft versions of the document. He also
+ worked on validating the proposed test cases in a virtual-machine-
+ based lab setting.
+
+Acknowledgments
+
+ The authors would like to thank Tomas Frankkila, Magnus Westerlund,
+ Kristofer Sandlund, Sergio Mena de la Cruz, and Mirja Kühlewind for
+ their valuable inputs and review comments regarding this document.
+
+Authors' Addresses
+
+ Zaheduzzaman Sarker
+ Ericsson AB
+ Torshamnsgatan 23
+ SE-164 83 Stockholm
+ Sweden
+
+ Phone: +46 10 717 37 43
+ Email: zaheduzzaman.sarker@ericsson.com
+
+
+ Xiaoqing Zhu
+ Cisco Systems
+ Building 4
+ 12515 Research Blvd
+ Austin, TX 78759
+ United States of America
+
+ Email: xiaoqzhu@cisco.com
+
+
+ Jiantao Fu
+ Cisco Systems
+ 771 Alder Drive
+ Milpitas, CA 95035
+ United States of America
+
+ Email: jianfu@cisco.com