summaryrefslogtreecommitdiff
path: root/doc/rfc/rfc5562.txt
diff options
context:
space:
mode:
Diffstat (limited to 'doc/rfc/rfc5562.txt')
-rw-r--r--doc/rfc/rfc5562.txt1851
1 files changed, 1851 insertions, 0 deletions
diff --git a/doc/rfc/rfc5562.txt b/doc/rfc/rfc5562.txt
new file mode 100644
index 0000000..ebd6cc6
--- /dev/null
+++ b/doc/rfc/rfc5562.txt
@@ -0,0 +1,1851 @@
+
+
+
+
+
+
+Network Working Group A. Kuzmanovic
+Request for Comments: 5562 A. Mondal
+Category: Experimental Northwestern University
+ S. Floyd
+ ICSI
+ K. Ramakrishnan
+ AT&T Labs Research
+ June 2009
+
+
+ Adding Explicit Congestion Notification (ECN) Capability
+ to TCP's SYN/ACK Packets
+
+Status of This Memo
+
+ This memo defines an Experimental Protocol for the Internet
+ community. It does not specify an Internet standard of any kind.
+ Discussion and suggestions for improvement are requested.
+ Distribution of this memo is unlimited.
+
+Copyright Notice
+
+ Copyright (c) 2009 IETF Trust and the persons identified as the
+ document authors. All rights reserved.
+
+ This document is subject to BCP 78 and the IETF Trust's Legal
+ Provisions Relating to IETF Documents in effect on the date of
+ publication of this document (http://trustee.ietf.org/license-info).
+ Please review these documents carefully, as they describe your rights
+ and restrictions with respect to this document.
+
+ This document may contain material from IETF Documents or IETF
+ Contributions published or made publicly available before November
+ 10, 2008. The person(s) controlling the copyright in some of this
+ material may not have granted the IETF Trust the right to allow
+ modifications of such material outside the IETF Standards Process.
+ Without obtaining an adequate license from the person(s) controlling
+ the copyright in such materials, this document may not be modified
+ outside the IETF Standards Process, and derivative works of it may
+ not be created outside the IETF Standards Process, except to format
+ it for publication as an RFC or to translate it into languages other
+ than English.
+
+
+
+
+
+
+
+
+
+Kuzmanovic, et al. Experimental [Page 1]
+
+RFC 5562 ECN and SYN/ACK Packets June 2009
+
+
+Abstract
+
+ The proposal in this document is Experimental. While it may be
+ deployed in the current Internet, it does not represent a consensus
+ that this is the best possible mechanism for the use of Explicit
+ Congestion Notification (ECN) in TCP SYN/ACK packets.
+
+ This document describes an optional, experimental modification to RFC
+ 3168 to allow TCP SYN/ACK packets to be ECN-Capable. For TCP, RFC
+ 3168 specifies setting an ECN-Capable codepoint on data packets, but
+ not on SYN and SYN/ACK packets. However, because of the high cost to
+ the TCP transfer of having a SYN/ACK packet dropped, with the
+ resulting retransmission timeout, this document describes the use of
+ ECN for the SYN/ACK packet itself, when sent in response to a SYN
+ packet with the two ECN flags set in the TCP header, indicating a
+ willingness to use ECN. Setting the initial TCP SYN/ACK packet as
+ ECN-Capable can be of great benefit to the TCP connection, avoiding
+ the severe penalty of a retransmission timeout for a connection that
+ has not yet started placing a load on the network. The TCP responder
+ (the sender of the SYN/ACK packet) must reply to a report of an ECN-
+ marked SYN/ACK packet by resending a SYN/ACK packet that is not ECN-
+ Capable. If the resent SYN/ACK packet is acknowledged, then the TCP
+ responder reduces its initial congestion window from two, three, or
+ four segments to one segment, thereby reducing the subsequent load
+ from that connection on the network. If instead the SYN/ACK packet
+ is dropped, or for some other reason the TCP responder does not
+ receive an acknowledgement in the specified time, the TCP responder
+ follows TCP standards for a dropped SYN/ACK packet (setting the
+ retransmission timer).
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+Kuzmanovic, et al. Experimental [Page 2]
+
+RFC 5562 ECN and SYN/ACK Packets June 2009
+
+
+Table of Contents
+
+ 1. Introduction ....................................................3
+ 2. Conventions and Terminology .....................................5
+ 3. Specification ...................................................6
+ 3.1. SYN/ACK Packets Dropped in the Network ....................7
+ 3.2. SYN/ACK Packets ECN-Marked in the Network .................8
+ 3.3. Management Interface .....................................10
+ 4. Discussion .....................................................11
+ 4.1. Flooding Attacks .........................................11
+ 4.2. The TCP SYN Packet .......................................11
+ 4.3. SYN/ACK Packets and Packet Size ..........................12
+ 4.4. Response to ECN-Marking of SYN/ACK Packets ...............12
+ 5. Related Work ...................................................14
+ 6. Performance Evaluation .........................................15
+ 6.1. The Costs and Benefits of Adding ECN-Capability ..........15
+ 6.2. An Evaluation of Different Responses to ECN-Marked
+ SYN/ACK Packets ..........................................16
+ 6.3. Experiments ..............................................17
+ 7. Security Considerations ........................................18
+ 7.1. "Bad" Routers or Middleboxes .............................18
+ 7.2. Congestion Collapse ......................................18
+ 8. Conclusions ....................................................19
+ 9. Acknowledgements ...............................................19
+ Appendix A. Report on Simulations .................................20
+ A.1. Simulations with RED in Packet Mode .......................20
+ A.2. Simulations with RED in Byte Mode .........................25
+ Appendix B. Issues of Incremental Deployment ......................28
+ Normative References ..............................................30
+ Informative References ............................................30
+
+1. Introduction
+
+ TCP's congestion control mechanism has primarily used packet loss as
+ the congestion indication, with packets dropped when buffers
+ overflow. With such tail-drop mechanisms, the packet delay can be
+ high, as the queue at bottleneck routers can be fairly large.
+ Dropping packets only when the queue overflows, and having TCP react
+ only to such losses, results in:
+
+ 1) significantly higher packet delay;
+
+ 2) unnecessarily many packet losses; and
+
+ 3) unfairness due to synchronization effects.
+
+
+
+
+
+
+Kuzmanovic, et al. Experimental [Page 3]
+
+RFC 5562 ECN and SYN/ACK Packets June 2009
+
+
+ The adoption of Active Queue Management (AQM) mechanisms allows
+ better control of bottleneck queues [RFC2309]. This use of AQM has
+ the following potential benefits:
+
+ 1) better control of the queue, with reduced queuing delay;
+
+ 2) fewer packet drops; and
+
+ 3) better fairness because of fewer synchronization effects.
+
+ With the adoption of ECN, performance may be further improved. When
+ the router detects congestion before buffer overflow, the router can
+ provide a congestion indication either by dropping a packet or by
+ setting the Congestion Experienced (CE) codepoint in the Explicit
+ Congestion Notification (ECN) field in the IP header [RFC3168]. The
+ IETF has standardized the use of the Congestion Experienced (CE)
+ codepoint in the IP header for routers to indicate congestion. For
+ incremental deployment and backwards compatibility, the RFC on ECN
+ [RFC3168] specifies that routers may mark ECN-Capable packets that
+ would otherwise have been dropped, using the Congestion Experienced
+ codepoint in the ECN field. The use of ECN allows TCP to react to
+ congestion while avoiding unnecessary retransmission timeouts. Thus,
+ using ECN has several benefits:
+
+ 1) For short transfers, a TCP connection's congestion window may be
+ small. For example, if the current window contains only one
+ packet, and that packet is dropped, TCP will have to wait for a
+ retransmission timeout to recover, reducing its overall
+ throughput. Similarly, if the current window contains only a few
+ packets and one of those packets is dropped, there might not be
+ enough duplicate acknowledgements for a fast retransmission, and
+ the sender of the data packet might have to wait for a delay of
+ several round-trip times (RTT) using Limited Transmit [RFC3042].
+ With the use of ECN, short flows are less likely to have packets
+ dropped, sometimes avoiding unnecessary delays or costly
+ retransmission timeouts.
+
+ 2) While longer flows may not see substantially improved throughput
+ with the use of ECN, they may experience lower loss. This may
+ benefit TCP applications that are latency- and loss-sensitive,
+ because of the avoidance of retransmissions.
+
+ RFC 3168 [RFC3168] specifies setting the ECN-Capable codepoint on TCP
+ data packets, but not on TCP SYN and SYN/ACK packets. RFC 3168
+ [RFC3168] specifies the negotiation of the use of ECN between the two
+ TCP endpoints in the TCP SYN and SYN-ACK exchange, using flags in the
+ TCP header. Erring on the side of being conservative, RFC 3168
+ [RFC3168] does not specify the use of ECN for the first SYN/ACK
+
+
+
+Kuzmanovic, et al. Experimental [Page 4]
+
+RFC 5562 ECN and SYN/ACK Packets June 2009
+
+
+ packet itself. However, because of the high cost to the TCP transfer
+ of having a SYN/ACK packet dropped, with the resulting retransmission
+ timeout, this document specifies the use of ECN for the SYN/ACK
+ packet itself. This can be of great benefit to the TCP connection,
+ avoiding the severe penalty of a retransmission timeout for a
+ connection that has not yet started placing a load on the network.
+ The sender of the SYN/ACK packet must respond to a report of an ECN-
+ marked SYN/ACK packet (a SYN/ACK packet with the CE codepoint set in
+ the ECN field in the IP header) by sending a non-ECN-Capable SYN/ACK
+ packet, and by reducing its initial congestion window from two,
+ three, or four segments to one segment, reducing the subsequent load
+ from that connection on the network.
+
+ The use of ECN for SYN/ACK packets has the following potential
+ benefits:
+
+ 1) Avoidance of a retransmission timeout;
+
+ 2) Improvement in the throughput of short connections.
+
+ This document specifies a modification to RFC 3168 [RFC3168] to allow
+ TCP SYN/ACK packets to be ECN-Capable. Section 3 contains the
+ specification of the change, while Section 4 discusses some of the
+ issues, and Section 5 discusses related work. Section 6 contains an
+ evaluation of the specified change.
+
+2. Conventions and Terminology
+
+ The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
+ "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
+ document are to be interpreted as described in [RFC2119].
+
+ We use the following terminology from RFC 3168 [RFC3168]:
+
+ The ECN field in the IP header:
+
+ o CE: the Congestion Experienced codepoint; and
+
+ o ECT: either one of the two ECN-Capable Transport codepoints.
+
+ The ECN flags in the TCP header:
+
+ o CWR: the Congestion Window Reduced flag; and
+
+ o ECE: the ECN-Echo flag.
+
+
+
+
+
+
+Kuzmanovic, et al. Experimental [Page 5]
+
+RFC 5562 ECN and SYN/ACK Packets June 2009
+
+
+ ECN-setup packets:
+
+ o ECN-setup SYN packet: a SYN packet with the ECE and CWR flags;
+
+ o ECN-setup SYN-ACK packet: a SYN-ACK packet with ECE but not CWR.
+
+ In this document, we use the terms "initiator" and "responder" to
+ refer to the sender of the SYN packet and of the SYN-ACK packet,
+ respectively.
+
+3. Specification
+
+ This section specifies the modification to RFC 3168 [RFC3168] to
+ allow TCP SYN/ACK packets to be ECN-Capable.
+
+ Section 6.1.1 of RFC 3168 [RFC3168] states that "A host MUST NOT set
+ ECT on SYN or SYN-ACK packets". In this section, we specify that a
+ TCP node may respond to an initial ECN-setup SYN packet by setting
+ ECT in the responding ECN-setup SYN/ACK packet, indicating to routers
+ that the SYN/ACK packet is ECN-Capable. This allows a congested
+ router along the path to mark the packet instead of dropping the
+ packet as an indication of congestion.
+
+ Assume that TCP node A transmits to TCP node B an ECN-setup SYN
+ packet, indicating willingness to use ECN for this connection. As
+ specified by RFC 3168 [RFC3168], if TCP node B is willing to use ECN,
+ node B responds with an ECN-setup SYN-ACK packet.
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+Kuzmanovic, et al. Experimental [Page 6]
+
+RFC 5562 ECN and SYN/ACK Packets June 2009
+
+
+3.1. SYN/ACK Packets Dropped in the Network
+
+ Figure 1 shows an interchange with the SYN/ACK packet dropped by a
+ congested router. Node B waits for a retransmission timeout, and
+ then retransmits the SYN/ACK packet.
+
+ ---------------------------------------------------------------
+ TCP Node A Router TCP Node B
+ (initiator) (responder)
+ ---------- ------ ----------
+
+ ECN-setup SYN packet --->
+ ECN-setup SYN packet --->
+
+ <--- ECN-setup SYN/ACK, possibly ECT
+ 3-second timer set
+ SYN/ACK dropped .
+ .
+ .
+ 3-second timer expires
+ <--- ECN-setup SYN/ACK, not ECT
+ <--- ECN-setup SYN/ACK
+ Data/ACK --->
+ Data/ACK --->
+ <--- Data (one to four segments)
+ ---------------------------------------------------------------
+
+ Figure 1: SYN exchange with the SYN/ACK packet dropped
+
+ If the SYN/ACK packet is dropped in the network, the responder (node
+ B) responds by waiting three seconds for the retransmission timer to
+ expire [RFC2988]. If a SYN/ACK packet with the ECT codepoint is
+ dropped, the responder should resend the SYN/ACK packet without the
+ ECN-Capable codepoint. (Although we are not aware of any middleboxes
+ that drop SYN/ACK packets that contain an ECN-Capable codepoint in
+ the IP header, we have learned to design our protocols defensively in
+ this regard [RFC3360].)
+
+ We note that if syn-cookies were used by the responder (node B) in
+ the exchange in Figure 1, the responder wouldn't set a timer upon
+ transmission of the SYN/ACK packet [SYN-COOK] [RFC4987]. In this
+ case, if the SYN/ACK packet was lost, the initiator (node A) would
+ have to timeout and retransmit the SYN packet in order to trigger
+ another SYN-ACK.
+
+
+
+
+
+
+
+Kuzmanovic, et al. Experimental [Page 7]
+
+RFC 5562 ECN and SYN/ACK Packets June 2009
+
+
+3.2. SYN/ACK Packets ECN-Marked in the Network
+
+ Figure 2 shows an interchange with the SYN/ACK packet sent as ECN-
+ Capable, and ECN-marked instead of dropped at the congested router.
+ This document specifies ECN+/TryOnce, which differs from the original
+ proposal for ECN+ in [ECN+]; with ECN+/TryOnce, if the TCP responder
+ is informed that the SYN/ACK was ECN-marked, the TCP responder
+ immediately sends a SYN/ACK packet that is not ECN-Capable. The TCP
+ responder is only allowed to send data packets after the TCP
+ initiator reports the receipt of a SYN/ACK packet that is not ECN-
+ marked.
+
+ ---------------------------------------------------------------
+ TCP Node A Router TCP Node B
+ (initiator) (responder)
+ ---------- ------ ----------
+
+ ECN-setup SYN packet --->
+ ECN-setup SYN packet --->
+
+ <--- ECN-setup SYN/ACK, ECT
+ 3-second timer set
+ <--- Sets CE on SYN/ACK
+ <--- ECN-setup SYN/ACK, CE
+
+ ACK, ECN-Echo --->
+ ACK, ECN-Echo --->
+ Window reduced to one segment.
+ <--- ECN-setup SYN/ACK, not ECT
+ <--- ECN-setup SYN/ACK
+
+ Data/ACK, ECT --->
+ Data/ACK, ECT --->
+ <--- Data, ECT (one segment only)
+ ---------------------------------------------------------------
+
+ Figure 2: SYN exchange with the SYN/ACK packet marked -
+ ECN+/TryOnce
+
+ If the initiator (node A) receives a SYN/ACK packet that has been
+ ECN-marked by the congested router, with the CE codepoint set, the
+ initiator restarts the retransmission timer. The initiator responds
+ to the ECN-marked SYN/ACK packet by setting the ECN-Echo flag in the
+ TCP header of the responding ACK packet. The initiator uses the
+ standard rules in setting the cumulative acknowledgement field in the
+ responding ACK packet.
+
+
+
+
+
+Kuzmanovic, et al. Experimental [Page 8]
+
+RFC 5562 ECN and SYN/ACK Packets June 2009
+
+
+ The initiator does not advance from the "SYN-Sent" to the
+ "Established" state until it receives a SYN/ACK packet that is not
+ ECN-marked.
+
+ When the responder (node B) receives the ECN-Echo packet reporting
+ the Congestion Experienced indication in the SYN/ACK packet, the
+ responder sets the initial congestion window to one segment, instead
+ of two segments as allowed by [RFC2581], or three or four segments
+ allowed by [RFC3390]. As illustrated in Figure 2, if the responder
+ (node B) receives an ECN-Echo packet informing it of a Congestion
+ Experienced indication on its SYN/ACK packet, the responder sends a
+ SYN/ACK packet that is not ECN-Capable, in addition to setting the
+ initial window to one segment. The responder does not advance the
+ send sequence number. The responder also sets the retransmission
+ timer. The responder follows RFC 2988 [RFC2988] in setting the RTO
+ (retransmission timeout).
+
+ The TCP hosts follow the standard specification for the response to
+ duplicate SYN/ACK packets (e.g., Section 3.4 of RFC 793 [RFC793]).
+
+ We note that the mechanism in this document differs from RFC 3168
+ [RFC3168], which specifies that "the sending TCP MUST restart the
+ retransmission timer on receiving the ECN-Echo packet when the
+ congestion window is one". RFC 3168 [RFC3168] does not allow SYN/ACK
+ packets to be ECN-Capable. RFC 3168 [RFC3168] specifies that in
+ response to an ECN-Echo packet, the TCP responder also sets the CWR
+ flag in the TCP header of the next data packet sent, to acknowledge
+ its receipt of and reaction to the ECN-Echo flag. In contrast, in
+ response to an ECN-Echo packet acknowledging the receipt of an ECN-
+ Capable SYN/ACK packet, the TCP responder doesn't set the CWR flag,
+ but simply sends a SYN/ACK packet that is not ECN-Capable. On
+ receiving the non-ECN-Capable SYN/ACK packet, the TCP initiator
+ clears the ECN-Echo flag on replying packets.
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+Kuzmanovic, et al. Experimental [Page 9]
+
+RFC 5562 ECN and SYN/ACK Packets June 2009
+
+
+ ---------------------------------------------------------------
+ TCP Node A Router TCP Node B
+ (initiator) (responder)
+ ---------- ------ ----------
+
+ ECN-setup SYN packet --->
+ ECN-setup SYN packet --->
+
+ <--- ECN-setup SYN/ACK, ECT
+ <--- Sets CE on SYN/ACK
+ <--- ECN-setup SYN/ACK, CE
+
+ ACK, ECN-Echo --->
+ ACK, ECN-Echo --->
+ Window reduced to one segment.
+
+ <--- ECN-setup SYN/ACK, not ECT
+ 3-second timer set
+ SYN/ACK dropped .
+ .
+ .
+ 3-second timer expires
+ <--- ECN-setup SYN/ACK, not ECT
+ <--- ECN-setup SYN/ACK, not ECT
+ Data/ACK, ECT --->
+ Data/ACK, ECT --->
+ <--- Data, ECT (one segment only)
+ ---------------------------------------------------------------
+
+ Figure 3: SYN exchange with the first SYN/ACK packet marked
+ and the second SYN/ACK packet dropped - ECN+/TryOnce
+
+ In contrast to Figure 2, Figure 3 shows an interchange where the
+ first SYN/ACK packet is ECN-marked and the second SYN/ACK packet is
+ dropped in the network. As in Figure 2, the TCP responder sets a
+ timer when the second SYN/ACK packet is sent. Figure 3 shows that if
+ the timer expires before the TCP responder receives an
+ acknowledgement for the other end, the TCP responder resends the
+ SYN/ACK packet, following the TCP standards.
+
+3.3. Management Interface
+
+ The TCP implementation using ECN-Capable SYN/ACK packets should
+ include a management interface to allow the use of ECN to be turned
+ off for SYN/ACK packets. This is to deal with possible backwards
+ compatibility problems such as those discussed in Appendix B.
+
+
+
+
+
+Kuzmanovic, et al. Experimental [Page 10]
+
+RFC 5562 ECN and SYN/ACK Packets June 2009
+
+
+4. Discussion
+
+ The rationale for the specification in this document is the
+ following. When node B receives a TCP SYN packet with ECN-Echo bit
+ set in the TCP header, this indicates that node A is ECN-Capable. If
+ node B is also ECN-Capable, there are no obstacles to immediately
+ setting one of the ECN-Capable codepoints in the IP header in the
+ responding TCP SYN/ACK packet.
+
+ There can be a great benefit in setting an ECN-Capable codepoint in
+ SYN/ACK packets, as is discussed further in [ECN+], and reported
+ briefly in Section 5 below. Congestion is most likely to occur in
+ the server-to-client direction. As a result, setting an ECN-Capable
+ codepoint in SYN/ACK packets can reduce the occurrence of three-
+ second retransmission timeouts resulting from the drop of SYN/ACK
+ packets.
+
+4.1. Flooding Attacks
+
+ Setting an ECN-Capable codepoint in the responding TCP SYN/ACK
+ packets does not raise any new or additional security
+ vulnerabilities. For example, provoking servers or hosts to send
+ SYN/ACK packets to a third party in order to perform a "SYN/ACK
+ flood" attack would be highly inefficient. Third parties would
+ immediately drop such packets, since they would know that they didn't
+ generate the TCP SYN packets in the first place. Moreover, such
+ SYN/ACK attacks would have the same signatures as the existing TCP
+ SYN attacks. Provoking servers or hosts to reply with SYN/ACK
+ packets in order to congest a certain link would also be highly
+ inefficient because SYN/ACK packets are small in size.
+
+ However, the addition of ECN-Capability to SYN/ACK packets could
+ allow SYN/ACK packets to persist for more hops along a network path
+ before being dropped, thus adding somewhat to the ability of a
+ SYN/ACK attack to flood a network link.
+
+4.2. The TCP SYN Packet
+
+ There are several reasons why an ECN-Capable codepoint must not be
+ set in the IP header of the initiating TCP SYN packet. First, when
+ the TCP SYN packet is sent, there are no guarantees that the other
+ TCP endpoint (node B in Figure 2) is ECN-Capable, or that it would be
+ able to understand and react if the ECN CE codepoint was set by a
+ congested router.
+
+
+
+
+
+
+
+Kuzmanovic, et al. Experimental [Page 11]
+
+RFC 5562 ECN and SYN/ACK Packets June 2009
+
+
+ Second, the ECN-Capable codepoint in TCP SYN packets could be misused
+ by malicious clients to "improve" the well-known TCP SYN attack. By
+ setting an ECN-Capable codepoint in TCP SYN packets, a malicious host
+ might be able to inject a large number of TCP SYN packets through a
+ potentially congested ECN-enabled router, congesting it even further.
+
+ For both these reasons, we continue the restriction that the TCP SYN
+ packet must not have the ECN-Capable codepoint in the IP header set.
+
+4.3. SYN/ACK Packets and Packet Size
+
+ There are a number of router buffer architectures that have smaller
+ dropping rates for small (SYN) packets than for large (data) packets.
+ For example, for a Drop-Tail queue in units of packets, where each
+ packet takes a single slot in the buffer regardless of packet size,
+ small and large packets are equally likely to be dropped. However,
+ for a Drop-Tail queue in units of bytes, small packets are less
+ likely to be dropped than are large ones. Similarly, for Random
+ Early Detection (RED) in packet mode, small and large packets are
+ equally likely to be dropped or marked, while for RED in byte mode, a
+ packet's chance of being dropped or marked is proportional to the
+ packet size in bytes.
+
+ For a congested router with an AQM mechanism in byte mode, where a
+ packet's chance of being dropped or marked is proportional to the
+ packet size in bytes, the drop or marking rate for TCP SYN/ACK
+ packets should generally be low. In this case, the benefit of making
+ SYN/ACK packets ECN-Capable should be similarly moderate. However,
+ for a congested router with a Drop-Tail queue in units of packets or
+ with an AQM mechanism in packet mode, and with no priority queuing
+ for smaller packets, small and large packets should have the same
+ probability of being dropped or marked. In such a case, making
+ SYN/ACK packets ECN-Capable should be of significant benefit.
+
+ We believe that there are a wide range of behaviors in the real world
+ in terms of the drop or mark behavior at routers as a function of
+ packet size (see Section 10 of [Tools]). We note that all of these
+ alternatives listed above are available in the NS simulator (Drop-
+ Tail queues are by default in units of packets, while the default for
+ RED queue management has been changed from packet mode to byte mode).
+
+4.4. Response to ECN-Marking of SYN/ACK Packets
+
+ One question is why TCP SYN/ACK packets should be treated differently
+ from other packets in terms of the end node's response to an ECN-
+ marked packet. Section 5 of RFC 3168 [RFC3168] specifies the
+ following:
+
+
+
+
+Kuzmanovic, et al. Experimental [Page 12]
+
+RFC 5562 ECN and SYN/ACK Packets June 2009
+
+
+ Upon the receipt by an ECN-Capable transport of a single CE
+ packet, the congestion control algorithms followed at the end-
+ systems MUST be essentially the same as the congestion control
+ response to a *single* dropped packet. For example, for ECN-
+ Capable TCP the source TCP is required to halve its congestion
+ window for any window of data containing either a packet drop or
+ an ECN indication.
+
+ In particular, Section 6.1.2 of RFC 3168 [RFC3168] specifies that
+ when the TCP congestion window consists of a single packet and that
+ packet is ECN-marked in the network, then the data sender must reduce
+ the sending rate below one packet per round-trip time, by waiting for
+ one RTO before sending another packet. If the RTO was set to the
+ average round-trip time, this would result in halving the sending
+ rate; because the RTO is in fact larger than the average round-trip
+ time, the sending rate is reduced to less than half of its previous
+ value.
+
+ TCP's congestion control response to the *dropping* of a SYN/ACK
+ packet is to wait a default time before sending another packet. This
+ document argues that ECN gives end-systems a wider range of possible
+ responses to the *marking* of a SYN/ACK packet, and that waiting a
+ default time before sending another packet is not the desired
+ response.
+
+ On the conservative end, one could assume an effective congestion
+ window of one packet for the SYN/ACK packet, and respond to an ECN-
+ marked SYN/ACK packet by reducing the sending rate to one packet
+ every two round-trip times. As an approximation, the TCP end node
+ could measure the round-trip time T between the sending of the
+ SYN/ACK packet and the receipt of the acknowledgement, and reply to
+ the acknowledgement of the ECN-marked SYN/ACK packet by waiting T
+ seconds before sending a data packet.
+
+ However, we note that for an ECN-marked SYN/ACK packet, halving the
+ *congestion window* is not the same as halving the *sending rate*;
+ there is no "sending rate" associated with an ECN-Capable SYN/ACK
+ packet, as such packets are only sent as the first packet in a
+ connection from that host. Further, a router's marking of a SYN/ACK
+ packet is not affected by any past history of that connection.
+
+ Adding ECN-Capability to SYN/ACK packets allows the response of the
+ responder setting the initial congestion window to one packet,
+ instead of its allowed default value of two, three, or four packets.
+ The responder sends a non-ECN-Capable SYN/ACK packet, and proceeds
+
+
+
+
+
+
+Kuzmanovic, et al. Experimental [Page 13]
+
+RFC 5562 ECN and SYN/ACK Packets June 2009
+
+
+ with a cautious sending rate of one data packet per round-trip time
+ after that SYN/ACK packet is acknowledged. This document argues that
+ this approach is useful to users, with no dangers of congestion
+ collapse or of starvation of competing traffic. This is discussed in
+ more detail below in Section 6.2.
+
+ We note that if the data transfer is entirely from node A to node B,
+ there is still a difference in performance between the original
+ mechanism ECN+ and the mechanism ECN+/TryOnce specified in this
+ document. In particular, with ECN+/TryOnce, the TCP originator does
+ not send data packets until it has received a non-ECN-marked SYN/ACK
+ packet from the other end.
+
+5. Related Work
+
+ The addition of ECN-Capability to TCP's SYN/ACK packets was initially
+ proposed in [ECN+]. The paper includes an extensive set of
+ simulation and testbed experiments to evaluate the effects of the
+ proposal, using several Active Queue Management (AQM) mechanisms,
+ including Random Early Detection (RED) [RED], Random Exponential
+ Marking (REM) [REM], and Proportional Integrator (PI) [PI]. The
+ performance measures were the end-to-end response times for each
+ request/response pair, and the aggregate throughput on the bottleneck
+ link. The end-to-end response time was computed as the time from the
+ moment when the request for the file is sent to the server, until
+ that file is successfully downloaded by the client.
+
+ The measurements from [ECN+] show that setting an ECN-Capable
+ codepoint in the IP packet header in TCP SYN/ACK packets
+ systematically improves performance with all evaluated AQM schemes.
+ When SYN/ACK packets at a congested router are ECN-marked instead of
+ dropped, this can avoid a long initial retransmission timeout,
+ improving the response time for the affected flow dramatically.
+
+ [ECN+] shows that the impact on aggregate throughput can also be
+ quite significant, because marking SYN ACK packets can prevent larger
+ flows from suffering long timeouts before being "admitted" into the
+ network. In addition, the testbed measurements from [ECN+] show that
+ web servers setting the ECN-Capable codepoint in TCP SYN/ACK packets
+ could serve more requests.
+
+ As a final step, [ECN+] explores the coexistence of flows that do and
+ don't set the ECN-Capable codepoint in TCP SYN/ACK packets. The
+ results in [ECN+] show that both types of flows can coexist, with
+ some performance degradation for flows that don't use ECN+. Flows
+
+
+
+
+
+
+Kuzmanovic, et al. Experimental [Page 14]
+
+RFC 5562 ECN and SYN/ACK Packets June 2009
+
+
+ that do use ECN+ improve their end-to-end performance. At the same
+ time, the performance degradation for flows that don't use ECN+, as a
+ result of the flows that do use ECN+, increases as a greater fraction
+ of flows use ECN+.
+
+6. Performance Evaluation
+
+6.1. The Costs and Benefits of Adding ECN-Capability
+
+ [ECN+] explores the costs and benefits of adding ECN-Capability to
+ SYN/ACK packets with both simulations and experiments. The addition
+ of ECN-Capability to SYN/ACK packets could be of significant benefit
+ for those ECN connections that would have had the SYN/ACK packet
+ dropped in the network, and for which the ECN-Capability would allow
+ the SYN/ACK to be marked rather than dropped.
+
+ The percent of SYN/ACK packets on a link can be quite high. In
+ particular, measurements on links dominated by web traffic indicate
+ that 15-20% of the packets can be SYN/ACK packets [SCJO01].
+
+ The benefit of adding ECN-Capability to SYN/ACK packets depends in
+ part on the size of the data transfer. The drop of a SYN/ACK packet
+ can increase the download time of a short file by an order of
+ magnitude, by requiring a three-second retransmission timeout. For
+ longer-lived flows, the effect of a dropped SYN/ACK packet on file
+ download time is less dramatic. However, even for longer-lived
+ flows, the addition of ECN-Capability to SYN/ACK packets can improve
+ the fairness among long-lived flows, as newly arriving flows would be
+ less likely to have to wait for retransmission timeouts.
+
+ One question that arises is what fraction of connections would see
+ the benefit from making SYN/ACK packets ECN-Capable in a particular
+ scenario. Specifically:
+
+ (1) What fraction of arriving SYN/ACK packets are dropped at the
+ congested router when the SYN/ACK packets are not ECN-Capable?
+
+ (2) Of those SYN/ACK packets that are dropped, what fraction would
+ have been ECN-marked instead of dropped if the SYN/ACK packets
+ had been ECN-Capable?
+
+ To answer (1), it is necessary to consider not only the level of
+ congestion but also the queue architecture at the congested link. As
+ described in Section 4 above, for some queue architectures, small
+ packets are less likely to be dropped than large ones. In such an
+ environment, SYN/ACK packets would have lower packet drop rates;
+ question (1) could not necessarily be inferred from the overall
+ packet drop rate, but could be answered by measuring the drop rate
+
+
+
+Kuzmanovic, et al. Experimental [Page 15]
+
+RFC 5562 ECN and SYN/ACK Packets June 2009
+
+
+ for SYN/ACK packets directly. In such an environment, adding ECN-
+ Capability to SYN/ACK packets would be of less dramatic benefit than
+ in environments where all packets are equally likely to be dropped
+ regardless of packet size.
+
+ As question (2) implies, even if all of the SYN/ACK packets were
+ ECN-Capable, there could still be some SYN/ACK packets dropped
+ instead of marked at the congested link; the full answer to question
+ (2) depends on the details of the queue management mechanism at the
+ router. If congestion is sufficiently bad, and the queue management
+ mechanism cannot prevent the buffer from overflowing, then SYN/ACK
+ packets will be dropped rather than marked upon buffer overflow
+ whether or not they are ECN-Capable.
+
+ For some AQM mechanisms, ECN-Capable packets are marked instead of
+ dropped any time this is possible, that is, any time the buffer is
+ not yet full. For other AQM mechanisms however, such as the RED
+ mechanism as recommended in [RED], packets are dropped rather than
+ marked when the packet drop/mark rate exceeds a certain threshold,
+ e.g., 10%, even if the packets are ECN-Capable. For a router with
+ such an AQM mechanism, when congestion is sufficiently severe to
+ cause a high drop/mark rate, some SYN/ACK packets would be dropped
+ instead of marked whether or not they were ECN-Capable.
+
+ Thus, the degree of benefit of adding ECN-Capability to SYN/ACK
+ packets depends not only on the overall packet drop rate in the
+ network, but also on the queue management architecture at the
+ congested link.
+
+6.2. An Evaluation of Different Responses to ECN-Marked SYN/ACK Packets
+
+ This document specifies that the end node responds to the report of
+ an ECN-marked SYN/ACK packet by setting the initial congestion window
+ to one segment, instead of its possible default value of two to four
+ segments, and resending a SYN/ACK packet that is not ECN-Capable. We
+ call this ECN+/TryOnce.
+
+ However, Section 4 discussed two other possible responses to an ECN-
+ marked SYN/ACK packet. In ECN+, the original proposal from [ECN+],
+ the end node responds to the report of an ECN-marked SYN/ACK packet
+ by setting the initial congestion window to one segment and
+ immediately sending a data packet, if it has one to send. In
+ ECN+/Wait, the end node responds to the report of an ECN-marked
+ SYN/ACK packet by setting the initial congestion window to one
+ segment and waiting an RTT before sending a data packet.
+
+
+
+
+
+
+Kuzmanovic, et al. Experimental [Page 16]
+
+RFC 5562 ECN and SYN/ACK Packets June 2009
+
+
+ Simulations comparing the performance with Standard ECN (without
+ ECN-marked SYN/ACK packets), ECN+, ECN+/Wait, and ECN/TryOnce show
+ little difference, in terms of aggregate congestion, between ECN+ and
+ ECN+/Wait. However, for some scenarios with queues that are packet-
+ based rather than byte-based, and with packet drop rates above 25%
+ without ECN+, the use of ECN+ or of ECN+/Wait can more than double
+ the packet drop rates to greater than 50%. The details are given in
+ Tables 1 and 3 of Appendix A below. ECN+/TryOnce does not increase
+ the packet drop rate in scenarios of high congestion. Therefore,
+ ECN+/TryOnce is superior to ECN+ or to ECN+/Wait, which both
+ significantly increase the packet drop rate in scenarios of high
+ congestion. At the same time, ECN+/TryOnce gives a performance
+ improvement similar to that of ECN+ or ECN+/Wait (Tables 2 and 4 of
+ Appendix A).
+
+ Our conclusions are that ECN+/TryOnce is safe, and has significant
+ benefits to the user, and avoids the problems of ECN+ or ECN+/Wait
+ under extreme levels of congestion. As a consequence, this document
+ specifies the use of ECN+/TryOnce.
+
+ Note: We only discovered the occasional congestion-related problems
+ of ECN+ and of ECN+/Wait when re-running the simulations with an
+ updated version of the ns-2 simulator, after the document had almost
+ completed the standardization process.
+
+6.3. Experiments
+
+ This section discusses experiments that would be useful before a
+ widespread deployment of ECN-Capability for TCP SYN/ACK packets.
+
+ Section 7.1 below discusses some of the known deployment problems of
+ ECN, in terms of routers or middleboxes that react inappropriately to
+ packets that use ECN codepoints in the IP or TCP packet headers. One
+ goal of a measurement study of ECN-Capability for TCP SYN/ACK packets
+ would be to determine if there were any routers or middleboxes that
+ react inappropriately to TCP SYN/ACK packets containing an ECN-
+ Capable or CE codepoint in the IP header. A second goal of a
+ measurement study would be to check the deployment status of older
+ TCP implementations that are ECN-Capable, but that don't respond to
+ ECN-Capability for SYN/ACK packets. (This is discussed in more
+ detail in Appendix B below.)
+
+ Following the discussion in Section 6.2, an experimental study could
+ explore the use of ECN-Capability for TCP SYN/ACK packets in highly
+ congested environments with ECN-Capable routers.
+
+
+
+
+
+
+Kuzmanovic, et al. Experimental [Page 17]
+
+RFC 5562 ECN and SYN/ACK Packets June 2009
+
+
+7. Security Considerations
+
+ TCP packets carrying the ECT codepoint in IP headers can be marked
+ rather than dropped by ECN-Capable routers. This raises several
+ security concerns that we discuss below.
+
+7.1. "Bad" Routers or Middleboxes
+
+ There are a number of known deployment problems from using ECN with
+ TCP traffic in the Internet. The first reported problem, dating back
+ to 2000, is of a small but decreasing number of routers or
+ middleboxes that reset a TCP connection in response to TCP SYN
+ packets using flags in the TCP header to negotiate ECN-Capability
+ [Kelson00] [RFC3360] [MAF05]. Dave Thaler reported at the March 2007
+ IETF of two new problems encountered by TCP connections using ECN;
+ the first of the two problems concerns routers that crash when a TCP
+ data packet arrives with the ECN field in the IP header with the
+ codepoint ECT(0) or ECT(1), indicating that an ECN-Capable connection
+ has been established [SBT07].
+
+ While there is no evidence that any routers or middleboxes drop
+ SYN/ACK packets that contain an ECN-Capable or CE codepoint in the IP
+ header, such behavior cannot be excluded. (There seems to be a
+ number of routers or middleboxes that drop TCP SYN packets that
+ contain known or unknown IP options (see figure 1 of [MAF05].) Thus,
+ as specified in Section 3, if a SYN/ACK packet with the ECT or CE
+ codepoint is dropped, the TCP node should resend the SYN/ACK packet
+ without the ECN-Capable codepoint. There is also no evidence that
+ any routers or middleboxes crash when a SYN/ACK arrives with an ECN-
+ Capable or CE codepoint in the IP header (over and above the routers
+ already known to crash when a data packet arrives with either ECT(0)
+ or ECT(1)), but we have not conducted any measurement studies of this
+ [F07].
+
+7.2. Congestion Collapse
+
+ Because TCP SYN/ACK packets carrying an ECT codepoint could be ECN-
+ marked instead of dropped at an ECN-Capable router, the concern is
+ whether this can either invoke congestion or worsen performance in
+ highly congested scenarios. However, after learning that a SYN/ACK
+ packet was ECN-marked, the responder sends a SYN/ACK packet that is
+ not ECN-Capable; if this SYN/ACK packet is dropped, the responder
+ then waits for a retransmission timeout, as specified in the TCP
+ standards. In addition, routers are free to drop rather than mark
+ arriving packets in times of high congestion, regardless of whether
+ the packets are ECN-Capable. When congestion is very high and a
+ router's buffer is full, the router has no choice but to drop rather
+ than to mark an arriving packet.
+
+
+
+Kuzmanovic, et al. Experimental [Page 18]
+
+RFC 5562 ECN and SYN/ACK Packets June 2009
+
+
+ The simulations reported in Appendix A show that even with demanding
+ traffic mixes dominated by short flows and high levels of congestion,
+ the aggregate packet dropping rates are not significantly different
+ with Standard ECN or with ECN+/TryOnce. However, in our simulations,
+ we have one scenario where ECN+ or ECN+/Wait results in a
+ significantly higher packet drop rate than ECN or ECN+/TryOnce
+ (Tables 1 and 3 in Appendix A below).
+
+8. Conclusions
+
+ This document specifies a modification to RFC 3168 [RFC3168] to allow
+ TCP nodes to send SYN/ACK packets as being ECN-Capable. Making the
+ SYN/ACK packet ECN-Capable avoids the high cost to a TCP transfer
+ when a SYN/ACK packet is dropped by a congested router, by avoiding
+ the resulting retransmission timeout. This improves the throughput
+ of short connections. This document specifies the ECN+/TryOnce
+ mechanism for ECN-Capability for SYN/ACK packets, where the sender of
+ the SYN/ACK packet responds to an ECN mark by reducing its initial
+ congestion window from two, three, or four segments to one segment,
+ and sending a SYN/ACK packet that is not ECN-Capable. The addition
+ of ECN-Capability to SYN/ACK packets is particularly beneficial in
+ the server-to-client direction, where congestion is more likely to
+ occur. In this case, the initial information provided by the ECN
+ marking in the SYN/ACK packet enables the server to appropriately
+ adjust the initial load it places on the network, while avoiding the
+ delay of a retransmission timeout.
+
+9. Acknowledgements
+
+ We thank Anil Agarwal, Mark Allman, Remi Denis-Courmont, Wesley Eddy,
+ Lars Eggert, Alfred Hoenes, Janardhan Iyengar, and Pasi Sarolahti for
+ feedback on earlier working drafts of this document. We thank Adam
+ Langley [L08] for contributing a patch for ECN+/TryOnce for the Linux
+ development tree.
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+Kuzmanovic, et al. Experimental [Page 19]
+
+RFC 5562 ECN and SYN/ACK Packets June 2009
+
+
+Appendix A. Report on Simulations
+
+ This section reports on simulations showing the costs of adding ECN+
+ in highly congested scenarios. This section also reports on
+ simulations for a comparative evaluation between ECN, ECN+,
+ ECN+/Wait, and ECN+/TryOnce.
+
+ The simulations are run with a range of file-size distributions,
+ using the PackMime traffic generator in the ns-2 simulator. They all
+ use a heavy-tailed distribution of file sizes. The simulations
+ reported in the tables below use a mean file size of 3 Kbytes, to
+ show the results with a traffic mix with a large number of small
+ transfers. Other simulations were run with mean file sizes of 5
+ Kbytes, 7 Kbytes, 14 Kbytes, and 17 Kbytes. The title of each chart
+ gives the targeted average load from the traffic generator. Because
+ the simulations use a heavy-tailed distribution of file sizes, and
+ run for only 85 seconds (including ten seconds of warm-up time), the
+ actual load is often much smaller than the targeted load. The
+ congested link is 100 Mbps. RED is run in gentle mode, and arriving
+ ECN-Capable packets are only dropped instead of marked if the buffer
+ is full (and the router has no choice).
+
+ We explore three possible mechanisms for a TCP node's response to a
+ report of an ECN-marked SYN/ACK packet. With ECN+, the TCP node
+ sends a data packet immediately (with an initial congestion window of
+ one segment). With ECN+/Wait, the TCP node waits a round-trip time
+ before sending a data packet; the responder already has one
+ measurement of the round-trip time when the acknowledgement for the
+ SYN/ACK packet is received. With ECN+/TryOnce, the mechanism
+ standardized in this document, the TCP responder replies to a report
+ of an ECN-marked SYN/ACK packet by sending a SYN/ACK packet that is
+ not ECN-Capable, and reducing the initial congestion window to one
+ segment.
+
+ The simulation scripts are available on [ECN-SYN], along with graphs
+ showing the distribution of response times for the TCP connections.
+
+A.1. Simulations with RED in Packet Mode
+
+ The simulations with RED in packet mode and with the queue in packets
+ show that ECN+ is useful in times of moderate or high congestion.
+ However, for the simulations with a target load of 125%, with a
+ packet loss rate of over 25% for ECN, ECN+ and ECN+/Wait both result
+ in a packet loss rate of over 50%. (In contrast, the packet loss
+ rate with ECN+/TryOnce is less than that of ECN alone.) For the
+ distribution of response times, the simulations show that ECN+,
+ ECN+/Wait, and ECN+/TryOnce all significantly improve the response
+ times, when compared to the response times with Standard ECN.
+
+
+
+Kuzmanovic, et al. Experimental [Page 20]
+
+RFC 5562 ECN and SYN/ACK Packets June 2009
+
+
+ Table 1 shows the congestion levels for simulations with RED in
+ packet mode, with a queue in packets. To explore a worst-case
+ scenario, these simulations use a traffic mix with an unrealistically
+ small flow size distribution, with a mean flow size of 3 Kbytes. For
+ each table showing a particular traffic load, the four rows show the
+ number of packets dropped, the number of packets ECN-marked, the
+ aggregate packet drop rate, and the aggregate throughput. The four
+ columns show the simulations with Standard ECN, ECN+, ECN+/Wait, and
+ ECN+/TryOnce.
+
+ These simulations were run with RED set to mark instead of drop
+ packets any time that the queue is not full. This is a worst-case
+ scenario for ECN+ and its variants. For the default implementation
+ of RED in the ns-2 simulator, when the average queue size exceeds a
+ configured threshold, the router drops all arriving packets. For
+ scenarios with this RED mechanism, it is less likely that ECN+ or one
+ of its variants would increase the average queue size above the
+ configured threshold.
+
+ The usefulness of ECN+: The first thing to observe is that for all of
+ the simulations, the use of ECN+ or ECN+/Wait significantly increases
+ the number of packets marked. In contrast, the use of ECN+/TryOnce
+ significantly increases the number of packets marked in the
+ simulations with moderate congestion, and gives a more moderate
+ increase in the number of packets marked for the simulations with
+ higher levels of congestion. However, the cumulative distribution
+ function (CDF) in Table 2 shows that ECN+, ECN+/Wait, and
+ ECN+/TryOnce all improve response times for all of the simulations,
+ with moderate or with larger levels of congestion.
+
+ Little increase in congestion, sometimes: The second thing to observe
+ is that for the simulations with low or moderate levels of congestion
+ (that is, with packet drop rates less than 10%), the use of ECN+,
+ ECN+/Wait, and ECN+/TryOnce all decrease the aggregate packet drop
+ rate relative to the simulations with ECN. This makes sense, since
+ with low or moderate levels of congestion, ECN+ allows SYN/ACK
+ packets to be marked instead of dropped, and the use of ECN+ doesn't
+ add to the aggregate congestion. However, for the simulations with
+ packet drop rates of 15% or higher with ECN, the use of ECN+ or
+ ECN+/Wait increases the aggregate packet drop rate, sometimes even
+ doubling it.
+
+ Comparing ECN+, ECN+/Wait, and ECN+/TryOnce: The aggregate packet
+ drop rate is generally higher with ECN+/Wait than with ECN+. Thus,
+ there is no congestion-related reason to prefer ECN+/Wait over ECN+.
+ In contrast, the aggregate packet drop rate with ECN+/TryOnce is
+ often significantly lower than the aggregate packet drop rate with
+ either ECN, ECN+, or ECN+/Wait.
+
+
+
+Kuzmanovic, et al. Experimental [Page 21]
+
+RFC 5562 ECN and SYN/ACK Packets June 2009
+
+
+ Target Load = 95%:
+ ECN ECN+ ECN+/Wait ECN+/TryOnce
+ ------- ------- ------- ----------
+ Dropped 20,516 11,226 11,735 16,755`
+ Marked 30,586 37,741 37,425 40,764
+ Loss rate 1.41% 0.78% 0.81% 1.02%
+ Throughput 81% 81% 81% 81%
+
+ Target Load = 110%:
+ ECN ECN+ ECN+/Wait ECN+/TryOnce
+ ------- ------- ------- ----------
+ Dropped 165,566 106,083 147,180 208,422
+ Marked 179,735 281,306 308,473 235,483
+ Loss rate 9.01% 6.12% 8.02% 6.89%
+ Throughput 92% 92% 92% 94%
+
+ Target Load = 125%:
+ ECN ECN+ ECN+/Wait ECN+/TryOnce
+ ------- ------- ------- ----------
+ Dropped 600,628 1,746,768 2,176,530 625,552
+ Marked 418,433 1,166,450 1,164,932 439,847
+ Loss rate 25.45% 51.73% 56.87% 18.31%
+ Throughput 94% 98% 97% 95%
+
+ Target Load = 150%
+ ECN ECN+ ECN+/Wait ECN+/TryOnce
+ ------- ------- ------- ----------
+ Dropped 1,449,945 1,565,0517 1,563,0801 1,351,637
+ Marked 669,840 583,378 591,315 684,715
+ Loss rate 46.7% 59.0% 59.0% 32.7%
+ Throughput 88% 94% 94% 92%
+
+ Table 1: Simulations with an average flow size of 3 Kbytes, a 100
+ Mbps link, RED in packet mode, queue in packets
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+Kuzmanovic, et al. Experimental [Page 22]
+
+RFC 5562 ECN and SYN/ACK Packets June 2009
+
+
+ Target Load = 95%:
+ TIME: 10 100 200 300 400 500 1000 2000 3000 4000 5000
+ ------------------------------------------------------
+ ECN: 0.00 0.07 0.26 0.51 0.82 0.96 0.97 0.97 0.97 1.00 1.00
+ ECN+: 0.00 0.07 0.27 0.53 0.85 0.99 1.00 1.00 1.00 1.00 1.00
+ Wait: 0.00 0.07 0.26 0.51 0.83 0.97 1.00 1.00 1.00 1.00 1.00
+ Once: 0.00 0.07 0.24 0.49 0.83 0.97 1.00 1.00 1.00 1.00 1.00
+
+ Target Load = 110%:
+ TIME: 10 100 200 300 400 500 1000 2000 3000 4000 5000
+ ------------------------------------------------------
+ ECN: 0.00 0.05 0.19 0.41 0.67 0.79 0.80 0.80 0.80 0.96 0.96
+ ECN+: 0.00 0.07 0.22 0.48 0.81 0.96 1.00 1.00 1.00 1.00 1.00
+ Wait: 0.00 0.05 0.18 0.38 0.64 0.77 0.95 1.00 1.00 1.00 1.00
+ Once: 0.00 0.06 0.19 0.42 0.70 0.86 0.95 0.96 0.96 0.99 0.99
+
+ Target Load = 125%:
+ TIME: 10 100 200 300 400 500 1000 2000 3000 4000 5000
+ ------------------------------------------------------
+ ECN: 0.00 0.04 0.13 0.27 0.46 0.56 0.58 0.59 0.59 0.82 0.82
+ ECN+: 0.00 0.06 0.18 0.33 0.58 0.76 0.97 0.99 0.99 1.00 1.00
+ Wait: 0.00 0.01 0.06 0.13 0.21 0.27 0.68 0.98 0.99 1.00 1.00
+ Once: 0.00 0.05 0.16 0.34 0.58 0.73 0.85 0.87 0.87 0.95 0.96
+
+ Target Load = 150%:
+ TIME: 10 100 200 300 400 500 1000 2000 3000 4000 5000
+ ------------------------------------------------------
+ ECN: 0.00 0.03 0.08 0.18 0.31 0.39 0.42 0.42 0.43 0.68 0.68
+ ECN+: 0.00 0.06 0.18 0.39 0.67 0.81 0.83 0.84 0.84 0.93 0.93
+ Wait: 0.00 0.06 0.18 0.39 0.67 0.81 0.83 0.84 0.84 0.93 0.94
+ Once: 0.00 0.04 0.13 0.27 0.46 0.59 0.72 0.75 0.75 0.88 0.88
+
+ Table 2: The cumulative distribution function (CDF) for transfer
+ times, for simulations with an average flow size of 3
+ Kbytes, a 100 Mbps link, RED in packet mode, queue in
+ packets (the graphs are available from
+ "http://www.icir.org/floyd/ecn-syn/")
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+Kuzmanovic, et al. Experimental [Page 23]
+
+RFC 5562 ECN and SYN/ACK Packets June 2009
+
+
+ Target Load = 95%
+ ECN ECN+ ECN+/Wait ECN+/TryOnce
+ ------- ------- ------- ----------
+ Dropped 8,448 6,362 7,740 14,107
+ Marked 9,891 16,787 17,456 16,132
+ Loss rate 5.5% 4.3% 5.0% 5.0%
+ Throughput 78% 78% 78% 81%
+
+ Target Load = 110%
+ ECN ECN+ ECN+/Wait ECN+/TryOnce
+ ------- ------- ------- ----------
+ Dropped 31,284 29,773 49,297 45,277
+ Marked 28,429 54,729 60,383 34,622
+ Loss rate 15.3% 15.2% 21.9% 13.6%
+ Throughput 97% 96% 96% 94%
+
+ Target Load = 125%
+ ECN ECN+ ECN+/Wait ECN+/TryOnce
+ ------- ------- ------- ----------
+ Dropped 61,433 176,682 214,096 75,612
+ Marked 44,408 119,728 117,301 49,442
+ Loss rate 25.4% 51.9% 56.0% 22.3%
+ Throughput 97% 98% 98% 96%
+
+ Target Load = 150%
+ ECN ECN+ ECN+/Wait ECN+/TryOnce
+ ------- ------- ------- ----------
+ Dropped 130,007 251,856 326,845 133,603
+ Marked 63,066 146,757 147,239 66,444
+ Loss rate 42.5% 61.3% 67.3% 31.7%
+ Throughput 93% 99% 99% 94%
+
+ Table 3: Simulations with an average flow size of 3 Kbytes, a 10
+ Mbps link, RED in packet mode, queue in packets
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+Kuzmanovic, et al. Experimental [Page 24]
+
+RFC 5562 ECN and SYN/ACK Packets June 2009
+
+
+ Target Load = 95%:
+ TIME: 10 100 200 300 400 500 1000 2000 3000 4000 5000
+ ------------------------------------------------------
+ ECN: 0.00 0.05 0.18 0.42 0.70 0.86 0.88 0.88 0.88 0.98 0.98
+ ECN+: 0.00 0.06 0.20 0.45 0.78 0.96 1.00 1.00 1.00 1.00 1.00
+ Wait: 0.00 0.05 0.18 0.40 0.68 0.84 0.96 1.00 1.00 1.00 1.00
+ Once: 0.00 0.05 0.18 0.40 0.71 0.88 0.96 0.97 0.97 0.99 0.99
+
+ Target Load = 110%:
+ TIME: 10 100 200 300 400 500 1000 2000 3000 4000 5000
+ ------------------------------------------------------
+ ECN: 0.00 0.03 0.13 0.29 0.52 0.66 0.69 0.69 0.69 0.91 0.91
+ ECN+: 0.00 0.05 0.17 0.36 0.66 0.88 0.98 0.99 1.00 1.00 1.00
+ Wait: 0.00 0.02 0.08 0.20 0.35 0.47 0.76 0.98 1.00 1.00 1.00
+ Once: 0.00 0.05 0.15 0.32 0.58 0.75 0.88 0.90 0.90 0.97 0.97
+
+
+ Target Load = 125%:
+ TIME: 10 100 200 300 400 500 1000 2000 3000 4000 5000
+ ------------------------------------------------------
+ ECN: 0.00 0.03 0.10 0.22 0.40 0.52 0.56 0.56 0.57 0.82 0.82
+ ECN+: 0.00 0.03 0.14 0.27 0.49 0.70 0.96 0.99 0.99 0.99 1.00
+ Wait: 0.00 0.00 0.03 0.07 0.12 0.18 0.50 0.94 0.99 0.99 1.00
+ Once: 0.00 0.04 0.13 0.28 0.51 0.66 0.81 0.84 0.84 0.94 0.94
+
+ Target Load = 150%:
+ TIME: 10 100 200 300 400 500 1000 2000 3000 4000 5000
+ ------------------------------------------------------
+ ECN: 0.00 0.02 0.07 0.15 0.28 0.38 0.42 0.42 0.43 0.67 0.68
+ ECN+: 0.00 0.00 0.00 0.00 0.01 0.05 0.68 0.83 0.95 0.97 0.98
+ Wait: 0.00 0.00 0.00 0.00 0.00 0.00 0.10 0.62 0.83 0.93 0.97
+ Once: 0.00 0.03 0.11 0.24 0.42 0.56 0.71 0.75 0.75 0.88 0.88
+
+ Table 4: The cumulative distribution function (CDF) for transfer
+ times, for simulations with an average flow size of 3
+ Kbytes, a 10 Mbps link, RED in packet mode, queue in
+ packets (the graphs are available from
+ "http://www.icir.org/floyd/ecn-syn/")
+
+A.2. Simulations with RED in Byte Mode
+
+ Table 5 below shows simulations with RED in byte mode and the queue
+ in bytes. There is no significant increase in aggregate congestion
+ with the use of ECN+, ECN+/Wait, or ECN+/TryOnce.
+
+ However, unlike the simulations with RED in packet mode, the
+ simulations with RED in byte mode show little benefit from the use of
+ ECN+ or ECN+/Wait, in that the packet marking rate with ECN+ or
+
+
+
+Kuzmanovic, et al. Experimental [Page 25]
+
+RFC 5562 ECN and SYN/ACK Packets June 2009
+
+
+ ECN+/Wait is not much different than the packet marking rate with
+ Standard ECN. This is because with RED in byte mode, small packets
+ like SYN/ACK packets are rarely dropped or marked -- that is, there
+ is no drawback from the use of ECN+ in these scenarios, but not much
+ need for ECN+ either, in a scenario where small packets are unlikely
+ to be dropped or marked.
+
+ Target Load = 95%
+ ECN ECN+ ECN+/Wait ECN+/TryOnce
+ ------- ------- ------- ----------
+ Dropped 766 446 427 408
+ Marked 32,683 34,289 33,412 31,892
+ Loss rate 0.05% 0.03% 0.03% 0.03%
+ Throughput 81% 81% 81% 81%
+
+ Target Load = 110%
+ ECN ECN+ ECN+/Wait ECN+/TryOnce
+ ------- ------- ------- ----------
+ Dropped 2,496 2,110 1,733 2,020
+ Marked 220,573 258,696 230,955 214,604
+ Loss rate 0.15% 0.13% 0.11% 0.11%
+ Throughput 92% 91% 92% 92%
+
+ Target Load = 125%
+ ECN ECN+ ECN+/Wait ECN+/TryOnce
+ ------- ------- ------- ----------
+ Dropped 20,032 13,555 13,979 16,918
+ Marked 725,165 726,992 726,823 615,235
+ Loss rate 1.11% 0.76% 0.78% 0.66%
+ Throughput 95% 95% 95% 96%
+
+ Target Load = 150%
+ ECN ECN+ ECN+/Wait ECN+/TryOnce
+ ------- ------- ------- ----------
+ Dropped 484,251 483,847 507,727 600,737
+ Marked 865,905 872,254 873,317 818,451
+ Loss rate 19.09% 19.13% 19.71% 12.66%
+ Throughput 99% 98% 99% 99%
+
+ Table 5: Simulations with an average flow size of 3 Kbytes, a 100
+ Mbps link, RED in byte mode, queue in bytes
+
+
+
+
+
+
+
+
+
+
+Kuzmanovic, et al. Experimental [Page 26]
+
+RFC 5562 ECN and SYN/ACK Packets June 2009
+
+
+ Target Load = 95%
+ ECN ECN+ ECN+/Wait ECN+/TryOnce
+ ------- ------- ------- ----------
+ Dropped 142 77 103 99
+ Marked 11,694 11,387 11,604 12,129
+ Loss rate 0.1% 0.1% 0.1% 0.1%
+ Throughput 78% 78% 78% 78%
+
+ Target Load = 110%
+ ECN ECN+ ECN+/Wait ECN+/TryOnce
+ ------- ------- ------- ----------
+ Dropped 338 210 247 274
+ Marked 41,676 40,412 44,173 36,265
+ Loss rate 0.2% 0.1% 0.1% 0.1%
+ Throughput 94% 94% 94% 96%
+
+ Target Load = 125%
+ ECN ECN+ ECN+/Wait ECN+/TryOnce
+ ------- ------- ------- ----------
+ Dropped 1,559 951 978 1,723
+ Marked 74,933 75,499 75,481 59,670
+ Loss rate 0.8% 0.5% 0.5% 0.6%
+ Throughput 99% 99% 99% 96%
+
+ Target Load = 150%
+ ECN ECN+ ECN+/Wait ECN+/TryOnce
+ ------- ------- ------- ----------
+ Dropped 2,374 1,528 1,515 4,848
+ Marked 85,739 86,428 86,144 81,350
+ Loss rate 1.2% 0.8% 0.8% 1.4%
+ Throughput 99% 98% 98% 98%
+
+ Table 6: Simulations with an average flow size of 3 Kbytes, a 10
+ Mbps link, RED in byte mode, queue in bytes
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+Kuzmanovic, et al. Experimental [Page 27]
+
+RFC 5562 ECN and SYN/ACK Packets June 2009
+
+
+Appendix B. Issues of Incremental Deployment
+
+ In order for TCP node B to send a SYN/ACK packet as ECN-Capable, node
+ B must have received an ECN-setup SYN packet from node A. However,
+ it is possible that node A supports ECN, but either ignores the CE
+ codepoint on received SYN/ACK packets, or ignores SYN/ACK packets
+ with the ECT or CE codepoint set. If the TCP initiator ignores the
+ CE codepoint on received SYN/ACK packets, this would mean that the
+ TCP responder would not respond to this congestion indication.
+ However, this seems to us an acceptable cost to pay in the
+ incremental deployment of ECN-Capability for TCP's SYN/ACK packets.
+ It would mean that the responder would not reduce the initial
+ congestion window from two, three, or four segments down to one
+ segment, as it should, and would not sent a non-ECN-Capable SYN/ACK
+ packet to complete the SYN exchange. However, the TCP end nodes
+ would still respond correctly to any subsequent CE indications on
+ data packets later on in the connection.
+
+ Figure 4 shows an interchange with the SYN/ACK packet ECN-marked, but
+ with the ECN mark ignored by the TCP originator.
+
+ ---------------------------------------------------------------
+ TCP Node A Router TCP Node B
+ (initiator) (responder)
+ ---------- ------ ----------
+
+ ECN-setup SYN packet --->
+ ECN-setup SYN packet --->
+
+ <--- ECN-setup SYN/ACK, ECT
+ <--- Sets CE on SYN/ACK
+ <--- ECN-setup SYN/ACK, CE
+
+ Data/ACK, No ECN-Echo --->
+ Data/ACK --->
+ <--- Data (up to four packets)
+ ---------------------------------------------------------------
+
+ Figure 4: SYN exchange with the SYN/ACK packet marked,
+ but with the ECN mark ignored by the TCP initiator
+
+ Thus, to be explicit, when a TCP connection includes an initiator
+ that supports ECN but *does not* support ECN-Capability for SYN/ACK
+ packets, in combination with a responder that *does* support ECN-
+ Capability for SYN/ACK packets, it is possible that the ECN-Capable
+ SYN/ACK packets will be marked rather than dropped in the network,
+ and that the responder will not learn about the ECN mark on the
+ SYN/ACK packet. This would not be a problem if most packets from the
+
+
+
+Kuzmanovic, et al. Experimental [Page 28]
+
+RFC 5562 ECN and SYN/ACK Packets June 2009
+
+
+ responder supporting ECN for SYN/ACK packets were in long-lived TCP
+ connections, but it would be more problematic if most of the packets
+ were from TCP connections consisting of four data packets, and the
+ TCP responder for these connections was ready to send its data
+ packets immediately after the SYN/ACK exchange. Of course, with
+ *severe* congestion, the SYN/ACK packets would likely be dropped
+ rather than ECN-marked at the congested router, preventing the TCP
+ responder from adding to the congestion by sending its initial window
+ of four data packets.
+
+ It is also possible that in some older TCP implementation, the
+ initiator would ignore arriving SYN/ACK packets that had the ECT or
+ CE codepoint set. This would result in a delay in connection setup
+ for that TCP connection, with the initiator re-sending the SYN packet
+ after a retransmission timeout. We are not aware of any TCP
+ implementations with this behavior.
+
+ One possibility for coping with problems of backwards compatibility
+ would be for TCP initiators to use a TCP flag that means "I
+ understand ECN-Capable SYN/ACK packets". If this document were to
+ standardize the use of such an "ECN-SYN" flag, then the TCP responder
+ would only send a SYN/ACK packet as ECN-Capable if the incoming SYN
+ packet had the "ECN-SYN" flag set. An ECN-SYN flag would prevent the
+ backwards compatibility problems described in the paragraphs above.
+
+ One drawback to the use of an ECN-SYN flag is that it would use one
+ of the four remaining reserved bits in the TCP header for a transient
+ backwards compatibility problem. This drawback is limited by the
+ fact that the "ECN-SYN" flag would be defined only for use with ECN-
+ setup SYN packets; that bit in the TCP header could be defined to
+ have other uses for other kinds of TCP packets.
+
+ Factors in deciding not to use an ECN-SYN flag include the following:
+
+ (1) The limited installed base: At the time that this document was
+ written, the TCP implementations in Microsoft Vista and Mac OS X
+ included ECN, but ECN was not enabled by default [SBT07]. Thus,
+ there was not a large deployed base of ECN-Capable TCP
+ implementations. This limits the scope of any backwards
+ compatibility problems.
+
+ (2) Limits to the scope of the problem: The backwards compatibility
+ problem would not be serious enough to cause congestion collapse;
+ with severe congestion, the buffer at the congested router will
+ overflow, and the congested router will drop rather than ECN-mark
+
+
+
+
+
+
+Kuzmanovic, et al. Experimental [Page 29]
+
+RFC 5562 ECN and SYN/ACK Packets June 2009
+
+
+ arriving SYN packets. Some active queue management mechanisms
+ might switch from packet-marking to packet-dropping in times of
+ high congestion before buffer overflow, as recommended in Section
+ 19.1 of RFC 3168 [RFC3168]. This helps to prevent congestion
+ collapse problems with the use of ECN.
+
+ (3) Detection of and response to backwards-compatibility problems: A
+ TCP responder such as a web server can't differentiate between a
+ SYN/ACK packet that is not ECN-marked in the network, and a
+ SYN/ACK packet that is ECN-marked, but where the ECN mark is
+ ignored by the TCP initiator. However, a TCP responder *can*
+ detect if a SYN/ACK packet is sent as ECN-capable and not
+ reported as ECN-marked, but data packets are dropped or marked
+ from the initial window of data. We will call this scenario
+ "initial-window-congestion". If a web server frequently
+ experienced initial-window-congestion (without SYN/ACK
+ congestion), then the web server *might* be experiencing
+ backwards compatibility problems with ECN-Capable SYN/ACK
+ packets, and could respond by not sending SYN/ACK packets as
+ ECN-Capable.
+
+Normative References
+
+ [RFC793] Postel, J., "Transmission Control Protocol", STD 7, RFC
+ 793, September 1981.
+
+ [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate
+ Requirement Levels", BCP 14, RFC 2119, March 1997.
+
+ [RFC2988] Paxson, V. and M. Allman, "Computing TCP's Retransmission
+ Timer", RFC 2988, November 2000.
+
+ [RFC3168] Ramakrishnan, K., Floyd, S., and D. Black, "The Addition
+ of Explicit Congestion Notification (ECN) to IP", RFC
+ 3168, September 2001.
+
+Informative References
+
+ [ECN+] A. Kuzmanovic, The Power of Explicit Congestion
+ Notification, SIGCOMM 2005.
+
+ [ECN-SYN] ECN-SYN web page with simulation scripts,
+ http://www.icir.org/floyd/ecn-syn.
+
+
+
+
+
+
+
+
+Kuzmanovic, et al. Experimental [Page 30]
+
+RFC 5562 ECN and SYN/ACK Packets June 2009
+
+
+ [F07] S. Floyd, "[BEHAVE] Response of firewalls and middleboxes
+ to TCP SYN packets that are ECN-Capable?", August 2, 2007,
+ email to the BEHAVE mailing list, http://www1.ietf.org/
+ mail-archive/web/behave/current/msg02644.html.
+
+ [Kelson00] Dax Kelson, "8% of the Internet unreachable!", September
+ 10, 2000, email to the Linux kernel mailing list,
+ http://lkml.indiana.edu/hypermail/linux/kernel/
+ 0009.1/0329.html.
+
+ [L08] A. Landley, "Re: [tcpm] I-D Action:draft-ietf-tcpm-
+ ecnsyn-06.txt", August 24, 2008, email to the tcpm mailing
+ list, http://www.ietf.org/
+ mail-archive/web/tcpm/current/msg03988.html.
+
+ [MAF05] A. Medina, M. Allman, and S. Floyd, "Measuring the
+ Evolution of Transport Protocols in the Internet", ACM
+ CCR, April 2005.
+
+ [PI] C. Hollot, V. Misra, W. Gong, and D. Towsley, "On
+ Designing Improved Controllers for AQM Routers Supporting
+ TCP Flows", April 1998.
+
+ [RED] Floyd, S., and Jacobson, V., "Random Early Detection
+ gateways for Congestion Avoidance", IEEE/ACM Transactions
+ on Networking, V.1 N.4, August 1993.
+
+ [REM] S. Athuraliya, V. H. Li, S. H. Low and Q. Yin, "REM:
+ Active Queue Management", IEEE Network, May 2001.
+
+ [RFC2309] Braden, B., Clark, D., Crowcroft, J., Davie, B., Deering,
+ S., Estrin, D., Floyd, S., Jacobson, V., Minshall, G.,
+ Partridge, C., Peterson, L., Ramakrishnan, K., Shenker,
+ S., Wroclawski, J., and L. Zhang, "Recommendations on
+ Queue Management and Congestion Avoidance in the
+ Internet", RFC 2309, April 1998.
+
+ [RFC2581] Allman, M., Paxson, V., and W. Stevens, "TCP Congestion
+ Control", RFC 2581, April 1999.
+
+ [RFC3042] Allman, M., Balakrishnan, H., and S. Floyd, "Enhancing
+ TCP's Loss Recovery Using Limited Transmit", RFC 3042,
+ January 2001.
+
+ [RFC3360] Floyd, S., "Inappropriate TCP Resets Considered Harmful",
+ BCP 60, RFC 3360, August 2002.
+
+
+
+
+
+Kuzmanovic, et al. Experimental [Page 31]
+
+RFC 5562 ECN and SYN/ACK Packets June 2009
+
+
+ [RFC3390] Allman, M., Floyd, S., and C. Partridge, "Increasing TCP's
+ Initial Window", RFC 3390, October 2002.
+
+ [RFC4987] Eddy, W., "TCP SYN Flooding Attacks and Common
+ Mitigations", RFC 4987, August 2007.
+
+ [SCJO01] F. Smith, F. Campos, K. Jeffay, and D. Ott, "What TCP/IP
+ Protocol Headers Can Tell us about the Web", SIGMETRICS,
+ June 2001.
+
+ [SYN-COOK] Dan J. Bernstein, SYN cookies, 1997, see also
+ <http://cr.yp.to/syncookies.html>.
+
+ [SBT07] M. Sridharan, D. Bansal, and D. Thaler, "Implementation
+ Report on Experiences with Various TCP RFCs", Presentation
+ in the TSVAREA, IETF 68, March 2007.
+ http://www3.ietf.org/proceedings/07mar/slides/tsvarea-
+ 3/sld6.htm.
+
+ [Tools] S. Floyd, Ed., and E. Kohler, Ed., "Tools for the
+ Evaluation of Simulation and Testbed Scenarios", Work in
+ Progress, February 2008.
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+Kuzmanovic, et al. Experimental [Page 32]
+
+RFC 5562 ECN and SYN/ACK Packets June 2009
+
+
+Authors' Addresses
+
+ Aleksandar Kuzmanovic
+ Northwestern University
+
+ Phone: +1 (847) 467-5519
+ EMail: akuzma@northwestern.edu
+ URL: http://cs.northwestern.edu/~akuzma
+
+
+ Amit Mondal
+ Northwestern University
+
+ Phone: +1 (847) 467-6455
+ EMail: a-mondal@northwestern.edu
+ URL: http://www.cs.northwestern.edu/~akm175/
+
+
+ Sally Floyd
+ ICIR (ICSI Center for Internet Research)
+
+ Phone: +1 (510) 666-2989
+ EMail: floyd@icir.org
+ URL: http://www.icir.org/floyd/
+
+
+ K. K. Ramakrishnan
+ AT&T Labs Research
+ Rm. A161
+ 180 Park Ave.
+ Florham Park, NJ 07932
+
+ Phone: +1 (973) 360-8764
+ EMail: kkrama@research.att.com
+ URL: http://www.research.att.com/info/kkrama
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+Kuzmanovic, et al. Experimental [Page 33]
+