summaryrefslogtreecommitdiff
path: root/doc/rfc/rfc3439.txt
diff options
context:
space:
mode:
Diffstat (limited to 'doc/rfc/rfc3439.txt')
-rw-r--r--doc/rfc/rfc3439.txt1571
1 files changed, 1571 insertions, 0 deletions
diff --git a/doc/rfc/rfc3439.txt b/doc/rfc/rfc3439.txt
new file mode 100644
index 0000000..ad2dcc0
--- /dev/null
+++ b/doc/rfc/rfc3439.txt
@@ -0,0 +1,1571 @@
+
+
+
+
+
+
+Network Working Group R. Bush
+Request for Comments: 3439 D. Meyer
+Updates: 1958 December 2002
+Category: Informational
+
+
+ Some Internet Architectural Guidelines and Philosophy
+
+Status of this Memo
+
+ This memo provides information for the Internet community. It does
+ not specify an Internet standard of any kind. Distribution of this
+ memo is unlimited.
+
+Copyright Notice
+
+ Copyright (C) The Internet Society (2002). All Rights Reserved.
+
+Abstract
+
+ This document extends RFC 1958 by outlining some of the philosophical
+ guidelines to which architects and designers of Internet backbone
+ networks should adhere. We describe the Simplicity Principle, which
+ states that complexity is the primary mechanism that impedes
+ efficient scaling, and discuss its implications on the architecture,
+ design and engineering issues found in large scale Internet
+ backbones.
+
+Table of Contents
+
+ 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 2
+ 2. Large Systems and The Simplicity Principle . . . . . . . . . 3
+ 2.1. The End-to-End Argument and Simplicity . . . . . . . . . 3
+ 2.2. Non-linearity and Network Complexity . . . . . . . . . . 3
+ 2.2.1. The Amplification Principle. . . . . . . . . . . . . . . 4
+ 2.2.2. The Coupling Principle . . . . . . . . . . . . . . . . . 5
+ 2.3. Complexity lesson from voice. . . . . . . . . . . . . . . 6
+ 2.4. Upgrade cost of complexity. . . . . . . . . . . . . . . . 7
+ 3. Layering Considered Harmful. . . . . . . . . . . . . . . . . 7
+ 3.1. Optimization Considered Harmful . . . . . . . . . . . . . 8
+ 3.2. Feature Richness Considered Harmful . . . . . . . . . . . 9
+ 3.3. Evolution of Transport Efficiency for IP. . . . . . . . . 9
+ 3.4. Convergence Layering. . . . . . . . . . . . . . . . . . . 9
+ 3.4.1. Note on Transport Protocol Layering. . . . . . . . . . . 11
+ 3.5. Second Order Effects . . . . . . . . . . . . . . . . . . 11
+ 3.6. Instantiating the EOSL Model with IP . . . . . . . . . . 12
+ 4. Avoid the Universal Interworking Function. . . . . . . . . . 12
+ 4.1. Avoid Control Plane Interworking . . . . . . . . . . . . . 13
+
+
+
+Bush, et. al. Informational [Page 1]
+
+RFC 3439 Internet Architectural Guidelines December 2002
+
+
+ 5. Packet versus Circuit Switching: Fundamental Differences . . 13
+ 5.1. Is PS is inherently more efficient than CS? . . . . . . . 13
+ 5.2. Is PS simpler than CS? . . . . . . . . . . . . . . . . . . 14
+ 5.2.1. Software/Firmware Complexity . . . . . . . . . . . . . . 15
+ 5.2.2. Macro Operation Complexity . . . . . . . . . . . . . . . 15
+ 5.2.3. Hardware Complexity. . . . . . . . . . . . . . . . . . . 15
+ 5.2.4. Power. . . . . . . . . . . . . . . . . . . . . . . . . . 16
+ 5.2.5. Density. . . . . . . . . . . . . . . . . . . . . . . . . 16
+ 5.2.6. Fixed versus variable costs. . . . . . . . . . . . . . . 16
+ 5.2.7. QoS. . . . . . . . . . . . . . . . . . . . . . . . . . . 17
+ 5.2.8. Flexibility. . . . . . . . . . . . . . . . . . . . . . . 17
+ 5.3. Relative Complexity . . . . . . . . . . . . . . . . . . . 17
+ 5.3.1. HBHI and the OPEX Challenge. . . . . . . . . . . . . . . 18
+ 6. The Myth of Over-Provisioning. . . . . . . . . . . . . . . . 18
+ 7. The Myth of Five Nines . . . . . . . . . . . . . . . . . . . 19
+ 8. Architectural Component Proportionality Law. . . . . . . . . 20
+ 8.1. Service Delivery Paths . . . . . . . . . . . . . . . . . . 21
+ 9. Conclusions. . . . . . . . . . . . . . . . . . . . . . . . . 21
+ 10. Security Considerations . . . . . . . . . . . . . . . . . . 22
+ 11. Acknowledgments . . . . . . . . . . . . . . . . . . . . . . 23
+ 12. References. . . . . . . . . . . . . . . . . . . . . . . . . 23
+ 13. Authors' Addresses. . . . . . . . . . . . . . . . . . . . . 27
+ 14. Full Copyright Statement. . . . . . . . . . . . . . . . . . 28
+
+1. Introduction
+
+ RFC 1958 [RFC1958] describes the underlying principles of the
+ Internet architecture. This note extends that work by outlining some
+ of the philosophical guidelines to which architects and designers of
+ Internet backbone networks should adhere. While many of the areas
+ outlined in this document may be controversial, the unifying
+ principle described here, controlling complexity as a mechanism to
+ control costs and reliability, should not be. Complexity in carrier
+ networks can derive from many sources. However, as stated in
+ [DOYLE2002], "Complexity in most systems is driven by the need for
+ robustness to uncertainty in their environments and component parts
+ far more than by basic functionality". The major thrust of this
+ document, then, is to raise awareness about the complexity of some of
+ our current architectures, and to examine the effect such complexity
+ will almost certainly have on the IP carrier industry's ability to
+ succeed.
+
+ The rest of this document is organized as follows: The first section
+ describes the Simplicity Principle and its implications for the
+ design of very large systems. The remainder of the document outlines
+ the high-level consequences of the Simplicity Principle and how it
+ should guide large scale network architecture and design approaches.
+
+
+
+
+Bush, et. al. Informational [Page 2]
+
+RFC 3439 Internet Architectural Guidelines December 2002
+
+
+2. Large Systems and The Simplicity Principle
+
+ The Simplicity Principle, which was perhaps first articulated by Mike
+ O'Dell, former Chief Architect at UUNET, states that complexity is
+ the primary mechanism which impedes efficient scaling, and as a
+ result is the primary driver of increases in both capital
+ expenditures (CAPEX) and operational expenditures (OPEX). The
+ implication for carrier IP networks then, is that to be successful we
+ must drive our architectures and designs toward the simplest possible
+ solutions.
+
+2.1. The End-to-End Argument and Simplicity
+
+ The end-to-end argument, which is described in [SALTZER] (as well as
+ in RFC 1958 [RFC1958]), contends that "end-to-end protocol design
+ should not rely on the maintenance of state (i.e., information about
+ the state of the end-to-end communication) inside the network. Such
+ state should be maintained only in the end points, in such a way that
+ the state can only be destroyed when the end point itself breaks."
+ This property has also been related to Clark's "fate-sharing" concept
+ [CLARK]. We can see that the end-to-end principle leads directly to
+ the Simplicity Principle by examining the so-called "hourglass"
+ formulation of the Internet architecture [WILLINGER2002]. In this
+ model, the thin waist of the hourglass is envisioned as the
+ (minimalist) IP layer, and any additional complexity is added above
+ the IP layer. In short, the complexity of the Internet belongs at
+ the edges, and the IP layer of the Internet should remain as simple
+ as possible.
+
+ Finally, note that the End-to-End Argument does not imply that the
+ core of the Internet will not contain and maintain state. In fact, a
+ huge amount coarse grained state is maintained in the Internet's core
+ (e.g., routing state). However, the important point here is that
+ this (coarse grained) state is almost orthogonal to the state
+ maintained by the end-points (e.g., hosts). It is this minimization
+ of interaction that contributes to simplicity. As a result,
+ consideration of "core vs. end-point" state interaction is crucial
+ when analyzing protocols such as Network Address Translation (NAT),
+ which reduce the transparency between network and hosts.
+
+2.2. Non-linearity and Network Complexity
+
+ Complex architectures and designs have been (and continue to be)
+ among the most significant and challenging barriers to building cost-
+ effective large scale IP networks. Consider, for example, the task
+ of building a large scale packet network. Industry experience has
+ shown that building such a network is a different activity (and hence
+ requires a different skill set) than building a small to medium scale
+
+
+
+Bush, et. al. Informational [Page 3]
+
+RFC 3439 Internet Architectural Guidelines December 2002
+
+
+ network, and as such doesn't have the same properties. In
+ particular, the largest networks exhibit, both in theory and in
+ practice, architecture, design, and engineering non-linearities which
+ are not exhibited at smaller scale. We call this Architecture,
+ Design, and Engineering (ADE) non-linearity. That is, systems such
+ as the Internet could be described as highly self-dissimilar, with
+ extremely different scales and levels of abstraction [CARLSON]. The
+ ADE non-linearity property is based upon two well-known principles
+ from non-linear systems theory [THOMPSON]:
+
+2.2.1. The Amplification Principle
+
+ The Amplification Principle states that there are non-linearities
+ which occur at large scale which do not occur at small to medium
+ scale.
+
+ COROLLARY: In many large networks, even small things can and do cause
+ huge events. In system-theoretic terms, in large systems such as
+ these, even small perturbations on the input to a process can
+ destabilize the system's output.
+
+ An important example of the Amplification Principle is non-linear
+ resonant amplification, which is a powerful process that can
+ transform dynamic systems, such as large networks, in surprising ways
+ with seemingly small fluctuations. These small fluctuations may
+ slowly accumulate, and if they are synchronized with other cycles,
+ may produce major changes. Resonant phenomena are examples of non-
+ linear behavior where small fluctuations may be amplified and have
+ influences far exceeding their initial sizes. The natural world is
+ filled with examples of resonant behavior that can produce system-
+ wide changes, such as the destruction of the Tacoma Narrows bridge
+ (due to the resonant amplification of small gusts of wind). Other
+ examples include the gaps in the asteroid belts and rings of Saturn
+ which are created by non-linear resonant amplification. Some
+ features of human behavior and most pilgrimage systems are influenced
+ by resonant phenomena involving the dynamics of the solar system,
+ such as solar days, the 27.3 day (sidereal) and 29.5 day (synodic)
+ cycles of the moon or the 365.25 day cycle of the sun.
+
+ In the Internet domain, it has been shown that increased inter-
+ connectivity results in more complex and often slower BGP routing
+ convergence [AHUJA]. A related result is that a small amount of
+ inter-connectivity causes the output of a routing mesh to be
+ significantly more complex than its input [GRIFFIN]. An important
+ method for reducing amplification is ensure that local changes have
+ only local effect (this is as opposed to systems in which local
+ changes have global effect). Finally, ATM provides an excellent
+ example of an amplification effect: if you lose one cell, you destroy
+
+
+
+Bush, et. al. Informational [Page 4]
+
+RFC 3439 Internet Architectural Guidelines December 2002
+
+
+ the entire packet (and it gets worse, as in the absence of mechanisms
+ such as Early Packet Discard [ROMANOV], you will continue to carry
+ the already damaged packet).
+
+ Another interesting example of amplification comes from the
+ engineering domain, and is described in [CARLSON]. They consider the
+ Boeing 777, which is a "fly-by-wire" aircraft, containing as many as
+ 150,000 subsystems and approximately 1000 CPUs. What they observe is
+ that while the 777 is robust to large-scale atmospheric disturbances,
+ turbulence boundaries, and variations in cargo loads (to name a few),
+ it could be catastrophically disabled my microscopic alterations in a
+ very few large CPUs (as the point out, fortunately this is a very
+ rare occurrence). This example illustrates the issue "that
+ complexity can amplify small perturbations, and the design engineer
+ must ensure such perturbations are extremely rare." [CARLSON]
+
+2.2.2. The Coupling Principle
+
+ The Coupling Principle states that as things get larger, they often
+ exhibit increased interdependence between components.
+
+ COROLLARY: The more events that simultaneously occur, the larger the
+ likelihood that two or more will interact. This phenomenon has also
+ been termed "unforeseen feature interaction" [WILLINGER2002].
+
+ Much of the non-linearity observed large systems is largely due to
+ coupling. This coupling has both horizontal and vertical
+ components. In the context of networking, horizontal coupling is
+ exhibited between the same protocol layer, while vertical coupling
+ occurs between layers.
+
+ Coupling is exhibited by a wide variety of natural systems, including
+ plasma macro-instabilities (hydro-magnetic, e.g., kink, fire-hose,
+ mirror, ballooning, tearing, trapped-particle effects) [NAVE], as
+ well as various kinds of electrochemical systems (consider the custom
+ fluorescent nucleotide synthesis/nucleic acid labeling problem
+ [WARD]). Coupling of clock physical periodicity has also been
+ observed [JACOBSON], as well as coupling of various types of
+ biological cycles.
+
+ Several canonical examples also exist in well known network systems.
+ Examples include the synchronization of various control loops, such
+ as routing update synchronization and TCP Slow Start synchronization
+ [FLOYD,JACOBSON]. An important result of these observations is that
+ coupling is intimately related to synchronization. Injecting
+ randomness into these systems is one way to reduce coupling.
+
+
+
+
+
+Bush, et. al. Informational [Page 5]
+
+RFC 3439 Internet Architectural Guidelines December 2002
+
+
+ Interestingly, in analyzing risk factors for the Public Switched
+ Telephone Network (PSTN), Charles Perrow decomposes the complexity
+ problem along two related axes, which he terms "interactions" and
+ "coupling" [PERROW]. Perrow cites interactions and coupling as
+ significant factors in determining the reliability of a complex
+ system (and in particular, the PSTN). In this model, interactions
+ refer to the dependencies between components (linear or non-linear),
+ while coupling refers to the flexibility in a system. Systems with
+ simple, linear interactions have components that affect only other
+ components that are functionally downstream. Complex system
+ components interact with many other components in different and
+ possibly distant parts of the system. Loosely coupled systems are
+ said to have more flexibility in time constraints, sequencing, and
+ environmental assumptions than do tightly coupled systems. In
+ addition, systems with complex interactions and tight coupling are
+ likely to have unforeseen failure states (of course, complex
+ interactions permit more complications to develop and make the system
+ hard to understand and predict); this behavior is also described in
+ [WILLINGER2002]. Tight coupling also means that the system has less
+ flexibility in recovering from failure states.
+
+ The PSTN's SS7 control network provides an interesting example of
+ what can go wrong with a tightly coupled complex system. Outages
+ such as the well publicized 1991 outage of AT&T's SS7 demonstrates
+ the phenomenon: the outage was caused by software bugs in the
+ switches' crash recovery code. In this case, one switch crashed due
+ to a hardware glitch. When this switch came back up, it (plus a
+ reasonably probable timing event) caused its neighbors to crash When
+ the neighboring switches came back up, they caused their neighbors to
+ crash, and so on [NEUMANN] (the root cause turned out to be a
+ misplaced 'break' statement; this is an excellent example of cross-
+ layer coupling). This phenomenon is similar to the phase-locking of
+ weakly coupled oscillators, in which random variations in sequence
+ times plays an important role in system stability [THOMPSON].
+
+2.3. Complexity lesson from voice
+
+ In the 1970s and 1980s, the voice carriers competed by adding
+ features which drove substantial increases in the complexity of the
+ PSTN, especially in the Class 5 switching infrastructure. This
+ complexity was typically software-based, not hardware driven, and
+ therefore had cost curves worse than Moore's Law. In summary, poor
+ margins on voice products today are due to OPEX and CAPEX costs not
+ dropping as we might expect from simple hardware-bound
+ implementations.
+
+
+
+
+
+
+Bush, et. al. Informational [Page 6]
+
+RFC 3439 Internet Architectural Guidelines December 2002
+
+
+2.4. Upgrade cost of complexity
+
+ Consider the cost of providing new features in a complex network.
+ The traditional voice network has little intelligence in its edge
+ devices (phone instruments), and a very smart core. The Internet has
+ smart edges, computers with operating systems, applications, etc.,
+ and a simple core, which consists of a control plane and packet
+ forwarding engines. Adding an new Internet service is just a matter
+ of distributing an application to the a few consenting desktops who
+ wish to use it. Compare this to adding a service to voice, where one
+ has to upgrade the entire core.
+
+3. Layering Considered Harmful
+
+ There are several generic properties of layering, or vertical
+ integration as applied to networking. In general, a layer as defined
+ in our context implements one or more of
+
+ Error Control: The layer makes the "channel" more reliable
+ (e.g., reliable transport layer)
+
+ Flow Control: The layer avoids flooding slower peer (e.g.,
+ ATM flow control)
+
+ Fragmentation: Dividing large data chunks into smaller
+ pieces, and subsequent reassembly (e.g., TCP
+ MSS fragmentation/reassembly)
+
+ Multiplexing: Allow several higher level sessions share
+ single lower level "connection" (e.g., ATM PVC)
+
+ Connection Setup: Handshaking with peer (e.g., TCP three-way
+ handshake, ATM ILMI)
+
+ Addressing/Naming: Locating, managing identifiers associated
+ with entities (e.g., GOSSIP 2 NSAP Structure
+ [RFC1629])
+
+ Layering of this type does have various conceptual and structuring
+ advantages. However, in the data networking context structured
+ layering implies that the functions of each layer are carried out
+ completely before the protocol data unit is passed to the next layer.
+ This means that the optimization of each layer has to be done
+ separately. Such ordering constraints are in conflict with efficient
+ implementation of data manipulation functions. One could accuse the
+ layered model (e.g., TCP/IP and ISO OSI) of causing this conflict.
+ In fact, the operations of multiplexing and segmentation both hide
+ vital information that lower layers may need to optimize their
+
+
+
+Bush, et. al. Informational [Page 7]
+
+RFC 3439 Internet Architectural Guidelines December 2002
+
+
+ performance. For example, layer N may duplicate lower level
+ functionality, e.g., error recovery hop-hop versus end-to-end error
+ recovery. In addition, different layers may need the same
+ information (e.g., time stamp): layer N may need layer N-2
+ information (e.g., lower layer packet sizes), and the like [WAKEMAN].
+ A related and even more ironic statement comes from Tennenhouse's
+ classic paper, "Layered Multiplexing Considered Harmful"
+ [TENNENHOUSE]: "The ATM approach to broadband networking is presently
+ being pursued within the CCITT (and elsewhere) as the unifying
+ mechanism for the support of service integration, rate adaptation,
+ and jitter control within the lower layers of the network
+ architecture. This position paper is specifically concerned with the
+ jitter arising from the design of the "middle" and "upper" layers
+ that operate within the end systems and relays of multi-service
+ networks (MSNs)."
+
+ As a result of inter-layer dependencies, increased layering can
+ quickly lead to violation of the Simplicity Principle. Industry
+ experience has taught us that increased layering frequently increases
+ complexity and hence leads to increases in OPEX, as is predicted by
+ the Simplicity Principle. A corollary is stated in RFC 1925
+ [RFC1925], section 2(5):
+
+ "It is always possible to agglutinate multiple separate problems
+ into a single complex interdependent solution. In most cases
+ this is a bad idea."
+
+ The first order conclusion then, is that horizontal (as opposed to
+ vertical) separation may be more cost-effective and reliable in the
+ long term.
+
+3.1. Optimization Considered Harmful
+
+ A corollary of the layering arguments above is that optimization can
+ also be considered harmful. In particular, optimization introduces
+ complexity, and as well as introducing tighter coupling between
+ components and layers.
+
+ An important and related effect of optimization is described by the
+ Law of Diminishing Returns, which states that if one factor of
+ production is increased while the others remain constant, the overall
+ returns will relatively decrease after a certain point [SPILLMAN].
+ The implication here is that trying to squeeze out efficiency past
+ that point only adds complexity, and hence leads to less reliable
+ systems.
+
+
+
+
+
+
+Bush, et. al. Informational [Page 8]
+
+RFC 3439 Internet Architectural Guidelines December 2002
+
+
+3.2. Feature Richness Considered Harmful
+
+ While adding any new feature may be considered a gain (and in fact
+ frequently differentiates vendors of various types of equipment), but
+ there is a danger. The danger is in increased system complexity.
+
+3.3. Evolution of Transport Efficiency for IP
+
+ The evolution of transport infrastructures for IP offers a good
+ example of how decreasing vertical integration has lead to various
+ efficiencies. In particular,
+
+ | IP over ATM over SONET -->
+ | IP over SONET over WDM -->
+ | IP over WDM
+ |
+ \|/
+ Decreasing complexity, CAPEX, OPEX
+
+ The key point here is that layers are removed resulting in CAPEX and
+ OPEX efficiencies.
+
+3.4. Convergence Layering
+
+ Convergence is related to the layering concepts described above in
+ that convergence is achieved via a "convergence layer". The end
+ state of the convergence argument is the concept of Everything Over
+ Some Layer (EOSL). Conduit, DWDM, fiber, ATM, MPLS, and even IP have
+ all been proposed as convergence layers. It is important to note
+ that since layering typically drives OPEX up, we expect convergence
+ will as well. This observation is again consistent with industry
+ experience.
+
+ There are many notable examples of convergence layer failure.
+ Perhaps the most germane example is IP over ATM. The immediate and
+ most obvious consequence of ATM layering is the so-called cell tax:
+ First, note that the complete answer on ATM efficiency is that it
+ depends upon packet size distributions. Let's assume that typical
+ Internet type traffic patterns, which tend to have high percentages
+ of packets at 40, 44, and 552 bytes. Recent data [CAIDA] shows that
+ about 95% of WAN bytes and 85% of packets are TCP. Much of this
+ traffic is composed of 40/44 byte packets.
+
+ Now, consider the case of a a DS3 backbone with PLCP turned on. Then
+ the maximum cell rate is 96,000 cells/sec. If you multiply this
+ value by the number of bits in the payload, you get: 96000 cells/sec
+ * 48 bytes/cell * 8 = 36.864 Mbps. This, however, is unrealistic
+ since it
+
+
+
+Bush, et. al. Informational [Page 9]
+
+RFC 3439 Internet Architectural Guidelines December 2002
+
+
+ assumes perfect payload packing. There are two other things that
+ contribute to the ATM overhead (cell tax): The wasted padding and the
+ 8 byte SNAP header.
+
+ It is the SNAP header which causes most of the problems (and you
+ can't do anything about this), forcing most small packets to consume
+ two cells, with the second cell to be mostly empty padding (this
+ interacts really poorly with the data quoted above, e.g., that most
+ packets are 40-44 byte TCP Ack packets). This causes a loss of about
+ another 16% from the 36.8 Mbps ideal throughput.
+
+ So the total throughput ends up being (for a DS3):
+
+ DS3 Line Rate: 44.736
+ PLCP Overhead - 4.032
+ Per Cell Header: - 3.840
+ SNAP Header & Padding: - 5.900
+ =========
+ 30.960 Mbps
+
+ Result: With a DS3 line rate of 44.736 Mbps, the total overhead is
+ about 31%.
+
+ Another way to look at this is that since a large fraction of WAN
+ traffic is comprised of TCP ACKs, one can make a different but
+ related calculation. IP over ATM requires:
+
+ IP data (40 bytes in this case)
+ 8 bytes SNAP
+ 8 bytes AAL5 stuff
+ 5 bytes for each cell
+ + as much more as it takes to fill out the last cell
+
+ On ATM, this becomes two cells - 106 bytes to convey 40 bytes of
+ information. The next most common size seems to be one of several
+ sizes in the 504-556 byte range - 636 bytes to carry IP, TCP, and a
+ 512 byte TCP payload - with messages larger than 1000 bytes running
+ third.
+
+ One would imagine that 87% payload (556 byte message size) is better
+ than 37% payload (TCP Ack size), but it's not the 95-98% that
+ customers are used to, and the predominance of TCP Acks skews the
+ average.
+
+
+
+
+
+
+
+
+Bush, et. al. Informational [Page 10]
+
+RFC 3439 Internet Architectural Guidelines December 2002
+
+
+3.4.1. Note on Transport Protocol Layering
+
+ Protocol layering models are frequently cast as "X over Y" models.
+ In these cases, protocol Y carries protocol X's protocol data units
+ (and possibly control data) over Y's data plane, i.e., Y is a
+ "convergence layer". Examples include Frame Relay over ATM, IP over
+ ATM, and IP over MPLS. While X over Y layering has met with only
+ marginal success [TENNENHOUSE,WAKEMAN], there have been a few notable
+ instances where efficiency can be and is gained. In particular, "X
+ over Y efficiencies" can be realized when there is a kind of
+ "isomorphism" between the X and Y (i.e., there is a small convergence
+ layer). In these cases X's data, and possibly control traffic, are
+ "encapsulated" and transported over Y. Examples include Frame Relay
+ over ATM, and Frame Relay, AAL5 ATM and Ethernet over L2TPv3
+ [L2TPV3]; the simplifying factors here are that there is no
+ requirement that a shared clock be recovered by the communicating end
+ points, and that control-plane interworking is minimized. An
+ alternative is to interwork the X and Y's control and data planes;
+ control-plane interworking is discussed below.
+
+3.5. Second Order Effects
+
+ IP over ATM provides an excellent example of unanticipated second
+ order effects. In particular, Romanov and Floyd's classic study on
+ TCP good-put [ROMANOV] on ATM showed that large UBR buffers (larger
+ than one TCP window size) are required to achieve reasonable
+ performance, that packet discard mechanisms (such as Early Packet
+ Discard, or EPD) improve the effective usage of the bandwidth and
+ that more elaborate service and drop strategies than FIFO+EPD, such
+ as per VC queuing and accounting, might be required at the bottleneck
+ to ensure both high efficiency and fairness. Though all studies
+ clearly indicate that a buffer size not less than one TCP window size
+ is required, the amount of extra buffer required naturally depends on
+ the packet discard mechanism used and is still an open issue.
+
+ Examples of this kind of problem with layering abound in practical
+ networking. Consider, for example, the effect of IP transport's
+ implicit assumptions of lower layers. In particular:
+
+ o Packet loss: TCP assumes that packet losses are indications of
+ congestion, but sometimes losses are from corruption on a wireless
+ link [RFC3115].
+
+ o Reordered packets: TCP assumes that significantly reordered
+ packets are indications of congestion. This is not always the
+ case [FLOYD2001].
+
+
+
+
+
+Bush, et. al. Informational [Page 11]
+
+RFC 3439 Internet Architectural Guidelines December 2002
+
+
+ o Round-trip times: TCP measures round-trip times, and assumes that
+ the lack of an acknowledgment within a period of time based on the
+ measured round-trip time is a packet loss, and therefore an
+ indication of congestion [KARN].
+
+ o Congestion control: TCP congestion control implicitly assumes that
+ all the packets in a flow are treated the same by the network, but
+ this is not always the case [HANDLEY].
+
+3.6. Instantiating the EOSL Model with IP
+
+ While IP is being proposed as a transport for almost everything, the
+ base assumption, that Everything over IP (EOIP) will result in OPEX
+ and CAPEX efficiencies, requires critical examination. In
+ particular, while it is the case that many protocols can be
+ efficiently transported over an IP network (specifically, those
+ protocols that do not need to recover synchronization between the
+ communication end points, such as Frame Relay, Ethernet, and AAL5
+ ATM), the Simplicity and Layering Principles suggest that EOIP may
+ not represent the most efficient convergence strategy for arbitrary
+ services. Rather, a more CAPEX and OPEX efficient convergence layer
+ might be much lower (again, this behavior is predicted by the
+ Simplicity Principle).
+
+ An example of where EOIP would not be the most OPEX and CAPEX
+ efficient transport would be in those cases where a service or
+ protocol needed SONET-like restoration times (e.g., 50ms). It is not
+ hard to imagine that it would cost more to build and operate an IP
+ network with this kind of restoration and convergence property (if
+ that were even possible) than it would to build the SONET network in
+ the first place.
+
+4. Avoid the Universal Interworking Function
+
+ While there have been many implementations of Universal Interworking
+ unction (UIWF), IWF approaches have been problematic at large scale.
+ his concern is codified in the Principle of Minimum Intervention
+ BRYANT]:
+
+ "To minimise the scope of information, and to improve the efficiency
+ of data flow through the Encapsulation Layer, the payload should,
+ where possible, be transported as received without modification."
+
+
+
+
+
+
+
+
+
+Bush, et. al. Informational [Page 12]
+
+RFC 3439 Internet Architectural Guidelines December 2002
+
+
+4.1. Avoid Control Plane Interworking
+
+ This corollary is best understood in the context of the integrated
+ solutions space. In this case, the architecture and design
+ frequently achieves the worst of all possible worlds. This is due to
+ the fact that such integrated solutions perform poorly at both ends
+ of the performance/CAPEX/OPEX spectrum: the protocols with the least
+ switching demand may have to bear the cost of the most expensive,
+ while the protocols with the most stringent requirements often must
+ make concessions to those with different requirements. Add to this
+ the various control plane interworking issues and you have a large
+ opportunity for failure. In summary, interworking functions should
+ be restricted to data plane interworking and encapsulations, and
+ these functions should be carried out at the edge of the network.
+
+ As described above, interworking models have been successful in those
+ cases where there is a kind of "isomorphism" between the layers being
+ interworked. The trade-off here, frequently described as the
+ "Integrated vs. Ships In the Night trade-off" has been examined at
+ various times and at various protocol layers. In general, there are
+ few cases in which such integrated solutions have proven efficient.
+ Multi-protocol BGP [RFC2283] is a subtly different but notable
+ exception. In this case, the control plane is independent of the
+ format of the control data. That is, no control plane data
+ conversion is required, in contrast with control plane interworking
+ models such as the ATM/IP interworking envisioned by some soft-switch
+ manufacturers, and the so-called "PNNI-MPLS SIN" interworking
+ [ATMMPLS].
+
+5. Packet versus Circuit Switching: Fundamental Differences
+
+ Conventional wisdom holds that packet switching (PS) is inherently
+ more efficient than circuit switching (CS), primarily because of the
+ efficiencies that can be gained by statistical multiplexing and the
+ fact that routing and forwarding decisions are made independently in
+ a hop-by-hop fashion [[MOLINERO2002]. Further, it is widely assumed
+ that IP is simpler that circuit switching, and hence should be more
+ economical to deploy and manage [MCK2002]. However, if one examines
+ these and related assumptions, a different picture emerges (see for
+ example [ODLYZKO98]). The following sections discuss these
+ assumptions.
+
+5.1. Is PS is inherently more efficient than CS?
+
+ It is well known that packet switches make efficient use of scarce
+ bandwidth [BARAN]. This efficiency is based on the statistical
+ multiplexing inherent in packet switching. However, we continue to
+ be puzzled by what is generally believed to be the low utilization of
+
+
+
+Bush, et. al. Informational [Page 13]
+
+RFC 3439 Internet Architectural Guidelines December 2002
+
+
+ Internet backbones. The first question we might ask is what is the
+ current average utilization of Internet backbones, and how does that
+ relate to the utilization of long distance voice networks? Odlyzko
+ and Coffman [ODLYZKO,COFFMAN] report that the average utilization of
+ links in the IP networks was in the range between 3% and 20%
+ (corporate intranets run in the 3% range, while commercial Internet
+ backbones run in the 15-20% range). On the other hand, the average
+ utilization of long haul voice lines is about 33%. In addition, for
+ 2002, the average utilization of optical networks (all services)
+ appears to be hovering at about 11%, while the historical average is
+ approximately 15% [ML2002]. The question then becomes why we see
+ such utilization levels, especially in light of the assumption that
+ PS is inherently more efficient than CS. The reasons cited by
+ Odlyzko and Coffman include:
+
+ (i). Internet traffic is extremely asymmetric and bursty, but
+ links are symmetric and of fixed capacity (i.e., don't know
+ the traffic matrix, or required link capacities);
+
+ (ii). It is difficult to predict traffic growth on a link, so
+ operators tend to add bandwidth aggressively;
+
+ (iii). Falling prices for coarser bandwidth granularity make it
+ appear more economical to add capacity in large increments.
+
+ Other static factors include protocol overhead, other kinds of
+ equipment granularity, restoration capacity, and provisioning lag
+ time all contribute to the need to "over-provision" [MC2001].
+
+5.2. Is PS simpler than CS?
+
+ The end-to-end principle can be interpreted as stating that the
+ complexity of the Internet belongs at the edges. However, today's
+ Internet backbone routers are extremely complex. Further, this
+ complexity scales with line rate. Since the relative complexity of
+ circuit and packet switching seems to have resisted direct analysis,
+ we instead examine several artifacts of packet and circuit switching
+ as complexity metrics. Among the metrics we might look at are
+ software complexity, macro operation complexity, hardware complexity,
+ power consumption, and density. Each of these metrics is considered
+ below.
+
+
+
+
+
+
+
+
+
+
+Bush, et. al. Informational [Page 14]
+
+RFC 3439 Internet Architectural Guidelines December 2002
+
+
+5.2.1. Software/Firmware Complexity
+
+ One measure of software/firmware complexity is the number of
+ instructions required to program the device. The typical software
+ image for an Internet router requires between eight and ten million
+ instructions (including firmware), whereas a typical transport switch
+ requires on average about three million instructions [MCK2002].
+
+ This difference in software complexity has tended to make Internet
+ routers unreliable, and has notable other second order effects (e.g.,
+ it may take a long time to reboot such a router). As another point
+ of comparison, consider that the AT&T (Lucent) 5ESS class 5 switch,
+ which has a huge number of calling features, requires only about
+ twice the number of lines of code as an Internet core router [EICK].
+
+ Finally, since routers are as much or more software than hardware
+ devices, another result of the code complexity is that the cost of
+ routers benefits less from Moore's Law than less software-intensive
+ devices. This causes a bandwidth/device trade-off that favors
+ bandwidth more than less software-intensive devices.
+
+5.2.2. Macro Operation Complexity
+
+ An Internet router's line card must perform many complex operations,
+ including processing the packet header, longest prefix match,
+ generating ICMP error messages, processing IP header options, and
+ buffering the packet so that TCP congestion control will be effective
+ (this typically requires a buffer of size proportional to the line
+ rate times the RTT, so a buffer will hold around 250 ms of packet
+ data). This doesn't include route and packet filtering, or any QoS
+ or VPN filtering.
+
+ On the other hand, a transport switch need only to map ingress time-
+ slots to egress time-slots and interfaces, and therefore can be
+ considerably less complex.
+
+5.2.3. Hardware Complexity
+
+ One measure of hardware complexity is the number of logic gates on a
+ line card [MOLINERO2002]. Consider the case of a high-speed Internet
+ router line card: An OC192 POS router line card contains at least 30
+ million gates in ASICs, at least one CPU, 300 Mbytes of packet
+ buffers, 2 Mbytes of forwarding table, and 10 Mbytes of other
+
+
+
+
+
+
+
+
+Bush, et. al. Informational [Page 15]
+
+RFC 3439 Internet Architectural Guidelines December 2002
+
+
+ state memory. On the other hand, a comparable transport switch line
+ card has 7.5 million logic gates, no CPU, no packet buffer, no
+ forwarding table, and an on-chip state memory. Rather, the line-card
+ of an electronic transport switch typically contains a SONET framer,
+ a chip to map ingress time-slots to egress time-slots, and an
+ interface to the switch fabric.
+
+5.2.4. Power
+
+ Since transport switches have traditionally been built from simpler
+ hardware components, they also consume less power [PMC].
+
+5.2.5. Density
+
+ The highest capacity transport switches have about four times the
+ capacity of an IP router [CISCO,CIENA], and sell for about one-third
+ as much per Gigabit/sec. Optical (OOO) technology pushes this
+ complexity difference further (e.g., tunable lasers, MEMs switches.
+ e.g., [CALIENT]), and DWDM multiplexers provide technology to build
+ extremely high capacity, low power transport switches.
+
+ A related metric is physical footprint. In general, by virtue of
+ their higher density, transport switches have a smaller "per-gigabit"
+ physical footprint.
+
+5.2.6. Fixed versus variable costs
+
+ Packet switching would seem to have high variable cost, meaning that
+ it costs more to send the n-th piece of information using packet
+ switching than it might in a circuit switched network. Much of this
+ advantage is due to the relatively static nature of circuit
+ switching, e.g., circuit switching can take advantage of of pre-
+ scheduled arrival of information to eliminate operations to be
+ performed on incoming information. For example, in the circuit
+ switched case, there is no need to buffer incoming information,
+ perform loop detection, resolve next hops, modify fields in the
+ packet header, and the like. Finally, many circuit switched networks
+ combine relatively static configuration with out-of-band control
+ planes (e.g., SS7), which greatly simplifies data-plane switching.
+ The bottom line is that as data rates get large, it becomes more and
+ more complex to switch packets, while circuit switching scales more
+ or less linearly.
+
+
+
+
+
+
+
+
+
+Bush, et. al. Informational [Page 16]
+
+RFC 3439 Internet Architectural Guidelines December 2002
+
+
+5.2.7. QoS
+
+ While the components of a complete solution for Internet QoS,
+ including call admission control, efficient packet classification,
+ and scheduling algorithms, have been the subject of extensive
+ research and standardization for more than 10 years, end-to-end
+ signaled QoS for the Internet has not become a reality.
+ Alternatively, QoS has been part of the circuit switched
+ infrastructure almost from its inception. On the other hand, QoS is
+ usually deployed to determine queuing disciplines to be used when
+ there is insufficient bandwidth to support traffic. But unlike voice
+ traffic, packet drop or severe delay may have a much more serious
+ effect on TCP traffic due to its congestion-aware feedback loop (in
+ particular, TCP backoff/slow start).
+
+5.2.8. Flexibility
+
+ A somewhat harder to quantify metric is the inherent flexibility of
+ the Internet. While the Internet's flexibility has led to its rapid
+ growth, this flexibility comes with a relatively high cost at the
+ edge: the need for highly trained support personnel. A standard rule
+ of thumb is that in an enterprise setting, a single support person
+ suffices to provide telephone service for a group, while you need ten
+ computer networking experts to serve the networking requirements of
+ the same group [ODLYZKO98A]. This phenomenon is also described in
+ [PERROW].
+
+5.3. Relative Complexity
+
+ The relative computational complexity of circuit switching as
+ compared to packet switching has been difficult to describe in formal
+ terms [PARK]. As such, the sections above seek to describe the
+ complexity in terms of observable artifacts. With this in mind, it
+ is clear that the fundamental driver producing the increased
+ complexities outlined above is the hop-by-hop independence (HBHI)
+ inherent in the IP architecture. This is in contrast to the end to
+ end architectures such as ATM or Frame Relay.
+
+ [WILLINGER2002] describes this phenomenon in terms of the robustness
+ requirement of the original Internet design, and how this requirement
+ has the driven complexity of the network. In particular, they
+ describe a "complexity/robustness" spiral, in which increases in
+ complexity create further and more serious sensitivities, which then
+ requires additional robustness (hence the spiral).
+
+
+
+
+
+
+
+Bush, et. al. Informational [Page 17]
+
+RFC 3439 Internet Architectural Guidelines December 2002
+
+
+ The important lesson of this section is that the Simplicity
+ Principle, while applicable to circuit switching as well as packet
+ switching, is crucial in controlling the complexity (and hence OPEX
+ and CAPEX properties) of packet networks. This idea is reinforced by
+ the observation that while packet switching is a younger, less mature
+ discipline than circuit switching, the trend in packet switches is
+ toward more complex line cards, while the complexity of circuit
+ switches appears to be scaling linearly with line rates and aggregate
+ capacity.
+
+5.3.1. HBHI and the OPEX Challenge
+
+ As a result of HBHI, we need to approach IP networks in a
+ fundamentally different way than we do circuit based networks. In
+ particular, the major OPEX challenge faced by the IP network is that
+ debugging of a large-scale IP network still requires a large degree
+ of expertise and understanding, again due to the hop-by-hop
+ independence inherent in a packet architecture (again, note that this
+ hop-by-hop independence is not present in virtual circuit networks
+ such as ATM or Frame Relay). For example, you may have to visit a
+ large set of your routers only to discover that the problem is
+ external to your own network. Further, the debugging tools used to
+ diagnose problems are also complex and somewhat primitive. Finally,
+ IP has to deal with people having problems with their DNS or their
+ mail or news or some new application, whereas this is usually not the
+ case for TDM/ATM/etc. In the case of IP, this can be eased by
+ improving automation (note that much of what we mention is customer
+ facing). In general, there are many variables external to the
+ network that effect OPEX.
+
+ Finally, it is important to note that the quantitative relationship
+ between CAPEX, OPEX, and a network's inherent complexity is not well
+ understood. In fact, there are no agreed upon and quantitative
+ metrics for describing a network's complexity, so a precise
+ relationship between CAPEX, OPEX, and complexity remains elusive.
+
+6. The Myth of Over-Provisioning
+
+ As noted in [MC2001] and elsewhere, much of the complexity we observe
+ in today's Internet is directed at increasing bandwidth utilization.
+ As a result, the desire of network engineers to keep network
+ utilization below 50% has been termed "over-provisioning". However,
+ this use of the term over-provisioning is a misnomer. Rather, in
+ modern Internet backbones the unused capacity is actually protection
+ capacity. In particular, one might view this as "1:1 protection at
+ the IP layer". Viewed in this way, we see that an IP network
+ provisioned to run at 50% utilization is no more over-provisioned
+ than the typical SONET network. However, the important advantages
+
+
+
+Bush, et. al. Informational [Page 18]
+
+RFC 3439 Internet Architectural Guidelines December 2002
+
+
+ that accrue to an IP network provisioned in this way include close to
+ speed of light delay and close to zero packet loss [FRALEIGH]. These
+ benefits can been seen as a "side-effect" of 1:1 protection
+ provisioning.
+
+ There are also other, system-theoretic reasons for providing 1:1-like
+ protection provisioning. Most notable among these reasons is that
+ packet-switched networks with in-band control loops can become
+ unstable and can experience oscillations and synchronization when
+ congested. Complex and non-linear dynamic interaction of traffic
+ means that congestion in one part of the network will spread to other
+ parts of the network. When routing protocol packets are lost due to
+ congestion or route-processor overload, it causes inconsistent
+ routing state, and this may result in traffic loops, black holes, and
+ lost connectivity. Thus, while statistical multiplexing can in
+ theory yield higher network utilization, in practice, to maintain
+ consistent performance and a reasonably stable network, the dynamics
+ of the Internet backbones favor 1:1 provisioning and its side effects
+ to keep the network stable and delay low.
+
+7. The Myth of Five Nines
+
+ Paul Baran, in his classic paper, "SOME PERSPECTIVES ON NETWORKS--
+ PAST, PRESENT AND FUTURE", stated that "The tradeoff curves between
+ cost and system reliability suggest that the most reliable systems
+ might be built of relatively unreliable and hence low cost elements,
+ if it is system reliability at the lowest overall system cost that is
+ at issue" [BARAN77].
+
+ Today we refer to this phenomenon as "the myth of five nines".
+ Specifically, so-called five nines reliability in packet network
+ elements is consider a myth for the following reasons: First, since
+ 80% of unscheduled outages are caused by people or process errors
+ [SCOTT], there is only a 20% window in which to optimize. Thus, in
+ order to increase component reliability, we add complexity
+ (optimization frequently leads to complexity), which is the root
+ cause of 80% of the unplanned outages. This effectively narrows the
+ 20% window (i.e., you increase the likelihood of people and process
+ failure). This phenomenon is also characterized as a
+ "complexity/robustness" spiral [WILLINGER2002], in which increases in
+ complexity create further and more serious sensitivities, which then
+ requires additional robustness, and so on (hence the spiral).
+
+ The conclusion, then is that while a system like the Internet can
+ reach five-nines-like reliability, it is undesirable (and likely
+ impossible) to try to make any individual component, especially the
+ most complex ones, reach that reliability standard.
+
+
+
+
+Bush, et. al. Informational [Page 19]
+
+RFC 3439 Internet Architectural Guidelines December 2002
+
+
+8. Architectural Component Proportionality Law
+
+ As noted in the previous section, the computational complexity of
+ packet switched networks such as the Internet has proven difficult to
+ describe in formal terms. However, an intuitive, high level
+ definition of architectural complexity might be that the complexity
+ of an architecture is proportional to its number of components, and
+ that the probability of achieving a stable implementation of an
+ architecture is inversely proportional to its number of components.
+ As described above, components include discrete elements such as
+ hardware elements, space and power requirements, as well as software,
+ firmware, and the protocols they implement.
+
+ Stated more abstractly:
+
+ Let
+
+ A be a representation of architecture A,
+
+ |A| be number of distinct components in the service
+ delivery path of architecture A,
+
+ w be a monotonically increasing function,
+
+ P be the probability of a stable implementation of an
+ architecture, and let
+
+ Then
+
+ Complexity(A) = O(w(|A|))
+ P(A) = O(1/w(|A|))
+
+ where
+
+ O(f) = {g:N->R | there exists c > 0 and n such that g(n)
+ < c*f(n)}
+
+ [That is, O(f) comprises the set of functions g for which
+ there exists a constant c and a number n, such that g(n) is
+ smaller or equal to c*f(n) for all n. That is, O(f) is the
+ set of all functions that do not grow faster than f,
+ disregarding constant factors]
+
+ Interestingly, the Highly Optimized Tolerance (HOT) model [HOT]
+ attempts to characterize complexity in general terms (HOT is one
+ recent attempt to develop a general framework for the study of
+ complexity, and is a member of a family of abstractions generally
+ termed "the new science of complexity" or "complex adaptive
+
+
+
+Bush, et. al. Informational [Page 20]
+
+RFC 3439 Internet Architectural Guidelines December 2002
+
+
+ systems"). Tolerance, in HOT semantics, means that "robustness in
+ complex systems is a constrained and limited quantity that must be
+ carefully managed and protected." One focus of the HOT model is to
+ characterize heavy-tailed distributions such as Complexity(A) in the
+ above example (other examples include forest fires, power outages,
+ and Internet traffic distributions). In particular, Complexity(A)
+ attempts to map the extreme heterogeneity of the parts of the system
+ (Internet), and the effect of their organization into highly
+ structured networks, with hierarchies and multiple scales.
+
+8.1. Service Delivery Paths
+
+ The Architectural Component Proportionality Law (ACPL) states that
+ the complexity of an architecture is proportional to its number of
+ components.
+
+ COROLLARY: Minimize the number of components in a service delivery
+ path, where the service delivery path can be a protocol path, a
+ software path, or a physical path.
+
+ This corollary is an important consequence of the ACPL, as the path
+ between a customer and the desired service is particularly sensitive
+ to the number and complexity of elements in the path. This is due to
+ the fact that the complexity "smoothing" that we find at high levels
+ of aggregation [ZHANG] is missing as you move closer to the edge, as
+ well as having complex interactions with backoffice and CRM systems.
+ Examples of architectures that haven't found a market due to this
+ effect include TINA-based CRM systems, CORBA/TINA based service
+ architectures. The basic lesson here was that the only possibilities
+ for deploying these systems were "Limited scale deployments (such) as
+ in Starvision can avoid coping with major unproven scalability
+ issues", or "Otherwise need massive investments (like the carrier-
+ grade ORB built almost from scratch)" [TINA]. In other words, these
+ systems had complex service delivery paths, and were too complex to
+ be feasibly deployed.
+
+9. Conclusions
+
+ This document attempts to codify long-understood Internet
+ architectural principles. In particular, the unifying principle
+ described here is best expressed by the Simplicity Principle, which
+ states complexity must be controlled if one hopes to efficiently
+ scale a complex object. The idea that simplicity itself can lead to
+ some form of optimality has been a common theme throughout history,
+ and has been stated in many other ways and along many dimensions.
+ For example, consider the maxim known as Occam's Razor, which was
+ formulated by the medieval English philosopher and Franciscan monk
+ William of Ockham (ca. 1285-1349), and states "Pluralitas non est
+
+
+
+Bush, et. al. Informational [Page 21]
+
+RFC 3439 Internet Architectural Guidelines December 2002
+
+
+ ponenda sine neccesitate" or "plurality should not be posited without
+ necessity." (hence Occam's Razor is sometimes called "the principle
+ of unnecessary plurality" and " the principle of simplicity"). A
+ perhaps more contemporary formulation of Occam's Razor states that
+ the simplest explanation for a phenomenon is the one preferred by
+ nature. Other formulations of the same idea can be found in the
+ KISS (Keep It Simple Stupid) principle and the Principle of Least
+ Astonishment (the assertion that the most usable system is the one
+ that least often leaves users astonished). [WILLINGER2002] provides
+ a more theoretical discussion of "robustness through simplicity", and
+ in discussing the PSTN, [KUHN87] states that in most systems, "a
+ trade-off can be made between simplicity of interactions and
+ looseness of coupling".
+
+ When applied to packet switched network architectures, the Simplicity
+ Principle has implications that some may consider heresy, e.g., that
+ highly converged approaches are likely to be less efficient than
+ "less converged" solutions. Otherwise stated, the "optimal"
+ convergence layer may be much lower in the protocol stack that is
+ conventionally believed. In addition, the analysis above leads to
+ several conclusions that are contrary to the conventional wisdom
+ surrounding packet networking. Perhaps most significant is the
+ belief that packet switching is simpler than circuit switching. This
+ belief has lead to conclusions such as "since packet is simpler than
+ circuit, it must cost less to operate". This study finds to the
+ contrary. In particular, by examining the metrics described above,
+ we find that packet switching is more complex than circuit switching.
+ Interestingly, this conclusion is borne out by the fact that
+ normalized OPEX for data networks is typically significantly greater
+ than for voice networks [ML2002].
+
+ Finally, the important conclusion of this work is that for packet
+ networks that are of the scale of today's Internet or larger, we must
+ strive for the simplest possible solutions if we hope to build cost
+ effective infrastructures. This idea is eloquently stated in
+ [DOYLE2002]: "The evolution of protocols can lead to a
+ robustness/complexity/fragility spiral where complexity added for
+ robustness also adds new fragilities, which in turn leads to new and
+ thus spiraling complexities". This is exactly the phenomenon that
+ the Simplicity Principle is designed to avoid.
+
+10. Security Considerations
+
+ This document does not directly effect the security of any existing
+ Internet protocol. However, adherence to the Simplicity Principle
+ does have a direct affect on our ability to implement secure systems.
+ In particular, a system's complexity grows, it becomes more
+ difficult to model and analyze, and hence it becomes more difficult
+
+
+
+Bush, et. al. Informational [Page 22]
+
+RFC 3439 Internet Architectural Guidelines December 2002
+
+
+ to find and understand the security implications inherent in its
+ architecture, design, and implementation.
+
+11. Acknowledgments
+
+ Many of the ideas for comparing the complexity of circuit switched
+ and packet switched networks were inspired by conversations with Nick
+ McKeown. Scott Bradner, David Banister, Steve Bellovin, Steward
+ Bryant, Christophe Diot, Susan Harris, Ananth Nagarajan, Andrew
+ Odlyzko, Pete and Natalie Whiting, and Lixia Zhang made many helpful
+ comments on early drafts of this document.
+
+12. References
+
+ [AHUJA] "The Impact of Internet Policy and Topology on
+ Delayed Routing Convergence", Labovitz, et. al.
+ Infocom, 2001.
+
+ [ATMMPLS] "ATM-MPLS Interworking Migration Complexities Issues
+ and Preliminary Assessment", School of
+ Interdisciplinary Computing and Engineering,
+ University of Missouri-Kansas City, April 2002
+
+ [BARAN] "On Distributed Communications", Paul Baran, Rand
+ Corporation Memorandum RM-3420-PR,
+ http://www.rand.org/publications/RM/RM3420", August,
+ 1964.
+
+ [BARAN77] "SOME PERSPECTIVES ON NETWORKS--PAST, PRESENT AND
+ FUTURE", Paul Baran, Information Processing 77,
+ North-Holland Publishing Company, 1977,
+
+ [BRYANT] "Protocol Layering in PWE3", Bryant et al, Work in
+ Progress.
+
+ [CAIDA] http://www.caida.org
+
+ [CALLIENT] http://www.calient.net/home.html
+
+ [CARLSON] "Complexity and Robustness", J.M. Carlson and John
+ Doyle, Proc. Natl. Acad. Sci. USA, Vol. 99, Suppl. 1,
+ 2538-2545, February 19, 2002.
+ http://www.pnas.org/cgi/doi/10.1073/pnas.012582499
+
+ [CIENA] "CIENA Multiwave CoreDiretor",
+ http://www.ciena.com/downloads/products/
+ coredirector.pdf
+
+
+
+
+Bush, et. al. Informational [Page 23]
+
+RFC 3439 Internet Architectural Guidelines December 2002
+
+
+ [CISCO] http://www.cisco.com
+
+ [CLARK] "The Design Philosophy of the DARPA Internet
+ Protocols", D. Clark, Proc. of the ACM SIGCOMM, 1988.
+
+ [COFFMAN] "Internet Growth: Is there a 'Moores Law' for Data
+ Traffic", K.G. Coffman and A.M. Odlyzko, pp. 47-93,
+ Handbook of Massive Data Stes, J. Elli, P. M.
+ Pardalos, and M. G. C. Resende, Editors. Kluwer,
+ 2002.
+
+ [DOYLE2002] "Robustness and the Internet: Theoretical
+ Foundations", John C. Doyle, et. al. Work in
+ Progress.
+
+ [EICK] "Visualizing Software Changes", S.G. Eick, et al,
+ National Institute of Statistical Sciences, Technical
+ Report 113, December 2000.
+
+ [MOLINERO2002] "TCP Switching: Exposing Circuits to IP", Pablo
+ Molinero-Fernandez and Nick McKeown, IEEE January,
+ 2002.
+
+ [FLOYD] "The Synchronization of Periodic Routing Messages",
+ Sally Floyd and Van Jacobson, IEEE ACM Transactions
+ on Networking, 1994.
+
+ [FLOYD2001] "A Report on Some Recent Developments in TCP
+ Congestion Control, IEEE Communications Magazine, S.
+ Floyd, April 2001.
+
+ [FRALEIGH] "Provisioning IP Backbone Networks to Support Delay-
+ Based Service Level Agreements", Chuck Fraleigh,
+ Fouad Tobagi, and Christophe Diot, 2002.
+
+ [GRIFFIN] "What is the Sound of One Route Flapping", Timothy G.
+ Griffin, IPAM Workshop on Large-Scale Communication
+ Networks: Topology, Routing, Traffic, and Control,
+ March, 2002.
+
+ [HANDLEY] "On Inter-layer Assumptions (A view from the
+ Transport Area), slides from a presentation at the
+ IAB workshop on Wireless Internetworking", M.
+ Handley, March 2000.
+
+ [HOT] J.M. Carlson and John Doyle, Phys. Rev. E 60, 1412-
+ 1427, 1999.
+
+
+
+
+Bush, et. al. Informational [Page 24]
+
+RFC 3439 Internet Architectural Guidelines December 2002
+
+
+ [ISO10589] "Intermediate System to Intermediate System
+ Intradomain Routing Exchange Protocol (IS-IS)".
+
+ [JACOBSON] "Congestion Avoidance and Control", Van Jacobson,
+ Proceedings of ACM Sigcomm 1988, pp. 273-288.
+
+ [KARN] "TCP vs Link Layer Retransmission" in P. Karn et al.,
+ Advice for Internet Subnetwork Designers, Work in
+ Progress.
+
+ [KUHN87] "Sources of Failure in the Public Switched Telephone
+ Network", D. Richard Kuhn, EEE Computer, Vol. 30, No.
+ 4, April, 1997.
+
+ [L2TPV3] Lan, J., et. al., "Layer Two Tunneling Protocol
+ (Version 3) -- L2TPv3", Work in Progress.
+
+ [MC2001] "U.S Communications Infrastructure at A Crossroads:
+ Opportunities Amid the Gloom", McKinsey&Company for
+ Goldman-Sachs, August 2001.
+
+ [MCK2002] Nick McKeown, personal communication, April, 2002.
+
+ [ML2002] "Optical Systems", Merril Lynch Technical Report,
+ April, 2002.
+
+ [NAVE] "The influence of mode coupling on the non-linear
+ evolution of tearing modes", M.F.F. Nave, et al, Eur.
+ Phys. J. D 8, 287-297.
+
+ [NEUMANN] "Cause of AT&T network failure", Peter G. Neumann,
+ http://catless.ncl.ac.uk/Risks/9.62.html#subj2
+
+ [ODLYZKO] "Data networks are mostly empty for good reason",
+ A.M. Odlyzko, IT Professional 1 (no. 2), pp. 67-69,
+ Mar/Apr 1999.
+
+ [ODLYZKO98A] "Smart and stupid networks: Why the Internet is like
+ Microsoft". A. M. Odlyzko, ACM Networker, 2(5),
+ December, 1998.
+
+ [ODLYZKO98] "The economics of the Internet: Utility, utilization,
+ pricing, and Quality of Service", A.M. Odlyzko, July,
+ 1998.
+ http://www.dtc.umn.edu/~odlyzko/doc/networks.html
+
+
+
+
+
+
+Bush, et. al. Informational [Page 25]
+
+RFC 3439 Internet Architectural Guidelines December 2002
+
+
+ [PARK] "The Internet as a Complex System: Scaling,
+ Complexity and Control", Kihong Park and Walter
+ Willinger, AT&T Research, 2002.
+
+ [PERROW] "Normal Accidents: Living with High Risk
+ Technologies", Basic Books, C. Perrow, New York,
+ 1984.
+
+ [PMC] "The Design of a 10 Gigabit Core Router
+ Architecture", PMC-Sierra, http://www.pmc-
+ sierra.com/products/diagrams/CoreRouter_lg.html
+
+ [RFC1629] Colella, R., Callon, R., Gardner, E. and Y. Rekhter,
+ "Guidelines for OSI NSAP Allocation in the Internet",
+ RFC 1629, May 1994.
+
+ [RFC1925] Callon, R., "The Twelve Networking Truths", RFC 1925,
+ 1 April 1996.
+
+ [RFC1958] Carpenter, B., Ed., "Architectural principles of the
+ Internet", RFC 1958, June 1996.
+
+ [RFC2283] Bates, T., Chandra, R., Katz, D. and Y. Rekhter,
+ "Multiprotocol Extensions for BGP4", RFC 2283,
+ February 1998.
+
+ [RFC3155] Dawkins, S., Montenegro, G., Kojo, M. and N. Vaidya,
+ "End-to-end Performance Implications of Links with
+ Errors", BCP 50, RFC 3155, May 2001.
+
+ [ROMANOV] "Dynamics of TCP over ATM Networks", A. Romanov, S.
+ Floyd, IEEE JSAC, vol. 13, No 4, pp.633-641, May
+ 1995.
+
+ [SALTZER] "End-To-End Arguments in System Design", J.H.
+ Saltzer, D.P. Reed, and D.D. Clark, ACM TOCS, Vol 2,
+ Number 4, November 1984, pp 277-288.
+
+ [SCOTT] "Making Smart Investments to Reduce Unplanned
+ Downtime", D. Scott, Tactical Guidelines, TG-07-4033,
+ Gartner Group Research Note, March 1999.
+
+ [SPILLMAN] "The Law of Diminishing Returns:, W. J. Spillman and
+ E. Lang, 1924.
+
+ [STALLINGS] "Data and Computer Communications (2nd Ed)", William
+ Stallings, Maxwell Macmillan, 1989.
+
+
+
+
+Bush, et. al. Informational [Page 26]
+
+RFC 3439 Internet Architectural Guidelines December 2002
+
+
+ [TENNENHOUSE] "Layered multiplexing considered harmful", D.
+ Tennenhouse, Proceedings of the IFIP Workshop on
+ Protocols for High-Speed Networks, Rudin ed., North
+ Holland Publishers, May 1989.
+
+ [THOMPSON] "Nonlinear Dynamics and Chaos". J.M.T. Thompson and
+ H.B. Stewart, John Wiley and Sons, 1994, ISBN
+ 0471909602.
+
+ [TINA] "What is TINA and is it useful for the TelCos?",
+ Paolo Coppo, Carlo A. Licciardi, CSELT, EURESCOM
+ Participants in P847 (FT, IT, NT, TI)
+
+ [WAKEMAN] "Layering considered harmful", Ian Wakeman, Jon
+ Crowcroft, Zheng Wang, and Dejan Sirovica, IEEE
+ Network, January 1992, p. 7-16.
+
+ [WARD] "Custom fluorescent-nucleotide synthesis as an
+ alternative method for nucleic acid labeling",
+ Octavian Henegariu*, Patricia Bray-Ward and David C.
+ Ward, Nature Biotech 18:345-348 (2000).
+
+ [WILLINGER2002] "Robustness and the Internet: Design and evolution",
+ Walter Willinger and John Doyle, 2002.
+
+ [ZHANG] "Impact of Aggregation on Scaling Behavior of
+ Internet Backbone Traffic", Sprint ATL Technical
+ Report TR02-ATL-020157 Zhi-Li Zhang, Vinay Ribeiroj,
+ Sue Moon, Christophe Diot, February, 2002.
+
+13. Authors' Addresses
+
+ Randy Bush
+ EMail: randy@psg.com
+
+ David Meyer
+ EMail: dmm@maoz.com
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+Bush, et. al. Informational [Page 27]
+
+RFC 3439 Internet Architectural Guidelines December 2002
+
+
+14. Full Copyright Statement
+
+ Copyright (C) The Internet Society (2002). All Rights Reserved.
+
+ This document and translations of it may be copied and furnished to
+ others, and derivative works that comment on or otherwise explain it
+ or assist in its implementation may be prepared, copied, published
+ and distributed, in whole or in part, without restriction of any
+ kind, provided that the above copyright notice and this paragraph are
+ included on all such copies and derivative works. However, this
+ document itself may not be modified in any way, such as by removing
+ the copyright notice or references to the Internet Society or other
+ Internet organizations, except as needed for the purpose of
+ developing Internet standards in which case the procedures for
+ copyrights defined in the Internet Standards process must be
+ followed, or as required to translate it into languages other than
+ English.
+
+ The limited permissions granted above are perpetual and will not be
+ revoked by the Internet Society or its successors or assigns.
+
+ This document and the information contained herein is provided on an
+ "AS IS" basis and THE INTERNET SOCIETY AND THE INTERNET ENGINEERING
+ TASK FORCE DISCLAIMS ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING
+ BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION
+ HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF
+ MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE.
+
+Acknowledgement
+
+ Funding for the RFC Editor function is currently provided by the
+ Internet Society.
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+Bush, et. al. Informational [Page 28]
+