summaryrefslogtreecommitdiff
path: root/doc/rfc/rfc9522.txt
diff options
context:
space:
mode:
authorThomas Voss <mail@thomasvoss.com> 2024-11-27 20:54:24 +0100
committerThomas Voss <mail@thomasvoss.com> 2024-11-27 20:54:24 +0100
commit4bfd864f10b68b71482b35c818559068ef8d5797 (patch)
treee3989f47a7994642eb325063d46e8f08ffa681dc /doc/rfc/rfc9522.txt
parentea76e11061bda059ae9f9ad130a9895cc85607db (diff)
doc: Add RFC documents
Diffstat (limited to 'doc/rfc/rfc9522.txt')
-rw-r--r--doc/rfc/rfc9522.txt4430
1 files changed, 4430 insertions, 0 deletions
diff --git a/doc/rfc/rfc9522.txt b/doc/rfc/rfc9522.txt
new file mode 100644
index 0000000..a441e16
--- /dev/null
+++ b/doc/rfc/rfc9522.txt
@@ -0,0 +1,4430 @@
+
+
+
+
+Internet Engineering Task Force (IETF) A. Farrel, Ed.
+Request for Comments: 9522 Old Dog Consulting
+Obsoletes: 3272 January 2024
+Category: Informational
+ISSN: 2070-1721
+
+
+ Overview and Principles of Internet Traffic Engineering
+
+Abstract
+
+ This document describes the principles of traffic engineering (TE) in
+ the Internet. The document is intended to promote better
+ understanding of the issues surrounding traffic engineering in IP
+ networks and the networks that support IP networking and to provide a
+ common basis for the development of traffic-engineering capabilities
+ for the Internet. The principles, architectures, and methodologies
+ for performance evaluation and performance optimization of
+ operational networks are also discussed.
+
+ This work was first published as RFC 3272 in May 2002. This document
+ obsoletes RFC 3272 by making a complete update to bring the text in
+ line with best current practices for Internet traffic engineering and
+ to include references to the latest relevant work in the IETF.
+
+Status of This Memo
+
+ This document is not an Internet Standards Track specification; it is
+ published for informational purposes.
+
+ This document is a product of the Internet Engineering Task Force
+ (IETF). It represents the consensus of the IETF community. It has
+ received public review and has been approved for publication by the
+ Internet Engineering Steering Group (IESG). Not all documents
+ approved by the IESG are candidates for any level of Internet
+ Standard; see Section 2 of RFC 7841.
+
+ Information about the current status of this document, any errata,
+ and how to provide feedback on it may be obtained at
+ https://www.rfc-editor.org/info/rfc9522.
+
+Copyright Notice
+
+ Copyright (c) 2024 IETF Trust and the persons identified as the
+ document authors. All rights reserved.
+
+ This document is subject to BCP 78 and the IETF Trust's Legal
+ Provisions Relating to IETF Documents
+ (https://trustee.ietf.org/license-info) in effect on the date of
+ publication of this document. Please review these documents
+ carefully, as they describe your rights and restrictions with respect
+ to this document. Code Components extracted from this document must
+ include Revised BSD License text as described in Section 4.e of the
+ Trust Legal Provisions and are provided without warranty as described
+ in the Revised BSD License.
+
+Table of Contents
+
+ 1. Introduction
+ 1.1. What is Internet Traffic Engineering?
+ 1.2. Components of Traffic Engineering
+ 1.3. Scope
+ 1.4. Terminology
+ 2. Background
+ 2.1. Context of Internet Traffic Engineering
+ 2.2. Network Domain Context
+ 2.3. Problem Context
+ 2.3.1. Congestion and Its Ramifications
+ 2.4. Solution Context
+ 2.4.1. Combating the Congestion Problem
+ 2.5. Implementation and Operational Context
+ 3. Traffic-Engineering Process Models
+ 3.1. Components of the Traffic-Engineering Process Model
+ 4. Taxonomy of Traffic-Engineering Systems
+ 4.1. Time-Dependent versus State-Dependent versus
+ Event-Dependent
+ 4.2. Offline versus Online
+ 4.3. Centralized versus Distributed
+ 4.3.1. Hybrid Systems
+ 4.3.2. Considerations for Software-Defined Networking
+ 4.4. Local versus Global
+ 4.5. Prescriptive versus Descriptive
+ 4.5.1. Intent-Based Networking
+ 4.6. Open-Loop versus Closed-Loop
+ 4.7. Tactical versus Strategic
+ 5. Review of TE Techniques
+ 5.1. Overview of IETF Projects Related to Traffic Engineering
+ 5.1.1. IETF TE Mechanisms
+ 5.1.2. IETF Approaches Relying on TE Mechanisms
+ 5.1.3. IETF Techniques Used by TE Mechanisms
+ 5.2. Content Distribution
+ 6. Recommendations for Internet Traffic Engineering
+ 6.1. Generic Non-functional Recommendations
+ 6.2. Routing Recommendations
+ 6.3. Traffic Mapping Recommendations
+ 6.4. Measurement Recommendations
+ 6.5. Policing, Planning, and Access Control
+ 6.6. Network Survivability
+ 6.6.1. Survivability in MPLS-Based Networks
+ 6.6.2. Protection Options
+ 6.7. Multi-Layer Traffic Engineering
+ 6.8. Traffic Engineering in Diffserv Environments
+ 6.9. Network Controllability
+ 7. Inter-Domain Considerations
+ 8. Overview of Contemporary TE Practices in Operational IP
+ Networks
+ 9. Security Considerations
+ 10. IANA Considerations
+ 11. Informative References
+ Appendix A. Summary of Changes since RFC 3272
+ A.1. RFC 3272
+ A.2. This Document
+ Acknowledgments
+ Contributors
+ Author's Address
+
+1. Introduction
+
+ This document describes the principles of Internet traffic
+ engineering (TE). The objective of the document is to articulate the
+ general issues and principles for Internet TE and, where appropriate,
+ to provide recommendations, guidelines, and options for the
+ development of preplanned (offline) and dynamic (online) Internet TE
+ capabilities and support systems.
+
+ Even though Internet TE is most effective when applied end-to-end,
+ the focus of this document is TE within a given domain (such as an
+ Autonomous System (AS)). However, because a preponderance of
+ Internet traffic tends to originate in one AS and terminate in
+ another, this document also provides an overview of aspects
+ pertaining to inter-domain TE.
+
+ This document provides terminology and a taxonomy for describing and
+ understanding common Internet TE concepts.
+
+ This work was first published as [RFC3272] in May 2002. This
+ document obsoletes [RFC3272] by making a complete update to bring the
+ text in line with best current practices for Internet TE and to
+ include references to the latest relevant work in the IETF. It is
+ worth noting around three-fifths of the RFCs referenced in this
+ document postdate the publication of [RFC3272]. Appendix A provides
+ a summary of changes between [RFC3272] and this document.
+
+1.1. What is Internet Traffic Engineering?
+
+ One of the most significant functions performed in the Internet is
+ the routing and forwarding of traffic from ingress nodes to egress
+ nodes. Therefore, one of the most distinctive functions performed by
+ Internet traffic engineering is the control and optimization of these
+ routing and forwarding functions, to steer traffic through the
+ network.
+
+ Internet traffic engineering is defined as that aspect of Internet
+ network engineering dealing with the issues of performance evaluation
+ and performance optimization of operational IP networks. Traffic
+ engineering encompasses the application of technology and scientific
+ principles to the measurement, characterization, modeling, and
+ control of Internet traffic [RFC2702] [AWD2].
+
+ It is the performance of the network as seen by end users of network
+ services that is paramount. The characteristics visible to end users
+ are the emergent properties of the network, which are the
+ characteristics of the network when viewed as a whole. A central
+ goal of the service provider, therefore, is to enhance the emergent
+ properties of the network while taking economic considerations into
+ account. This is accomplished by addressing traffic-oriented
+ performance requirements while utilizing network resources without
+ excessive waste and in a reliable way. Traffic-oriented performance
+ measures include delay, delay variation, packet loss, and throughput.
+
+ Internet TE responds to network events (such as link or node
+ failures, reported or predicted network congestion, planned
+ maintenance, service degradation, planned changes in the traffic
+ matrix, etc.). Aspects of capacity management respond at intervals
+ ranging from days to years. Routing control functions operate at
+ intervals ranging from milliseconds to days. Packet-level processing
+ functions operate at very fine levels of temporal resolution (up to
+ milliseconds) while reacting to statistical measures of the real-time
+ behavior of traffic.
+
+ Thus, the optimization aspects of TE can be viewed from a control
+ perspective and can be both proactive and reactive. In the proactive
+ case, the TE control system takes preventive action to protect
+ against predicted unfavorable future network states, for example, by
+ engineering backup paths. It may also take action that will lead to
+ a more desirable future network state. In the reactive case, the
+ control system responds to correct issues and adapt to network
+ events, such as routing after failure.
+
+ Another important objective of Internet TE is to facilitate reliable
+ network operations [RFC2702]. Reliable network operations can be
+ facilitated by providing mechanisms that enhance network integrity
+ and by embracing policies emphasizing network survivability. This
+ reduces the vulnerability of services to outages arising from errors,
+ faults, and failures occurring within the network infrastructure.
+
+ The optimization aspects of TE can be achieved through capacity
+ management and traffic management. In this document, capacity
+ management includes capacity planning, routing control, and resource
+ management. Network resources of particular interest include link
+ bandwidth, buffer space, and computational resources. In this
+ document, traffic management includes:
+
+ 1. Nodal traffic control functions, such as traffic conditioning,
+ queue management, and scheduling.
+
+ 2. Other functions that regulate the flow of traffic through the
+ network or that arbitrate access to network resources between
+ different packets or between different traffic flows.
+
+ One major challenge of Internet TE is the realization of automated
+ control capabilities that adapt quickly and cost-effectively to
+ significant changes in network state, while still maintaining
+ stability of the network. Performance evaluation can assess the
+ effectiveness of TE methods, and the results of this evaluation can
+ be used to identify existing problems, guide network reoptimization,
+ and aid in the prediction of potential future problems. However,
+ this process can also be time-consuming and may not be suitable to
+ act on short-lived changes in the network.
+
+ Performance evaluation can be achieved in many different ways. The
+ most notable techniques include analytic methods, simulation, and
+ empirical methods based on measurements.
+
+ Traffic engineering comes in two flavors:
+
+ * A background process that constantly monitors traffic and network
+ conditions and optimizes the use of resources to improve
+ performance.
+
+ * A form of a pre-planned traffic distribution that is considered
+ optimal.
+
+ In the latter case, any deviation from the optimum distribution
+ (e.g., caused by a fiber cut) is reverted upon repair without further
+ optimization. However, this form of TE relies upon the notion that
+ the planned state of the network is optimal. Hence, there are two
+ levels of TE in such a mode:
+
+ * The TE-planning task to enable optimum traffic distribution.
+
+ * The routing and forwarding tasks that keep traffic flows attached
+ to the pre-planned distribution.
+
+ As a general rule, TE concepts and mechanisms must be sufficiently
+ specific and well-defined to address known requirements but
+ simultaneously flexible and extensible to accommodate unforeseen
+ future demands (see Section 6.1).
+
+1.2. Components of Traffic Engineering
+
+ As mentioned in Section 1.1, Internet traffic engineering provides
+ performance optimization of IP networks while utilizing network
+ resources economically and reliably. Such optimization is supported
+ at the control/controller level and within the data/forwarding plane.
+
+ The key elements required in any TE solution are as follows:
+
+ 1. Policy
+
+ 2. Path steering
+
+ 3. Resource management
+
+ Some TE solutions rely on these elements to a lesser or greater
+ extent. Debate remains about whether a solution can truly be called
+ "TE" if it does not include all of these elements. For the sake of
+ this document, we assert that all TE solutions must include some
+ aspects of all of these elements. Other solutions can be classed as
+ "partial TE" and also fall in scope of this document.
+
+ Policy allows for the selection of paths (including next hops) based
+ on information beyond basic reachability. Early definitions of
+ routing policy, e.g., [RFC1102] and [RFC1104], discuss routing policy
+ being applied to restrict access to network resources at an aggregate
+ level. BGP is an example of a commonly used mechanism for applying
+ such policies; see [RFC4271] and [RFC8955]. In the TE context,
+ policy decisions are made within the control plane or by controllers
+ in the management plane and govern the selection of paths. Examples
+ can be found in [RFC4655] and [RFC5394]. TE solutions may cover the
+ mechanisms to distribute and/or enforce policies, but definition of
+ specific policies is left to the network operator.
+
+ Path steering is the ability to forward packets using more
+ information than just knowledge of the next hop. Examples of path
+ steering include IPv4 source routes [RFC0791], RSVP-TE explicit
+ routes [RFC3209], Segment Routing (SR) [RFC8402], and Service
+ Function Chaining [RFC7665]. Path steering for TE can be supported
+ via control plane protocols, by encoding in the data plane headers,
+ or by a combination of the two. This includes when control is
+ provided by a controller using a network-facing control protocol.
+
+ Resource management provides resource-aware control and forwarding.
+ Examples of resources are bandwidth, buffers, and queues, all of
+ which can be managed to control loss and latency.
+
+ Resource reservation is the control aspect of resource management.
+ It provides for domain-wide consensus about which network resources
+ are used by a particular flow. This determination may be made at a
+ very coarse or very fine level. Note that this consensus exists at
+ the network control or controller level but not within the data
+ plane. It may be composed purely of accounting/bookkeeping, but it
+ typically includes an ability to admit, reject, or reclassify a flow
+ based on policy. Such accounting can be done based on any
+ combination of a static understanding of resource requirements and
+ the use of dynamic mechanisms to collect requirements (e.g., via
+ RSVP-TE [RFC3209]) and resource availability (e.g., via OSPF
+ extensions for GMPLS [RFC4203]).
+
+ Resource allocation is the data plane aspect of resource management.
+ It provides for the allocation of specific node and link resources to
+ specific flows. Example resources include buffers, policing, and
+ rate-shaping mechanisms that are typically supported via queuing.
+ Resource allocation also includes the matching of a flow (i.e., flow
+ classification) to a particular set of allocated resources. The
+ method of flow classification and granularity of resource management
+ is technology-specific. Examples include Diffserv with dropping and
+ remarking [RFC4594], MPLS-TE [RFC3209], GMPLS-based Label Switched
+ Paths (LSPs) [RFC3945], as well as controller-based solutions
+ [RFC8453]. This level of resource control, while optional, is
+ important in networks that wish to support network congestion
+ management policies to control or regulate the offered traffic to
+ deliver different levels of service and alleviate network congestion
+ problems. It is also important in networks that wish to control the
+ latency experienced by specific traffic flows.
+
+1.3. Scope
+
+ The scope of this document is intra-domain TE because this is the
+ practical level of TE technology that exists in the Internet at the
+ time of writing. That is, this document describes TE within a given
+ AS in the Internet. This document discusses concepts pertaining to
+ intra-domain traffic control, including such issues as routing
+ control, micro and macro resource allocation, and control
+ coordination problems that arise consequently.
+
+ This document describes and characterizes techniques already in use
+ or in advanced development for Internet TE. The way these techniques
+ fit together is discussed and scenarios in which they are useful are
+ identified.
+
+ Although the emphasis in this document is on intra-domain traffic
+ engineering, an overview of the high-level considerations pertaining
+ to inter-domain TE is provided in Section 7. Inter-domain Internet
+ TE is crucial to the performance enhancement of the world-wide
+ Internet infrastructure.
+
+ Whenever possible, relevant requirements from existing IETF documents
+ and other sources are incorporated by reference.
+
+1.4. Terminology
+
+ This section provides terminology that is useful for Internet TE.
+ The definitions presented apply to this document. These terms may
+ have other meanings elsewhere.
+
+ Busy hour: A one-hour period within a specified interval of time
+ (typically 24 hours) in which the traffic load in a network or
+ sub-network is greatest.
+
+ Congestion: A state of a network resource in which the traffic
+ incident on the resource exceeds its output capacity over an
+ interval of time. A small amount of congestion may be beneficial
+ to ensure that network resources are run at full capacity, and
+ this may be particularly true at the network edge where it is
+ desirable to ensure that user traffic is served as much as
+ possible. Within the network, if congestion is allowed to build
+ (such as when input traffic exceeds output traffic in a sustained
+ way), it will have a negative effect on user traffic.
+
+ Congestion avoidance: An approach to congestion management that
+ attempts to obviate the occurrence of congestion. It is chiefly
+ relevant to network congestion, although it may form a part of
+ demand-side congestion management.
+
+ Congestion response: An approach to congestion management that
+ attempts to remedy congestion problems that have already occurred.
+
+ Constraint-based routing: A class of routing protocols that takes
+ specified traffic attributes, network constraints, and policy
+ constraints into account when making routing decisions.
+ Constraint-based routing is applicable to traffic aggregates as
+ well as flows. It is a generalization of QoS-based routing.
+
+ Demand-side congestion management: A congestion management scheme
+ that addresses congestion problems by regulating or conditioning
+ the offered load.
+
+ Effective bandwidth: The minimum amount of bandwidth that can be
+ assigned to a flow or traffic aggregate in order to deliver
+ "acceptable service quality" to the flow or traffic aggregate.
+ See [KELLY] for a more mathematical definition.
+
+ Egress node: The device (router) at which traffic leaves a network
+ toward a destination (host, server, etc.) or to another network.
+
+ End-to-end: This term is context-dependent and often applies to the
+ life of a traffic flow from original source to final destination.
+ In contrast, edge-to-edge is often used to describe the traffic
+ flow from the entry of a domain or network to the exit of that
+ domain or network. However, in some contexts (for example, where
+ there is a service interface between a network and the client of
+ that network or where a path traverses multiple domains under the
+ control of a single process), end-to-end is used to refer to the
+ full operation of the service that may be composed of concatenated
+ edge-to-edge operations. Thus, in the context of TE, the term
+ "end-to-end" may refer to the full TE path but not to the complete
+ path of the traffic from source application to ultimate
+ destination.
+
+ Hotspot: A network element or subsystem that is in a considerably
+ higher state of congestion than others.
+
+ Ingress node: The device (router) at which traffic enters a network
+ from a source (host) or from another network.
+
+ Metric: A parameter defined in terms of standard units of
+ measurement.
+
+ Measurement methodology: A repeatable measurement technique used to
+ derive one or more metrics of interest.
+
+ Network congestion: Congestion within the network at a specific node
+ or a specific link that is sufficiently extreme that it results in
+ unacceptable queuing delay or packet loss. Network congestion can
+ negatively impact end-to-end or edge-to-edge traffic flows, so TE
+ schemes may be deployed to balance traffic in the network and
+ deliver congestion avoidance.
+
+ Network survivability: The capability to provide a prescribed level
+ of QoS for existing services after a given number of failures
+ occur within the network.
+
+ Offered load: Offered load is also sometimes called "offered traffic
+ load". It is a measure of the amount of traffic being presented
+ to be carried across a network compared to the capacity of the
+ network to carry it. This term derives from queuing theory, and
+ an offered load of 1 indicates that the network can carry, but
+ only just manage to carry, all of the traffic presented to it.
+
+ Offline traffic engineering: A traffic engineering system that
+ exists outside of the network.
+
+ Online traffic engineering: A traffic-engineering system that exists
+ within the network, typically implemented on or as adjuncts to
+ operational network elements.
+
+ Performance measures: Metrics that provide quantitative or
+ qualitative measures of the performance of systems or subsystems
+ of interest.
+
+ Performance metric: A performance parameter defined in terms of
+ standard units of measurement.
+
+ Provisioning: The process of assigning or configuring network
+ resources to meet certain requests.
+
+ Quality of Service (QoS): QoS [RFC3198] refers to the mechanisms
+ used within a network to achieve specific goals for the delivery
+ of traffic for a particular service according to the parameters
+ specified in a Service Level Agreement. "Quality" is
+ characterized by service availability, delay, jitter, throughput,
+ and packet loss ratio. At a network resource level, "Quality of
+ Service" refers to a set of capabilities that allow a service
+ provider to prioritize traffic, control bandwidth, and network
+ latency.
+
+ QoS routing: Class of routing systems that selects paths to be used
+ by a flow based on the QoS requirements of the flow.
+
+ Service Level Agreement (SLA): A contract between a provider and a
+ customer that guarantees specific levels of performance and
+ reliability at a certain cost.
+
+ Service Level Objective (SLO): A key element of an SLA between a
+ provider and a customer. SLOs are agreed upon as a means of
+ measuring the performance of the service provider and are outlined
+ as a way of avoiding disputes between the two parties based on
+ misunderstanding.
+
+ Stability: An operational state in which a network does not
+ oscillate in a disruptive manner from one mode to another mode.
+
+ Supply-side congestion management: A congestion management scheme
+ that provisions additional network resources to address existing
+ and/or anticipated congestion problems.
+
+ Traffic characteristic: A description of the temporal behavior or a
+ description of the attributes of a given traffic flow or traffic
+ aggregate.
+
+ Traffic-engineering system: A collection of objects, mechanisms, and
+ protocols that are used together to accomplish traffic-engineering
+ objectives.
+
+ Traffic flow: A stream of packets between two endpoints that can be
+ characterized in a certain way. A common classification for a
+ traffic flow selects packets with the five-tuple of source and
+ destination addresses, source and destination ports, and protocol
+ ID. Flows may be very small and transient, ranging to very large.
+ The TE techniques described in this document are likely to be more
+ effective when applied to large flows. Traffic flows may be
+ aggregated and treated as a single unit in some forms of TE,
+ making it possible to apply TE to the smaller flows that comprise
+ the aggregate.
+
+ Traffic mapping: Traffic mapping is the assignment of traffic
+ workload onto (pre-established) paths to meet certain
+ requirements.
+
+ Traffic matrix: A representation of the traffic demand between a set
+ of origin and destination abstract nodes. An abstract node can
+ consist of one or more network elements.
+
+ Traffic monitoring: The process of observing traffic characteristics
+ at a given point in a network and collecting the traffic
+ information for analysis and further action.
+
+ Traffic trunk: An aggregation of traffic flows belonging to the same
+ class that are forwarded through a common path. A traffic trunk
+ may be characterized by an ingress and egress node and a set of
+ attributes that determine its behavioral characteristics and
+ requirements from the network.
+
+ Workload: Workload is also sometimes called "traffic workload". It
+ is an evaluation of the amount of work that must be done in a
+ network in order to facilitate the traffic demand. Colloquially,
+ it is the answer to, "How busy is the network?"
+
+2. Background
+
+ The Internet aims to convey IP packets from ingress nodes to egress
+ nodes efficiently, expeditiously, and economically. Furthermore, in
+ a multi-class service environment (e.g., Diffserv capable networks;
+ see Section 5.1.1.2), the resource-sharing parameters of the network
+ must be appropriately determined and configured according to
+ prevailing policies and service models to resolve resource contention
+ issues arising from mutual interference between packets traversing
+ the network. Thus, consideration must be given to resolving
+ competition for network resources between traffic flows belonging to
+ the same service class (intra-class contention resolution) and
+ traffic flows belonging to different classes (inter-class contention
+ resolution).
+
+2.1. Context of Internet Traffic Engineering
+
+ The context of Internet traffic engineering includes the following
+ sub-contexts:
+
+ 1. A network domain context that defines the scope under
+ consideration and, in particular, the situations in which the TE
+ problems occur. The network domain context includes network
+ structure, policies, characteristics, constraints, quality
+ attributes, and optimization criteria.
+
+ 2. A problem context defining the general and concrete issues that
+ TE addresses. The problem context includes identification,
+ abstraction of relevant features, representation, formulation,
+ specification of the requirements on the solution space, and
+ specification of the desirable features of acceptable solutions.
+
+ 3. A solution context suggesting how to address the issues
+ identified by the problem context. The solution context includes
+ analysis, evaluation of alternatives, prescription, and
+ resolution.
+
+ 4. An implementation and operational context in which the solutions
+ are instantiated. The implementation and operational context
+ includes planning, organization, and execution.
+
+ The context of Internet TE and the different problem scenarios are
+ discussed in the following subsections.
+
+2.2. Network Domain Context
+
+ IP networks range in size from small clusters of routers situated
+ within a given location to thousands of interconnected routers,
+ switches, and other components distributed all over the world.
+
+ At the most basic level of abstraction, an IP network can be
+ represented as a distributed dynamic system consisting of:
+
+ * a set of interconnected resources that provide transport services
+ for IP traffic subject to certain constraints
+
+ * a demand system representing the offered load to be transported
+ through the network
+
+ * a response system consisting of network processes, protocols, and
+ related mechanisms that facilitate the movement of traffic through
+ the network (see also [AWD2])
+
+ The network elements and resources may have specific characteristics
+ restricting the manner in which the traffic demand is handled.
+ Additionally, network resources may be equipped with traffic control
+ mechanisms managing the way in which the demand is serviced. Traffic
+ control mechanisms may be used to:
+
+ * control packet processing activities within a given resource
+
+ * arbitrate contention for access to the resource by different
+ packets
+
+ * regulate traffic behavior through the resource
+
+ A configuration management and provisioning system may allow the
+ settings of the traffic control mechanisms to be manipulated by
+ external or internal entities in order to exercise control over the
+ way in which the network elements respond to internal and external
+ stimuli.
+
+ The details of how the network carries packets are specified in the
+ policies of the network administrators and are installed through
+ network configuration management and policy-based provisioning
+ systems. Generally, the types of service provided by the network
+ also depend upon the technology and characteristics of the network
+ elements and protocols, the prevailing service and utility models,
+ and the ability of the network administrators to translate policies
+ into network configurations.
+
+ Internet networks have two significant characteristics:
+
+ * They provide real-time services.
+
+ * Their operating environments are very dynamic.
+
+ The dynamic characteristics of IP and IP/MPLS networks can be
+ attributed in part to fluctuations in demand, the interaction between
+ various network protocols and processes, the rapid evolution of the
+ infrastructure that demands the constant inclusion of new
+ technologies and new network elements, and the transient and
+ persistent faults that occur within the system.
+
+ Packets contend for the use of network resources as they are conveyed
+ through the network. A network resource is considered to be
+ congested if, for an interval of time, the arrival rate of packets
+ exceeds the output capacity of the resource. Network congestion may
+ result in some of the arriving packets being delayed or even dropped.
+
+ Network congestion increases transit delay and delay variation, may
+ lead to packet loss, and reduces the predictability of network
+ services. Clearly, while congestion may be a useful tool at ingress
+ edge nodes, network congestion is highly undesirable. Combating
+ network congestion at a reasonable cost is a major objective of
+ Internet TE, although it may need to be traded with other objectives
+ to keep the costs reasonable.
+
+ Efficient sharing of network resources by multiple traffic flows is a
+ basic operational premise for the Internet. A fundamental challenge
+ in network operation is to increase resource utilization while
+ minimizing the possibility of congestion.
+
+ The Internet has to function in the presence of different classes of
+ traffic with different service requirements. This requirement is
+ clarified in the architecture for Differentiated Services (Diffserv)
+ [RFC2475]. That document describes how packets can be grouped into
+ behavior aggregates such that each aggregate has a common set of
+ behavioral characteristics or a common set of delivery requirements.
+ Delivery requirements of a specific set of packets may be specified
+ explicitly or implicitly. Two of the most important traffic delivery
+ requirements are:
+
+ * Capacity constraints can be expressed statistically as peak rates,
+ mean rates, burst sizes, or as some deterministic notion of
+ effective bandwidth.
+
+ * QoS requirements can be expressed in terms of:
+
+ - integrity constraints, such as packet loss
+
+ - temporal constraints, such as timing restrictions for the
+ delivery of each packet (delay) and timing restrictions for the
+ delivery of consecutive packets belonging to the same traffic
+ stream (delay variation)
+
+2.3. Problem Context
+
+ There are several problems associated with operating a network like
+ those described in the previous section. This section analyzes the
+ problem context in relation to TE. The identification, abstraction,
+ representation, and measurement of network features relevant to TE
+ are significant issues.
+
+ A particular challenge is to formulate the problems that traffic
+ engineering attempts to solve. For example:
+
+ * How to identify the requirements on the solution space
+
+ * How to specify the desirable features of solutions
+
+ * How to actually solve the problems
+
+ * How to measure and characterize the effectiveness of solutions
+
+ Another class of problems is how to measure and estimate relevant
+ network state parameters. Effective TE relies on a good estimate of
+ the offered traffic load as well as a view of the underlying topology
+ and associated resource constraints. Offline planning requires a
+ full view of the topology of the network or partial network that is
+ being planned.
+
+ Still another class of problem is how to characterize the state of
+ the network and how to evaluate its performance. The performance
+ evaluation problem is two-fold: one aspect relates to the evaluation
+ of the system-level performance of the network, and the other aspect
+ relates to the evaluation of resource-level performance, which
+ restricts attention to the performance analysis of individual network
+ resources.
+
+ In this document, we refer to the system-level characteristics of the
+ network as the "macro-states" and the resource-level characteristics
+ as the "micro-states." The system-level characteristics are also
+ known as the emergent properties of the network. Correspondingly, we
+ refer to the TE schemes dealing with network performance optimization
+ at the systems level as "macro-TE" and the schemes that optimize at
+ the individual resource level as "micro-TE." Under certain
+ circumstances, the system-level performance can be derived from the
+ resource-level performance using appropriate rules of composition,
+ depending upon the particular performance measures of interest.
+
+ Another fundamental class of problem concerns how to effectively
+ optimize network performance. Performance optimization may entail
+ translating solutions for specific TE problems into network
+ configurations. Optimization may also entail some degree of resource
+ management control, routing control, and capacity augmentation.
+
+2.3.1. Congestion and Its Ramifications
+
+ Network congestion is one of the most significant problems in an
+ operational IP context. A network element is said to be congested if
+ it experiences sustained overload over an interval of time. Although
+ congestion at the edge of the network may be beneficial in ensuring
+ that the network delivers as much traffic as possible, network
+ congestion almost always results in degradation of service quality to
+ end users. Congestion avoidance and response schemes can include
+ demand-side policies and supply-side policies. Demand-side policies
+ may restrict access to congested resources or dynamically regulate
+ the demand to alleviate the overload situation. Supply-side policies
+ may expand or augment network capacity to better accommodate offered
+ traffic. Supply-side policies may also reallocate network resources
+ by redistributing traffic over the infrastructure. Traffic
+ redistribution and resource reallocation serve to increase the
+ effective capacity of the network.
+
+ The emphasis of this document is primarily on congestion management
+ schemes falling within the scope of the network, rather than on
+ congestion management systems dependent upon sensitivity and
+ adaptivity from end systems. That is, the aspects that are
+ considered in this document with respect to congestion management are
+ those solutions that can be provided by control entities operating on
+ the network and by the actions of network administrators and network
+ operations systems.
+
+2.4. Solution Context
+
+ The solution context for Internet TE involves analysis, evaluation of
+ alternatives, and choice between alternative courses of action.
+ Generally, the solution context is based on making inferences about
+ the current or future state of the network and making decisions that
+ may involve a preference between alternative sets of action. More
+ specifically, the solution context demands reasonable estimates of
+ traffic workload, characterization of network state, derivation of
+ solutions that may be implicitly or explicitly formulated, and
+ possibly instantiation of a set of control actions. Control actions
+ may involve the manipulation of parameters associated with routing,
+ control over tactical capacity acquisition, and control over the
+ traffic management functions.
+
+ The following list of instruments may be applicable to the solution
+ context of Internet TE:
+
+ * A set of policies, objectives, and requirements (which may be
+ context dependent) for network performance evaluation and
+ performance optimization.
+
+ * A collection of online and, in some cases, possibly offline tools
+ and mechanisms for measurement, characterization, modeling,
+ control of traffic, control over the placement and allocation of
+ network resources, as well as control over the mapping or
+ distribution of traffic onto the infrastructure.
+
+ * A set of constraints on the operating environment, the network
+ protocols, and the TE system itself.
+
+ * A set of quantitative and qualitative techniques and methodologies
+ for abstracting, formulating, and solving TE problems.
+
+ * A set of administrative control parameters that may be manipulated
+ through a configuration management system. Such a system may
+ itself include a configuration control subsystem, a configuration
+ repository, a configuration accounting subsystem, and a
+ configuration auditing subsystem.
+
+ * A set of guidelines for network performance evaluation,
+ performance optimization, and performance improvement.
+
+ Determining traffic characteristics through measurement or estimation
+ is very useful within the realm of the TE solution space. Traffic
+ estimates can be derived from customer subscription information,
+ traffic projections, traffic models, and actual measurements. The
+ measurements may be performed at different levels, e.g., at the
+ traffic-aggregate level or at the flow level. Measurements at the
+ flow level or on small traffic aggregates may be performed at edge
+ nodes, when traffic enters and leaves the network. Measurements for
+ large traffic aggregates may be performed within the core of the
+ network.
+
+ To conduct performance studies and to support planning of existing
+ and future networks, a routing analysis may be performed to determine
+ the paths the routing protocols will choose for various traffic
+ demands and to ascertain the utilization of network resources as
+ traffic is routed through the network. Routing analysis captures the
+ selection of paths through the network, the assignment of traffic
+ across multiple feasible routes, and the multiplexing of IP traffic
+ over traffic trunks (if such constructs exist) and over the
+ underlying network infrastructure. A model of network topology is
+ necessary to perform routing analysis. A network topology model may
+ be extracted from:
+
+ * network architecture documents
+
+ * network designs
+
+ * information contained in router configuration files
+
+ * routing databases such as the link-state database of an Interior
+ Gateway Protocol (IGP)
+
+ * routing tables
+
+ * automated tools that discover and collate network topology
+ information
+
+ Topology information may also be derived from servers that monitor
+ network state and from servers that perform provisioning functions.
+
+ Routing in operational IP networks can be administratively controlled
+ at various levels of abstraction, including the manipulation of BGP
+ attributes and IGP metrics. For path-oriented technologies such as
+ MPLS, routing can be further controlled by the manipulation of
+ relevant TE parameters, resource parameters, and administrative
+ policy constraints. Within the context of MPLS, the path of an
+ explicitly routed LSP can be computed and established in various
+ ways, including:
+
+ * manually
+
+ * automatically and online using constraint-based routing processes
+ implemented on Label Switching Routers (LSRs)
+
+ * automatically and offline using constraint-based routing entities
+ implemented on external TE support systems
+
+2.4.1. Combating the Congestion Problem
+
+ Minimizing congestion is a significant aspect of Internet traffic
+ engineering. This subsection gives an overview of the general
+ approaches that have been used or proposed to combat congestion.
+
+ Congestion management policies can be categorized based upon the
+ following criteria (see [YARE95] for a more detailed taxonomy of
+ congestion control schemes):
+
+ 1. Congestion Management Based on Response Timescales
+
+ * Long (weeks to months): Expanding network capacity by adding
+ new equipment, routers, and links takes time and is
+ comparatively costly. Capacity planning needs to take this
+ into consideration. Network capacity is expanded based on
+ estimates or forecasts of future traffic development and
+ traffic distribution. These upgrades are typically carried
+ out over weeks, months, or maybe even years.
+
+ * Medium (minutes to days): Several control policies fall within
+ the medium timescale category. Examples include:
+
+ a. Adjusting routing protocol parameters to route traffic
+ away from or towards certain segments of the network.
+
+ b. Setting up or adjusting explicitly routed LSPs in MPLS
+ networks to route traffic trunks away from possibly
+ congested resources or toward possibly more favorable
+ routes.
+
+ c. Reconfiguring the logical topology of the network to make
+ it correlate more closely with the spatial traffic
+ distribution using, for example, an underlying path-
+ oriented technology such as MPLS LSPs or optical channel
+ trails.
+
+ When these schemes are adaptive, they rely on measurement
+ systems. A measurement system monitors changes in traffic
+ distribution, traffic loads, and network resource utilization
+ and then provides feedback to the online or offline TE
+ mechanisms and tools so that they can trigger control actions
+ within the network. The TE mechanisms and tools can be
+ implemented in a distributed or centralized fashion. A
+ centralized scheme may have full visibility into the network
+ state and may produce more optimal solutions. However,
+ centralized schemes are prone to single points of failure and
+ may not scale as well as distributed schemes. Moreover, the
+ information utilized by a centralized scheme may be stale and
+ might not reflect the actual state of the network. It is not
+ an objective of this document to make a recommendation between
+ distributed and centralized schemes; that is a choice that
+ network administrators must make based on their specific
+ needs.
+
+ * Short (minutes or less): This category includes packet-level
+ processing functions and events that are recorded on the order
+ of several round-trip times. It also includes router
+ mechanisms such as passive and active buffer management. All
+ of these mechanisms are used to control congestion or signal
+ congestion to end systems so that they can adaptively regulate
+ the rate at which traffic is injected into the network. A
+ well-known active queue management scheme, especially for
+ responsive traffic such as TCP, is Random Early Detection
+ (RED) [FLJA93]. During congestion (but before the queue is
+ filled), the RED scheme chooses arriving packets to "mark"
+ according to a probabilistic algorithm that takes into account
+ the average queue size. A router that does not utilize
+ Explicit Congestion Notification (ECN) [RFC3168] can simply
+ drop marked packets to alleviate congestion and implicitly
+ notify the receiver about the congestion. On the other hand,
+ if the router and the end hosts support ECN, they can set the
+ ECN field in the packet header, and the end host can act on
+ this information. Several variations of RED have been
+ proposed to support different drop precedence levels in multi-
+ class environments [RFC2597]. RED provides congestion
+ avoidance that is better than or equivalent to Tail-Drop (TD)
+ queue management (drop arriving packets only when the queue is
+ full). Importantly, RED reduces the possibility of
+ retransmission bursts becoming synchronized within the network
+ and improves fairness among different responsive traffic
+ sessions. However, RED by itself cannot prevent congestion
+ and unfairness caused by sources unresponsive to RED, e.g.,
+ some misbehaved greedy connections. Other schemes have been
+ proposed to improve performance and fairness in the presence
+ of unresponsive traffic. Some of those schemes (such as
+ Longest Queue Drop (LQD) and Dynamic Soft Partitioning with
+ Random Drop (RND) [SLDC98]) were proposed as theoretical
+ frameworks and are typically not available in existing
+ commercial products, while others (such as Approximate Fair
+ Dropping (AFD) [AFD03]) have seen some implementation. Advice
+ on the use of Active Queue Management (AQM) schemes is
+ provided in [RFC7567]. [RFC7567] recommends self-tuning AQM
+ algorithms like those that the IETF has published in
+ [RFC8290], [RFC8033], [RFC8034], and [RFC9332], but RED is
+ still appropriate for links with stable bandwidth, if
+ configured carefully.
+
+ 2. Reactive versus Preventive Congestion Management Schemes
+
+ * Reactive (recovery) congestion management policies react to
+ existing congestion problems. All the policies described
+ above for the short and medium timescales can be categorized
+ as being reactive. They are based on monitoring and
+ identifying congestion problems that exist in the network and
+ on the initiation of relevant actions to ease a situation.
+ Reactive congestion management schemes may also be preventive.
+
+ * Preventive (predictive/avoidance) policies take proactive
+ action to prevent congestion based on estimates and
+ predictions of future congestion problems (e.g., traffic
+ matrix forecasts). Some of the policies described for the
+ long and medium timescales fall into this category.
+ Preventive policies do not necessarily respond immediately to
+ existing congestion problems. Instead, forecasts of traffic
+ demand and workload distribution are considered, and action
+ may be taken to prevent potential future congestion problems.
+ The schemes described for the short timescale can also be used
+ for congestion avoidance because dropping or marking packets
+ before queues actually overflow would trigger corresponding
+ responsive traffic sources to slow down. Preventive
+ congestion management schemes may also be reactive.
+
+ 3. Supply-Side versus Demand-Side Congestion Management Schemes
+
+ * Supply-side congestion management policies increase the
+ effective capacity available to traffic in order to control or
+ reduce congestion. This can be accomplished by increasing
+ capacity or by balancing distribution of traffic over the
+ network. Capacity planning aims to provide a physical
+ topology and associated link bandwidths that match or exceed
+ estimated traffic workload and traffic distribution, subject
+ to traffic forecasts and budgetary (or other) constraints. If
+ the actual traffic distribution does not fit the topology
+ derived from capacity planning, then the traffic can be mapped
+ onto the topology by using routing control mechanisms, by
+ applying path-oriented technologies (e.g., MPLS LSPs and
+ optical channel trails) to modify the logical topology or by
+ employing some other load redistribution mechanisms.
+
+ * Demand-side congestion management policies control or regulate
+ the offered traffic to alleviate congestion problems. For
+ example, some of the short timescale mechanisms described
+ earlier as well as policing and rate-shaping mechanisms
+ attempt to regulate the offered load in various ways.
+
+2.5. Implementation and Operational Context
+
+ The operational context of Internet TE is characterized by constant
+ changes that occur at multiple levels of abstraction. The
+ implementation context demands effective planning, organization, and
+ execution. The planning aspects may involve determining prior sets
+ of actions to achieve desired objectives. Organizing involves
+ arranging and assigning responsibility to the various components of
+ the TE system and coordinating the activities to accomplish the
+ desired TE objectives. Execution involves measuring and applying
+ corrective or perfective actions to attain and maintain desired TE
+ goals.
+
+3. Traffic-Engineering Process Models
+
+ This section describes a generic process model that captures the
+ high-level practical aspects of Internet traffic engineering in an
+ operational context. The process model is described as a sequence of
+ actions that must be carried out to optimize the performance of an
+ operational network (see also [RFC2702] and [AWD2]). This process
+ model may be enacted explicitly or implicitly, by a software process
+ or by a human.
+
+ The TE process model is iterative [AWD2]. The four phases of the
+ process model described below are repeated as a continual sequence:
+
+ 1. Define the relevant control policies that govern the operation of
+ the network.
+
+ 2. Acquire measurement data from the operational network.
+
+ 3. Analyze the network state and characterize the traffic workload.
+ Proactive analysis identifies potential problems that could
+ manifest in the future. Reactive analysis identifies existing
+ problems and determines their causes.
+
+ 4. Optimize the performance of the network. This involves a
+ decision process that selects and implements a set of actions
+ from a set of alternatives given the results of the three
+ previous steps. Optimization actions may include the use of
+ techniques to control the offered traffic and to control the
+ distribution of traffic across the network.
+
+3.1. Components of the Traffic-Engineering Process Model
+
+ The key components of the traffic-engineering process model are as
+ follows:
+
+ 1. Measurement is crucial to the TE function. The operational state
+ of a network can only be conclusively determined through
+ measurement. Measurement is also critical to the optimization
+ function because it provides feedback data that is used by TE
+ control subsystems. This data is used to adaptively optimize
+ network performance in response to events and stimuli originating
+ within and outside the network. Measurement in support of the TE
+ function can occur at different levels of abstraction. For
+ example, measurement can be used to derive packet-level
+ characteristics, flow-level characteristics, user- or customer-
+ level characteristics, traffic-aggregate characteristics,
+ component-level characteristics, and network-wide
+ characteristics.
+
+ 2. Modeling, analysis, and simulation are important aspects of
+ Internet TE. Modeling involves constructing an abstract or
+ physical representation that depicts relevant traffic
+ characteristics and network attributes. A network model is an
+ abstract representation of the network that captures relevant
+ network features, attributes, and characteristics. Network
+ simulation tools are extremely useful for TE. Because of the
+ complexity of realistic quantitative analysis of network
+ behavior, certain aspects of network performance studies can only
+ be conducted effectively using simulation.
+
+ 3. Network performance optimization involves resolving network
+ issues by transforming such issues into concepts that enable a
+ solution, identification of a solution, and implementation of the
+ solution. Network performance optimization can be corrective or
+ perfective. In corrective optimization, the goal is to remedy a
+ problem that has occurred or that is incipient. In perfective
+ optimization, the goal is to improve network performance even
+ when explicit problems do not exist and are not anticipated.
+
+4. Taxonomy of Traffic-Engineering Systems
+
+ This section presents a short taxonomy of traffic-engineering systems
+ constructed based on TE styles and views as listed below and
+ described in greater detail in the following subsections of this
+ document:
+
+ * Time-Dependent versus State-Dependent versus Event-Dependent
+
+ * Offline versus Online
+
+ * Centralized versus Distributed
+
+ * Local versus Global Information
+
+ * Prescriptive versus Descriptive
+
+ * Open-Loop versus Closed-Loop
+
+ * Tactical versus Strategic
+
+4.1. Time-Dependent versus State-Dependent versus Event-Dependent
+
+ Traffic-engineering methodologies can be classified as time-
+ dependent, state-dependent, or event-dependent. All TE schemes are
+ considered to be dynamic in this document. Static TE implies that no
+ TE methodology or algorithm is being applied -- it is a feature of
+ network planning but lacks the reactive and flexible nature of TE.
+
+ In time-dependent TE, historical information based on periodic
+ variations in traffic (such as time of day) is used to pre-program
+ routing and other TE control mechanisms. Additionally, customer
+ subscription or traffic projection may be used. Pre-programmed
+ routing plans typically change on a relatively long timescale (e.g.,
+ daily). Time-dependent algorithms do not attempt to adapt to short-
+ term variations in traffic or changing network conditions. An
+ example of a time-dependent algorithm is a centralized optimizer
+ where the input to the system is a traffic matrix and multi-class QoS
+ requirements as described in [MR99]. Another example of such a
+ methodology is the application of data mining to Internet traffic
+ [AJ19], which enables the use of various machine learning algorithms
+ to identify patterns within historically collected datasets about
+ Internet traffic and to extract information in order to guide
+ decision-making and improve efficiency and productivity of
+ operational processes.
+
+ State-dependent TE adapts the routing plans based on the current
+ state of the network, which provides additional information on
+ variations in actual traffic (i.e., perturbations from regular
+ variations) that could not be predicted using historical information.
+ Constraint-based routing is an example of state-dependent TE
+ operating in a relatively long timescale. An example of operating in
+ a relatively short timescale is a load-balancing algorithm described
+ in [MATE]. The state of the network can be based on parameters
+ flooded by the routers. Another approach is for a particular router
+ performing adaptive TE to send probe packets along a path to gather
+ the state of that path. [RFC6374] defines protocol extensions to
+ collect performance measurements from MPLS networks. Another
+ approach is for a management system to gather the relevant
+ information directly from network elements using telemetry data
+ collection publication/subscription techniques [RFC7923]. Timely
+ gathering and distribution of state information is critical for
+ adaptive TE. While time-dependent algorithms are suitable for
+ predictable traffic variations, state-dependent algorithms may be
+ needed to increase network efficiency and to provide resilience to
+ adapt to changes in network state.
+
+ Event-dependent TE methods can also be used for TE path selection.
+ Event-dependent TE methods are distinct from time-dependent and
+ state-dependent TE methods in the manner in which paths are selected.
+ These algorithms are adaptive and distributed in nature, and they
+ typically use learning models to find good paths for TE in a network.
+ While state-dependent TE models typically use available-link-
+ bandwidth (ALB) flooding [E.360.1] for TE path selection, event-
+ dependent TE methods do not require ALB flooding. Rather, event-
+ dependent TE methods typically search out capacity by learning
+ models, as in the success-to-the-top (STT) method [RFC6601]. ALB
+ flooding can be resource intensive, since it requires link bandwidth
+ to carry routing protocol link-state advertisements and processor
+ capacity to process those advertisements; in addition, the overhead
+ of the ALB advertisements and their processing can limit the size of
+ the area and AS. Modeling results suggest that event-dependent TE
+ methods could lead to a reduction in ALB flooding overhead without
+ loss of network throughput performance [TE-QoS-ROUTING].
+
+ A fully functional TE system is likely to use all aspects of time-
+ dependent, state-dependent, and event-dependent methodologies as
+ described in Section 4.3.1.
+
+4.2. Offline versus Online
+
+ Traffic engineering requires the computation of routing plans. The
+ computation may be performed offline or online. The computation can
+ be done offline for scenarios where routing plans need not be
+ executed in real time. For example, routing plans computed from
+ forecast information may be computed offline. Typically, offline
+ computation is also used to perform extensive searches on multi-
+ dimensional solution spaces.
+
+ Online computation is required when the routing plans must adapt to
+ changing network conditions as in state-dependent algorithms. Unlike
+ offline computation (which can be computationally demanding), online
+ computation is geared toward relatively simple and fast calculations
+ to select routes, fine-tune the allocations of resources, and perform
+ load balancing.
+
+4.3. Centralized versus Distributed
+
+ Under centralized control, there is a central authority that
+ determines routing plans and perhaps other TE control parameters on
+ behalf of each router. The central authority periodically collects
+ network-state information from all routers and sends routing
+ information to the routers. The update cycle for information
+ exchange in both directions is a critical parameter directly
+ impacting the performance of the network being controlled.
+ Centralized control may need high processing power and high bandwidth
+ control channels.
+
+ Distributed control determines route selection by each router
+ autonomously based on the router's view of the state of the network.
+ The network state information may be obtained by the router using a
+ probing method or distributed by other routers on a periodic basis
+ using link-state advertisements. Network state information may also
+ be disseminated under exception conditions. Examples of protocol
+ extensions used to advertise network link-state information are
+ defined in [RFC5305], [RFC6119], [RFC7471], [RFC8570], and [RFC8571].
+ See also Section 5.1.3.9.
+
+4.3.1. Hybrid Systems
+
+ In practice, most TE systems will be a hybrid of central and
+ distributed control. For example, a popular MPLS approach to TE is
+ to use a central controller based on an active, stateful Path
+ Computation Element (PCE) but to use routing and signaling protocols
+ to make local decisions at routers within the network. Local
+ decisions may be able to respond more quickly to network events but
+ may result in conflicts with decisions made by other routers.
+
+ Network operations for TE systems may also use a hybrid of offline
+ and online computation. TE paths may be precomputed based on stable-
+ state network information and planned traffic demands but may then be
+ modified in the active network depending on variations in network
+ state and traffic load. Furthermore, responses to network events may
+ be precomputed offline to allow rapid reactions without further
+ computation or may be derived online depending on the nature of the
+ events.
+
+4.3.2. Considerations for Software-Defined Networking
+
+ As discussed in Section 5.1.2.2, one of the main drivers for
+ Software-Defined Networking (SDN) is a decoupling of the network
+ control plane from the data plane [RFC7149]. However, SDN may also
+ combine centralized control of resources and facilitate application-
+ to-network interaction via an Application Programming Interface
+ (API), such as the one described in [RFC8040]. Combining these
+ features provides a flexible network architecture that can adapt to
+ the network requirements of a variety of higher-layer applications, a
+ concept often referred to as the "programmable network" [RFC7426].
+
+ The centralized control aspect of SDN helps improve network resource
+ utilization compared with distributed network control, where local
+ policy may often override network-wide optimization goals. In an SDN
+ environment, the data plane forwards traffic to its desired
+ destination. However, before traffic reaches the data plane, the
+ logically centralized SDN control plane often determines the path the
+ application traffic will take in the network. Therefore, the SDN
+ control plane needs to be aware of the underlying network topology,
+ capabilities, and current node and link resource state.
+
+ Using a PCE-based SDN control framework [RFC7491], the available
+ network topology may be discovered by running a passive instance of
+ OSPF or IS-IS, or via BGP Link State (BGP-LS) [RFC9552]), to generate
+ a Traffic Engineering Database (TED) (see Section 5.1.3.14). The PCE
+ is used to compute a path (see Section 5.1.3.11) based on the TED and
+ available bandwidth, and further path optimization may be based on
+ requested objective functions [RFC5541]. When a suitable path has
+ been computed, the programming of the explicit network path may be
+ either performed using a signaling protocol that traverses the length
+ of the path [RFC3209] or performed per-hop with each node being
+ directly programmed [RFC8283] by the SDN controller.
+
+ By utilizing a centralized approach to network control, additional
+ network benefits are also available, including Global Concurrent
+ Optimization (GCO) [RFC5557]. A GCO path computation request will
+ simultaneously use the network topology and a set of new path
+ signaling requests, along with their respective constraints, for
+ optimal placement in the network. Correspondingly, a GCO-based
+ computation may be applied to recompute existing network paths to
+ groom traffic and to mitigate congestion.
+
+4.4. Local versus Global
+
+ Traffic-engineering algorithms may require local and global network-
+ state information.
+
+ Local information is the state of a portion of the domain. Examples
+ include the bandwidth and packet loss rate of a particular path or
+ the state and capabilities of a network link. Local state
+ information may be sufficient for certain instances of distributed
+ control TE.
+
+ Global information is the state of the entire TE domain. Examples
+ include a global traffic matrix and loading information on each link
+ throughout the domain of interest. Global state information is
+ typically required with centralized control. Distributed TE systems
+ may also need global information in some cases.
+
+4.5. Prescriptive versus Descriptive
+
+ TE systems may also be classified as prescriptive or descriptive.
+
+ Prescriptive traffic engineering evaluates alternatives and
+ recommends a course of action. Prescriptive TE can be further
+ categorized as either corrective or perfective. Corrective TE
+ prescribes a course of action to address an existing or predicted
+ anomaly. Perfective TE prescribes a course of action to evolve and
+ improve network performance even when no anomalies are evident.
+
+ Descriptive traffic engineering, on the other hand, characterizes the
+ state of the network and assesses the impact of various policies
+ without recommending any particular course of action.
+
+4.5.1. Intent-Based Networking
+
+ One way to express a service request is through "intent". Intent-
+ Based Networking aims to produce networks that are simpler to manage
+ and operate, requiring only minimal intervention. Intent is defined
+ in [RFC9315] as follows:
+
+ | A set of operational goals (that a network should meet) and
+ | outcomes (that a network is supposed to deliver) defined in a
+ | declarative manner without specifying how to achieve or implement
+ | them.
+
+ Intent provides data and functional abstraction so that users and
+ operators do not need to be concerned with low-level device
+ configuration or the mechanisms used to achieve a given intent. This
+ approach can be conceptually easier for a user but may be less
+ expressive in terms of constraints and guidelines.
+
+ Intent-Based Networking is applicable to TE because many of the high-
+ level objectives may be expressed as intent (for example, load
+ balancing, delivery of services, and robustness against failures).
+ The intent is converted by the management system into TE actions
+ within the network.
+
+4.6. Open-Loop versus Closed-Loop
+
+ Open-loop traffic-engineering control is where control action does
+ not use feedback information from the current network state.
+ However, the control action may use its own local information for
+ accounting purposes.
+
+ Closed-loop traffic-engineering control is where control action
+ utilizes feedback information from the network state. The feedback
+ information may be in the form of current measurement or recent
+ historical records.
+
+4.7. Tactical versus Strategic
+
+ Tactical traffic engineering aims to address specific performance
+ problems (such as hotspots) that occur in the network from a tactical
+ perspective, without consideration of overall strategic imperatives.
+ Without proper planning and insights, tactical TE tends to be ad hoc
+ in nature.
+
+ Strategic traffic-engineering approaches the TE problem from a more
+ organized and systematic perspective, taking into consideration the
+ immediate and longer-term consequences of specific policies and
+ actions.
+
+5. Review of TE Techniques
+
+ This section briefly reviews different TE-related approaches proposed
+ and implemented in telecommunications and computer networks using
+ IETF protocols and architectures. These approaches are organized
+ into three categories:
+
+ * TE mechanisms that adhere to the definition provided in
+ Section 1.2
+
+ * Approaches that rely upon those TE mechanisms
+
+ * Techniques that are used by those TE mechanisms and approaches
+
+ The discussion is not intended to be comprehensive. It is primarily
+ intended to illuminate existing approaches to TE in the Internet. A
+ historic overview of TE in telecommunications networks was provided
+ in Section 4 of [RFC3272], and Section 4.6 of that document presented
+ an outline of some early approaches to TE conducted in other
+ standards bodies. It is out of the scope of this document to provide
+ an analysis of the history of TE or an inventory of TE-related
+ efforts conducted by other Standards Development Organizations
+ (SDOs).
+
+5.1. Overview of IETF Projects Related to Traffic Engineering
+
+ This subsection reviews a number of IETF activities pertinent to
+ Internet traffic engineering. Some of these technologies are widely
+ deployed, others are mature but have seen less deployment, and some
+ are unproven or are still under development.
+
+5.1.1. IETF TE Mechanisms
+
+5.1.1.1. Integrated Services
+
+ The IETF developed the Integrated Services (Intserv) model that
+ requires resources, such as bandwidth and buffers, to be reserved a
+ priori for a given traffic flow to ensure that the QoS requested by
+ the traffic flow is satisfied. The Intserv model includes additional
+ components beyond those used in the best-effort model such as packet
+ classifiers, packet schedulers, and admission control. A packet
+ classifier is used to identify flows that are to receive a certain
+ level of service. A packet scheduler handles the scheduling of
+ service to different packet flows to ensure that QoS commitments are
+ met. Admission control is used to determine whether a router has the
+ necessary resources to accept a new flow.
+
+ The main issue with the Intserv model has been scalability [RFC2998],
+ especially in large public IP networks that may potentially have
+ millions of active traffic flows in transit concurrently. Pre-
+ Congestion Notification (PCN) [RFC5559] solves the scaling problems
+ of Intserv by using measurement-based admission control (and flow
+ termination to handle failures) between edge nodes. Nodes between
+ the edges of the internetwork have no per-flow operations, and the
+ edge nodes can use the Resource Reservation Protocol (RSVP) per-flow
+ or per-aggregate.
+
+ A notable feature of the Intserv model is that it requires explicit
+ signaling of QoS requirements from end systems to routers [RFC2753].
+ RSVP performs this signaling function and is a critical component of
+ the Intserv model. RSVP is described in Section 5.1.3.2.
+
+5.1.1.2. Differentiated Services
+
+ The goal of Differentiated Services (Diffserv) within the IETF was to
+ devise scalable mechanisms for categorization of traffic into
+ behavior aggregates, which ultimately allows each behavior aggregate
+ to be treated differently, especially when there is a shortage of
+ resources, such as link bandwidth and buffer space [RFC2475]. One of
+ the primary motivations for Diffserv was to devise alternative
+ mechanisms for service differentiation in the Internet that mitigate
+ the scalability issues encountered with the Intserv model.
+
+ Diffserv uses the Differentiated Services field in the IP header (the
+ DS field) consisting of six bits in what was formerly known as the
+ Type of Service (TOS) octet. The DS field is used to indicate the
+ forwarding treatment that a packet should receive at a transit node
+ [RFC2474]. Diffserv includes the concept of Per-Hop Behavior (PHB)
+ groups. Using the PHBs, several classes of services can be defined
+ using different classification, policing, shaping, and scheduling
+ rules.
+
+ For an end user of network services to utilize Diffserv provided by
+ its Internet Service Provider (ISP), it may be necessary for the user
+ to have an SLA with the ISP. An SLA may explicitly or implicitly
+ specify a Traffic Conditioning Agreement (TCA) that defines
+ classifier rules as well as metering, marking, discarding, and
+ shaping rules.
+
+ Packets are classified and possibly policed and shaped at the ingress
+ to a Diffserv network. When a packet traverses the boundary between
+ different Diffserv domains, the DS field of the packet may be re-
+ marked according to existing agreements between the domains.
+
+ Diffserv allows only a finite number of service classes to be
+ specified by the DS field. The main advantage of the Diffserv
+ approach relative to the Intserv model is scalability. Resources are
+ allocated on a per-class basis, and the amount of state information
+ is proportional to the number of classes rather than to the number of
+ application flows.
+
+ Once the network has been planned and the packets have been marked at
+ the network edge, the Diffserv model deals with traffic management
+ issues on a per-hop basis. The Diffserv control model consists of a
+ collection of micro-TE control mechanisms. Other TE capabilities,
+ such as capacity management (including routing control), are also
+ required in order to deliver acceptable service quality in Diffserv
+ networks. The concept of "Per-Domain Behaviors" has been introduced
+ to better capture the notion of Diffserv across a complete domain
+ [RFC3086].
+
+ Diffserv procedures can also be applied in an MPLS context. See
+ Section 6.8 for more information.
+
+5.1.1.3. SR Policy
+
+ SR Policy [RFC9256] is an evolution of SR (see Section 5.1.3.12) to
+ enhance the TE capabilities of SR. It is a framework that enables
+ instantiation of an ordered list of segments on a node for
+ implementing a source routing policy with a specific intent for
+ traffic steering from that node.
+
+ An SR Policy is identified through the tuple <headend, color,
+ endpoint>. The headend is the IP address of the node where the
+ policy is instantiated. The endpoint is the IP address of the
+ destination of the policy. The color is an index that associates the
+ SR Policy with an intent (e.g., low latency).
+
+ The headend node is notified of SR Policies and associated SR paths
+ via configuration or by extensions to protocols such as the Path
+ Computation Element Communication Protocol (PCEP) [RFC8664] or BGP
+ [SR-TE-POLICY]. Each SR path consists of a segment list (an SR
+ source-routed path), and the headend uses the endpoint and color
+ parameters to classify packets to match the SR Policy and so
+ determine along which path to forward them. If an SR Policy is
+ associated with a set of SR paths, each is associated with a weight
+ for weighted load balancing. Furthermore, multiple SR Policies may
+ be associated with a set of SR paths to allow multiple traffic flows
+ to be placed on the same paths.
+
+ An SR Binding SID (BSID) may also be associated with each candidate
+ path associated with an SR Policy or with the SR Policy itself. The
+ headend node installs a BSID-keyed entry in the forwarding plane and
+ assigns it the action of steering packets that match the entry to the
+ selected path of the SR Policy. This steering can be done in various
+ ways:
+
+ SID Steering: Incoming packets have an active Segment Identifier
+ (SID) matching a local BSID at the headend.
+
+ Per-destination Steering: Incoming packets match a BGP/Service
+ route, which indicates an SR Policy.
+
+ Per-flow Steering: Incoming packets match a forwarding array (for
+ example, the classic 5-tuple), which indicates an SR Policy.
+
+ Policy-based Steering: Incoming packets match a routing policy,
+ which directs them to an SR Policy.
+
+5.1.1.4. Layer 4 Transport-Based TE
+
+ In addition to IP-based TE mechanisms, Layer 4 transport-based TE
+ approaches can be considered in specific deployment contexts (e.g.,
+ data centers and multi-homing). For example, the 3GPP defines the
+ Access Traffic Steering, Switching, and Splitting (ATSSS) [ATSSS]
+ service functions as follows:
+
+ Access Traffic Steering: This is the selection of an access network
+ for a new flow and the transfer of the traffic of that flow over
+ the selected access network.
+
+ Access Traffic Switching: This is the migration of all packets of an
+ ongoing flow from one access network to another access network.
+ Only one access network is in use at a time.
+
+ Access Traffic Splitting: This is about forwarding the packets of a
+ flow across multiple access networks simultaneously.
+
+ The control plane is used to provide hosts and specific network
+ devices with a set of policies that specify which flows are eligible
+ to use the ATSSS service. The traffic that matches an ATSSS policy
+ can be distributed among the available access networks following one
+ of the following four modes:
+
+ Active-Standby: The traffic is forwarded via a specific access
+ (called "active access") and switched to another access (called
+ "standby access") when the active access is unavailable.
+
+ Priority-based: Network accesses are assigned priority levels that
+ indicate which network access is to be used first. The traffic
+ associated with the matching flow will be steered onto the network
+ access with the highest priority until congestion is detected.
+ Then, the overflow will be forwarded over the next highest
+ priority access.
+
+ Load-Balancing: The traffic is distributed among the available
+ access networks following a distribution ratio (e.g., 75% to 25%).
+
+ Smallest Delay: The traffic is forwarded via the access that
+ presents the smallest round-trip time (RTT).
+
+ For resource management purposes, hosts and network devices support
+ means such as congestion control, RTT measurement, and packet
+ scheduling.
+
+ For TCP traffic, Multipath TCP [RFC8684] and the 0-RTT Convert
+ Protocol [RFC8803] are used to provide the ATSSS service.
+
+ Multipath QUIC [QUIC-MULTIPATH] and Proxying UDP in HTTP [RFC9298]
+ are used to provide the ATSSS service for UDP traffic. Note that
+ QUIC [RFC9000] supports the switching and steering functions.
+ Indeed, QUIC supports a connection migration procedure that allows
+ peers to change their Layer 4 transport coordinates (IP addresses,
+ port numbers) without breaking the underlying QUIC connection.
+
+ Extensions to the Datagram Congestion Control Protocol (DCCP)
+ [RFC4340] to support multipath operations are defined in
+ [MULTIPATH-DCCP].
+
+5.1.1.5. Deterministic Networking
+
+ Deterministic Networking (DetNet) [RFC8655] is an architecture for
+ applications with critical timing and reliability requirements. The
+ layered architecture particularly focuses on developing DetNet
+ service capabilities in the data plane [RFC8938]. The DetNet service
+ sub-layer provides a set of Packet Replication, Elimination, and
+ Ordering Functions (PREOF) to provide end-to-end service assurance.
+ The DetNet forwarding sub-layer provides corresponding forwarding
+ assurance (low packet loss, bounded latency, and in-order delivery)
+ functions using resource allocations and explicit route mechanisms.
+
+ The separation into two sub-layers allows a greater flexibility to
+ adapt DetNet capability over a number of TE data plane mechanisms,
+ such as IP, MPLS, and SR. More importantly, it interconnects IEEE
+ 802.1 Time Sensitive Networking (TSN) [RFC9023] deployed in Industry
+ Control and Automation Systems (ICAS).
+
+ DetNet can be seen as a specialized branch of TE, since it sets up
+ explicit optimized paths with allocation of resources as requested.
+ A DetNet application can express its QoS attributes or traffic
+ behavior using any combination of DetNet functions described in sub-
+ layers. They are then distributed and provisioned using well-
+ established control and provisioning mechanisms adopted for traffic
+ engineering.
+
+ In DetNet, a considerable amount of state information is required to
+ maintain per-flow queuing disciplines and resource reservation for a
+ large number of individual flows. This can be quite challenging for
+ network operations during network events, such as faults, change in
+ traffic volume, or reprovisioning. Therefore, DetNet recommends
+ support for aggregated flows; however, it still requires a large
+ amount of control signaling to establish and maintain DetNet flows.
+
+ Note that DetNet might suffer from some of the scalability concerns
+ described for Intserv in Section 5.1.1.1, but the scope of DetNet's
+ deployment scenarios is smaller and therefore less exposed to scaling
+ issues.
+
+5.1.2. IETF Approaches Relying on TE Mechanisms
+
+5.1.2.1. Application-Layer Traffic Optimization
+
+ This document describes various TE mechanisms available in the
+ network. However, in general, distributed applications
+ (particularly, bandwidth-greedy P2P applications that are used for
+ file sharing, for example) cannot directly use those techniques. As
+ per [RFC5693], applications could greatly improve traffic
+ distribution and quality by cooperating with external services that
+ are aware of the network topology. Addressing the Application-Layer
+ Traffic Optimization (ALTO) problem means, on the one hand, deploying
+ an ALTO service to provide applications with information regarding
+ the underlying network (e.g., basic network location structure and
+ preferences of network paths) and, on the other hand, enhancing
+ applications in order to use such information to perform better-than-
+ random selection of the endpoints with which they establish
+ connections.
+
+ The basic function of ALTO is based on abstract maps of a network.
+ These maps provide a simplified view, yet enough information about a
+ network for applications to effectively utilize them. Additional
+ services are built on top of the maps. [RFC7285] describes a
+ protocol implementing the ALTO services as an information-publishing
+ interface that allows a network to publish its network information to
+ network applications. This information can include network node
+ locations, groups of node-to-node connectivity arranged by cost
+ according to configurable granularities, and end-host properties.
+ The information published by the ALTO Protocol should benefit both
+ the network and the applications. The ALTO Protocol uses a REST-ful
+ design and encodes its requests and responses using JSON [RFC8259]
+ with a modular design by dividing ALTO information publication into
+ multiple ALTO services (e.g., the Map Service, the Map-Filtering
+ Service, the Endpoint Property Service, and the Endpoint Cost
+ Service).
+
+ [RFC8189] defines a new service that allows an ALTO Client to
+ retrieve several cost metrics in a single request for an ALTO
+ filtered cost map and endpoint cost map. [RFC8896] extends the ALTO
+ cost information service so that applications decide not only "where"
+ to connect but also "when". This is useful for applications that
+ need to perform bulk data transfer and would like to schedule these
+ transfers during an off-peak hour, for example. [RFC9439] introduces
+ network performance metrics, including network delay, jitter, packet
+ loss rate, hop count, and bandwidth. The ALTO server may derive and
+ aggregate such performance metrics from BGP-LS (see
+ Section 5.1.3.10), IGP-TE (see Section 5.1.3.9), or management tools
+ and then expose the information to allow applications to determine
+ "where" to connect based on network performance criteria. The ALTO
+ Working Group is evaluating the use of network TE properties while
+ making application decisions for new use cases such as edge computing
+ and data-center interconnect.
+
+5.1.2.2. Network Virtualization and Abstraction
+
+ One of the main drivers for SDN [RFC7149] is a decoupling of the
+ network control plane from the data plane. This separation has been
+ achieved for TE networks with the development of MPLS and GMPLS (see
+ Sections 5.1.3.3 and 5.1.3.5, respectively) and the PCE (see
+ Section 5.1.3.11). One of the advantages of SDN is its logically
+ centralized control regime that allows a full view of the underlying
+ networks. Centralized control in SDN helps improve network resource
+ utilization compared with distributed network control.
+
+ Abstraction and Control of TE Networks (ACTN) [RFC8453] defines a
+ hierarchical SDN architecture that describes the functional entities
+ and methods for the coordination of resources across multiple
+ domains, to provide composite traffic-engineered services. ACTN
+ facilitates composed, multi-domain connections and provides them to
+ the user. ACTN is focused on:
+
+ * Abstraction of the underlying network resources and how they are
+ provided to higher-layer applications and customers.
+
+ * Virtualization of underlying resources for use by the customer,
+ application, or service. The creation of a virtualized
+ environment allows operators to view and control multi-domain
+ networks as a single virtualized network.
+
+ * Presentation to customers of networks as a virtual network via
+ open and programmable interfaces.
+
+ The ACTN managed infrastructure is built from traffic-engineered
+ network resources, which may include statistical packet bandwidth,
+ physical forwarding-plane sources (such as wavelengths and time
+ slots), and forwarding and cross-connect capabilities. The type of
+ network virtualization seen in ACTN allows customers and applications
+ (tenants) to utilize and independently control allocated virtual
+ network resources as if they were physically their own resource. The
+ ACTN network is sliced, with tenants being given a different partial
+ and abstracted topology view of the physical underlying network.
+
+5.1.2.3. Network Slicing
+
+ An IETF Network Slice is a logical network topology connecting a
+ number of endpoints using a set of shared or dedicated network
+ resources [NETWORK-SLICES]. The resources are used to satisfy
+ specific SLOs specified by the consumer.
+
+ IETF Network Slices are not, of themselves, TE constructs. However,
+ a network operator that offers IETF Network Slices is likely to use
+ many TE tools in order to manage their network and provide the
+ services.
+
+ IETF Network Slices are defined such that they are independent of the
+ underlying infrastructure connectivity and technologies used. From a
+ customer's perspective, an IETF Network Slice looks like a VPN
+ connectivity matrix with additional information about the level of
+ service that the customer requires between the endpoints. From an
+ operator's perspective, the IETF Network Slice looks like a set of
+ routing or tunneling instructions with the network resource
+ reservations necessary to provide the required service levels as
+ specified by the SLOs. The concept of an IETF Network Slice is
+ consistent with an enhanced VPN [ENHANCED-VPN].
+
+5.1.3. IETF Techniques Used by TE Mechanisms
+
+5.1.3.1. Constraint-Based Routing
+
+ Constraint-based routing refers to a class of routing systems that
+ compute routes through a network subject to the satisfaction of a set
+ of constraints and requirements. In the most general case,
+ constraint-based routing may also seek to optimize overall network
+ performance while minimizing costs.
+
+ The constraints and requirements may be imposed by the network itself
+ or by administrative policies. Constraints may include bandwidth,
+ hop count, delay, and policy instruments such as resource class
+ attributes. Constraints may also include domain-specific attributes
+ of certain network technologies and contexts that impose restrictions
+ on the solution space of the routing function. Path-oriented
+ technologies such as MPLS have made constraint-based routing feasible
+ and attractive in public IP networks.
+
+ The concept of constraint-based routing within the context of MPLS-TE
+ requirements in IP networks was first described in [RFC2702] and led
+ to developments such as MPLS-TE [RFC3209] as described in
+ Section 5.1.3.3.
+
+ Unlike QoS-based routing (for example, see [RFC2386], [MA], and
+ [PERFORMANCE-ROUTING]) that generally addresses the issue of routing
+ individual traffic flows to satisfy prescribed flow-based QoS
+ requirements subject to network resource availability, constraint-
+ based routing is applicable to traffic aggregates as well as flows
+ and may be subject to a wide variety of constraints that may include
+ policy restrictions.
+
+5.1.3.1.1. IGP Flexible Algorithms
+
+ The normal approach to routing in an IGP network relies on the IGPs
+ deriving "shortest paths" over the network based solely on the IGP
+ metric assigned to the links. Such an approach is often limited:
+ traffic may tend to converge toward the destination, possibly causing
+ congestion, and it is not possible to steer traffic onto paths
+ depending on the end-to-end qualities demanded by the applications.
+
+ To overcome this limitation, various sorts of TE have been widely
+ deployed (as described in this document), where the TE component is
+ responsible for computing the path based on additional metrics and/or
+ constraints. Such paths (or tunnels) need to be installed in the
+ routers' forwarding tables in addition to, or as a replacement for,
+ the original paths computed by IGPs. The main drawbacks of these TE
+ approaches are the additional complexity of protocols and management
+ and the state that may need to be maintained within the network.
+
+ IGP Flexible Algorithms [RFC9350] allow IGPs to construct constraint-
+ based paths over the network by computing constraint-based next hops.
+ The intent of Flexible Algorithms is to reduce TE complexity by
+ letting an IGP perform some basic TE computation capabilities.
+ Flexible Algorithm includes a set of extensions to the IGPs that
+ enable a router to send TLVs that:
+
+ * describe a set of constraints on the topology
+
+ * identify calculation-type
+
+ * describe a metric-type that is to be used to compute the best
+ paths through the constrained topology
+
+ A given combination of calculation-type, metric-type, and constraints
+ is known as a Flexible Algorithm Definition (FAD). A router that
+ sends such a set of TLVs also assigns a specific identifier (the
+ Flexible Algorithm) to the specified combination of calculation-type,
+ metric-type, and constraints.
+
+ There are two use cases for Flexible Algorithm: in IP networks
+ [RFC9502] and in SR networks [RFC9350]. In the first case, Flexible
+ Algorithm computes paths to an IPv4 or IPv6 address; in the second
+ case, Flexible Algorithms computes paths to a Prefix SID (see
+ Section 5.1.3.12).
+
+ Examples of where Flexible Algorithms can be useful include:
+
+ * Expansion of the function of IP performance metrics [RFC5664]
+ where specific constraint-based routing (Flexible Algorithm) can
+ be instantiated within the network based on the results of
+ performance measurement.
+
+ * The formation of an "underlay" network using Flexible Algorithms,
+ and the realization of an "overlay" network using TE techniques.
+ This approach can leverage the nested combination of Flexible
+ Algorithm and TE extensions for IGP (see Section 5.1.3.9).
+
+ * Flexible Algorithms in SR-MPLS (Section 5.1.3.12) can be used as a
+ base to easily build a TE-like topology without TE components on
+ routers or the use of a PCE (see Section 5.1.3.11).
+
+ * The support for network slices [NETWORK-SLICES] where the SLOs of
+ a particular IETF Network Slice can be guaranteed by a Flexible
+ Algorithm or where a Filtered Topology [NETWORK-SLICES] can be
+ created as a TE-like topology using a Flexible Algorithm.
+
+5.1.3.2. RSVP
+
+ RSVP is a soft-state signaling protocol [RFC2205]. It supports
+ receiver-initiated establishment of resource reservations for both
+ multicast and unicast flows. RSVP was originally developed as a
+ signaling protocol within the Integrated Services framework (see
+ Section 5.1.1.1) for applications to communicate QoS requirements to
+ the network and for the network to reserve relevant resources to
+ satisfy the QoS requirements [RFC2205].
+
+ In RSVP, the traffic sender or source node sends a Path message to
+ the traffic receiver with the same source and destination addresses
+ as the traffic that the sender will generate. The Path message
+ contains:
+
+ * A sender traffic specification describing the characteristics of
+ the traffic
+
+ * A sender template specifying the format of the traffic
+
+ * An optional advertisement specification that is used to support
+ the concept of One Pass With Advertising (OPWA) [RFC2205]
+
+ Every intermediate router along the path forwards the Path message to
+ the next hop determined by the routing protocol. Upon receiving a
+ Path message, the receiver responds with a Resv message that includes
+ a flow descriptor used to request resource reservations. The Resv
+ message travels to the sender or source node in the opposite
+ direction along the path that the Path message traversed. Every
+ intermediate router along the path can reject or accept the
+ reservation request of the Resv message. If the request is rejected,
+ the rejecting router will send an error message to the receiver, and
+ the signaling process will terminate. If the request is accepted,
+ link bandwidth and buffer space are allocated for the flow, and the
+ related flow state information is installed in the router.
+
+ One of the issues with the original RSVP specification [RFC2205] was
+ scalability. This was because reservations were required for micro-
+ flows, so that the amount of state maintained by network elements
+ tended to increase linearly with the number of traffic flows. These
+ issues are described in [RFC2961], which also modifies and extends
+ RSVP to mitigate the scaling problems to make RSVP a versatile
+ signaling protocol for the Internet. For example, RSVP has been
+ extended to reserve resources for aggregation of flows [RFC3175], to
+ set up MPLS explicit LSPs (see Section 5.1.3.3), and to perform other
+ signaling functions within the Internet. [RFC2961] also describes a
+ mechanism to reduce the amount of Refresh messages required to
+ maintain established RSVP sessions.
+
+5.1.3.3. MPLS
+
+ MPLS is a forwarding scheme that also includes extensions to
+ conventional IP control plane protocols. MPLS extends the Internet
+ routing model and enhances packet forwarding and path control
+ [RFC3031].
+
+ At the ingress to an MPLS domain, LSRs classify IP packets into
+ Forwarding Equivalence Classes (FECs) based on a variety of factors,
+ including, e.g., a combination of the information carried in the IP
+ header of the packets and the local routing information maintained by
+ the LSRs. An MPLS label stack entry is then prepended to each packet
+ according to their FECs. The MPLS label stack entry is 32 bits long
+ and contains a 20-bit label field.
+
+ An LSR makes forwarding decisions by using the label prepended to
+ packets as the index into a local Next Hop Label Forwarding Entry
+ (NHLFE). The packet is then processed as specified in the NHLFE.
+ The incoming label may be replaced by an outgoing label (label swap),
+ and the packet may be forwarded to the next LSR. Before a packet
+ leaves an MPLS domain, its MPLS label may be removed (label pop). An
+ LSP is the path between an ingress LSR and an egress LSR through
+ which a labeled packet traverses. The path of an explicit LSP is
+ defined at the originating (ingress) node of the LSP. MPLS can use a
+ signaling protocol such as RSVP or the Label Distribution Protocol
+ (LDP) to set up LSPs.
+
+ MPLS is a powerful technology for Internet TE because it supports
+ explicit LSPs that allow constraint-based routing to be implemented
+ efficiently in IP networks [AWD2]. The requirements for TE over MPLS
+ are described in [RFC2702]. Extensions to RSVP to support
+ instantiation of explicit LSP are discussed in [RFC3209] and
+ Section 5.1.3.4.
+
+5.1.3.4. RSVP-TE
+
+ RSVP-TE is a protocol extension of RSVP (Section 5.1.3.2) for traffic
+ engineering. The base specification is found in [RFC3209]. RSVP-TE
+ enables the establishment of traffic-engineered MPLS LSPs (TE LSPs),
+ using loose or strict paths and taking into consideration network
+ constraints such as available bandwidth. The extension supports
+ signaling LSPs on explicit paths that could be administratively
+ specified or computed by a suitable entity (such as a PCE,
+ Section 5.1.3.11) based on QoS and policy requirements, taking into
+ consideration the prevailing network state as advertised by the IGP
+ extension for IS-IS in [RFC5305], for OSPFv2 in [RFC3630], and for
+ OSPFv3 in [RFC5329]. RSVP-TE enables the reservation of resources
+ (for example, bandwidth) along the path.
+
+ RSVP-TE includes the ability to preempt LSPs based on priorities and
+ uses link affinities to include or exclude links from the LSPs. The
+ protocol is further extended to support Fast Reroute (FRR) [RFC4090],
+ Diffserv [RFC4124], and bidirectional LSPs [RFC7551]. RSVP-TE
+ extensions for support for GMPLS (see Section 5.1.3.5) are specified
+ in [RFC3473].
+
+ Requirements for point-to-multipoint (P2MP) MPLS-TE LSPs are
+ documented in [RFC4461], and signaling protocol extensions for
+ setting up P2MP MPLS-TE LSPs via RSVP-TE are defined in [RFC4875],
+ where a P2MP LSP comprises multiple source-to-leaf (S2L) sub-LSPs.
+ To determine the paths for P2MP LSPs, selection of the branch points
+ (based on capabilities, network state, and policies) is key [RFC5671]
+
+ RSVP-TE has evolved to provide real-time dynamic metrics for path
+ selection for low-latency paths using extensions to IS-IS [RFC8570]
+ and OSPF [RFC7471] based on performance measurements for the Simple
+ Two-Way Active Measurement Protocol (STAMP) [RFC8972] and the Two-Way
+ Active Measurement Protocol (TWAMP) [RFC5357].
+
+ RSVP-TE has historically been used when bandwidth was constrained;
+ however, as bandwidth has increased, RSVP-TE has developed into a
+ bandwidth management tool to provide bandwidth efficiency and
+ proactive resource management.
+
+5.1.3.5. Generalized MPLS (GMPLS)
+
+ GMPLS extends MPLS control protocols to encompass time-division
+ (e.g., Synchronous Optical Network / Synchronous Digital Hierarchy
+ (SONET/SDH), Plesiochronous Digital Hierarchy (PDH), and Optical
+ Transport Network (OTN)), wavelength (lambdas), and spatial switching
+ (e.g., incoming port or fiber to outgoing port or fiber) and
+ continues to support packet switching. GMPLS provides a common set
+ of control protocols for all of these layers (including some
+ technology-specific extensions), each of which has a distinct data or
+ forwarding plane. GMPLS covers both the signaling and the routing
+ part of that control plane and is based on the TE extensions to MPLS
+ (see Section 5.1.3.4).
+
+ In GMPLS [RFC3945], the original MPLS architecture is extended to
+ include LSRs whose forwarding planes rely on circuit switching and
+ therefore cannot forward data based on the information carried in
+ either packet or cell headers. Specifically, such LSRs include
+ devices where the switching is based on time slots, wavelengths, or
+ physical ports. These additions impact basic LSP properties: how
+ labels are requested and communicated, the unidirectional nature of
+ MPLS LSPs, how errors are propagated, and information provided for
+ synchronizing the ingress and egress LSRs [RFC3473].
+
+5.1.3.6. IP Performance Metrics (IPPM)
+
+ The IETF IP Performance Metrics (IPPM) Working Group has developed a
+ set of standard metrics that can be used to monitor the quality,
+ performance, and reliability of Internet services. These metrics can
+ be applied by network operators, end users, and independent testing
+ groups to provide users and service providers with a common
+ understanding of the performance and reliability of the Internet
+ component clouds they use/provide [RFC2330]. The criteria for
+ performance metrics developed by the IPPM Working Group are described
+ in [RFC2330]. Examples of performance metrics include one-way packet
+ loss [RFC7680], one-way delay [RFC7679], and connectivity measures
+ between two nodes [RFC2678]. Other metrics include second-order
+ measures of packet loss and delay.
+
+ Some of the performance metrics specified by the IPPM Working Group
+ are useful for specifying SLAs. SLAs are sets of SLOs negotiated
+ between users and service providers, wherein each objective is a
+ combination of one or more performance metrics, possibly subject to
+ certain constraints.
+
+ The IPPM Working Group also designs measurement techniques and
+ protocols to obtain these metrics.
+
+5.1.3.7. Flow Measurement
+
+ The IETF Real Time Flow Measurement (RTFM) Working Group produced an
+ architecture that defines a method to specify traffic flows as well
+ as a number of components for flow measurement (meters, meter
+ readers, and managers) [RFC2722]. A flow measurement system enables
+ network traffic flows to be measured and analyzed at the flow level
+ for a variety of purposes. As noted in [RFC2722], a flow measurement
+ system can be very useful in the following contexts:
+
+ * understanding the behavior of existing networks
+
+ * planning for network development and expansion
+
+ * quantification of network performance
+
+ * verifying the quality of network service
+
+ * attribution of network usage to users
+
+ A flow measurement system consists of meters, meter readers, and
+ managers. A meter observes packets passing through a measurement
+ point, classifies them into groups, accumulates usage data (such as
+ the number of packets and bytes for each group), and stores the usage
+ data in a flow table. A group may represent any collection of user
+ applications, hosts, networks, etc. A meter reader gathers usage
+ data from various meters so it can be made available for analysis. A
+ manager is responsible for configuring and controlling meters and
+ meter readers. The instructions received by a meter from a manager
+ include flow specifications, meter control parameters, and sampling
+ techniques. The instructions received by a meter reader from a
+ manager include the address of the meter whose data are to be
+ collected, the frequency of data collection, and the types of flows
+ to be collected.
+
+ IP Flow Information Export (IPFIX) [RFC5470] defines an architecture
+ that is very similar to the RTFM architecture and includes Metering,
+ Exporting, and Collecting Processes. [RFC5472] describes the
+ applicability of IPFIX and makes a comparison with RTFM, pointing out
+ that, architecturally, while RTM talks about devices, IPFIX deals
+ with processes to clarify that multiple of those processes may be co-
+ located on the same machine. The IPFIX protocol [RFC7011] is widely
+ implemented.
+
+5.1.3.8. Endpoint Congestion Management
+
+ [RFC3124] provides a set of congestion control mechanisms for the use
+ of transport protocols. It also allows the development of mechanisms
+ for unifying congestion control across a subset of an endpoint's
+ active unicast connections (called a "congestion group"). A
+ congestion manager continuously monitors the state of the path for
+ each congestion group under its control. The manager uses that
+ information to instruct a scheduler on how to partition bandwidth
+ among the connections of that congestion group.
+
+ The concepts described in [RFC3124] and the lessons that can be
+ learned from that work found a home in HTTP/2 [RFC9113] and QUIC
+ [RFC9000], while [RFC9040] describes TCP control block
+ interdependence that is a core construct underpinning the congestion
+ manager defined in [RFC3124].
+
+5.1.3.9. TE Extensions to the IGPs
+
+ [RFC5305] describes the extensions to the Intermediate System to
+ Intermediate System (IS-IS) protocol to support TE. Similarly,
+ [RFC3630] specifies TE extensions for OSPFv2, and [RFC5329] has the
+ same description for OSPFv3.
+
+ IS-IS and OSPF share the common concept of TE extensions to
+ distribute TE parameters, such as link type and ID, local and remote
+ IP addresses, TE metric, maximum bandwidth, maximum reservable
+ bandwidth, unreserved bandwidth, and admin group. The information
+ distributed by the IGPs in this way can be used to build a view of
+ the state and capabilities of a TE network (see Section 5.1.3.14).
+
+ The difference between IS-IS and OSPF is in the details of how they
+ encode and transmit the TE parameters:
+
+ * IS-IS uses the Extended IS Reachability TLV (type 22), the
+ Extended IP Reachability TLV (type 135), and the Traffic
+ Engineering router ID TLV (type 134). These TLVs use specific
+ sub-TLVs described in [RFC8570] to carry the TE parameters.
+
+ * OSPFv2 uses Opaque LSA [RFC5250] type 10, and OSPFv3 uses the
+ Intra-Area-TE-LSA. In both OSPF cases, two top-level TLVs are
+ used (Router Address and Link TLVs), and these use sub-TLVs to
+ carry the TE parameters (as defined in [RFC7471] for OSPFv2 and
+ [RFC5329] for OSPFv3).
+
+5.1.3.10. BGP - Link State
+
+ In a number of environments, a component external to a network is
+ called upon to perform computations based on the network topology and
+ current state of the connections within the network, including TE
+ information. This is information typically distributed by IGP
+ routing protocols within the network (see Section 5.1.3.9).
+
+ BGP (see also Section 7) is one of the essential routing protocols
+ that glues the Internet together. BGP-LS [RFC9552] is a mechanism by
+ which link-state and TE information can be collected from networks
+ and shared with external components using the BGP routing protocol.
+ The mechanism is applicable to physical and virtual IGP links and is
+ subject to policy control.
+
+ Information collected by BGP-LS can be used, for example, to
+ construct the TED (Section 5.1.3.14) for use by the PCE (see
+ Section 5.1.3.11) or may be used by ALTO servers (see
+ Section 5.1.2.1).
+
+5.1.3.11. Path Computation Element
+
+ Constraint-based path computation is a fundamental building block for
+ TE in MPLS and GMPLS networks. Path computation in large, multi-
+ domain networks is complex and may require special computational
+ components and cooperation between the elements in different domains.
+ The PCE [RFC4655] is an entity (component, application, or network
+ node) that is capable of computing a network path or route based on a
+ network graph and applying computational constraints.
+
+ Thus, a PCE can provide a central component in a TE system operating
+ on the TED (see Section 5.1.3.14) with delegated responsibility for
+ determining paths in MPLS, GMPLS, or SR networks. The PCE uses the
+ Path Computation Element Communication Protocol (PCEP) [RFC5440] to
+ communicate with Path Computation Clients (PCCs), such as MPLS LSRs,
+ to answer their requests for computed paths or to instruct them to
+ initiate new paths [RFC8281] and maintain state about paths already
+ installed in the network [RFC8231].
+
+ PCEs form key components of a number of TE systems. More information
+ about the applicability of PCEs can be found in [RFC8051], while
+ [RFC6805] describes the application of PCEs to determining paths
+ across multiple domains. PCEs also have potential uses in
+ Abstraction and Control of TE Networks (ACTN) (see Section 5.1.2.2),
+ Centralized Network Control [RFC8283], and SDN (see Section 4.3.2).
+
+5.1.3.12. Segment Routing (SR)
+
+ The SR architecture [RFC8402] leverages the source routing and
+ tunneling paradigms. The path a packet takes is defined at the
+ ingress, and the packet is tunneled to the egress.
+
+ In a protocol realization, an ingress node steers a packet using a
+ set of instructions, called "segments", that are included in an SR
+ header prepended to the packet: a label stack in MPLS case, and a
+ series of 128-bit SIDs in the IPv6 case.
+
+ Segments are identified by SIDs. There are four types of SIDs that
+ are relevant for TE.
+
+ * Prefix SID: A SID that is unique within the routing domain and is
+ used to identify a prefix.
+
+ * Node SID: A Prefix SID with the "N" bit set to identify a node.
+
+ * Adjacency SID: Identifies a unidirectional adjacency.
+
+ * Binding SID: A Binding SID has two purposes:
+
+ 1. To advertise the mappings of prefixes to SIDs/Labels
+
+ 2. To advertise a path available for a Forwarding Equivalence
+ Class (FEC)
+
+ A segment can represent any instruction, topological or service-
+ based. SIDs can be looked up in a global context (domain-wide) as
+ well as in some other contexts (see, for example, "context labels" in
+ Section 3 of [RFC5331]).
+
+ The application of policy to SR can make SR into a TE mechanism, as
+ described in Section 5.1.1.3.
+
+5.1.3.13. Tree Engineering for Bit Index Explicit Replication
+
+ Bit Index Explicit Replication (BIER) [RFC8279] specifies an
+ encapsulation for multicast forwarding that can be used on MPLS or
+ Ethernet transports. A mechanism known as Tree Engineering for Bit
+ Index Explicit Replication (BIER-TE) [RFC9262] provides a component
+ that could be used to build a traffic-engineered multicast system.
+ BIER-TE does not of itself offer full traffic engineering, and the
+ abbreviation "TE" does not, in this case, refer to traffic
+ engineering.
+
+ In BIER-TE, path steering is supported via the definition of a
+ bitstring attached to each packet that determines how the packet is
+ forwarded and replicated within the network. Thus, this bitstring
+ steers the traffic within the network and forms an element of a
+ traffic-engineering system. A central controller that is aware of
+ the capabilities and state of the network as well as the demands of
+ the various traffic flows is able to select multicast paths that take
+ account of the available resources and demands. Therefore, this
+ controller is responsible for the policy elements of traffic
+ engineering.
+
+ Resource management has implications for the forwarding plane beyond
+ the steering of packets defined for BIER-TE. These include the
+ allocation of buffers to meet the requirements of admitted traffic
+ and may include policing and/or rate-shaping mechanisms achieved via
+ various forms of queuing. This level of resource control, while
+ optional, is important in networks that wish to support congestion
+ management policies to control or regulate the offered traffic to
+ deliver different levels of service and alleviate congestion
+ problems. It is also important in networks that wish to control
+ latencies experienced by specific traffic flows.
+
+5.1.3.14. Network TE State Definition and Presentation
+
+ The network states that are relevant to TE need to be stored in the
+ system and presented to the user. The TED is a collection of all TE
+ information about all TE nodes and TE links in the network. It is an
+ essential component of a TE system, such as MPLS-TE [RFC2702] or
+ GMPLS [RFC3945]. In order to formally define the data in the TED and
+ to present the data to the user, the data modeling language YANG
+ [RFC7950] can be used as described in [RFC8795].
+
+5.1.3.15. System Management and Control Interfaces
+
+ The TE control system needs to have a management interface that is
+ human-friendly and a control interface that is programmable for
+ automation. The Network Configuration Protocol (NETCONF) [RFC6241]
+ and the RESTCONF protocol [RFC8040] provide programmable interfaces
+ that are also human-friendly. These protocols use XML- or JSON-
+ encoded messages. When message compactness or protocol bandwidth
+ consumption needs to be optimized for the control interface, other
+ protocols, such as Group Communication for the Constrained
+ Application Protocol (CoAP) [RFC7390] or gRPC [GRPC], are available,
+ especially when the protocol messages are encoded in a binary format.
+ Along with any of these protocols, the data modeling language YANG
+ [RFC7950] can be used to formally and precisely define the interface
+ data.
+
+ PCEP [RFC5440] is another protocol that has evolved to be an option
+ for the TE system control interface. PCEP messages are TLV based;
+ they are not defined by a data-modeling language such as YANG.
+
+5.2. Content Distribution
+
+ The Internet is dominated by client-server interactions, principally
+ web traffic and multimedia streams, although in the future, more
+ sophisticated media servers may become dominant. The location and
+ performance of major information servers have a significant impact on
+ the traffic patterns within the Internet as well as on the perception
+ of service quality by end users.
+
+ A number of dynamic load-balancing techniques have been devised to
+ improve the performance of replicated information servers. These
+ techniques can cause spatial traffic characteristics to become more
+ dynamic in the Internet because information servers can be
+ dynamically picked based upon the location of the clients, the
+ location of the servers, the relative utilization of the servers, the
+ relative performance of different networks, and the relative
+ performance of different parts of a network. This process of
+ assignment of distributed servers to clients is called "traffic
+ directing". It is an application-layer function.
+
+ Traffic-directing schemes that allocate servers in multiple
+ geographically dispersed locations to clients may require empirical
+ network performance statistics to make more effective decisions. In
+ the future, network measurement systems may need to provide this type
+ of information.
+
+ When congestion exists in the network, traffic-directing and traffic-
+ engineering systems should act in a coordinated manner. This topic
+ is for further study.
+
+ The issues related to location and replication of information
+ servers, particularly web servers, are important for Internet traffic
+ engineering because these servers contribute a substantial proportion
+ of Internet traffic.
+
+6. Recommendations for Internet Traffic Engineering
+
+ This section describes high-level recommendations for traffic
+ engineering in the Internet in general terms.
+
+ The recommendations describe the capabilities needed to solve a TE
+ problem or to achieve a TE objective. Broadly speaking, these
+ recommendations can be categorized as either functional or non-
+ functional recommendations:
+
+ * Functional recommendations describe the functions that a traffic-
+ engineering system should perform. These functions are needed to
+ realize TE objectives by addressing traffic-engineering problems.
+
+ * Non-functional recommendations relate to the quality attributes or
+ state characteristics of a TE system. These recommendations may
+ contain conflicting assertions and may sometimes be difficult to
+ quantify precisely.
+
+ The subsections that follow first summarize the non-functional
+ requirements and then detail the functional requirements.
+
+6.1. Generic Non-functional Recommendations
+
+ The generic non-functional recommendations for Internet traffic
+ engineering are listed in the paragraphs that follow. In a given
+ context, some of these recommendations may be critical while others
+ may be optional. Therefore, prioritization may be required during
+ the development phase of a TE system to tailor it to a specific
+ operational context.
+
+ Automation: Whenever feasible, a TE system should automate as many
+ TE functions as possible to minimize the amount of human effort
+ needed to analyze and control operational networks. Automation is
+ particularly important in large-scale public networks because of
+ the high cost of the human aspects of network operations and the
+ high risk of network problems caused by human errors. Automation
+ may additionally benefit from feedback from the network that
+ indicates the state of network resources and the current load in
+ the network. Further, placing intelligence into components of the
+ TE system could enable automation to be more dynamic and
+ responsive to changes in the network.
+
+ Flexibility: A TE system should allow for changes in optimization
+ policy. In particular, a TE system should provide sufficient
+ configuration options so that a network administrator can tailor
+ the system to a particular environment. It may also be desirable
+ to have both online and offline TE subsystems that can be
+ independently enabled and disabled. TE systems that are used in
+ multi-class networks should also have options to support class-
+ based performance evaluation and optimization.
+
+ Interoperability: Whenever feasible, TE systems and their components
+ should be developed with open standards-based interfaces to allow
+ interoperation with other systems and components.
+
+ Scalability: Public networks continue to grow rapidly with respect
+ to network size and traffic volume. Therefore, to remain
+ applicable as the network evolves, a TE system should be scalable.
+ In particular, a TE system should remain functional as the network
+ expands with regard to the number of routers and links and with
+ respect to the number of flows and the traffic volume. A TE
+ system should have a scalable architecture, should not adversely
+ impair other functions and processes in a network element, and
+ should not consume too many network resources when collecting and
+ distributing state information or when exerting control.
+
+ Security: Security is a critical consideration in TE systems. Such
+ systems typically exert control over functional aspects of the
+ network to achieve the desired performance objectives. Therefore,
+ adequate measures must be taken to safeguard the integrity of the
+ TE system. Adequate measures must also be taken to protect the
+ network from vulnerabilities that originate from security breaches
+ and other impairments within the TE system.
+
+ Simplicity: A TE system should be as simple as possible. Simplicity
+ in user interface does not necessarily imply that the TE system
+ will use naive algorithms. When complex algorithms and internal
+ structures are used, the user interface should hide such
+ complexities from the network administrator as much as possible.
+
+ Stability: Stability refers to the resistance of the network to
+ oscillate (flap) in a disruptive manner from one state to another,
+ which may result in traffic being routed first one way and then
+ another without satisfactory resolution of the underlying TE
+ issues and with continued changes that do not settle down.
+ Stability is a very important consideration in TE systems that
+ respond to changes in the state of the network. State-dependent
+ TE methodologies typically include a trade-off between
+ responsiveness and stability. It is strongly recommended that
+ when a trade-off between responsiveness and stability is needed,
+ it should be made in favor of stability (especially in public IP
+ backbone networks).
+
+ Usability: Usability is a human aspect of TE systems. It refers to
+ the ease with which a TE system can be deployed and operated. In
+ general, it is desirable to have a TE system that can be readily
+ deployed in an existing network. It is also desirable to have a
+ TE system that is easy to operate and maintain.
+
+ Visibility: Mechanisms should exist as part of the TE system to
+ collect statistics from the network and to analyze these
+ statistics to determine how well the network is functioning.
+ Derived statistics (such as traffic matrices, link utilization,
+ latency, packet loss, and other performance measures of interest)
+ that are determined from network measurements can be used as
+ indicators of prevailing network conditions. The capabilities of
+ the various components of the routing system are other examples of
+ status information that should be observable.
+
+6.2. Routing Recommendations
+
+ Routing control is a significant aspect of Internet traffic
+ engineering. Routing impacts many of the key performance measures
+ associated with networks, such as throughput, delay, and utilization.
+ Generally, it is very difficult to provide good service quality in a
+ wide area network without effective routing control. A desirable TE
+ routing system is one that takes traffic characteristics and network
+ constraints into account during route selection while maintaining
+ stability.
+
+ Shortest Path First (SPF) IGPs are based on shortest path algorithms
+ and have limited control capabilities for TE [RFC2702] [AWD2]. These
+ limitations include:
+
+ 1. Pure SPF protocols do not take network constraints and traffic
+ characteristics into account during route selection. For
+ example, IGPs always select the shortest paths based on link
+ metrics assigned by administrators, so load sharing cannot be
+ performed across paths of different costs. Note that link
+ metrics are assigned following a range of operator-selected
+ policies that might reflect preference for the use of some links
+ over others; therefore, "shortest" might not be purely a measure
+ of distance. Using shortest paths to forward traffic may cause
+ the following problems:
+
+ * If traffic from a source to a destination exceeds the capacity
+ of a link along the shortest path, the link (and hence the
+ shortest path) becomes congested while a longer path between
+ these two nodes may be under-utilized.
+
+ * The shortest paths from different sources can overlap at some
+ links. If the total traffic from the sources exceeds the
+ capacity of any of these links, congestion will occur.
+
+ * Problems can also occur because traffic demand changes over
+ time, but network topology and routing configuration cannot be
+ changed as rapidly. This causes the network topology and
+ routing configuration to become sub-optimal over time, which
+ may result in persistent congestion problems.
+
+ 2. The Equal-Cost Multipath (ECMP) capability of SPF IGPs supports
+ sharing of traffic among equal-cost paths. However, ECMP
+ attempts to divide the traffic as equally as possible among the
+ equal-cost shortest paths. Generally, ECMP does not support
+ configurable load-sharing ratios among equal-cost paths. The
+ result is that one of the paths may carry significantly more
+ traffic than other paths because it may also carry traffic from
+ other sources. This situation can result in congestion along the
+ path that carries more traffic. Weighted ECMP (WECMP) (see, for
+ example, [EVPN-UNEQUAL-LB]) provides some mitigation.
+
+ 3. Modifying IGP metrics to control traffic routing tends to have
+ network-wide effects. Consequently, undesirable and
+ unanticipated traffic shifts can be triggered as a result. Work
+ described in Section 8 may be capable of better control [FT00]
+ [FT01].
+
+ Because of these limitations, capabilities are needed to enhance the
+ routing function in IP networks. Some of these capabilities are
+ summarized below:
+
+ * Constraint-based routing computes routes to fulfill requirements
+ subject to constraints. This can be useful in public IP backbones
+ with complex topologies. Constraints may include bandwidth, hop
+ count, delay, and administrative policy instruments, such as
+ resource class attributes [RFC2702] [RFC2386]. This makes it
+ possible to select routes that satisfy a given set of
+ requirements. Routes computed by constraint-based routing are not
+ necessarily the shortest paths. Constraint-based routing works
+ best with path-oriented technologies that support explicit
+ routing, such as MPLS.
+
+ * Constraint-based routing can also be used as a way to distribute
+ traffic onto the infrastructure, including for best-effort
+ traffic. For example, congestion problems caused by uneven
+ traffic distribution may be avoided or reduced by knowing the
+ reservable bandwidth attributes of the network links and by
+ specifying the bandwidth requirements for path selection.
+
+ * A number of enhancements to the link-state IGPs allow them to
+ distribute additional state information required for constraint-
+ based routing. The extensions to OSPF are described in [RFC3630],
+ and the extensions to IS-IS are described in [RFC5305]. Some of
+ the additional topology state information includes link
+ attributes, such as reservable bandwidth and link resource class
+ attribute (an administratively specified property of the link).
+ The resource class attribute concept is defined in [RFC2702]. The
+ additional topology state information is carried in new TLVs and
+ sub-TLVs in IS-IS [RFC5305] or in the Opaque LSA in OSPF
+ [RFC3630].
+
+ * An enhanced link-state IGP may flood information more frequently
+ than a normal IGP. This is because even without changes in
+ topology, changes in reservable bandwidth or link affinity can
+ trigger the enhanced IGP to initiate flooding. A trade-off
+ between the timeliness of the information flooded and the flooding
+ frequency is typically implemented using a threshold based on the
+ percentage change of the advertised resources to avoid excessive
+ consumption of link bandwidth and computational resources and to
+ avoid instability in the TED.
+
+ * In a TE system, it is also desirable for the routing subsystem to
+ make the load-splitting ratio among multiple paths (with equal
+ cost or different cost) configurable. This capability gives
+ network administrators more flexibility in the control of traffic
+ distribution across the network. It can be very useful for
+ avoiding/relieving congestion in certain situations. Examples can
+ be found in [XIAO] and [EVPN-UNEQUAL-LB].
+
+ * The routing system should also have the capability to control the
+ routes of subsets of traffic without affecting the routes of other
+ traffic if sufficient resources exist for this purpose. This
+ capability allows a more refined control over the distribution of
+ traffic across the network. For example, the ability to move
+ traffic away from its original path to another path (without
+ affecting other traffic paths) allows the traffic to be moved from
+ resource-poor network segments to resource-rich segments. Path-
+ oriented technologies, such as MPLS-TE, inherently support this
+ capability as discussed in [AWD2].
+
+ * Additionally, the routing subsystem should be able to select
+ different paths for different classes of traffic (or for different
+ traffic behavior aggregates) if the network supports multiple
+ classes of service (different behavior aggregates).
+
+6.3. Traffic Mapping Recommendations
+
+ Traffic mapping is the assignment of traffic workload onto (pre-
+ established) paths to meet certain requirements. Thus, while
+ constraint-based routing deals with path selection, traffic mapping
+ deals with the assignment of traffic to established paths that may
+ have been generated by constraint-based routing or by some other
+ means. Traffic mapping can be performed by time-dependent or state-
+ dependent mechanisms, as described in Section 4.1.
+
+ Two important aspects of the traffic mapping function are the ability
+ to establish multiple paths between an originating node and a
+ destination node and the capability to distribute the traffic across
+ those paths according to configured policies. A precondition for
+ this scheme is the existence of flexible mechanisms to partition
+ traffic and then assign the traffic partitions onto the parallel
+ paths (described as "parallel traffic trunks" in [RFC2702]). When
+ traffic is assigned to multiple parallel paths, it is recommended
+ that special care should be taken to ensure proper ordering of
+ packets belonging to the same application (or traffic flow) at the
+ destination node of the parallel paths.
+
+ Mechanisms that perform the traffic mapping functions should aim to
+ map the traffic onto the network infrastructure to minimize
+ congestion. If the total traffic load cannot be accommodated, or if
+ the routing and mapping functions cannot react fast enough to
+ changing traffic conditions, then a traffic mapping system may use
+ short timescale congestion control mechanisms (such as queue
+ management, scheduling, etc.) to mitigate congestion. Thus,
+ mechanisms that perform the traffic mapping functions complement
+ existing congestion control mechanisms. In an operational network,
+ traffic should be mapped onto the infrastructure such that intra-
+ class and inter-class resource contention are minimized (see
+ Section 2).
+
+ When traffic mapping techniques that depend on dynamic state feedback
+ (e.g., MPLS Adaptive Traffic Engineering (MATE) [MATE] and suchlike)
+ are used, special care must be taken to guarantee network stability.
+
+6.4. Measurement Recommendations
+
+ The importance of measurement in TE has been discussed throughout
+ this document. A TE system should include mechanisms to measure and
+ collect statistics from the network to support the TE function.
+ Additional capabilities may be needed to help in the analysis of the
+ statistics. The actions of these mechanisms should not adversely
+ affect the accuracy and integrity of the statistics collected. The
+ mechanisms for statistical data acquisition should also be able to
+ scale as the network evolves.
+
+ Traffic statistics may be classified according to long-term or short-
+ term timescales. Long-term traffic statistics are very useful for
+ traffic engineering. Long-term traffic statistics may periodically
+ record network workload (such as hourly, daily, and weekly variations
+ in traffic profiles) as well as traffic trends. Aspects of the
+ traffic statistics may also describe class of service characteristics
+ for a network supporting multiple classes of service. Analysis of
+ the long-term traffic statistics may yield other information such as
+ busy-hour characteristics, traffic growth patterns, persistent
+ congestion problems, hotspots, and imbalances in link utilization
+ caused by routing anomalies.
+
+ A mechanism for constructing traffic matrices for both long-term and
+ short-term traffic statistics should be in place. In multi-service
+ IP networks, the traffic matrices may be constructed for different
+ service classes. Each element of a traffic matrix represents a
+ statistic about the traffic flow between a pair of abstract nodes.
+ An abstract node may represent a router, a collection of routers, or
+ a site in a VPN.
+
+ Traffic statistics should provide reasonable and reliable indicators
+ of the current state of the network on the short-term scale. Some
+ short-term traffic statistics may reflect link utilization and link
+ congestion status. Examples of congestion indicators include
+ excessive packet delay, packet loss, and high resource utilization.
+ Examples of mechanisms for distributing this kind of information
+ include SNMP, probing tools, FTP, IGP link-state advertisements,
+ NETCONF/RESTCONF, etc.
+
+6.5. Policing, Planning, and Access Control
+
+ The recommendations in Sections 6.2 and 6.3 may be sub-optimal or
+ even ineffective if the amount of traffic flowing on a route or path
+ exceeds the capacity of the resource on that route or path. Several
+ approaches can be used to increase the performance of TE systems:
+
+ * The fundamental approach is some form of planning where traffic is
+ steered onto paths so that it is distributed across the available
+ resources. This planning may be centralized or distributed and
+ must be aware of the planned traffic volumes and available
+ resources. However, this approach is only of value if the traffic
+ that is presented conforms to the planned traffic volumes.
+
+ * Traffic flows may be policed at the edges of a network. This is a
+ simple way to ensure that the actual traffic volumes are
+ consistent with the planned volumes. Some form of measurement
+ (see Section 6.4) is used to determine the rate of arrival of
+ traffic, and excess traffic could be discarded. Alternatively,
+ excess traffic could be forwarded as best-effort within the
+ network. However, this approach is only completely effective if
+ the planning is stringent and network-wide and if a harsh approach
+ is taken to disposing of excess traffic.
+
+ * Resource-based admission control is the process whereby network
+ nodes decide whether to grant access to resources. The basis for
+ the decision on a packet-by-packet basis is the determination of
+ the flow to which the packet belongs. This information is
+ combined with policy instructions that have been locally
+ configured or installed through the management or control planes.
+ The end result is that a packet may be allowed to access (or use)
+ specific resources on the node if, and only if, the flow to which
+ the packet belongs conforms to the policy.
+
+ Combining some elements of all three of these measures is advisable
+ to achieve a better TE system.
+
+6.6. Network Survivability
+
+ Network survivability refers to the capability of a network to
+ maintain service continuity in the presence of faults. This can be
+ accomplished by promptly recovering from network impairments and
+ maintaining the required QoS for existing services after recovery.
+ Survivability is an issue of great concern within the Internet
+ community due to the demand to carry mission-critical traffic, real-
+ time traffic, and other high-priority traffic over the Internet.
+ Survivability can be addressed at the device level by developing
+ network elements that are more reliable and at the network level by
+ incorporating redundancy into the architecture, design, and operation
+ of networks. It is recommended that a philosophy of robustness and
+ survivability should be adopted in the architecture, design, and
+ operation of TE used to control IP networks (especially public IP
+ networks). Because different contexts may demand different levels of
+ survivability, the mechanisms developed to support network
+ survivability should be flexible so that they can be tailored to
+ different needs. A number of tools and techniques have been
+ developed to enable network survivability, including MPLS Fast
+ Reroute [RFC4090], Topology Independent Loop-free Alternate Fast
+ Reroute for Segment Routing [SR-TI-LFA], RSVP-TE Extensions in
+ Support of End-to-End GMPLS Recovery [RFC4872], and GMPLS Segment
+ Recovery [RFC4873].
+
+ The impact of service outages varies significantly for different
+ service classes depending on the duration of the outage, which can
+ vary from milliseconds (with minor service impact) to seconds (with
+ possible call drops for IP telephony and session timeouts for
+ connection-oriented transactions) to minutes and hours (with
+ potentially considerable social and business impact). Outages of
+ different durations have different impacts depending on the nature of
+ the traffic flows that are interrupted.
+
+ Failure protection and restoration capabilities are available in
+ multiple layers as network technologies have continued to evolve.
+ Optical networks are capable of providing dynamic ring and mesh
+ restoration functionality at the wavelength level. At the SONET/SDH
+ layer, survivability capability is provided with Automatic Protection
+ Switching (APS) as well as self-healing ring and mesh architectures.
+ Similar functionality is provided by Layer 2 technologies such as
+ Ethernet.
+
+ Rerouting is used at the IP layer to restore service following link
+ and node outages. Rerouting at the IP layer occurs after a period of
+ routing convergence, which may require seconds to minutes to
+ complete. Path-oriented technologies such as MPLS [RFC3469] can be
+ used to enhance the survivability of IP networks in a potentially
+ cost-effective manner.
+
+ An important aspect of multi-layer survivability is that technologies
+ at different layers may provide protection and restoration
+ capabilities at different granularities in terms of timescales and at
+ different bandwidth granularities (from the level of packets to that
+ of wavelengths). Protection and restoration capabilities can also be
+ sensitive to different service classes and different network utility
+ models. Coordinating different protection and restoration
+ capabilities across multiple layers in a cohesive manner to ensure
+ network survivability is maintained at reasonable cost is a
+ challenging task. Protection and restoration coordination across
+ layers may not always be feasible, because networks at different
+ layers may belong to different administrative domains.
+
+ Some of the general recommendations for protection and restoration
+ coordination are as follows:
+
+ * Protection and restoration capabilities from different layers
+ should be coordinated to provide network survivability in a
+ flexible and cost-effective manner. Avoiding duplication of
+ functions in different layers is one way to achieve the
+ coordination. Escalation of alarms and other fault indicators
+ from lower to higher layers may also be performed in a coordinated
+ manner. The order of timing of restoration triggers from
+ different layers is another way to coordinate multi-layer
+ protection/restoration.
+
+ * Network capacity reserved in one layer to provide protection and
+ restoration is not available to carry traffic in a higher layer:
+ it is not visible as spare capacity in the higher layer. Placing
+ protection/restoration functions in many layers may increase
+ redundancy and robustness, but it can result in significant
+ inefficiencies in network resource utilization. Careful planning
+ is needed to balance the trade-off between the desire for
+ survivability and the optimal use of resources.
+
+ * It is generally desirable to have protection and restoration
+ schemes that are intrinsically bandwidth efficient.
+
+ * Failure notifications throughout the network should be timely and
+ reliable if they are to be acted on as triggers for effective
+ protection and restoration actions.
+
+ * Alarms and other fault monitoring and reporting capabilities
+ should be provided at the right network layers so that the
+ protection and restoration actions can be taken in those layers.
+
+6.6.1. Survivability in MPLS-Based Networks
+
+ Because MPLS is path-oriented, it has the potential to provide faster
+ and more predictable protection and restoration capabilities than
+ conventional hop-by-hop routed IP systems. Protection types for MPLS
+ networks can be divided into four categories:
+
+ Link Protection: The objective of link protection is to protect an
+ LSP from the failure of a given link. Under link protection, a
+ protection or backup LSP (the secondary LSP) follows a path that
+ is disjoint from the path of the working or operational LSP (the
+ primary LSP) at the particular link where link protection is
+ required. When the protected link fails, traffic on the working
+ LSP is switched to the protection LSP at the headend of the failed
+ link. As a local repair method, link protection can be fast.
+ This form of protection may be most appropriate in situations
+ where some network elements along a given path are known to be
+ less reliable than others.
+
+ Node Protection: The objective of node protection is to protect an
+ LSP from the failure of a given node. Under node protection, the
+ secondary LSP follows a path that is disjoint from the path of the
+ primary LSP at the particular node where node protection is
+ required. The secondary LSP is also disjoint from the primary LSP
+ at all links attached to the node to be protected. When the
+ protected node fails, traffic on the working LSP is switched over
+ to the protection LSP at the upstream LSR directly connected to
+ the failed node. Node protection covers a slightly larger part of
+ the network compared to link protection but is otherwise
+ fundamentally the same.
+
+ Path Protection: The goal of LSP path protection (or end-to-end
+ protection) is to protect an LSP from any failure along its routed
+ path. Under path protection, the path of the protection LSP is
+ completely disjoint from the path of the working LSP. The
+ advantage of path protection is that the backup LSP protects the
+ working LSP from all possible link and node failures along the
+ path, except for failures of ingress or egress LSR. Additionally,
+ path protection may be more efficient in terms of resource usage
+ than link or node protection applied at every hop along the path.
+ However, path protection may be slower than link and node
+ protection because the fault notifications have to be propagated
+ further.
+
+ Segment Protection: An MPLS domain may be partitioned into multiple
+ subdomains (protection domains). Path protection is applied to
+ the path of each LSP as it crosses the domain from its ingress to
+ the domain to where it egresses the domain. In cases where an LSP
+ traverses multiple protection domains, a protection mechanism
+ within a domain only needs to protect the segment of the LSP that
+ lies within the domain. Segment protection will generally be
+ faster than end-to-end path protection because recovery generally
+ occurs closer to the fault, and the notification doesn't have to
+ propagate as far.
+
+ See [RFC3469] and [RFC6372] for a more comprehensive discussion of
+ MPLS-based recovery.
+
+6.6.2. Protection Options
+
+ Another issue to consider is the concept of protection options. We
+ use notation such as "m:n protection", where m is the number of
+ protection LSPs used to protect n working LSPs. In all cases except
+ 1+1 protection, the resources associated with the protection LSPs can
+ be used to carry preemptable best-effort traffic when the working LSP
+ is functioning correctly.
+
+ 1:1 protection: One working LSP is protected/restored by one
+ protection LSP. Traffic is sent only on the protected LSP until
+ the protection/restoration event switches the traffic to the
+ protection LSP.
+
+ 1:n protection: One protection LSP is used to protect/restore n
+ working LSPs. Traffic is sent only on the n protected working
+ LSPs until the protection/restoration event switches the traffic
+ from one failed LSP to the protection LSP. Only one failed LSP
+ can be restored at any time.
+
+ n:1 protection: One working LSP is protected/restored by n
+ protection LSPs, possibly with load splitting across the
+ protection LSPs. This may be especially useful when it is not
+ feasible to find one path for the backup that can satisfy the
+ bandwidth requirement of the primary LSP.
+
+ 1+1 protection: Traffic is sent concurrently on both the working LSP
+ and a protection LSP. The egress LSR selects one of the two LSPs
+ based on local policy (usually based on traffic integrity). When
+ a fault disrupts the traffic on one LSP, the egress switches to
+ receive traffic from the other LSP. This approach is expensive in
+ how it consumes network but recovers from failures most rapidly.
+
+6.7. Multi-Layer Traffic Engineering
+
+ Networks are often implemented as layers. A layer relationship may
+ represent the interaction between technologies (for example, an IP
+ network operated over an optical network) or the relationship between
+ different network operators (for example, a customer network operated
+ over a service provider's network). Note that a multi-layer network
+ does not imply the use of multiple technologies, although some form
+ of encapsulation is often applied.
+
+ Multi-layer traffic engineering presents a number of challenges
+ associated with scalability and confidentiality. These issues are
+ addressed in [RFC7926], which discusses the sharing of information
+ between domains through policy filters, aggregation, abstraction, and
+ virtualization. That document also discusses how existing protocols
+ can support this scenario with special reference to BGP-LS (see
+ Section 5.1.3.10).
+
+ PCE (see Section 5.1.3.11) is also a useful tool for multi-layer
+ networks as described in [RFC6805], [RFC8685], and [RFC5623].
+ Signaling techniques for multi-layer TE are described in [RFC6107].
+
+ See also Section 6.6 for examination of multi-layer network
+ survivability.
+
+6.8. Traffic Engineering in Diffserv Environments
+
+ Increasing requirements to support multiple classes of traffic in the
+ Internet, such as best-effort and mission-critical data, call for IP
+ networks to differentiate traffic according to some criteria and to
+ give preferential treatment to certain types of traffic. Large
+ numbers of flows can be aggregated into a few behavior aggregates
+ based on some criteria based on common performance requirements in
+ terms of packet loss ratio, delay, and jitter or in terms of common
+ fields within the IP packet headers.
+
+ Differentiated Services (Diffserv) [RFC2475] can be used to ensure
+ that SLAs defined to differentiate between traffic flows are met.
+ Classes of service can be supported in a Diffserv environment by
+ concatenating Per-Hop Behaviors (PHBs) along the routing path. A PHB
+ is the forwarding behavior that a packet receives at a Diffserv-
+ compliant node, and it can be configured at each router. PHBs are
+ delivered using buffer-management and packet-scheduling mechanisms
+ and require that the ingress nodes use traffic classification,
+ marking, policing, and shaping.
+
+ TE can complement Diffserv to improve utilization of network
+ resources. TE can be operated on an aggregated basis across all
+ service classes [RFC3270] or on a per-service-class basis. The
+ former is used to provide better distribution of the traffic load
+ over the network resources (see [RFC3270] for detailed mechanisms to
+ support aggregate TE). The latter case is discussed below since it
+ is specific to the Diffserv environment, with so-called Diffserv-
+ aware traffic engineering [RFC4124].
+
+ For some Diffserv networks, it may be desirable to control the
+ performance of some service classes by enforcing relationships
+ between the traffic workload contributed by each service class and
+ the amount of network resources allocated or provisioned for that
+ service class. Such relationships between demand and resource
+ allocation can be enforced using a combination of, for example:
+
+ * TE mechanisms on a per-service-class basis that enforce the
+ relationship between the amount of traffic contributed by a given
+ service class and the resources allocated to that class.
+
+ * Mechanisms that dynamically adjust the resources allocated to a
+ given service class to relate to the amount of traffic contributed
+ by that service class.
+
+ It may also be desirable to limit the performance impact of high-
+ priority traffic on relatively low-priority traffic. This can be
+ achieved, for example, by controlling the percentage of high-priority
+ traffic that is routed through a given link. Another way to
+ accomplish this is to increase link capacities appropriately so that
+ lower-priority traffic can still enjoy adequate service quality.
+ When the ratio of traffic workload contributed by different service
+ classes varies significantly from router to router, it may not be
+ enough to rely on conventional IGP routing protocols or on TE
+ mechanisms that are not sensitive to different service classes.
+ Instead, it may be desirable to perform TE, especially routing
+ control and mapping functions, on a per-service-class basis. One way
+ to accomplish this in a domain that supports both MPLS and Diffserv
+ is to define class-specific LSPs and to map traffic from each class
+ onto one or more LSPs that correspond to that service class. An LSP
+ corresponding to a given service class can then be routed and
+ protected/restored in a class-dependent manner, according to specific
+ policies.
+
+ Performing TE on a per-class basis may require per-class parameters
+ to be distributed. It is common to have some classes share some
+ aggregate constraints (e.g., maximum bandwidth requirement) without
+ enforcing the constraint on each individual class. These classes can
+ be grouped into class types, and per-class-type parameters can be
+ distributed to improve scalability. This also allows better
+ bandwidth sharing between classes in the same class type. A class
+ type is a set of classes that satisfy the following two conditions:
+
+ * Classes in the same class type have common aggregate requirements
+ to satisfy required performance levels.
+
+ * There is no requirement to be enforced at the level of an
+ individual class in the class type. Note that it is,
+ nevertheless, still possible to implement some priority policies
+ for classes in the same class type to permit preferential access
+ to the class type bandwidth through the use of preemption
+ priorities.
+
+ See [RFC4124] for detailed requirements on Diffserv-aware TE.
+
+6.9. Network Controllability
+
+ Offline and online (see Section 4.2) TE considerations are of limited
+ utility if the network cannot be controlled effectively to implement
+ the results of TE decisions and to achieve the desired network
+ performance objectives.
+
+ Capacity augmentation is a coarse-grained solution to TE issues.
+ However, it is simple, may be applied through creating parallel links
+ that form part of an ECMP scheme, and may be advantageous if
+ bandwidth is abundant and cheap. However, bandwidth is not always
+ abundant and cheap, and additional capacity might not always be the
+ best solution. Adjustments of administrative weights and other
+ parameters associated with routing protocols provide finer-grained
+ control, but this approach is difficult to use and imprecise because
+ of the way the routing protocols interact across the network.
+
+ Control mechanisms can be manual (e.g., static configuration),
+ partially automated (e.g., scripts), or fully automated (e.g.,
+ policy-based management systems). Automated mechanisms are
+ particularly useful in large-scale networks. Multi-vendor
+ interoperability can be facilitated by standardized management tools
+ (e.g., YANG models) to support the control functions required to
+ address TE objectives.
+
+ Network control functions should be secure, reliable, and stable as
+ these are often needed to operate correctly in times of network
+ impairments (e.g., during network congestion or attacks).
+
+7. Inter-Domain Considerations
+
+ Inter-domain TE is concerned with performance optimization for
+ traffic that originates in one administrative domain and terminates
+ in a different one.
+
+ BGP [RFC4271] is the standard exterior gateway protocol used to
+ exchange routing information between ASes in the Internet. BGP
+ includes a decision process that calculates the preference for routes
+ to a given destination network. There are two fundamental aspects to
+ inter-domain TE using BGP:
+
+ Route Propagation: Controlling the import and export of routes
+ between ASes and controlling the redistribution of routes between
+ BGP and other protocols within an AS.
+
+ Best-path selection: Selecting the best path when there are multiple
+ candidate paths to a given destination network. This is performed
+ by the BGP decision process, which selects the preferred exit
+ points out of an AS toward specific destination networks by taking
+ a number of different considerations into account. The BGP path
+ selection process can be influenced by manipulating the attributes
+ associated with the process, including NEXT_HOP, LOCAL_PREF,
+ AS_PATH, ORIGIN, MULTI_EXIT_DISC (MED), IGP metric, etc.
+
+ Most BGP implementations provide constructs that facilitate the
+ implementation of complex BGP policies based on pre-configured
+ logical conditions. These can be used to control import and export
+ of incoming and outgoing routes, control redistribution of routes
+ between BGP and other protocols, and influence the selection of best
+ paths by manipulating the attributes (either standardized or local to
+ the implementation) associated with the BGP decision process.
+
+ When considering inter-domain TE with BGP, note that the outbound
+ traffic exit point is controllable, whereas the interconnection point
+ where inbound traffic is received typically is not. Therefore, it is
+ up to each individual network to implement TE strategies that deal
+ with the efficient delivery of outbound traffic from its customers to
+ its peering points. The vast majority of TE policy is based on a
+ "closest exit" strategy, which offloads inter-domain traffic at the
+ nearest outbound peering point towards the destination AS. Most
+ methods of manipulating the point at which inbound traffic enters are
+ either ineffective or not accepted in the peering community.
+
+ Inter-domain TE with BGP is generally effective, but it is usually
+ applied in a trial-and-error fashion because a TE system usually only
+ has a view of the available network resources within one domain (an
+ AS in this case). A systematic approach for inter-domain TE requires
+ cooperation between the domains. Further, what may be considered a
+ good solution in one domain may not necessarily be a good solution in
+ another. Moreover, it is generally considered inadvisable for one
+ domain to permit a control process from another domain to influence
+ the routing and management of traffic in its network.
+
+ MPLS-TE tunnels (LSPs) can add a degree of flexibility in the
+ selection of exit points for inter-domain routing by applying the
+ concept of relative and absolute metrics. If BGP attributes are
+ defined such that the BGP decision process depends on IGP metrics to
+ select exit points for inter-domain traffic, then some inter-domain
+ traffic destined to a given peer network can be made to prefer a
+ specific exit point by establishing a TE tunnel between the router
+ making the selection and the peering point via a TE tunnel and
+ assigning the TE tunnel a metric that is smaller than the IGP cost to
+ all other peering points. RSVP-TE protocol extensions for inter-
+ domain MPLS and GMPLS are described in [RFC5151].
+
+ Similarly to intra-domain TE, inter-domain TE is best accomplished
+ when a traffic matrix can be derived to depict the volume of traffic
+ from one AS to another.
+
+ Layer 4 multipath transport protocols are designed to move traffic
+ between domains and to allow some influence over the selection of the
+ paths. To be truly effective, these protocols would require
+ visibility of paths and network conditions in other domains, but that
+ information may not be available, might not be complete, and is not
+ necessarily trustworthy.
+
+8. Overview of Contemporary TE Practices in Operational IP Networks
+
+ This section provides an overview of some TE practices in IP
+ networks. The focus is on aspects of control of the routing function
+ in operational contexts. The intent here is to provide an overview
+ of the commonly used practices; the discussion is not intended to be
+ exhaustive.
+
+ Service providers apply many of the TE mechanisms described in this
+ document to optimize the performance of their IP networks, although
+ others choose to not use any of them. These techniques include
+ capacity planning, including adding ECMP options, for long
+ timescales; routing control using IGP metrics and MPLS, as well as
+ path planning and path control using MPLS and SR for medium
+ timescales; and traffic management mechanisms for short timescales.
+
+ * Capacity planning is an important component of how a service
+ provider plans an effective IP network. These plans may take the
+ following aspects into account: location of any new links or
+ nodes, WECMP algorithms, existing and predicted traffic patterns,
+ costs, link capacity, topology, routing design, and survivability.
+
+ * Performance optimization of operational networks is usually an
+ ongoing process in which traffic statistics, performance
+ parameters, and fault indicators are continually collected from
+ the network. This empirical data is analyzed and used to trigger
+ TE mechanisms. Tools that perform what-if analysis can also be
+ used to assist the TE process by reviewing scenarios before a new
+ set of configurations are implemented in the operational network.
+
+ * Real-time intra-domain TE using the IGP is done by increasing the
+ OSPF or IS-IS metric of a congested link until enough traffic has
+ been diverted away from that link. This approach has some
+ limitations as discussed in Section 6.2. Intra-domain TE
+ approaches [RR94] [FT00] [FT01] [WANG] take traffic matrix,
+ network topology, and network performance objectives as input and
+ produce link metrics and load-sharing ratios. These processes
+ open the possibility for intra-domain TE with IGP to be done in a
+ more systematic way.
+
+ Administrators of MPLS-TE networks specify and configure link
+ attributes and resource constraints such as maximum reservable
+ bandwidth and resource class attributes for the links in the domain.
+ A link-state IGP that supports TE extensions (IS-IS-TE or OSPF-TE) is
+ used to propagate information about network topology and link
+ attributes to all routers in the domain. Network administrators
+ specify the LSPs that are to originate at each router. For each LSP,
+ the network administrator specifies the destination node and the
+ attributes of the LSP that indicate the requirements that are to be
+ satisfied during the path selection process. The attributes may
+ include an explicit path for the LSP to follow, or the originating
+ router may use a local constraint-based routing process to compute
+ the path of the LSP. RSVP-TE is used as a signaling protocol to
+ instantiate the LSPs. By assigning proper bandwidth values to links
+ and LSPs, congestion caused by uneven traffic distribution can be
+ avoided or mitigated.
+
+ The bandwidth attributes of an LSP relate to the bandwidth
+ requirements of traffic that flows through the LSP. The traffic
+ attribute of an LSP can be modified to accommodate persistent shifts
+ in demand (traffic growth or reduction). If network congestion
+ occurs due to unexpected events, existing LSPs can be rerouted to
+ alleviate the situation, or the network administrator can configure
+ new LSPs to divert some traffic to alternative paths. The reservable
+ bandwidth of the congested links can also be reduced to force some
+ LSPs to be rerouted to other paths. A traffic matrix in an MPLS
+ domain can also be estimated by monitoring the traffic on LSPs. Such
+ traffic statistics can be used for a variety of purposes including
+ network planning and network optimization.
+
+ Network management and planning systems have evolved and assumed a
+ lot of the responsibility for determining traffic paths in TE
+ networks. This allows a network-wide view of resources and
+ facilitates coordination of the use of resources for all traffic
+ flows in the network. Initial solutions using a PCE to perform path
+ computation on behalf of network routers have given way to an
+ approach that follows the SDN architecture. A stateful PCE is able
+ to track all of the LSPs in the network and can redistribute them to
+ make better use of the available resources. Such a PCE can form part
+ of a network orchestrator that uses PCEP or some other configuration
+ and management interface to instruct the signaling protocol or
+ directly program the routers.
+
+ SR leverages a centralized TE controller and either an MPLS or IPv6
+ forwarding plane but does not need to use a signaling protocol or
+ management plane protocol to reserve resources in the routers. All
+ resource reservation is logical within the controller and is not
+ distributed to the routers. Packets are steered through the network
+ using SR, and this may have configuration and operational scaling
+ benefits.
+
+ As mentioned in Section 7, there is usually no direct control over
+ the distribution of inbound traffic to a domain. Therefore, the main
+ goal of inter-domain TE is to optimize the distribution of outbound
+ traffic between multiple inter-domain links. When operating a
+ geographically widespread network (such as for a multi-national or
+ global network provider), maintaining the ability to operate the
+ network in a regional fashion where desired, while continuing to take
+ advantage of the benefits of a globally interconnected network, also
+ becomes an important objective.
+
+ Inter-domain TE with BGP begins with the placement of multiple
+ peering interconnection points that are in close proximity to traffic
+ sources/destinations and offer lowest-cost paths across the network
+ between the peering points and the sources/destinations. Some
+ location-decision problems that arise in association with inter-
+ domain routing are discussed in [AWD5].
+
+ Once the locations of the peering interconnects have been determined
+ and implemented, the network operator decides how best to handle the
+ routes advertised by the peer, as well as how to propagate the peer's
+ routes within their network. One way to engineer outbound traffic
+ flows in a network with many peering interconnects is to create a
+ hierarchy of peers. Generally, the shortest AS paths will be chosen
+ to forward traffic, but BGP metrics can be used to prefer some peers
+ and so favor particular paths. Preferred peers are those peers
+ attached through peering interconnects with the most available
+ capacity. Changes may be needed, for example, to deal with a
+ "problem peer" who is difficult to work with on upgrades or is
+ charging high prices for connectivity to their network. In that
+ case, the peer may be given a reduced preference. This type of
+ change can affect a large amount of traffic and is only used after
+ other methods have failed to provide the desired results.
+
+ When there are multiple exit points toward a given peer, and only one
+ of them is congested, it is not necessary to shift traffic away from
+ the peer entirely, but only from the one congested connection. This
+ can be achieved by using passive IGP metrics, AS_PATH filtering, or
+ prefix filtering.
+
+9. Security Considerations
+
+ In general, TE mechanisms are security neutral, and this document
+ does not introduce new security issues.
+
+ Network security is, of course, an important issue, and TE mechanisms
+ can have benefits and drawbacks:
+
+ * TE may use tunnels that can slightly help protect traffic from
+ inspection and that, in some cases, can be secured using
+ encryption.
+
+ * TE puts traffic onto predictable paths within the network that may
+ make it easier to find and attack.
+
+ * TE often increases the complexity of operation and management of
+ the network, which may lead to errors that compromise security.
+
+ * TE enables traffic to be steered onto more secure links or to more
+ secure parts of the network.
+
+ * TE can be used to steer traffic through network nodes that are
+ able to provide additional security functions.
+
+ The consequences of attacks on the control and management protocols
+ used to operate TE networks can be significant:
+
+ * Traffic can be hijacked to pass through specific nodes that
+ perform inspection or even to be delivered to the wrong place.
+
+ * Traffic can be steered onto paths that deliver quality that is
+ below the desired quality.
+
+ * Networks can be congested or have resources on key links consumed.
+
+ Thus, it is important to use adequate protection mechanisms, such as
+ authentication, on all protocols used to deliver TE.
+
+ Certain aspects of a network may be deduced from the details of the
+ TE paths that are used. For example, the link connectivity of the
+ network and the quality and load on individual links may be inferred
+ from knowing the paths of traffic and the requirements they place on
+ the network (for example, by seeing the control messages or through
+ path-trace techniques). Such knowledge can be used to launch
+ targeted attacks (for example, taking down critical links) or can
+ reveal commercially sensitive information (for example, whether a
+ network is close to capacity). Therefore, network operators may
+ choose techniques that mask or hide information from within the
+ network.
+
+ External control interfaces that are introduced to provide additional
+ control and management of TE systems (see Section 5.1.2) provide
+ flexibility to management and to customers, but they do so at the
+ risk of exposing the internals of a network to potentially malicious
+ actors. The protocols used at these interfaces must be secured to
+ protect against snooping and modification, and use of the interfaces
+ must be authenticated.
+
+10. IANA Considerations
+
+ This document has no IANA actions.
+
+11. Informative References
+
+ [AFD03] Pan, R., Breslau, L., Prabhakar, B., and S. Shenker,
+ "Approximate fairness through differential dropping", ACM
+ SIGCOMM Computer Communication Review, Volume 33, Issue 2,
+ Pages 23-39, DOI 10.1145/956981.956985, April 2003,
+ <https://dl.acm.org/doi/10.1145/956981.956985>.
+
+ [AJ19] Adekitan, A., Abolade, J., and O. Shobayo, "Data mining
+ approach for predicting the daily Internet data traffic of
+ a smart university", Journal of Big Data, Volume 6, Number
+ 1, Page 1, DOI 10.1186/s40537-019-0176-5, February 2019,
+ <https://journalofbigdata.springeropen.com/track/
+ pdf/10.1186/s40537-019-0176-5.pdf>.
+
+ [ATSSS] 3GPP, "Study on access traffic steering, switch and
+ splitting support in the 5G System (5GS) architecture",
+ Release 16, 3GPP TR 23.793, December 2018,
+ <https://www.3gpp.org/ftp//Specs/
+ archive/23_series/23.793/23793-g00.zip>.
+
+ [AWD2] Awduche, D., "MPLS and traffic engineering in IP
+ networks", IEEE Communications Magazine, Volume 37, Issue
+ 12, Pages 42-47, DOI 10.1109/35.809383, December 1999,
+ <https://ieeexplore.ieee.org/document/809383>.
+
+ [AWD5] Awduche, D., "An approach to optimal peering between
+ autonomous systems in the Internet", Proceedings 7th
+ International Conference on Computer Communications and
+ Networks (Cat. No. 98EX226),
+ DOI 10.1109/ICCCN.1998.998795, October 1998,
+ <https://ieeexplore.ieee.org/document/998795>.
+
+ [E.360.1] ITU-T, "Framework for QoS routing and related traffic
+ engineering methods for IP-, ATM-, and TDM-based
+ multiservice networks", ITU-T Recommendation E.360.1, May
+ 2002, <https://www.itu.int/rec/T-REC-E.360.1-200205-I/en>.
+
+ [ENHANCED-VPN]
+ Dong, J., Bryant, S., Li, Z., Miyasaka, T., and Y. Lee, "A
+ Framework for NRP-based Enhanced Virtual Private Network",
+ Work in Progress, Internet-Draft, draft-ietf-teas-
+ enhanced-vpn-17, 25 December 2023,
+ <https://datatracker.ietf.org/doc/html/draft-ietf-teas-
+ enhanced-vpn-17>.
+
+ [Err309] RFC Errata, Erratum ID 309, RFC 3272,
+ <https://www.rfc-editor.org/errata/eid309>.
+
+ [EVPN-UNEQUAL-LB]
+ Malhotra, N., Ed., Sajassi, A., Rabadan, J., Drake, J.,
+ Lingala, A., and S. Thoria, "Weighted Multi-Path
+ Procedures for EVPN Multi-Homing", Work in Progress,
+ Internet-Draft, draft-ietf-bess-evpn-unequal-lb-21, 7
+ December 2023, <https://datatracker.ietf.org/doc/html/
+ draft-ietf-bess-evpn-unequal-lb-21>.
+
+ [FLJA93] Floyd, S. and V. Jacobson, "Random Early Detection
+ Gateways for Congestion Avoidance", IEEE/ACM Transactions
+ on Networking, Volume 1, Issue 4, Pages 397-413,
+ DOI 10.1109/90.251892, August 1993,
+ <https://www.icir.org/floyd/papers/early.twocolumn.pdf>.
+
+ [FT00] Fortz, B. and M. Thorup, "Internet Traffic Engineering by
+ Optimizing OSPF Weights", Proceedings IEEE INFOCOM 2000,
+ DOI 10.1109/INFCOM.2000.832225, March 2000,
+ <https://www.cs.cornell.edu/courses/cs619/2004fa/
+ documents/ospf_opt.pdf>.
+
+ [FT01] Fortz, B. and M. Thorup, "Optimizing OSPF/IS-IS Weights in
+ a Changing World", IEEE Journal on Selected Areas in
+ Communications, DOI 10.1109/JSAC.2002.1003042, May 2002,
+ <https://ieeexplore.ieee.org/document/1003042>.
+
+ [GRPC] gRPC Authors, "gRPC: A high performance, open source
+ universal RPC framework", <https://grpc.io>.
+
+ [KELLY] Kelly, F., "Notes on effective bandwidths", Oxford
+ University Press, 1996.
+
+ [MA] Ma, Q., "Quality-of-Service Routing in Integrated Services
+ Networks", Ph.D. Dissertation, Carnegie Mellon University,
+ CMU-CS-98-138, January 1998,
+ <https://apps.dtic.mil/sti/pdfs/ADA352299.pdf>.
+
+ [MATE] Elwalid, A., Jin, C., Low, S., and I. Widjaja, "MATE: MPLS
+ Adaptive Traffic Engineering", Proceedings IEEE INFOCOM
+ 2001, Conference on Computer Communications, Twentieth
+ Annual Joint Conference of the IEEE Computer and
+ Communications Society (Cat. No. 01CH37213),
+ DOI 10.1109/INFCOM.2001.916625, August 2002,
+ <https://www.yumpu.com/en/document/view/35140398/mate-
+ mpls-adaptive-traffic-engineering-infocom-ieee-xplore/8>.
+
+ [MR99] Mitra, D. and K.G. Ramakrishnan, "A case study of
+ multiservice, multipriority traffic engineering design for
+ data networks", Seamless Interconnection for Universal
+ Services, Global Telecommunications Conference,
+ GLOBECOM'99, (Cat. No. 99CH37042),
+ DOI 10.1109/GLOCOM.1999.830281, December 1999,
+ <https://ieeexplore.ieee.org/document/830281>.
+
+ [MULTIPATH-DCCP]
+ Amend, M., Ed., Brunstrom, A., Kassler, A., Rakocevic, V.,
+ and S. Johnson, "DCCP Extensions for Multipath Operation
+ with Multiple Addresses", Work in Progress, Internet-
+ Draft, draft-ietf-tsvwg-multipath-dccp-11, 12 October
+ 2023, <https://datatracker.ietf.org/doc/html/draft-ietf-
+ tsvwg-multipath-dccp-11>.
+
+ [NETWORK-SLICES]
+ Farrel, A., Ed., Drake, J., Ed., Rokui, R., Homma, S.,
+ Makhijani, K., Contreras, L. M., and J. Tantsura, "A
+ Framework for Network Slices in Networks Built from IETF
+ Technologies", Work in Progress, Internet-Draft, draft-
+ ietf-teas-ietf-network-slices-25, 14 September 2023,
+ <https://datatracker.ietf.org/doc/html/draft-ietf-teas-
+ ietf-network-slices-25>.
+
+ [PERFORMANCE-ROUTING]
+ Xu, X., Hegde, S., Talaulikar, K., Boucadair, M., and C.
+ Jacquenet, "Performance-based BGP Routing Mechanism", Work
+ in Progress, Internet-Draft, draft-ietf-idr-performance-
+ routing-03, 22 December 2020,
+ <https://datatracker.ietf.org/doc/html/draft-ietf-idr-
+ performance-routing-03>.
+
+ [QUIC-MULTIPATH]
+ Liu, Y., Ed., Ma, Y., Ed., De Coninck, Q., Ed.,
+ Bonaventure, O., Huitema, C., and M. Kühlewind, Ed.,
+ "Multipath Extension for QUIC", Work in Progress,
+ Internet-Draft, draft-ietf-quic-multipath-06, 23 October
+ 2023, <https://datatracker.ietf.org/doc/html/draft-ietf-
+ quic-multipath-06>.
+
+ [RFC0791] Postel, J., "Internet Protocol", STD 5, RFC 791,
+ DOI 10.17487/RFC0791, September 1981,
+ <https://www.rfc-editor.org/info/rfc791>.
+
+ [RFC1102] Clark, D., "Policy routing in Internet protocols",
+ RFC 1102, DOI 10.17487/RFC1102, May 1989,
+ <https://www.rfc-editor.org/info/rfc1102>.
+
+ [RFC1104] Braun, H., "Models of policy based routing", RFC 1104,
+ DOI 10.17487/RFC1104, June 1989,
+ <https://www.rfc-editor.org/info/rfc1104>.
+
+ [RFC2205] Braden, R., Ed., Zhang, L., Berson, S., Herzog, S., and S.
+ Jamin, "Resource ReSerVation Protocol (RSVP) -- Version 1
+ Functional Specification", RFC 2205, DOI 10.17487/RFC2205,
+ September 1997, <https://www.rfc-editor.org/info/rfc2205>.
+
+ [RFC2330] Paxson, V., Almes, G., Mahdavi, J., and M. Mathis,
+ "Framework for IP Performance Metrics", RFC 2330,
+ DOI 10.17487/RFC2330, May 1998,
+ <https://www.rfc-editor.org/info/rfc2330>.
+
+ [RFC2386] Crawley, E., Nair, R., Rajagopalan, B., and H. Sandick, "A
+ Framework for QoS-based Routing in the Internet",
+ RFC 2386, DOI 10.17487/RFC2386, August 1998,
+ <https://www.rfc-editor.org/info/rfc2386>.
+
+ [RFC2474] Nichols, K., Blake, S., Baker, F., and D. Black,
+ "Definition of the Differentiated Services Field (DS
+ Field) in the IPv4 and IPv6 Headers", RFC 2474,
+ DOI 10.17487/RFC2474, December 1998,
+ <https://www.rfc-editor.org/info/rfc2474>.
+
+ [RFC2475] Blake, S., Black, D., Carlson, M., Davies, E., Wang, Z.,
+ and W. Weiss, "An Architecture for Differentiated
+ Services", RFC 2475, DOI 10.17487/RFC2475, December 1998,
+ <https://www.rfc-editor.org/info/rfc2475>.
+
+ [RFC2597] Heinanen, J., Baker, F., Weiss, W., and J. Wroclawski,
+ "Assured Forwarding PHB Group", RFC 2597,
+ DOI 10.17487/RFC2597, June 1999,
+ <https://www.rfc-editor.org/info/rfc2597>.
+
+ [RFC2678] Mahdavi, J. and V. Paxson, "IPPM Metrics for Measuring
+ Connectivity", RFC 2678, DOI 10.17487/RFC2678, September
+ 1999, <https://www.rfc-editor.org/info/rfc2678>.
+
+ [RFC2702] Awduche, D., Malcolm, J., Agogbua, J., O'Dell, M., and J.
+ McManus, "Requirements for Traffic Engineering Over MPLS",
+ RFC 2702, DOI 10.17487/RFC2702, September 1999,
+ <https://www.rfc-editor.org/info/rfc2702>.
+
+ [RFC2722] Brownlee, N., Mills, C., and G. Ruth, "Traffic Flow
+ Measurement: Architecture", RFC 2722,
+ DOI 10.17487/RFC2722, October 1999,
+ <https://www.rfc-editor.org/info/rfc2722>.
+
+ [RFC2753] Yavatkar, R., Pendarakis, D., and R. Guerin, "A Framework
+ for Policy-based Admission Control", RFC 2753,
+ DOI 10.17487/RFC2753, January 2000,
+ <https://www.rfc-editor.org/info/rfc2753>.
+
+ [RFC2961] Berger, L., Gan, D., Swallow, G., Pan, P., Tommasi, F.,
+ and S. Molendini, "RSVP Refresh Overhead Reduction
+ Extensions", RFC 2961, DOI 10.17487/RFC2961, April 2001,
+ <https://www.rfc-editor.org/info/rfc2961>.
+
+ [RFC2998] Bernet, Y., Ford, P., Yavatkar, R., Baker, F., Zhang, L.,
+ Speer, M., Braden, R., Davie, B., Wroclawski, J., and E.
+ Felstaine, "A Framework for Integrated Services Operation
+ over Diffserv Networks", RFC 2998, DOI 10.17487/RFC2998,
+ November 2000, <https://www.rfc-editor.org/info/rfc2998>.
+
+ [RFC3031] Rosen, E., Viswanathan, A., and R. Callon, "Multiprotocol
+ Label Switching Architecture", RFC 3031,
+ DOI 10.17487/RFC3031, January 2001,
+ <https://www.rfc-editor.org/info/rfc3031>.
+
+ [RFC3086] Nichols, K. and B. Carpenter, "Definition of
+ Differentiated Services Per Domain Behaviors and Rules for
+ their Specification", RFC 3086, DOI 10.17487/RFC3086,
+ April 2001, <https://www.rfc-editor.org/info/rfc3086>.
+
+ [RFC3124] Balakrishnan, H. and S. Seshan, "The Congestion Manager",
+ RFC 3124, DOI 10.17487/RFC3124, June 2001,
+ <https://www.rfc-editor.org/info/rfc3124>.
+
+ [RFC3168] Ramakrishnan, K., Floyd, S., and D. Black, "The Addition
+ of Explicit Congestion Notification (ECN) to IP",
+ RFC 3168, DOI 10.17487/RFC3168, September 2001,
+ <https://www.rfc-editor.org/info/rfc3168>.
+
+ [RFC3175] Baker, F., Iturralde, C., Le Faucheur, F., and B. Davie,
+ "Aggregation of RSVP for IPv4 and IPv6 Reservations",
+ RFC 3175, DOI 10.17487/RFC3175, September 2001,
+ <https://www.rfc-editor.org/info/rfc3175>.
+
+ [RFC3198] Westerinen, A., Schnizlein, J., Strassner, J., Scherling,
+ M., Quinn, B., Herzog, S., Huynh, A., Carlson, M., Perry,
+ J., and S. Waldbusser, "Terminology for Policy-Based
+ Management", RFC 3198, DOI 10.17487/RFC3198, November
+ 2001, <https://www.rfc-editor.org/info/rfc3198>.
+
+ [RFC3209] Awduche, D., Berger, L., Gan, D., Li, T., Srinivasan, V.,
+ and G. Swallow, "RSVP-TE: Extensions to RSVP for LSP
+ Tunnels", RFC 3209, DOI 10.17487/RFC3209, December 2001,
+ <https://www.rfc-editor.org/info/rfc3209>.
+
+ [RFC3270] Le Faucheur, F., Ed., Wu, L., Davie, B., Davari, S.,
+ Vaananen, P., Krishnan, R., Cheval, P., and J. Heinanen,
+ "Multi-Protocol Label Switching (MPLS) Support of
+ Differentiated Services", RFC 3270, DOI 10.17487/RFC3270,
+ May 2002, <https://www.rfc-editor.org/info/rfc3270>.
+
+ [RFC3272] Awduche, D., Chiu, A., Elwalid, A., Widjaja, I., and X.
+ Xiao, "Overview and Principles of Internet Traffic
+ Engineering", RFC 3272, DOI 10.17487/RFC3272, May 2002,
+ <https://www.rfc-editor.org/info/rfc3272>.
+
+ [RFC3469] Sharma, V., Ed. and F. Hellstrand, Ed., "Framework for
+ Multi-Protocol Label Switching (MPLS)-based Recovery",
+ RFC 3469, DOI 10.17487/RFC3469, February 2003,
+ <https://www.rfc-editor.org/info/rfc3469>.
+
+ [RFC3473] Berger, L., Ed., "Generalized Multi-Protocol Label
+ Switching (GMPLS) Signaling Resource ReserVation Protocol-
+ Traffic Engineering (RSVP-TE) Extensions", RFC 3473,
+ DOI 10.17487/RFC3473, January 2003,
+ <https://www.rfc-editor.org/info/rfc3473>.
+
+ [RFC3630] Katz, D., Kompella, K., and D. Yeung, "Traffic Engineering
+ (TE) Extensions to OSPF Version 2", RFC 3630,
+ DOI 10.17487/RFC3630, September 2003,
+ <https://www.rfc-editor.org/info/rfc3630>.
+
+ [RFC3945] Mannie, E., Ed., "Generalized Multi-Protocol Label
+ Switching (GMPLS) Architecture", RFC 3945,
+ DOI 10.17487/RFC3945, October 2004,
+ <https://www.rfc-editor.org/info/rfc3945>.
+
+ [RFC4090] Pan, P., Ed., Swallow, G., Ed., and A. Atlas, Ed., "Fast
+ Reroute Extensions to RSVP-TE for LSP Tunnels", RFC 4090,
+ DOI 10.17487/RFC4090, May 2005,
+ <https://www.rfc-editor.org/info/rfc4090>.
+
+ [RFC4124] Le Faucheur, F., Ed., "Protocol Extensions for Support of
+ Diffserv-aware MPLS Traffic Engineering", RFC 4124,
+ DOI 10.17487/RFC4124, June 2005,
+ <https://www.rfc-editor.org/info/rfc4124>.
+
+ [RFC4203] Kompella, K., Ed. and Y. Rekhter, Ed., "OSPF Extensions in
+ Support of Generalized Multi-Protocol Label Switching
+ (GMPLS)", RFC 4203, DOI 10.17487/RFC4203, October 2005,
+ <https://www.rfc-editor.org/info/rfc4203>.
+
+ [RFC4271] Rekhter, Y., Ed., Li, T., Ed., and S. Hares, Ed., "A
+ Border Gateway Protocol 4 (BGP-4)", RFC 4271,
+ DOI 10.17487/RFC4271, January 2006,
+ <https://www.rfc-editor.org/info/rfc4271>.
+
+ [RFC4340] Kohler, E., Handley, M., and S. Floyd, "Datagram
+ Congestion Control Protocol (DCCP)", RFC 4340,
+ DOI 10.17487/RFC4340, March 2006,
+ <https://www.rfc-editor.org/info/rfc4340>.
+
+ [RFC4461] Yasukawa, S., Ed., "Signaling Requirements for Point-to-
+ Multipoint Traffic-Engineered MPLS Label Switched Paths
+ (LSPs)", RFC 4461, DOI 10.17487/RFC4461, April 2006,
+ <https://www.rfc-editor.org/info/rfc4461>.
+
+ [RFC4594] Babiarz, J., Chan, K., and F. Baker, "Configuration
+ Guidelines for DiffServ Service Classes", RFC 4594,
+ DOI 10.17487/RFC4594, August 2006,
+ <https://www.rfc-editor.org/info/rfc4594>.
+
+ [RFC4655] Farrel, A., Vasseur, J.-P., and J. Ash, "A Path
+ Computation Element (PCE)-Based Architecture", RFC 4655,
+ DOI 10.17487/RFC4655, August 2006,
+ <https://www.rfc-editor.org/info/rfc4655>.
+
+ [RFC4872] Lang, J.P., Ed., Rekhter, Y., Ed., and D. Papadimitriou,
+ Ed., "RSVP-TE Extensions in Support of End-to-End
+ Generalized Multi-Protocol Label Switching (GMPLS)
+ Recovery", RFC 4872, DOI 10.17487/RFC4872, May 2007,
+ <https://www.rfc-editor.org/info/rfc4872>.
+
+ [RFC4873] Berger, L., Bryskin, I., Papadimitriou, D., and A. Farrel,
+ "GMPLS Segment Recovery", RFC 4873, DOI 10.17487/RFC4873,
+ May 2007, <https://www.rfc-editor.org/info/rfc4873>.
+
+ [RFC4875] Aggarwal, R., Ed., Papadimitriou, D., Ed., and S.
+ Yasukawa, Ed., "Extensions to Resource Reservation
+ Protocol - Traffic Engineering (RSVP-TE) for Point-to-
+ Multipoint TE Label Switched Paths (LSPs)", RFC 4875,
+ DOI 10.17487/RFC4875, May 2007,
+ <https://www.rfc-editor.org/info/rfc4875>.
+
+ [RFC5151] Farrel, A., Ed., Ayyangar, A., and JP. Vasseur, "Inter-
+ Domain MPLS and GMPLS Traffic Engineering -- Resource
+ Reservation Protocol-Traffic Engineering (RSVP-TE)
+ Extensions", RFC 5151, DOI 10.17487/RFC5151, February
+ 2008, <https://www.rfc-editor.org/info/rfc5151>.
+
+ [RFC5250] Berger, L., Bryskin, I., Zinin, A., and R. Coltun, "The
+ OSPF Opaque LSA Option", RFC 5250, DOI 10.17487/RFC5250,
+ July 2008, <https://www.rfc-editor.org/info/rfc5250>.
+
+ [RFC5305] Li, T. and H. Smit, "IS-IS Extensions for Traffic
+ Engineering", RFC 5305, DOI 10.17487/RFC5305, October
+ 2008, <https://www.rfc-editor.org/info/rfc5305>.
+
+ [RFC5329] Ishiguro, K., Manral, V., Davey, A., and A. Lindem, Ed.,
+ "Traffic Engineering Extensions to OSPF Version 3",
+ RFC 5329, DOI 10.17487/RFC5329, September 2008,
+ <https://www.rfc-editor.org/info/rfc5329>.
+
+ [RFC5331] Aggarwal, R., Rekhter, Y., and E. Rosen, "MPLS Upstream
+ Label Assignment and Context-Specific Label Space",
+ RFC 5331, DOI 10.17487/RFC5331, August 2008,
+ <https://www.rfc-editor.org/info/rfc5331>.
+
+ [RFC5357] Hedayat, K., Krzanowski, R., Morton, A., Yum, K., and J.
+ Babiarz, "A Two-Way Active Measurement Protocol (TWAMP)",
+ RFC 5357, DOI 10.17487/RFC5357, October 2008,
+ <https://www.rfc-editor.org/info/rfc5357>.
+
+ [RFC5394] Bryskin, I., Papadimitriou, D., Berger, L., and J. Ash,
+ "Policy-Enabled Path Computation Framework", RFC 5394,
+ DOI 10.17487/RFC5394, December 2008,
+ <https://www.rfc-editor.org/info/rfc5394>.
+
+ [RFC5440] Vasseur, JP., Ed. and JL. Le Roux, Ed., "Path Computation
+ Element (PCE) Communication Protocol (PCEP)", RFC 5440,
+ DOI 10.17487/RFC5440, March 2009,
+ <https://www.rfc-editor.org/info/rfc5440>.
+
+ [RFC5470] Sadasivan, G., Brownlee, N., Claise, B., and J. Quittek,
+ "Architecture for IP Flow Information Export", RFC 5470,
+ DOI 10.17487/RFC5470, March 2009,
+ <https://www.rfc-editor.org/info/rfc5470>.
+
+ [RFC5472] Zseby, T., Boschi, E., Brownlee, N., and B. Claise, "IP
+ Flow Information Export (IPFIX) Applicability", RFC 5472,
+ DOI 10.17487/RFC5472, March 2009,
+ <https://www.rfc-editor.org/info/rfc5472>.
+
+ [RFC5541] Le Roux, JL., Vasseur, JP., and Y. Lee, "Encoding of
+ Objective Functions in the Path Computation Element
+ Communication Protocol (PCEP)", RFC 5541,
+ DOI 10.17487/RFC5541, June 2009,
+ <https://www.rfc-editor.org/info/rfc5541>.
+
+ [RFC5557] Lee, Y., Le Roux, JL., King, D., and E. Oki, "Path
+ Computation Element Communication Protocol (PCEP)
+ Requirements and Protocol Extensions in Support of Global
+ Concurrent Optimization", RFC 5557, DOI 10.17487/RFC5557,
+ July 2009, <https://www.rfc-editor.org/info/rfc5557>.
+
+ [RFC5559] Eardley, P., Ed., "Pre-Congestion Notification (PCN)
+ Architecture", RFC 5559, DOI 10.17487/RFC5559, June 2009,
+ <https://www.rfc-editor.org/info/rfc5559>.
+
+ [RFC5623] Oki, E., Takeda, T., Le Roux, JL., and A. Farrel,
+ "Framework for PCE-Based Inter-Layer MPLS and GMPLS
+ Traffic Engineering", RFC 5623, DOI 10.17487/RFC5623,
+ September 2009, <https://www.rfc-editor.org/info/rfc5623>.
+
+ [RFC5664] Halevy, B., Welch, B., and J. Zelenka, "Object-Based
+ Parallel NFS (pNFS) Operations", RFC 5664,
+ DOI 10.17487/RFC5664, January 2010,
+ <https://www.rfc-editor.org/info/rfc5664>.
+
+ [RFC5671] Yasukawa, S. and A. Farrel, Ed., "Applicability of the
+ Path Computation Element (PCE) to Point-to-Multipoint
+ (P2MP) MPLS and GMPLS Traffic Engineering (TE)", RFC 5671,
+ DOI 10.17487/RFC5671, October 2009,
+ <https://www.rfc-editor.org/info/rfc5671>.
+
+ [RFC5693] Seedorf, J. and E. Burger, "Application-Layer Traffic
+ Optimization (ALTO) Problem Statement", RFC 5693,
+ DOI 10.17487/RFC5693, October 2009,
+ <https://www.rfc-editor.org/info/rfc5693>.
+
+ [RFC6107] Shiomoto, K., Ed. and A. Farrel, Ed., "Procedures for
+ Dynamically Signaled Hierarchical Label Switched Paths",
+ RFC 6107, DOI 10.17487/RFC6107, February 2011,
+ <https://www.rfc-editor.org/info/rfc6107>.
+
+ [RFC6119] Harrison, J., Berger, J., and M. Bartlett, "IPv6 Traffic
+ Engineering in IS-IS", RFC 6119, DOI 10.17487/RFC6119,
+ February 2011, <https://www.rfc-editor.org/info/rfc6119>.
+
+ [RFC6241] Enns, R., Ed., Bjorklund, M., Ed., Schoenwaelder, J., Ed.,
+ and A. Bierman, Ed., "Network Configuration Protocol
+ (NETCONF)", RFC 6241, DOI 10.17487/RFC6241, June 2011,
+ <https://www.rfc-editor.org/info/rfc6241>.
+
+ [RFC6372] Sprecher, N., Ed. and A. Farrel, Ed., "MPLS Transport
+ Profile (MPLS-TP) Survivability Framework", RFC 6372,
+ DOI 10.17487/RFC6372, September 2011,
+ <https://www.rfc-editor.org/info/rfc6372>.
+
+ [RFC6374] Frost, D. and S. Bryant, "Packet Loss and Delay
+ Measurement for MPLS Networks", RFC 6374,
+ DOI 10.17487/RFC6374, September 2011,
+ <https://www.rfc-editor.org/info/rfc6374>.
+
+ [RFC6601] Ash, G., Ed. and D. McDysan, "Generic Connection Admission
+ Control (GCAC) Algorithm Specification for IP/MPLS
+ Networks", RFC 6601, DOI 10.17487/RFC6601, April 2012,
+ <https://www.rfc-editor.org/info/rfc6601>.
+
+ [RFC6805] King, D., Ed. and A. Farrel, Ed., "The Application of the
+ Path Computation Element Architecture to the Determination
+ of a Sequence of Domains in MPLS and GMPLS", RFC 6805,
+ DOI 10.17487/RFC6805, November 2012,
+ <https://www.rfc-editor.org/info/rfc6805>.
+
+ [RFC7011] Claise, B., Ed., Trammell, B., Ed., and P. Aitken,
+ "Specification of the IP Flow Information Export (IPFIX)
+ Protocol for the Exchange of Flow Information", STD 77,
+ RFC 7011, DOI 10.17487/RFC7011, September 2013,
+ <https://www.rfc-editor.org/info/rfc7011>.
+
+ [RFC7149] Boucadair, M. and C. Jacquenet, "Software-Defined
+ Networking: A Perspective from within a Service Provider
+ Environment", RFC 7149, DOI 10.17487/RFC7149, March 2014,
+ <https://www.rfc-editor.org/info/rfc7149>.
+
+ [RFC7285] Alimi, R., Ed., Penno, R., Ed., Yang, Y., Ed., Kiesel, S.,
+ Previdi, S., Roome, W., Shalunov, S., and R. Woundy,
+ "Application-Layer Traffic Optimization (ALTO) Protocol",
+ RFC 7285, DOI 10.17487/RFC7285, September 2014,
+ <https://www.rfc-editor.org/info/rfc7285>.
+
+ [RFC7390] Rahman, A., Ed. and E. Dijk, Ed., "Group Communication for
+ the Constrained Application Protocol (CoAP)", RFC 7390,
+ DOI 10.17487/RFC7390, October 2014,
+ <https://www.rfc-editor.org/info/rfc7390>.
+
+ [RFC7426] Haleplidis, E., Ed., Pentikousis, K., Ed., Denazis, S.,
+ Hadi Salim, J., Meyer, D., and O. Koufopavlou, "Software-
+ Defined Networking (SDN): Layers and Architecture
+ Terminology", RFC 7426, DOI 10.17487/RFC7426, January
+ 2015, <https://www.rfc-editor.org/info/rfc7426>.
+
+ [RFC7471] Giacalone, S., Ward, D., Drake, J., Atlas, A., and S.
+ Previdi, "OSPF Traffic Engineering (TE) Metric
+ Extensions", RFC 7471, DOI 10.17487/RFC7471, March 2015,
+ <https://www.rfc-editor.org/info/rfc7471>.
+
+ [RFC7491] King, D. and A. Farrel, "A PCE-Based Architecture for
+ Application-Based Network Operations", RFC 7491,
+ DOI 10.17487/RFC7491, March 2015,
+ <https://www.rfc-editor.org/info/rfc7491>.
+
+ [RFC7551] Zhang, F., Ed., Jing, R., and R. Gandhi, Ed., "RSVP-TE
+ Extensions for Associated Bidirectional Label Switched
+ Paths (LSPs)", RFC 7551, DOI 10.17487/RFC7551, May 2015,
+ <https://www.rfc-editor.org/info/rfc7551>.
+
+ [RFC7567] Baker, F., Ed. and G. Fairhurst, Ed., "IETF
+ Recommendations Regarding Active Queue Management",
+ BCP 197, RFC 7567, DOI 10.17487/RFC7567, July 2015,
+ <https://www.rfc-editor.org/info/rfc7567>.
+
+ [RFC7665] Halpern, J., Ed. and C. Pignataro, Ed., "Service Function
+ Chaining (SFC) Architecture", RFC 7665,
+ DOI 10.17487/RFC7665, October 2015,
+ <https://www.rfc-editor.org/info/rfc7665>.
+
+ [RFC7679] Almes, G., Kalidindi, S., Zekauskas, M., and A. Morton,
+ Ed., "A One-Way Delay Metric for IP Performance Metrics
+ (IPPM)", STD 81, RFC 7679, DOI 10.17487/RFC7679, January
+ 2016, <https://www.rfc-editor.org/info/rfc7679>.
+
+ [RFC7680] Almes, G., Kalidindi, S., Zekauskas, M., and A. Morton,
+ Ed., "A One-Way Loss Metric for IP Performance Metrics
+ (IPPM)", STD 82, RFC 7680, DOI 10.17487/RFC7680, January
+ 2016, <https://www.rfc-editor.org/info/rfc7680>.
+
+ [RFC7923] Voit, E., Clemm, A., and A. Gonzalez Prieto, "Requirements
+ for Subscription to YANG Datastores", RFC 7923,
+ DOI 10.17487/RFC7923, June 2016,
+ <https://www.rfc-editor.org/info/rfc7923>.
+
+ [RFC7926] Farrel, A., Ed., Drake, J., Bitar, N., Swallow, G.,
+ Ceccarelli, D., and X. Zhang, "Problem Statement and
+ Architecture for Information Exchange between
+ Interconnected Traffic-Engineered Networks", BCP 206,
+ RFC 7926, DOI 10.17487/RFC7926, July 2016,
+ <https://www.rfc-editor.org/info/rfc7926>.
+
+ [RFC7950] Bjorklund, M., Ed., "The YANG 1.1 Data Modeling Language",
+ RFC 7950, DOI 10.17487/RFC7950, August 2016,
+ <https://www.rfc-editor.org/info/rfc7950>.
+
+ [RFC8033] Pan, R., Natarajan, P., Baker, F., and G. White,
+ "Proportional Integral Controller Enhanced (PIE): A
+ Lightweight Control Scheme to Address the Bufferbloat
+ Problem", RFC 8033, DOI 10.17487/RFC8033, February 2017,
+ <https://www.rfc-editor.org/info/rfc8033>.
+
+ [RFC8034] White, G. and R. Pan, "Active Queue Management (AQM) Based
+ on Proportional Integral Controller Enhanced (PIE) for
+ Data-Over-Cable Service Interface Specifications (DOCSIS)
+ Cable Modems", RFC 8034, DOI 10.17487/RFC8034, February
+ 2017, <https://www.rfc-editor.org/info/rfc8034>.
+
+ [RFC8040] Bierman, A., Bjorklund, M., and K. Watsen, "RESTCONF
+ Protocol", RFC 8040, DOI 10.17487/RFC8040, January 2017,
+ <https://www.rfc-editor.org/info/rfc8040>.
+
+ [RFC8051] Zhang, X., Ed. and I. Minei, Ed., "Applicability of a
+ Stateful Path Computation Element (PCE)", RFC 8051,
+ DOI 10.17487/RFC8051, January 2017,
+ <https://www.rfc-editor.org/info/rfc8051>.
+
+ [RFC8189] Randriamasy, S., Roome, W., and N. Schwan, "Multi-Cost
+ Application-Layer Traffic Optimization (ALTO)", RFC 8189,
+ DOI 10.17487/RFC8189, October 2017,
+ <https://www.rfc-editor.org/info/rfc8189>.
+
+ [RFC8231] Crabbe, E., Minei, I., Medved, J., and R. Varga, "Path
+ Computation Element Communication Protocol (PCEP)
+ Extensions for Stateful PCE", RFC 8231,
+ DOI 10.17487/RFC8231, September 2017,
+ <https://www.rfc-editor.org/info/rfc8231>.
+
+ [RFC8259] Bray, T., Ed., "The JavaScript Object Notation (JSON) Data
+ Interchange Format", STD 90, RFC 8259,
+ DOI 10.17487/RFC8259, December 2017,
+ <https://www.rfc-editor.org/info/rfc8259>.
+
+ [RFC8279] Wijnands, IJ., Ed., Rosen, E., Ed., Dolganow, A.,
+ Przygienda, T., and S. Aldrin, "Multicast Using Bit Index
+ Explicit Replication (BIER)", RFC 8279,
+ DOI 10.17487/RFC8279, November 2017,
+ <https://www.rfc-editor.org/info/rfc8279>.
+
+ [RFC8281] Crabbe, E., Minei, I., Sivabalan, S., and R. Varga, "Path
+ Computation Element Communication Protocol (PCEP)
+ Extensions for PCE-Initiated LSP Setup in a Stateful PCE
+ Model", RFC 8281, DOI 10.17487/RFC8281, December 2017,
+ <https://www.rfc-editor.org/info/rfc8281>.
+
+ [RFC8283] Farrel, A., Ed., Zhao, Q., Ed., Li, Z., and C. Zhou, "An
+ Architecture for Use of PCE and the PCE Communication
+ Protocol (PCEP) in a Network with Central Control",
+ RFC 8283, DOI 10.17487/RFC8283, December 2017,
+ <https://www.rfc-editor.org/info/rfc8283>.
+
+ [RFC8290] Hoeiland-Joergensen, T., McKenney, P., Taht, D., Gettys,
+ J., and E. Dumazet, "The Flow Queue CoDel Packet Scheduler
+ and Active Queue Management Algorithm", RFC 8290,
+ DOI 10.17487/RFC8290, January 2018,
+ <https://www.rfc-editor.org/info/rfc8290>.
+
+ [RFC8402] Filsfils, C., Ed., Previdi, S., Ed., Ginsberg, L.,
+ Decraene, B., Litkowski, S., and R. Shakir, "Segment
+ Routing Architecture", RFC 8402, DOI 10.17487/RFC8402,
+ July 2018, <https://www.rfc-editor.org/info/rfc8402>.
+
+ [RFC8453] Ceccarelli, D., Ed. and Y. Lee, Ed., "Framework for
+ Abstraction and Control of TE Networks (ACTN)", RFC 8453,
+ DOI 10.17487/RFC8453, August 2018,
+ <https://www.rfc-editor.org/info/rfc8453>.
+
+ [RFC8570] Ginsberg, L., Ed., Previdi, S., Ed., Giacalone, S., Ward,
+ D., Drake, J., and Q. Wu, "IS-IS Traffic Engineering (TE)
+ Metric Extensions", RFC 8570, DOI 10.17487/RFC8570, March
+ 2019, <https://www.rfc-editor.org/info/rfc8570>.
+
+ [RFC8571] Ginsberg, L., Ed., Previdi, S., Wu, Q., Tantsura, J., and
+ C. Filsfils, "BGP - Link State (BGP-LS) Advertisement of
+ IGP Traffic Engineering Performance Metric Extensions",
+ RFC 8571, DOI 10.17487/RFC8571, March 2019,
+ <https://www.rfc-editor.org/info/rfc8571>.
+
+ [RFC8655] Finn, N., Thubert, P., Varga, B., and J. Farkas,
+ "Deterministic Networking Architecture", RFC 8655,
+ DOI 10.17487/RFC8655, October 2019,
+ <https://www.rfc-editor.org/info/rfc8655>.
+
+ [RFC8664] Sivabalan, S., Filsfils, C., Tantsura, J., Henderickx, W.,
+ and J. Hardwick, "Path Computation Element Communication
+ Protocol (PCEP) Extensions for Segment Routing", RFC 8664,
+ DOI 10.17487/RFC8664, December 2019,
+ <https://www.rfc-editor.org/info/rfc8664>.
+
+ [RFC8684] Ford, A., Raiciu, C., Handley, M., Bonaventure, O., and C.
+ Paasch, "TCP Extensions for Multipath Operation with
+ Multiple Addresses", RFC 8684, DOI 10.17487/RFC8684, March
+ 2020, <https://www.rfc-editor.org/info/rfc8684>.
+
+ [RFC8685] Zhang, F., Zhao, Q., Gonzalez de Dios, O., Casellas, R.,
+ and D. King, "Path Computation Element Communication
+ Protocol (PCEP) Extensions for the Hierarchical Path
+ Computation Element (H-PCE) Architecture", RFC 8685,
+ DOI 10.17487/RFC8685, December 2019,
+ <https://www.rfc-editor.org/info/rfc8685>.
+
+ [RFC8795] Liu, X., Bryskin, I., Beeram, V., Saad, T., Shah, H., and
+ O. Gonzalez de Dios, "YANG Data Model for Traffic
+ Engineering (TE) Topologies", RFC 8795,
+ DOI 10.17487/RFC8795, August 2020,
+ <https://www.rfc-editor.org/info/rfc8795>.
+
+ [RFC8803] Bonaventure, O., Ed., Boucadair, M., Ed., Gundavelli, S.,
+ Seo, S., and B. Hesmans, "0-RTT TCP Convert Protocol",
+ RFC 8803, DOI 10.17487/RFC8803, July 2020,
+ <https://www.rfc-editor.org/info/rfc8803>.
+
+ [RFC8896] Randriamasy, S., Yang, R., Wu, Q., Deng, L., and N.
+ Schwan, "Application-Layer Traffic Optimization (ALTO)
+ Cost Calendar", RFC 8896, DOI 10.17487/RFC8896, November
+ 2020, <https://www.rfc-editor.org/info/rfc8896>.
+
+ [RFC8938] Varga, B., Ed., Farkas, J., Berger, L., Malis, A., and S.
+ Bryant, "Deterministic Networking (DetNet) Data Plane
+ Framework", RFC 8938, DOI 10.17487/RFC8938, November 2020,
+ <https://www.rfc-editor.org/info/rfc8938>.
+
+ [RFC8955] Loibl, C., Hares, S., Raszuk, R., McPherson, D., and M.
+ Bacher, "Dissemination of Flow Specification Rules",
+ RFC 8955, DOI 10.17487/RFC8955, December 2020,
+ <https://www.rfc-editor.org/info/rfc8955>.
+
+ [RFC8972] Mirsky, G., Min, X., Nydell, H., Foote, R., Masputra, A.,
+ and E. Ruffini, "Simple Two-Way Active Measurement
+ Protocol Optional Extensions", RFC 8972,
+ DOI 10.17487/RFC8972, January 2021,
+ <https://www.rfc-editor.org/info/rfc8972>.
+
+ [RFC9000] Iyengar, J., Ed. and M. Thomson, Ed., "QUIC: A UDP-Based
+ Multiplexed and Secure Transport", RFC 9000,
+ DOI 10.17487/RFC9000, May 2021,
+ <https://www.rfc-editor.org/info/rfc9000>.
+
+ [RFC9023] Varga, B., Ed., Farkas, J., Malis, A., and S. Bryant,
+ "Deterministic Networking (DetNet) Data Plane: IP over
+ IEEE 802.1 Time-Sensitive Networking (TSN)", RFC 9023,
+ DOI 10.17487/RFC9023, June 2021,
+ <https://www.rfc-editor.org/info/rfc9023>.
+
+ [RFC9040] Touch, J., Welzl, M., and S. Islam, "TCP Control Block
+ Interdependence", RFC 9040, DOI 10.17487/RFC9040, July
+ 2021, <https://www.rfc-editor.org/info/rfc9040>.
+
+ [RFC9113] Thomson, M., Ed. and C. Benfield, Ed., "HTTP/2", RFC 9113,
+ DOI 10.17487/RFC9113, June 2022,
+ <https://www.rfc-editor.org/info/rfc9113>.
+
+ [RFC9256] Filsfils, C., Talaulikar, K., Ed., Voyer, D., Bogdanov,
+ A., and P. Mattes, "Segment Routing Policy Architecture",
+ RFC 9256, DOI 10.17487/RFC9256, July 2022,
+ <https://www.rfc-editor.org/info/rfc9256>.
+
+ [RFC9262] Eckert, T., Ed., Menth, M., and G. Cauchie, "Tree
+ Engineering for Bit Index Explicit Replication (BIER-TE)",
+ RFC 9262, DOI 10.17487/RFC9262, October 2022,
+ <https://www.rfc-editor.org/info/rfc9262>.
+
+ [RFC9298] Schinazi, D., "Proxying UDP in HTTP", RFC 9298,
+ DOI 10.17487/RFC9298, August 2022,
+ <https://www.rfc-editor.org/info/rfc9298>.
+
+ [RFC9315] Clemm, A., Ciavaglia, L., Granville, L. Z., and J.
+ Tantsura, "Intent-Based Networking - Concepts and
+ Definitions", RFC 9315, DOI 10.17487/RFC9315, October
+ 2022, <https://www.rfc-editor.org/info/rfc9315>.
+
+ [RFC9332] De Schepper, K., Briscoe, B., Ed., and G. White, "Dual-
+ Queue Coupled Active Queue Management (AQM) for Low
+ Latency, Low Loss, and Scalable Throughput (L4S)",
+ RFC 9332, DOI 10.17487/RFC9332, January 2023,
+ <https://www.rfc-editor.org/info/rfc9332>.
+
+ [RFC9350] Psenak, P., Ed., Hegde, S., Filsfils, C., Talaulikar, K.,
+ and A. Gulko, "IGP Flexible Algorithm", RFC 9350,
+ DOI 10.17487/RFC9350, February 2023,
+ <https://www.rfc-editor.org/info/rfc9350>.
+
+ [RFC9439] Wu, Q., Yang, Y., Lee, Y., Dhody, D., Randriamasy, S., and
+ L. Contreras, "Application-Layer Traffic Optimization
+ (ALTO) Performance Cost Metrics", RFC 9439,
+ DOI 10.17487/RFC9439, August 2023,
+ <https://www.rfc-editor.org/info/rfc9439>.
+
+ [RFC9502] Britto, W., Hegde, S., Kaneriya, P., Shetty, R., Bonica,
+ R., and P. Psenak, "IGP Flexible Algorithm in IP
+ Networks", RFC 9502, DOI 10.17487/RFC9502, November 2023,
+ <https://www.rfc-editor.org/info/rfc9502>.
+
+ [RFC9552] Talaulikar, K., Ed., "Distribution of Link-State and
+ Traffic Engineering Information Using BGP", RFC 9552,
+ DOI 10.17487/RFC9552, December 2023,
+ <https://www.rfc-editor.org/info/rfc9552>.
+
+ [RR94] Rodrigues, M. and K.G. Ramakrishnan, "Optimal routing in
+ shortest-path data networks", Bell Labs Technical Journal,
+ Volume 6, Issue 1, Pages 117-138, DOI 10.1002/bltj.2267,
+ August 2002,
+ <https://onlinelibrary.wiley.com/doi/abs/10.1002/
+ bltj.2267>.
+
+ [SLDC98] Suter, B., Lakshman, T.V., Stiliadis, D., and A.K.
+ Choudhury, "Design considerations for supporting TCP with
+ per-flow queueing", Proceedings IEEE INFOCOM '98,
+ DOI 10.1109/INFCOM.1998.659666, April 1998,
+ <https://ieeexplore.ieee.org/document/659666>.
+
+ [SR-TE-POLICY]
+ Previdi, S., Filsfils, C., Talaulikar, K., Ed., Mattes,
+ P., and D. Jain, "Advertising Segment Routing Policies in
+ BGP", Work in Progress, Internet-Draft, draft-ietf-idr-
+ segment-routing-te-policy-26, 23 October 2023,
+ <https://datatracker.ietf.org/doc/html/draft-ietf-idr-
+ segment-routing-te-policy-26>.
+
+ [SR-TI-LFA]
+ Bashandy, A., Litkowski, S., Filsfils, C., Francois, P.,
+ Decraene, B., and D. Voyer, "Topology Independent Fast
+ Reroute using Segment Routing", Work in Progress,
+ Internet-Draft, draft-ietf-rtgwg-segment-routing-ti-lfa-
+ 13, 16 January 2024,
+ <https://datatracker.ietf.org/doc/html/draft-ietf-rtgwg-
+ segment-routing-ti-lfa-13>.
+
+ [TE-QoS-ROUTING]
+ Ash, G., "Traffic Engineering & QoS Methods for IP-, ATM-,
+ & Based Multiservice Networks", Work in Progress,
+ Internet-Draft, draft-ietf-tewg-qos-routing-04, October
+ 2001, <https://datatracker.ietf.org/doc/html/draft-ietf-
+ tewg-qos-routing-04>.
+
+ [WANG] Wang, Y., Wang, Z., and L. Zhang, "Internet traffic
+ engineering without full mesh overlaying", Proceedings
+ IEEE INFOCOM 2001, DOI 10.1109/INFCOM.2001.916782, April
+ 2001, <https://ieeexplore.ieee.org/document/916782>.
+
+ [XIAO] Xiao, X., Hannan, A., Bailey, B., and L. Ni, "Traffic
+ Engineering with MPLS in the Internet", IEEE Network,
+ Volume 14, Issue 2, Pages 28-33, DOI 10.1109/65.826369,
+ March 2000,
+ <https://courses.cs.washington.edu/courses/cse561/02au/
+ papers/xiao-mpls-net00.pdf>.
+
+ [YARE95] Yang, C. and A. Reddy, "A Taxonomy for Congestion Control
+ Algorithms in Packet Switching Networks", IEEE Network,
+ Pages 34-45, DOI 10.1109/65.397042, August 1995,
+ <https://ieeexplore.ieee.org/document/397042>.
+
+Appendix A. Summary of Changes since RFC 3272
+
+ The changes to this document since [RFC3272] are substantial and not
+ easily summarized as section-by-section changes. The material in the
+ document has been moved around considerably, some of it removed, and
+ new text added.
+
+ The approach taken here is to list the contents of both [RFC3272] and
+ this document saying, respectively, where the text has been placed
+ and where the text came from.
+
+A.1. RFC 3272
+
+ * Section 1.0 ("Introduction"): Edited in place in Section 1.
+
+ - Section 1.1 ("What is Internet Traffic Engineering?"): Edited
+ in place in Section 1.1.
+
+ - Section 1.2 ("Scope"): Moved to Section 1.3.
+
+ - Section 1.3 ("Terminology"): Moved to Section 1.4 with some
+ obsolete terms removed and a little editing.
+
+ * Section 2.0 ("Background"): Retained as Section 2 with some text
+ removed.
+
+ - Section 2.1 ("Context of Internet Traffic Engineering"):
+ Retained as Section 2.1.
+
+ - Section 2.2 ("Network Context"): Rewritten as Section 2.2.
+
+ - Section 2.3 ("Problem Context"): Rewritten as Section 2.3.
+
+ o Section 2.3.1 ("Congestion and its Ramifications"): Retained
+ as Section 2.3.1.
+
+ - Section 2.4 ("Solution Context"): Edited as Section 2.4.
+
+ o Section 2.4.1 ("Combating the Congestion Problem"):
+ Reformatted as Section 2.4.1.
+
+ - Section 2.5 ("Implementation and Operational Context"):
+ Retained as Section 2.5.
+
+ * Section 3.0 ("Traffic Engineering Process Model"): Retained as
+ Section 3.
+
+ - Section 3.1 ("Components of the Traffic Engineering Process
+ Model"): Retained as Section 3.1.
+
+ - Section 3.2 ("Measurement"): Merged into Section 3.1.
+
+ - Section 3.3 ("Modeling, Analysis, and Simulation"): Merged into
+ Section 3.1.
+
+ - Section 3.4 ("Optimization"): Merged into Section 3.1.
+
+ * Section 4.0 ("Historical Review and Recent Developments"):
+ Retained as Section 5, but the very historic aspects have been
+ deleted.
+
+ - Section 4.1 ("Traffic Engineering in Classical Telephone
+ Networks"): Deleted.
+
+ - Section 4.2 ("Evolution of Traffic Engineering in the
+ Internet"): Deleted.
+
+ - Section 4.3 ("Overlay Model"): Deleted.
+
+ - Section 4.4 ("Constraint-Based Routing"): Retained as
+ Section 5.1.3.1, but moved into Section 5.1.
+
+ - Section 4.5 ("Overview of Other IETF Projects Related to
+ Traffic Engineering"): Retained as Section 5.1 with many new
+ subsections.
+
+ o Section 4.5.1 ("Integrated Services"): Retained as
+ Section 5.1.1.1.
+
+ o Section 4.5.2 ("RSVP"): Retained as Section 5.1.3.2 with
+ some edits.
+
+ o Section 4.5.3 ("Differentiated Services"): Retained as
+ Section 5.1.1.2.
+
+ o Section 4.5.4 ("MPLS"): Retained as Section 5.1.3.3.
+
+ o Section 4.5.5 ("IP Performance Metrics"): Retained as
+ Section 5.1.3.6.
+
+ o Section 4.5.6 ("Flow Measurement"): Retained as
+ Section 5.1.3.7 with some reformatting.
+
+ o Section 4.5.7 ("Endpoint Congestion Management"): Retained
+ as Section 5.1.3.8.
+
+ - Section 4.6 ("Overview of ITU Activities Related to Traffic
+ Engineering"): Deleted.
+
+ - Section 4.7 ("Content Distribution"): Retained as Section 5.2.
+
+ * Section 5.0 ("Taxonomy of Traffic Engineering Systems"): Retained
+ as Section 4.
+
+ - Section 5.1 ("Time-Dependent Versus State-Dependent"): Retained
+ as Section 4.1.
+
+ - Section 5.2 ("Offline Versus Online"): Retained as Section 4.2.
+
+ - Section 5.3 ("Centralized Versus Distributed"): Retained as
+ Section 4.3 with additions.
+
+ - Section 5.4 ("Local Versus Global"): Retained as Section 4.4.
+
+ - Section 5.5 ("Prescriptive Versus Descriptive"): Retained as
+ Section 4.5 with additions.
+
+ - Section 5.6 ("Open-Loop Versus Closed-Loop"): Retained as
+ Section 4.6.
+
+ - Section 5.7 ("Tactical vs Strategic"): Retained as Section 4.7.
+
+ * Section 6.0 ("Recommendations for Internet Traffic Engineering"):
+ Retained as Section 6.
+
+ - Section 6.1 ("Generic Non-functional Recommendations"):
+ Retained as Section 6.1.
+
+ - Section 6.2 ("Routing Recommendations"): Retained as
+ Section 6.2 with edits.
+
+ - Section 6.3 ("Traffic Mapping Recommendations"): Retained as
+ Section 6.3.
+
+ - Section 6.4 ("Measurement Recommendations"): Retained as
+ Section 6.4.
+
+ - Section 6.5 ("Network Survivability"): Retained as Section 6.6.
+
+ o Section 6.5.1 ("Survivability in MPLS Based Networks"):
+ Retained as Section 6.6.1.
+
+ o Section 6.5.2 ("Protection Option"): Retained as
+ Section 6.6.2.
+
+ - Section 6.6 ("Traffic Engineering in Diffserv Environments"):
+ Retained as Section 6.8 with edits.
+
+ - Section 6.7 ("Network Controllability"): Retained as
+ Section 6.9.
+
+ * Section 7.0 ("Inter-Domain Considerations"): Retained as
+ Section 7.
+
+ * Section 8.0 ("Overview of Contemporary TE Practices in Operational
+ IP Networks"): Retained as Section 8.
+
+ * Section 9.0 ("Conclusion"): Removed.
+
+ * Section 10.0 ("Security Considerations"): Retained as Section 9
+ with considerable new text.
+
+A.2. This Document
+
+ * Section 1: Based on Section 1 of [RFC3272].
+
+ - Section 1.1: Based on Section 1.1 of [RFC3272].
+
+ - Section 1.2: New for this document.
+
+ - Section 1.3: Based on Section 1.2 of [RFC3272].
+
+ - Section 1.4: Based on Section 1.3 of [RFC3272].
+
+ * Section 2: Based on Section 2 of [RFC3272].
+
+ - Section 2.1: Based on Section 2.1 of [RFC3272].
+
+ - Section 2.2: Based on Section 2.2 of [RFC3272].
+
+ - Section 2.3: Based on Section 2.3 of [RFC3272].
+
+ o Section 2.3.1: Based on Section 2.3.1 of [RFC3272].
+
+ - Section 2.4: Based on Section 2.4 of [RFC3272].
+
+ o Section 2.4.1: Based on Section 2.4.1 of [RFC3272].
+
+ - Section 2.5: Based on Section 2.5 of [RFC3272].
+
+ * Section 3: Based on Section 3 of [RFC3272].
+
+ - Section 3.1: Based on Sections 3.1, 3.2, 3.3, and 3.4 of
+ [RFC3272].
+
+ * Section 4: Based on Section 5 of [RFC3272].
+
+ - Section 4.1: Based on Section 5.1 of [RFC3272].
+
+ - Section 4.2: Based on Section 5.2 of [RFC3272].
+
+ - Section 4.3: Based on Section 5.3 of [RFC3272].
+
+ o Section 4.3.1: New for this document.
+
+ o Section 4.3.2: New for this document.
+
+ - Section 4.4: Based on Section 5.4 of [RFC3272].
+
+ - Section 4.5: Based on Section 5.5 of [RFC3272].
+
+ o Section 4.5.1: New for this document.
+
+ - Section 4.6: Based on Section 5.6 of [RFC3272].
+
+ - Section 4.7: Based on Section 5.7 of [RFC3272].
+
+ * Section 5: Based on Section 4 of [RFC3272].
+
+ - Section 5.1: Based on Section 4.5 of [RFC3272].
+
+ o Section 5.1.1.1: Based on Section 4.5.1 of [RFC3272].
+
+ o Section 5.1.1.2: Based on Section 4.5.3 of [RFC3272].
+
+ o Section 5.1.1.3: New for this document.
+
+ o Section 5.1.1.4: New for this document.
+
+ o Section 5.1.1.5: New for this document.
+
+ o Section 5.1.2.1: New for this document.
+
+ o Section 5.1.2.2: New for this document.
+
+ o Section 5.1.2.3: New for this document.
+
+ o Section 5.1.3.1: Based on Section 4.4 of [RFC3272].
+
+ + Section 5.1.3.1.1: New for this document.
+
+ o Section 5.1.3.2: Based on Section 4.5.2 of [RFC3272].
+
+ o Section 5.1.3.3: Based on Section 4.5.4 of [RFC3272].
+
+ o Section 5.1.3.4: New for this document.
+
+ o Section 5.1.3.5: New for this document.
+
+ o Section 5.1.3.6: Based on Section 4.5.5 of [RFC3272].
+
+ o Section 5.1.3.7: Based on Section 4.5.6 of [RFC3272].
+
+ o Section 5.1.3.8: Based on Section 4.5.7 of [RFC3272].
+
+ o Section 5.1.3.9: New for this document.
+
+ o Section 5.1.3.10: New for this document.
+
+ o Section 5.1.3.11: New for this document.
+
+ o Section 5.1.3.12: New for this document.
+
+ o Section 5.1.3.13: New for this document.
+
+ o Section 5.1.3.14: New for this document.
+
+ o Section 5.1.3.15: New for this document.
+
+ - Section 5.2: Based on Section 4.7 of [RFC3272].
+
+ * Section 6: Based on Section 6 of [RFC3272].
+
+ - Section 6.1: Based on Section 6.1 of [RFC3272].
+
+ - Section 6.2: Based on Section 6.2 of [RFC3272].
+
+ - Section 6.3: Based on Section 6.3 of [RFC3272].
+
+ - Section 6.4: Based on Section 6.4 of [RFC3272].
+
+ - Section 6.5: New for this document.
+
+ - Section 6.6: Based on Section 6.5 of [RFC3272].
+
+ o Section 6.6.1: Based on Section 6.5.1 of [RFC3272].
+
+ o Section 6.6.2: Based on Section 6.5.2 of [RFC3272].
+
+ - Section 6.7: New for this document.
+
+ - Section 6.8: Based on Section 6.6 of [RFC3272].
+
+ - Section 6.9: Based on Section 6.7 of [RFC3272].
+
+ * Section 7: Based on Section 7 of [RFC3272].
+
+ * Section 8: Based on Section 8 of [RFC3272].
+
+ * Section 9: Based on Section 10 of [RFC3272].
+
+Acknowledgments
+
+ Much of the text in this document is derived from [RFC3272]. The
+ editor and contributors to this document would like to express their
+ gratitude to all involved in that work. Although the source text has
+ been edited in the production of this document, the original authors
+ should be considered as contributors to this work. They were:
+
+ Daniel O. Awduche
+ Movaz Networks
+
+
+ Angela Chiu
+ Celion Networks
+
+
+ Anwar Elwalid
+ Lucent Technologies
+
+
+ Indra Widjaja
+ Bell Labs, Lucent Technologies
+
+
+ XiPeng Xiao
+ Redback Networks
+
+
+ The acknowledgements in [RFC3272] were as below. All people who
+ helped in the production of that document also need to be thanked for
+ the carry-over into this new document.
+
+ | The authors would like to thank Jim Boyle for inputs on the
+ | recommendations section, Francois Le Faucheur for inputs on
+ | Diffserv aspects, Blaine Christian for inputs on measurement,
+ | Gerald Ash for inputs on routing in telephone networks and for
+ | text on event-dependent TE methods, Steven Wright for inputs on
+ | network controllability, and Jonathan Aufderheide for inputs on
+ | inter-domain TE with BGP. Special thanks to Randy Bush for
+ | proposing the TE taxonomy based on "tactical vs strategic"
+ | methods. The subsection describing an "Overview of ITU Activities
+ | Related to Traffic Engineering" was adapted from a contribution by
+ | Waisum Lai. Useful feedback and pointers to relevant materials
+ | were provided by J. Noel Chiappa. Additional comments were
+ | provided by Glenn Grotefeld during the working last call process.
+ | Finally, the authors would like to thank Ed Kern, the TEWG co-
+ | chair, for his comments and support.
+
+ The early draft versions of this document were produced by the TEAS
+ Working Group's RFC3272bis Design Team. The full list of members of
+ this team is:
+
+ Acee Lindem
+ Adrian Farrel
+ Aijun Wang
+ Daniele Ceccarelli
+ Dieter Beller
+ Jeff Tantsura
+ Julien Meuric
+ Liu Hua
+ Loa Andersson
+ Luis Miguel Contreras
+ Martin Horneffer
+ Tarek Saad
+ Xufeng Liu
+
+ The production of this document includes a fix to the original text
+ resulting from an errata report #309 [Err309] by Jean-Michel
+ Grimaldi.
+
+ The editor of this document would also like to thank Dhruv Dhody,
+ Gyan Mishra, Joel Halpern, Dave Taht, John Scudder, Rich Salz, Behcet
+ Sarikaya, Bob Briscoe, Erik Kline, Jim Guichard, Martin Duke, and
+ Roman Danyliw for review comments.
+
+ This work is partially supported by the European Commission under
+ Horizon 2020 grant agreement number 101015857 Secured autonomic
+ traffic management for a Tera of SDN flows (Teraflow).
+
+Contributors
+
+ The following people contributed substantive text to this document:
+
+ Gert Grammel
+ Email: ggrammel@juniper.net
+
+
+ Loa Andersson
+ Email: loa@pi.nu
+
+
+ Xufeng Liu
+ Email: xufeng.liu.ietf@gmail.com
+
+
+ Lou Berger
+ Email: lberger@labn.net
+
+
+ Jeff Tantsura
+ Email: jefftant.ietf@gmail.com
+
+
+ Daniel King
+ Email: daniel@olddog.co.uk
+
+
+ Boris Hassanov
+ Email: bhassanov@yandex-team.ru
+
+
+ Kiran Makhijani
+ Email: kiranm@futurewei.com
+
+
+ Dhruv Dhody
+ Email: dhruv.ietf@gmail.com
+
+
+ Mohamed Boucadair
+ Email: mohamed.boucadair@orange.com
+
+
+Author's Address
+
+ Adrian Farrel (editor)
+ Old Dog Consulting
+ Email: adrian@olddog.co.uk