From 4bfd864f10b68b71482b35c818559068ef8d5797 Mon Sep 17 00:00:00 2001 From: Thomas Voss Date: Wed, 27 Nov 2024 20:54:24 +0100 Subject: doc: Add RFC documents --- doc/rfc/rfc6847.txt | 731 ++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 731 insertions(+) create mode 100644 doc/rfc/rfc6847.txt (limited to 'doc/rfc/rfc6847.txt') diff --git a/doc/rfc/rfc6847.txt b/doc/rfc/rfc6847.txt new file mode 100644 index 0000000..817a383 --- /dev/null +++ b/doc/rfc/rfc6847.txt @@ -0,0 +1,731 @@ + + + + + + +Independent Submission D. Melman +Request for Comments: 6847 T. Mizrahi +Category: Informational Marvell +ISSN: 2070-1721 D. Eastlake 3rd + Huawei + January 2013 + + + Fibre Channel over Ethernet (FCoE) over + Transparent Interconnection of Lots of Links (TRILL) + +Abstract + + Fibre Channel over Ethernet (FCoE) and Transparent Interconnection of + Lots of Links (TRILL) are two emerging standards in the data center + environment. While these two protocols are seemingly unrelated, they + have a very similar behavior in the forwarding plane, as both perform + hop-by-hop forwarding over Ethernet, modifying the packet's Media + Access Control (MAC) addresses at each hop. This document describes + an architecture for the integrated deployment of these two protocols. + +Status of This Memo + + This document is not an Internet Standards Track specification; it is + published for informational purposes. + + This is a contribution to the RFC Series, independently of any other + RFC stream. The RFC Editor has chosen to publish this document at + its discretion and makes no statement about its value for + implementation or deployment. Documents approved for publication by + the RFC Editor are not a candidate for any level of Internet + Standard; see Section 2 of RFC 5741. + + Information about the current status of this document, any errata, + and how to provide feedback on it may be obtained at + http://www.rfc-editor.org/info/rfc6847. + +Copyright Notice + + Copyright (c) 2013 IETF Trust and the persons identified as the + document authors. All rights reserved. + + This document is subject to BCP 78 and the IETF Trust's Legal + Provisions Relating to IETF Documents + (http://trustee.ietf.org/license-info) in effect on the date of + publication of this document. Please review these documents + carefully, as they describe your rights and restrictions with respect + to this document. + + + +Melman, et al. Informational [Page 1] + +RFC 6847 FCoE over TRILL January 2013 + + +Table of Contents + + 1. Introduction ................................................. 2 + 2. Abbreviations ................................................ 3 + 3. FCoE over TRILL .............................................. 4 + 3.1. FCoE over a TRILL Cloud ................................. 4 + 3.2. FCoE over an RBridge .................................... 5 + 3.2.1. FCRB ............................................... 5 + 3.2.2. Topology ........................................... 7 + 3.2.3. The FCRB Flow ..................................... 8 + 3.2.3.1. Example - ENode to ENode ..................... 8 + 3.2.3.1.1. Forwarding from A to C in Dense Mode .... 9 + 3.2.3.1.2. Forwarding from A to C in Sparse Mode ... 9 + 3.2.3.2. Example - ENode to Native FC Node ............ 10 + 3.2.3.3. Example - ENode to ENode with Non-FCRB EoR ... 10 + 3.2.3.4. Example - FCoE Control Traffic through an FCRB 11 + 4. Security Considerations ..................................... 12 + 5. Acknowledgments ............................................. 12 + 6. References .................................................. 12 + 6.1. Normative References ................................... 12 + 6.2. Informative References ................................. 12 + +1. Introduction + + Data center networks are rapidly evolving towards a consolidated + approach, in which Ethernet is used as the common infrastructure for + all types of traffic. Storage traffic was traditionally dominated by + the Fibre Channel (FC) protocol suite. At the intersection between + these two technologies a new technology was born, Fibre Channel over + Ethernet (FCoE), in which native FC packets are encapsulated with an + FCoE encapsulation over an Ethernet header. FCoE is specified in + [FC-BB-5]. (A future version of FCoE is under development and is + expected to be specified in a document to be referred to as FC-BB-6; + however, this is a work in progress and is beyond the scope of this + document.) + + Traffic between two FCoE end nodes (ENodes) is forwarded through one + or more FCoE Forwarders (FCFs). An FCF takes a forwarding decision + based on the Fibre Channel destination ID (D_ID), and enforces + security policies between ENodes, also known as zoning. Once an FCF + takes a forwarding decision, it modifies the source and destination + MAC addresses of the packet, to reflect the path to the next-hop FCF + or ENode. An FCoE virtual link is an Ethernet link between an ENode + and an FCF, or between two FCFs. An FCoE virtual link may traverse + one or more Layer 2 bridges. FCFs use a routing protocol called + Fabric Shortest Path First (FSPF) to find the optimal path to each + + + + + +Melman, et al. Informational [Page 2] + +RFC 6847 FCoE over TRILL January 2013 + + + destination. An FCF typically has one or more native Fibre Channel + interfaces, allowing it to communicate with native Fibre Channel + devices, e.g., storage arrays. + + TRILL [TRILL] is a protocol for transparent least-cost routing, where + Routing Bridges (RBridges) forward traffic to their destination based + on a least-cost route, using a TRILL encapsulation header. RBridges + route TRILL-encapsulated packets based on the egress RBridge nickname + in the TRILL header. An RBridge routes a TRILL-encapsulated packet + after modifying its MAC addresses to reflect the path to the next-hop + RBridge and decrementing a Hop Count field. + + TRILL and FCoE bear a strong resemblance in their forwarding planes. + Both protocols take a routing decision based on protocol addresses + above Layer 2, and both modify the Ethernet MAC addresses on a per- + hop basis. Each of the protocols uses its own routing protocol + rather than using any type of bridging protocol, such as the spanning + tree protocol [802.1Q] or the Shortest Path Bridging protocol + [802.1Q]. + + FCoE and TRILL are both targeted at the data center environment, and + their concurrent deployment is self-evident. This document describes + an architecture for the integrated deployment of these two protocols. + +2. Abbreviations + + DCB Data Center Bridging + + ENode FCoE Node such as server or storage array + + EoR End of Row + + FC Fibre Channel + + FCF FCoE Forwarder + + FCoE Fibre Channel over Ethernet + + FCRB FCF over RBridge + + FIP FCoE Initialization Protocol + + FSPF Fabric Shortest Path First + + LAN Local Area Network + + RBridge Routing Bridge + + + + +Melman, et al. Informational [Page 3] + +RFC 6847 FCoE over TRILL January 2013 + + + SAN Storage Area Network + + ToR Top of Rack + + TRILL Transparent Interconnection of Lots of Links + + WAN Wide Area Network + +3. FCoE over TRILL + +3.1. FCoE over a TRILL Cloud + + The simplest approach for running FCoE traffic over a TRILL network + is presented in Figure 1. The figure illustrates a TRILL-enabled + network, in which FCoE traffic is transparently forwarded over the + TRILL cloud. The figure illustrates two ENodes, a Server and an FCoE + Storage Array, an FCF, and a native Fibre Channel SAN connected to + the FCF. + + FCoE traffic between the two ENodes is sent from the first ENode over + the TRILL cloud to the FCF, and then back through the TRILL cloud to + the second ENode. + + +---+ + | |_________ + | | \ ___ _ + +---+ \/ \_/ \__ _ __ + FCoE Storage _/ \ / \_/ \_ + Array / TRILL / +---+ \_ \ + (ENode A) \_ Cloud /________| |____/ SAN _/ + / \ | | \__ _/ + \__/\_ ___/ +---+ \_/ + +---+ / \_/ FCF + | |________/ + | | + +---+ + Server + (ENode B) + + Figure 1. The "Separate Cloud" Approach + + The configuration in Figure 1 separates the TRILL cloud(s) and the + FCoE cloud(s). The TRILL cloud routes FCoE traffic as standard + Ethernet traffic, and appears to the ENodes and FCF as an Ethernet + LAN. FCoE traffic routed over the TRILL cloud includes FCoE data + frames, as well as FCoE control traffic, including FCoE + + + + + +Melman, et al. Informational [Page 4] + +RFC 6847 FCoE over TRILL January 2013 + + + Initialization Protocol (FIP) frames. To eliminate frame loss due to + queue overflow, the switches in any TRILL Cloud used with FCoE would + likely implement and use the relevant DCB protocols [TRILLPFC] + [TRILLCN]. + + The main drawbacks of the Separate Cloud approach are that RBridges + and FCFs are separate nodes in the network, resulting in more cabling + and boxes, and that communication between ENodes usually requires + traversing the TRILL cloud twice, so there are twice as many hops. + As mentioned above, data center networking is converging towards a + consolidated and cost-effective approach, where the same + infrastructure and equipment are used for both data and storage + traffic, and where high efficiency and minimal number of hops are + important factors when designing the network topology. + + The Separate Cloud approach is presented as background to clarify the + motivation to develop an alternative approach with a higher level of + integration. + +3.2. FCoE over an RBridge + +3.2.1. FCRB + + Rather than using the Separate Cloud approach discussed in Section + 3.1, an alternate approach is presented, where each switch + incorporates both an FCF entity and an RBridge entity. This + consolidated entity is referred to as FCoE-forwarder-over-RBridge + (FCRB). + + Figure 2 illustrates an FCRB and its main building blocks. An FCRB + can be functionally viewed as two independent entities: + + o An FCoE Forwarder (FCF) entity. + + o An RBridge entity. + + The FCF entity is connected to one of the ports of the RBridge, and + appears to the RBridge as a native Ethernet host. A detailed + description of the interaction between the layers is presented in + Section 3.2.3. + + Note: In this document, the term "FCF" is used slightly differently + than defined in [FC-BB-5] to emphasize the concept that an FCRB is + logically similar to an RBridge cascaded to an FCF. In the + terminology defined in [FC-BB-5], an FCRB would be referred to as an + FCF, and the FCF building block in Figure 2 would be referred to as + an FC switching element. + + + + +Melman, et al. Informational [Page 5] + +RFC 6847 FCoE over TRILL January 2013 + + + +-------------------+ + |FCRB | + | +-----------+ | Native FC + | | FCF |------ Interface + | +-----+-----+ | + | | | + | +-----+-----+ | + | | RBridge | | + | +-+-+---+-+-+ | + | | | | | | + +-----|-|---|-|-----+ + FCoE/ / | | | + +---+ Ethernet / / | | FCoE / Ethernet + | |___________________/ / | | over TRILL ___ _ + | | / | | / \_/ \__ + +---+ / | \_____________ _/ \ + FCoE Storage / \_______________/ TRILL / + Array / \_ Cloud / + (ENode A) / / \ + / \__/\_ ___/ + +---+ / \_/ + | |______________/ + | | + +---+ + Server + (ENode B) + + Figure 2. FCRB Entity in the Network + + The FCRB entity maintains layer independence between the TRILL and + FCoE protocols, while enabling both protocols on the same network. + + Note that FCoE traffic is always forwarded through an FCF and cannot + be forwarded directly between two ENodes. Thus, FCoE traffic between + ENodes A and B in the topology in Figure 1 is forwarded through the + path + + ENode A-->TRILL cloud-->FCF-->TRILL cloud-->ENode B + + As opposed to the topology in Figure 1, the FCF in Figure 2 is + adjacent to ENodes A and B. In Figure 2, the FCRB is connected to + ENodes A and B, and functions as the edge RBridge that connects these + two nodes to the TRILL cloud, as well as the FCF that forwards + traffic between these two nodes. Thus, traffic between ENodes A and + B in the topology in Figure 2 is forwarded through the path + + ENode A-->FCRB-->ENode B + + + + +Melman, et al. Informational [Page 6] + +RFC 6847 FCoE over TRILL January 2013 + + + Hence, the usage of FCRB entities allows TRILL and FCoE to use common + infrastructure and equipment, as opposed to requiring separate + infrastructure as shown in the Separate Cloud topology presented in + Figure 1. + +3.2.2. Topology + + The network configuration illustrated in Figure 3 shows a typical + topology of a data center network. Servers are hierarchically + connected through Top-of-Rack (ToR) switches, also known as access + switches, and each set of racks is aggregated through an End-of-Row + (EoR) switch. The EoR switches are aggregated to the core switches, + which may be connected to other clouds, such as an external WAN or a + native FC SAN. + _ __ _ __ + / \_/ \_ / \_/ \_ + \_ \ \_ \ .... + / SAN _/ / WAN _/ + \__ _/ \__ _/ + \_/ \_/ + | | + | | + +------+ +------+ + Core | | | | + FCoE over | | | | + RBridge | | | | + (FCRB) +------+ +------+ + | \___ ___/ | + | \ / | + | \/ | + EoR +----+_______/\_______+----+ + FCoE over | | | | + RBridge | | | | + (FCRB) +----+ +----+ + / \ / \ + / \ / \ + ToR +---+ +---+ +---+ +---+ + FCoE over | | | | | | | | + RBridge | | | | | | | | + (FCRB) +---+ +---+ +---+ +---+ + / \ / \ / \ / \ + / \ / \ / \ / \ + +-+ +-+ +-+ +-+ +-+ +-+ +-+ +-+ + Servers/ | | | | | | | | | | | | | | | | + ENodes +-+ +-+ +-+ +-+ +-+ +-+ +-+ +-+ + A B C D E F G H + + Figure 3. FCoE over RBridge Topology + + + +Melman, et al. Informational [Page 7] + +RFC 6847 FCoE over TRILL January 2013 + + + Note that in the example in Figure 3, all the ToR, EoR, and core + switches are FCRB entities, but it is also possible for some of the + network nodes to be pure RBridges, creating a topology in which FCRBs + are interconnected through TRILL clouds. + +3.2.3. The FCRB Flow + +3.2.3.1. Example - ENode to ENode + + FCoE traffic sent between the two ENodes A and B in Figure 3 is + transmitted through the ToR FCRB, since A and B are connected to the + same ToR. Traffic between ENodes A and C must be forwarded through + the EoR FCRB. + + The FCoE jargon distinguishes between two deployment modes: + + o Sparse mode: an FCoE packet sent between two FCFs may be forwarded + over several hops of a Layer 2 network, allowing the underlying + Layer 2 network to determine the path between the two FCFs. + + o Dense mode: each node along the path between two FCFs is also an + FCF, and the network is configured such that the forwarding + decision at each hop is taken at the FCF layer, allowing the path + between the two FCFs to be based on the FSPF routing protocol. + + Figure 4 illustrates the traffic between ENodes A and C, which are + not connected to the same ToR. The following two subsections + describe the forwarding procedure in the Dense mode and in the Sparse + mode, respectively. + + +--------+ +--------+ +--------+ +--------+ +--------+ + | FCoE |.....| FCF |.....| FCF |.....| FCF |.....| FCoE | + | ENode | +--------+ +--------+ +--------+ | ENode | + | | |RBridge |.....|RBridge |.....|RBridge | | | + +--------+ +--------+ +--------+ +--------+ +--------+ + |Ethernet|<===>|Ethernet|<===>|Ethernet|<===>|Ethernet|<===>|Ethernet| + +--------+ +--------+ +--------+ +--------+ +--------+ + Server ToR 1 EoR ToR 2 FCoE Storage + ENode A FCRB FCRB FCRB Array + ENode C + + Figure 4. Traffic between two ENodes - Example + + + + + + + + + +Melman, et al. Informational [Page 8] + +RFC 6847 FCoE over TRILL January 2013 + + +3.2.3.1.1. Forwarding from A to C in Dense Mode + + o FCoE traffic from A is sent to ToR 1 over the Ethernet interface. + The destination MAC address is the address of the FCF entity at + ToR 1. + + o ToR 1: + + o The packet is forwarded to the FCF entity at the ToR. Thus, + forwarding between ENode A and the FCF at the ToR is + analogous to forwarding between two Ethernet hosts. + + o The FCF entity at the ToR takes a forwarding decision based + on the FC addresses. This decision is based on the FSPF + routing protocol at the FCF layer. The FCF entity at the + ToR forwards the packet to the FCF entity in the EoR. + + o The FCF then updates the destination MAC address of the + packet to the address of the EoR FCF. + + o The packet is forwarded to the RBridge entity, where it is + encapsulated in a TRILL header, and sent to the RBridge at + the EoR over a single hop of the TRILL network. + + o The RBridge entity in the EoR FCRB, acting as the egress RBridge, + decapsulates the TRILL header and forwards the FCoE packet to the + FCF entity. From this point, the forwarding process is similar to + the one described above for the ToR. + + o A similar forwarding process takes place at the next-hop ToR FCRB, + where the FCRB finally forwards the FCoE packet to the target, + ENode C. + +3.2.3.1.2. Forwarding from A to C in Sparse Mode + + o Traffic is forwarded to ToR 1, as described in Section 3.2.3.1.1. + + o The FCF in ToR 1, based on an FSPF forwarding decision, forwards + the packet to the FCF in ToR 2. The destination MAC address of + the FCoE packet is updated, reflecting the FCF in ToR 2. The + RBridge entity in ToR 2 adds a TRILL encapsulation, with an egress + RBridge nickname representing ToR 2. + + o The packet reaches the EoR. The RBridge entity in the EoR routes + the packet to the RBridge entity in ToR 2. + + o The packet reaches ToR 2. From this point on, the process is + identical to the one described in Section 3.2.3.1.1. + + + +Melman, et al. Informational [Page 9] + +RFC 6847 FCoE over TRILL January 2013 + + +3.2.3.2. Example - ENode to Native FC Node + ++--------+ +--------+ +--------+ +---------+ +--------+ +| FCoE |.....| FCF |.....| FCF |.....| FCF |.....| FC | +| ENode | +--------+ +--------+ +----+----+ |protocol| +| | |RBridge |.....|RBridge |.....| RB | | | stack | ++--------+ +--------+ +--------+ +----+ FC | | | +|Ethernet|<===>|Ethernet|<===>|Ethernet|<===>|Eth | |<===>| | ++--------+ +--------+ +--------+ +----+----+ +--------+ + Server ToR EoR Core Native FC + ENode FCRB FCRB FCRB Storage Array + + Figure 5. Example of Traffic between an + ENode and a Native FC Storage Array + + Figure 5 illustrates a second example, where traffic is sent between + an ENode and an FC Storage Array, based on the network topology in + Figure 3. + + o FCoE traffic from the ENode is sent to the ToR over the Ethernet + interface. The forwarding process through the ToR FCRB and + through the EoR is similar to the corresponding steps in Section + 3.2.3.1. + + o When the packet reaches the core FCRB, the egress RBridge entity + decapsulates the TRILL header and forwards the FCoE packet to the + FCF entity. The packet is then forwarded as a native FC packet + through the FC interface to the native FC node. + +3.2.3.3. Example - ENode to ENode with Non-FCRB EoR + + The example illustrated in Figure 6 is similar to the one shown in + Figure 4, except that the EoR is an RBridge rather than an FCRB. + ++--------+ +--------+ +--------+ +--------+ +| FCoE |.....| FCF |....................| FCF |.....| FCoE | +| ENode | +--------+ +--------+ +--------+ | ENode | +| | |RBridge |.....|RBridge |.....|RBridge | | | ++--------+ +--------+ +--------+ +--------+ +--------+ +|Ethernet|<===>|Ethernet|<===>|Ethernet|<===>|Ethernet|<===>|Ethernet| ++--------+ +--------+ +--------+ +--------+ +--------+ + Server ToR 1 EoR ToR 2 FCoE Storage + ENode A FCRB FCRB FCRB Array + ENode C + + Figure 6. Example of Traffic between Two ENodes + + + + + +Melman, et al. Informational [Page 10] + +RFC 6847 FCoE over TRILL January 2013 + + + An FCoE packet sent from ENode A to C is forwarded as follows: + + o The packet is sent to the FCF in ToR 1, as in the previous + example. + + o The FCF in ToR 1 takes a forwarding decision based on the FC + addresses and forwards the packet to the next-hop FCF, which + resides in ToR 2. This forwarding decision is taken at the FCF + layer and is based on the FSPF routing protocol. + + o The packet is then forwarded to the RBridge entity in ToR 1, where + it is encapsulated in a TRILL encapsulation, and forwarded to the + RBridge at ToR 2. The packet is routed over the TRILL cloud + through the RBridge at the EoR. The path through the TRILL cloud + is determined by TRILL's IS-IS routing protocol. + + o Once the packet reaches ToR 2, it is forwarded in a similar manner + to the description in Section 3.2.3.1. + + This example demonstrates that it is possible to have a hybrid + network, in which some of the nodes are FCRBs and some of the nodes + are RBridges. The forwarding procedure in this example is somewhat + similar to the sparse-mode forwarding described in Section 3.2.3.1.2. + +3.2.3.4. Example - FCoE Control Traffic through an FCRB + + The previous subsections focused on the data plane, i.e., storage + data exchanges transported over an FCoE encapsulation. FCoE also + requires control and management traffic that is used for initializing + sessions (i.e., FIP), distributing routing information (i.e., FSPF), + and administering and managing fabric. + + The FCoE Initialization Protocol (FIP) uses Ethernet frames with a + dedicated Ethertype, allowing the FCF to distinguish these frames + from other traffic. FIP uses both unicast and multicast traffic. + The following example describes the forwarding scheme of a multicast + FIP packet sent through the network depicted in Figure 4: + + o ENode A generates a multicast frame to a multicast MAC address + that represents all the FCFs (All-FCF-MAC). + + o The packet is forwarded to the ToR FCRB node. The RBridge entity + forwards a copy of the packet to its FCF entity, and also sends + the packet through the TRILL cloud as a multicast TRILL + encapsulated packet. + + + + + + +Melman, et al. Informational [Page 11] + +RFC 6847 FCoE over TRILL January 2013 + + + o Each of the FCRBs then receives the packet, forwards a copy to its + FCF entity, and forwards the packet through the TRILL network, + allowing all the FCFs to receive the packet. + + While FIP packets have a dedicated Ethertype and frame format, other + types of FCoE control and management frames use the same FCoE + encapsulation as FCoE data traffic. Thus, the forwarding scheme for + such control traffic is similar to the examples described in the + previous subsections, with the exception that these frames can be + sent between ENodes, between FCFs, or between ENodes and FCFs. + +4. Security Considerations + + For general TRILL security considerations, see [TRILL]. + + For general FCoE security considerations, see Annex D of [FC-BB-5]. + + There are no additional security implications imposed by this + document. + +5. Acknowledgments + + The authors gratefully acknowledge Ayandeh Siamack and David Black + for their helpful comments. The authors also thank the T11 committee + for reviewing the document, and in particular Pat Thaler and Joe + White for their useful input. + +6. References + +6.1. Normative References + + [TRILL] Perlman, R., Eastlake 3rd, D., Dutt, D., Gai, S., and A. + Ghanwani, "Routing Bridges (RBridges): Base Protocol + Specification", RFC 6325, July 2011. + + [FC-BB-5] ANSI INCITS 462: "Information Technology - Fibre Channel - + Backbone - 5 (FC-BB-5)", May 2010. + +6.2. Informative References + + [802.1Q] "IEEE Standard for Local and metropolitan area networks - + Media Access Control (MAC) Bridges and Virtual Bridged + Local Area Networks", IEEE Std 802.1Q(tm), 2012 Edition, + October 2012. + + + + + + + +Melman, et al. Informational [Page 12] + +RFC 6847 FCoE over TRILL January 2013 + + + [TRILLPFC] Eastlake 3rd, D., Wadekar, M., Ghanwani, A., Agarwal, P., + and T. Mizrahi, "TRILL: Support of IEEE 802.1 Priority- + based Flow Control and Enhanced Transmission Selection", + Work in Progress, January 2013. + + [TRILLCN] Eastlake 3rd, D., Wadekar, M., Ghanwani, A., Agarwal, P., + and T. Mizrahi, "TRILL: Support of IEEE 802.1 Congestion + Notification", Work in Progress, January 2013. + +Authors' Addresses + + David Melman + Marvell + 6 Hamada St. + Yokneam, 20692 Israel + + EMail: davidme@marvell.com + + + Tal Mizrahi + Marvell + 6 Hamada St. + Yokneam, 20692 Israel + + EMail: talmi@marvell.com + + + Donald Eastlake 3rd + Huawei USA R&D + 155 Beaver Street + Milford, MA 01757 USA + + Phone: +1-508-333-2270 + EMail: d3e3e3@gmail.com + + + + + + + + + + + + + + + + + +Melman, et al. Informational [Page 13] + -- cgit v1.2.3