INTERNET-DRAFT Luyuan Fang Intended Status: Informational Microsoft Expires: January 9, 2017 Luis M. Contreras Telefonica Daniel Voyer Bell Canada July 8, 2016 BGP/MPLS IP VPN Data Center Interconnect draft-fang-bess-data-center-interconnect-01 Abstract This document discusses two categories of inter-connections of BGP/MPLS IP VPN and Data Center (DC) overlay networks. In the first category, DC overlay virtual network is built with BGP/MPLS IP VPN (IP VPN) technologies, the inter-connection of IP VPN in the DC to IP VPN in the WAN enables end-to-end IP VPN connectivity. In the second category, DC overlay network uses non IP VPN overlay technologies, the inter-connection of any overlay virtual network in the DC to IP VPN in the WAN provides end user connectivity through stitching of different overlay technologies. Status of this Memo This Internet-Draft is submitted to IETF in full conformance with the provisions of BCP 78 and BCP 79. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF), its areas, and its working groups. Note that other groups may also distribute working documents as Internet-Drafts. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." The list of current Internet-Drafts can be accessed at http://www.ietf.org/1id-abstracts.html The list of Internet-Draft Shadow Directories can be accessed at http://www.ietf.org/shadow.html Fang et al. Expires [Page 1] INTERNET DRAFT Copyright and License Notice Copyright (c) 2016 IETF Trust and the persons identified as the document authors. All rights reserved. This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (http://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Simplified BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Simplified BSD License. Table of Contents 1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 2 1.1 Terminology . . . . . . . . . . . . . . . . . . . . . . . . 3 2. Use Cases . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 2.1 Case 1: End-to-end BGP IP VPN cloud inter-connection . . . . 4 2.2 Case 2: Hybrid cloud inter-connection . . . . . . . . . . . 4 3. Architecture reference models . . . . . . . . . . . . . . . . . 5 3.1 BGP/MPLS IP VPN Inter-AS model . . . . . . . . . . . . . . . 5 3.2 BGP/MPLS IP VPN Gateway PE to DC vCE Model . . . . . . . . . 6 3.3 Hybrid inter-connection model . . . . . . . . . . . . . . . 7 3.4 Cloud Provider as a VPN customer . . . . . . . . . . . . . . 7 4. Inter-connect IP VPN between DC and WAN . . . . . . . . . . . . 8 4.1 Existing Inter-AS options and DCI gap analysis . . . . . . . 8 4.1.1 Option A pros and cons . . . . . . . . . . . . . . . . . 8 4.1.2 Option B pros and cons . . . . . . . . . . . . . . . . . 9 4.1.3 Option C pros and cons . . . . . . . . . . . . . . . . . 9 4.1.4 Use of RTC . . . . . . . . . . . . . . . . . . . . . . . 10 4.2 Carriers' carrier model . . . . . . . . . . . . . . . . . . 10 5. Inter-connect IP VPN and non-IP VPN overlay networks . . . . . 11 6. Connection in NFV environments . . . . . . . . . . . . . . . . 12 7. Security Considerations . . . . . . . . . . . . . . . . . . . . 12 8. IANA Considerations . . . . . . . . . . . . . . . . . . . . . . 12 9. References . . . . . . . . . . . . . . . . . . . . . . . . . . 12 9.1 Normative References . . . . . . . . . . . . . . . . . . . 12 9.2 Informative References . . . . . . . . . . . . . . . . . . 13 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 13 1 Introduction With the growth of cloud services, the need of inter-connecting DC Fang et al. Expires [Page 2] INTERNET DRAFT overlay networks and Enterprise BGP/MPLS IP VPNs in the Wide Area Network (WAN) arises. Two categories of inter-connections of BGP/MPLS IP VPN [RFC4364] and Data Center (DC) overlay networks are discussed in this document. In the first category, DC overlay virtual network is built with BGP/MPLS IP VPN (IP VPN) technologies, the inter-connection of IP VPN in the DC to IP VPN in the WAN enables end-to-end IP VPN connectivity for Virtual Private Cloud (VPC) services. In the second category, DC overlay network uses non IP VPN overlay technologies, the inter- connection of any overlay virtual network in the DC to IP VPN in the WAN provides end user connectivity through stitching of different overlay technologies. This document discusses use cases of the inter-connection of BGP/MPLS VPN to Data Centers, the general requirements, and the proposed solutions for the inter-connections. 1.1 Terminology The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC 2119 [RFC2119]. Term Definition ----------- -------------------------------------------------- AS Autonomous System ASBR Autonomous System Border Router BGP Border Gateway Protocol CE Customer Edge GRE Generic Routing Encapsulation Hypervisor Virtual Machine Manager I2RS Interface to Routing System MP-BGP Multi-Protocol Border Gateway Protocol NVGRE Network Virtualization using GRE QoS Quality of Service RD Route Distinguisher RR Route Reflector RT Route Target RTC RT Constraint SDN Software Defined Network ToR Top-of-Rack switch vCE virtual Customer Edge Router VM Virtual Machine VN Virtual Network VPC Virtual Private Cloud vPE virtual Provider Edge Router Fang et al. Expires [Page 3] INTERNET DRAFT VPN Virtual Private Network VXLAN Virtual eXtensible Local Area Network WAN Wide Area Network 2. Use Cases 2.1 Case 1: End-to-end BGP IP VPN cloud inter-connection SPs have large deployments of BGP/MPLS IP VPNs. Many SPs are interested to extend the IP VPN capabilities into their DCs to provide end-to-end native BGP IP VPN services to their enterprise customers. BGP IP VPN provides routing isolation, rich policy control, and QoS support. The technologies developed to extend BGP IP VPN into DC servers or ToR are work in progress in IETF, [I-D.fang-l3vpn-virtual-pe],and [I-D.ietf-l3vpn-end-system]. The WAN and DC may be managed by the same or different administrative domains. One special case of the cloud inter-connection is the extension across the network of large DCs, typically concentrated in central areas of a network, being complemented by one or more micro-DCs, usually close to the access points of presence. ,-----. ( ') +------+ .--(. '.---. +------+ | | ( ' ' ) | | | DC_a +---( IP/MPLS WAN )---+ DC_b | | | (. .) | | +------+ '-( ( .) +------+ ''--' '-''--'' Figure 1. Cloud Inter-Connection use case 2.2 Case 2: Hybrid cloud inter-connection Inter-connecting network SPs Enterprise IP VPNs to Cloud/Content providers DCs for expanded services. The inter-connection between the SP BGP/MPLS IP VPNs and the cloud provider networks is needed to provide the service end-to-end. The inter-connection of different types of providers can be BGP/MPLS IP VPN to other VPN or overlay technologies which may be virtualized or non-virtualized. Fang et al. Expires [Page 4] INTERNET DRAFT 3. Architecture reference models The architecture reference models described below focus on the inter-connection aspects. Although the intra-DC implementation is not within the scope of this discussion, the intra-DC technology has a direct impact to inter-DC connection. Therefore, various models are illustrated. 3.1 BGP/MPLS IP VPN Inter-AS model The BGP/MPLS IP VPNs are implemented in both the WAN network and the Data Center. A customer VPN, for example VPNA in figure 2, consists of enterprise remote sites and VMs supporting applications in the DC. The IP VPN implementation is using vPE technology in DC. The two segments of the VPNs are inter-connected through ASBRs facing each other in the respective networks. ,-----. ,-----. ( ') ( ') .--(. '.---. .-.(. '.---. ( ' ' +-----+ +-----+ ) ( IP/MPLS WAN |ASBR1|---|ASBR2| DC Network ) (. +-----+ +-----+ .) +-----+ ( .) ( ( +-----+ | PE1 |-.-' '-''--'' ''--' '-''-|vPE2 | .----.-.----. .----.-.----. |VRFA| |VRFB| |VRFA| |VRFB| '----' '----' '----' '----' / \ / \ \ +---+ +---+ .---. .---. .---. |CE1| |CE2| |VM1| |VM2| |VM3| +---+ +---+ '---' '---' '---' (VPNA) (VPNB) ( VPNA ) (VPNB) Figure 2. BGP/MPLS IP VPN Inter-Connection with ASBR in each network One boarding ASBR can be shared for the inter-connection of the two networks, especially if the WAN and DC belong to the same provider. Figure 3 illustrates this shared ASBR model. Fang et al. Expires [Page 5] INTERNET DRAFT ,----.. ,-----. ( ') ( ') .--(. '.----. .-.(. '.---. ( ' ' +------+ ) ( IP/MPLS WAN | ASBR | DC Network ) (. +------+ .) +-----+ ( .) ( ( +-----+ | PE1 |-.-' '-''---'' ''--' '-''-|vPE2 | .----.-.----. .----.-.----. |VRFA| |VRFB| |VRFA| |VRFB| '----' '----' '----' '----' / \ / \ \ +---+ +---+ .---. .---. .---. |CE1| |CE2| |VM1| |VM2| |VM3| +---+ +---+ '---' '---' '---' (VPNA) (VPNB) ( VPNA ) (VPNB) Figure 3. BGP/MPLS IP VPN Inter-Connection with share ASBR 3.2 BGP/MPLS IP VPN Gateway PE to DC vCE Model A simple virtual CE (vCE) [I-D.fang-l3vpn-virtual-ce] model can be used to inter-connect client containers to the DC Gateway which function as PE. This model is used by SPs to provide managed services, when scale can meet the service requirement. ,----.. ,-----. ( ') ( ') .--(. '.----. .----. '.----. ( ' ' +-----|VRFA| +----+ ( IP/MPLS WAN |GW/PE'----' DC Network |vCE4| (. +-----|VRFB| +----+ +-----+ ( )' '----'.( )-' | (VPNB) | PE1 |-.-' '-''- ' '--' '+----+ .---. .----.-.----. |vCE3| |VM3| |VRFA| |VRFB| (VPNA) +----+ '---' '----' '----' / \ / \ .---..---. +---+ +---+ |VM1||VM2| |CE1| |CE2| '---''---' +---+ +---+ (VPNA) (VPNB) Figure 4. BGP/MPLS IP VPN GW/PE to vCEs Fang et al. Expires [Page 6] INTERNET DRAFT without BGP/MPLS IP VPN in the DC 3.3 Hybrid inter-connection model The BGP/MPLS IP VPNs are implemented in the WAN network, and non BGP/MPLS IP VPN Overlay are implemented in the DC. The connection of the two networks is outside of the technologies for Inter-AS connections for BGP IP VPNs. This model includes many variations depending on the specific technologies used in the DC overlay. Figure 5 provides a general view of this inter-connecting model with ASBR on the MPLS WAN side, and the DC GW on the DC side. It is also viable to use one shared ASBR/GW for the inter-connection, especially if the WAN and the DC belong to the same provider. ,-----. ,-----. ( ') ( ') .--(. '.---. .-.(. '.---. ( ' ' +-----+ +-----+ ) ( IP/MPLS WAN |ASBR |---|DC GW| DC Network ) (. +-----+ +-----+ .) +-----+ ( .) ( ( +-----+ | PE1 |-.-' '-''--'' ''--' '-''-| NVE | .----.-.----. +-----+ |VRFA| |VRFB| / \ \ '----' '----' .---. .---. .---. / \ |VM1| |VM2| |VM3| +---+ +---+ .---. .---. .---. |CE1| |CE2| ( TenantA) (TenantB) +---+ +---+ (VPNA) (VPNB) Figure 5. BGP/MPLS IP VPN Inter-Connection with non BGP/MPLS IP VPN Overlay in DC 3.4 Cloud Provider as a VPN customer The Cloud Provider can establish its own VPN for internal services between dispersed DCs across the network. The SP should facilitate the BGP connectivity between DCs without requiring the knowledge respect to the the detailed routing information of the Cloud Provider VPNs. Fang et al. Expires [Page 7] INTERNET DRAFT (VPN A) (VPN B) (VPN A) (VPN B) .---. .---. .---. .---. |VM1| |VM2| |VMi| |VMj| '---' '---' '---' '---' / / / / .----. .----. .----. .----. |VRFA| |VRFB| ,-----. |VRFA| |VRFB| '----'-'----' ( ') '----'-'----' +-|vPE|-+ .--(. '.---. +-|vPE|-+ | +---+ | ( ' ' ) | +--+ | | : +--+ ( IP/MPLS WAN ) +--+ : | | :..|CE|--(. .)--|CE|..: | | +--+ '-( ( .) +--+ | | DC_a | ''--' '-''--'' | DC_b | +-------+ +-------+ Figure 6. BGP/MPLS IP VPN Inter-Connection with SP as carrier's carrier of Cloud Provider 4. Inter-connect IP VPN between DC and WAN 4.1 Existing Inter-AS options and DCI gap analysis The inter-AS options described in [RFC4364] can be used for DC inter- connection. Option A, B, and C must be supported. 4.1.1 Option A pros and cons In Option A: back-to-back VRF. The PE-ASBR in one AS performs MPLS or IP VPN de-encapsulation and transmits packets to the peer PE-ASBR in the adjacent AS. The peer PE-ASBR performs MPLS or IP VPN encapsulation on the customer IPv4/IPv6 packets received, and transmits the packet through the IP backbone of the AS. VPN service providers exchange routes across a back-to-back VRF connection. Each VRF instance represents a separate VPN client, and it is configured on a separate PE-ASBR interface, allowing a PE-ASBR to communicate with its peer PE-ASBR as if the peer was a CE router. Pros: This is the most secure option among options A, B, and C. And it is the simplest model from operation perspective. Each PE-ASBR is treating the other as a CE. Cons: This option suffers from scaling limitations, because per Inter-AS VPN VRF and interface are needed on the PE-ASBR. Option A has been commonly used in BGP/MPLS VPN Inter-Provider inter- Fang et al. Expires [Page 8] INTERNET DRAFT connections because of the security considerations and its clear operational demarcation. DCI considerations: This is a simple way to connect DC and WAN if both sides are of small scale. Scale will be the major concern for DC inter-connect if large scale support is needed. Even if the DC scale is small, there are major concerns on receiving relevant routes (potentially a large number) from the WAN side, and Vice Versa. 4.1.2 Option B pros and cons In Option B: EBGP redistribution of labeled VPN-IPv4/IPv6 routes between the neighboring ASes. ASes exchange VPN routing information (routes and labels) to establish connections. To control connections between ASes, the PE routers and EBGP border edge routers maintain a label forwarding information base (LFIB). The LFIB manages the labels and routes that the PE routers and EBGP border edge routers receive during the exchange of VPN information. The ASes exchange VPN routing information, such as, the destination network, the next hop field associated with the distributing router, a local MPLS label, and an RD. ASBRs are configured to change the next hop (next-hop-self) when sending VPN-IPv4 NLRIs to the IBGP neighbors; the ASBRs must allocate a new label when they forward the NLRI to the IBGP neighbors. Pros: It provides improved scalability when compared with option A, since it removes the needs of per Inter-AS VPN VRF and interface on the ASBR. Cons: vanilla version of Option B is considered less secure in comparison with Option A, due to the dynamic routing information exchange that is involved. The ASBR scaling may still be an issue because ASBR must maintain all VPN routes. Option B is commonly used within single provider or for inter- provider connections. DCI considerations: Option B is one viable option to be used in DC inter-connection. However, it has the same scale concerns as other options because of the potentially large number of routes exchanged between the WAN and the DC. 4.1.3 Option C pros and cons In option C: Multihop eBGP redistribution of labeled VPN-IPv4/Ipv6 routes between source and destination ASs, with eBGP redistribution of labeled IPv4/IPv6 routes from AS to neighboring AS. The ASBRs need only to exchange host routes (/32 or /128) to the PE routers involved in the VPN, with the labels needed to get there. A Label Switch Path Fang et al. Expires [Page 9] INTERNET DRAFT (LSP) is built from the ingress PE router in one AS to the egress PE in the other AS (using Loopback addresses). VPN traffic uses this LSP to reach the other AS. From data plane's perspective, the ASBRs act as P routers, with no knowledge about the VPNs concerned. Between the two inter-connecting ASBRs, the VPN traffic is treated just as between two P routers, each VPN data packet is pre-pended with the VPN label and then with an egress-PE label. Option C can be further scaled by using route reflectors (RRs) in each AS. Pros:It is the most scalable option among all three. ASBR is no longer a bottle neck for VPN routes scaling as in Option B. Cons: Major security issues as IGP reachability must be exchanged between the inter-connecting ASes. Option C has seen used within a single SP for inter-AS connections. Using RR for VPN routes exchange is the common approach. DCI consideration: Option C SHOULD NOT be used for any DCI which is between two different providers for security reasons. In this option, though ASBR is not longer the scaling bottleneck, the scaling issues still call for careful design, as the total numbers of VRFs, VPN interfaces, and the VPN routes being exchanged between the two ASes can be very large. 4.1.4 Use of RTC RT constraint [RFC4684] function must be used to only distribute the IP VPN routes of a VPN from one AS to another under the condition that they both support that VPN in each of the AS. This is one most important function for scalable solution. However, all IP VPN routes are exchanged between the two ASes (e.g. WAN and DC) as long as they have to support the same VPNs. The potential IP VPN routes distribution can still be very substantial in large WAN and DC deployment. Additional aggregation and abstraction mechnisms can be used to avoid large numbers of VPN routes being exchanges across the border between the interconnecting WAN and the DC in either directions. 4.2 Carriers' carrier model A BGP/MPLS IP VPN is implemented between DCs in the WAN side. The DCs connected through the WAN belong to the same AS. Now the CE routers in the DCs are responsible of establishing a BGP session among them for advertising between DCs the routes external to the DCs, since now such responsibility lays on the Cloud/Content provider. Fang et al. Expires [Page 10] INTERNET DRAFT The Cloud Provider can establish its own VPN services that can be extended including dispersed DCs. The SP will not be involved in the routing of the VPN services implemented in the DCs; it will just provide MPLS connectivity between them. In the same way, the DC will not require to routing information from the SP (only the corresponding local PE). Pros: The SP does not require getting knowledge of the routes for the carried VPN. Cons: The equipment within the DCs should support MPLS. DCI considerations: The equipment within the DCs should support MPLS. 5. Inter-connect IP VPN and non-IP VPN overlay networks As one significant instance of the hybrid use-case described in section 2.2, a DC may support a multi-tenant virtualized service network using IP based DC overlay encapsulations such as VXLAN [I-D.mahalingam-dutt-dcops-vxlan] or NVGRE [I-D.sridharan-virtualization-nvgre]. Different deployment models may be used within the DC depending on the DC provider's functional and operational requirements. When an IP DC overlay is terminated at the DC Gateway router and traffic directed into a BGP/MPLS IP VPN, the DC Gateway router performs MPLS encapsulation towards the WAN and IP overlay based forwarding within the DC. The inter-connection mechanisms between the DC and the WAN may fall into two categories: 1. VRF Termination The overlay based virtual network terminates into a BGP IP VPN VRF at the DC-WAN Gateway router. Both the internal routes of the DC as well as the external routes received from the WAN router can be installed in the VRF forwarding table at the DC Gateway router. The DC Gateway performs an IP lookup, appropriate MPLS or IP encapsulation, and forward traffic. The DC Gateway router peers with the WAN router using one of the existing inter-AS mechanisms described above. The DC Gateway functions as an IP-VPN ASBR with local VRFs; for example, packets still undergo an IP forwarding lookup. Fang et al. Expires [Page 11] INTERNET DRAFT 2. DC-VN and IP VPN Inter-working In this case, the DC Gateway router performs a direct translation between VN-IDs and IP VPN labels while switching packets between the DC and WAN interfaces without performing an IP lookup. The forwarding table at the DC Gateway router is set up to do a VN-ID or label lookup and derive the output label or VN-ID. The DC Gateway Router acts as an Inter-AS Option B ASBR peering with other ASBRs. 6. Connection in NFV environments NFV environments are constituted of NFV Infrastructure (NFVI) Points of Presence (PoPs) where Virtual Network Functions (VNFs) are deployed. Such VNFs usually require interconnection to the WAN, where BGP/MPLS services are typically in place. The NFVI PoPs have gateways at the border which will be used as stitching points between the internal services in the NFVI PoP (connecting the VNF) and the external VPNs interconnecting such PoPs. Details on potential ways of achieving this connectivity are left for further versions of the draft. 7. Security Considerations BGP/MPLS Inter-AS security threats and defense techniques described in RFC 4111 [RFC4111] are applicable for the WAN/DC inter- connections. In addition, the protocols between the Gateway routers and the controller/orchestrator MUST be mutually authenticated. Given the potentially very large scale and the dynamic nature in the cloud/DC environment, the choice of key management mechanisms need to be further studied. 8. IANA Considerations None. 9. References 9.1 Normative References [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, March 1997. [RFC4364] Rosen, E. and Y. Rekhter, "BGP/MPLS IP Virtual Private Fang et al. Expires [Page 12] INTERNET DRAFT Networks (VPNs)", RFC 4364, February 2006. [RFC4684] Marques, P., Bonica, R., Fang, L., Martini, L., Raszuk, R., Patel, K., and J. Guichard, "Constrained Route Distribution for Border Gateway Protocol/MultiProtocol Label Switching (BGP/MPLS) Internet Protocol (IP) Virtual Private Networks (VPNs)", RFC 4684, November 2006. 9.2 Informative References [RFC4111] Fang, L., Ed., "Security Framework for Provider- Provisioned Virtual Private Networks (PPVPNs)", RFC 4111, July 2005. [I-D.ietf-l3vpn-end-system] Marques, P., Fang, L., Pan, P., Shukla, A., Napierala, M., "BGP-signaled end-system IP/VPNs", draft-ietf-l3vpn-end-system, work in progress. [I-D.fang-l3vpn-virtual-pe] Fang, L., Ward, D., Fernando, R., Napierala, M., Bitar, N., Rao, D., Rijsman, B., So, N., "BGP IP VPN Virtual PE", draft-fang-l3vpn-virtual-pe, work in progress. [I-D.fang-l3vpn-virtual-ce] Fang, L., Evans, J., Ward, D., Fernando, R., Mullooly, J., So, N., Bitar., N., Napierala, M., "BGP IP VPN Virtual PE", draft-fang-l3vpn-virtual-ce, work in progress. [I-D.mahalingam-dutt-dcops-vxlan]: Mahalingam, M, Dutt, D.., et al., "A Framework for Overlaying Virtualized Layer 2 Networks over Layer 3 Networks" draft-mahalingam-dutt-dcops-vxlan, work in progress. [I-D.sridharan-virtualization-nvgre]: SridharanNetwork, M., et al., "Virtualization using Generic Routing Encapsulation", draft-sridharan-virtualization-nvgre, work in progress. Authors' Addresses Luyuan Fang Microsoft 5600 148th Ave NE Redmond, WA 98052 Email: lufang@microsoft.com Luis M. Contreras Telefonica Ronda de la Comunicacion, s/n Fang et al. Expires [Page 13] INTERNET DRAFT Sur-3 building, 3rd floor Madrid 28050 Spain Email: luismiguel.contrerasmurillo@telefonica.com URI: http://people.tid.es/LuisM.Contreras/ Daniel Voyer Bell Canada Email: daniel.voyer@bell.ca Fang et al. Expires [Page 14]