Elsevier

Computer Networks

Volume 57, Issue 4, 13 March 2013, Pages 954-975
Computer Networks

BGP-XM: BGP eXtended Multipath for transit Autonomous Systems

https://doi.org/10.1016/j.comnet.2012.11.011Get rights and content

Abstract

Multipath interdomain routing has been proposed to enable flexible traffic engineering for transit Autonomos Systems (ASes). Yet, there is a lack of solutions providing maximal path diversity and backwards compatibility at the same time. The BGP-XM (Border Gateway Protocol-eXtended Multipath) extension presented in this paper is a complete and flexible approach to solve many of the limitations of previous BGP multipath solutions. ASes can benefit from multipath capabilities starting with a single upgraded router, and without any coordination with other ASes. BGP-XM defines an algorithm to merge into regular BGP updates information from paths which may even traverse different ASes. This algorithm can be combined with different multipath selection algorithms, such as the K-BESTRO (K-Best Route Optimizer) tunable selection algorithm proposed in this paper. A stability analysis and stable policy guidelines are provided. The performance evaluation of BGP-XM, running over an Internet-like topology, shows that high path diversity can be achieved even for limited deployments of the multipath mechanism. Further results for large-scale deployments reveal that the extension is suitable for large deployment since it shows a low impact in the AS path length and in the routing table size.

Introduction

The Internet topology is inherently redundant. Intra-site topologies show high redundancy in their interconnection [1], [2]. Autonomous Systems (ASes) are usually connected to multiple provider ASes [3] to leverage fault tolerance. Internet’s increasingly richer connection degree is the result of the quest for improved performance and lower transit costs. Even after removing all the paths which are not usable according to the business relationships established among the connected ASes, a large number of paths traversing different sequences of ASes exists between most of the Internet sites [4]. Additional redundancy comes from the use of multiple links between pairs of neighboring ASes [5].

For years, the networking community has sought for flexible and simple ways to use the largest number of available paths in order to improve availability and perform traffic engineering. In the same way as technologies such as MPLS (MultiProtocol Label Switching) achieve that flexibility in the intra-domain scope, the management of traffic exchanged between domains could be further improved if inter-domain routing policies are changed dynamically.

As reported in [6], this occurs in today’s Internet for the particular case of egress traffic in stub ASes. Another example are the mechanisms to coordinate traffic engineering across multiple links between two peering ASes proposed in [7], [8]. These solutions share in common that there is no need to advertise to other domains the routing changes due to the traffic engineering.

Nevertheless, transit ASes must advertise their selected paths to other ASes using BGP (Border Gateway Protocol). In the extreme case, a change in a route selected by a transit AS, which wishes to modify its outgoing traffic pattern, may result in undesired changes for its incoming traffic, because of subsequent routing decisions made by other ASes [9]. Moreover, even if the effects for the incoming traffic are negligible, the deployment of this strategy is likely to stress BGP routers all over the Internet, since they will have to cope with frequent routing changes. In addition to the harm caused by route recomputation, more undesired interactions may appear if ASes perform active path monitoring as in route flap damping [10].

Multipath routing has been proposed to achieve flexible and frequent load balancing without path recomputation [11], [12]. Should the routing protocol make available multiple paths to a destination, a router is able to balance traffic across any combination of available paths without being forced to send additional routing messages. This should provide transit ASes with finer control over their outgoing traffic without incurring in the aforementioned problems.

Nevertheless, upgrading the current inter-domain routing to support multipath is far from trivial, especially due to the need of incremental deployments. Current proposals for multipath inter-domain routing are based on the advertisement of additional paths, others than the BGP best path, by means of a parallel negotiation between routers [13] or new BGP capabilities [14]. Unfortunately, their deployment requires the support of the mechanism in two or more neighboring ASes.

Other proposals [15], [16] advocate the utilization of multiple paths while maintaining the BGP advertising scheme, thus announcing only one of them as in-use. Yet, withholding the advertisement of in-use paths requires additional mechanisms to ensure loop-freeness.

The taxonomy is completed with some commercial router implementations, which already allow the use of multiple paths as long as all of them share most of their BGP attributes with the BGP best path [17]. In particular, all the routes to a destination can only differ in their IGP NEXT_HOP attribute. The obtained path diversity mostly result from the use of different links between consecutive ASes.

We claim that a multipath routing mechanism enabling inter-domain traffic engineering must fulfill the following requirements:

Allow high path diversity. The mechanism should impose as few restrictions as possible to the selection of multiple routes. In particular, it must not be restricted to select paths with the same sequence of traversed ASes or paths received from the same neighboring AS, since transit ASes may require moving traffic freely from one neighboring AS to another, e.g., to offload traffic to another provider or to obtain better performance.

Be backwards compatible. The mechanism must allow the selection of multiple routes being advertised by (unipath) BGP routers. Additionally, current BGP routers must be able to receive and use any set of routes created by a multipath router. To fulfill the latter, any set of routes selected by the multipath router must be expressed in a BGP-compatible format. The mechanism must not require any data plane modification in devices other than the multipath-upgraded routers. Hosts and routers must be able to exploit multipath routing without including multipath-specific information in data packets. Therefore, the resulting multipath service can be described as a multipath next-hop routing, i.e., every router takes forwarding decisions independently of the decisions taken by other routers.

Be incrementally deployable. In order to start providing multiple routes, the mechanism must not require being simultaneously deployed in different ASes, or in all the routers of an AS. Enabling multipath operation in a single router should be enough to disclose the multiple paths available to the router.

Be highly configurable. The selection of multiple routes must be tunable. For example, it must allow controlling the size of routing tables, the number of alternative paths per prefix or the prefixes for which multipath routing is enabled. In addition, the multipath mechanism must be configurable to limit the route stretch, i.e., the length difference between the longest and the shortest path to a destination, measured in the number of traversed ASes.

Seamlessly preserve usual business relationships. The current business model for inter-domain connectivity results in some widely used relationships such as transit, peering, siblings or partial transit [18]. The relationships define preference, import and export filtering rules for route advertisements. Despite the increment in the number of routes, the effort put in the configuration of a multipath mechanism should be close to that for configuring BGP.

Preserve the effects of usual traffic engineering techniques. An AS administrator may choose from a set of techniques to determine how its traffic exits from the AS, and to influence the path followed by the ingress traffic. Domain’s inclinations for outbound traffic are usually controlled by associating explicit preferences to routes. The most popular tools to influence the path for the incoming traffic, besides the injection of more specific prefixes, are the use of pre-agreed COMMUNITY values to inform other ASes about local preferences, the artificial increase of the length of the AS_PATH attribute to make the path less attractive, and the use of inter-AS metrics (MED attribute) [19]. Any multipath solution must allow both the domain deploying multipath and the rest of the domains to continue using these tools in a similar way as they currently do.

Generate loop-free paths. Resulting routes must be loop-free. Note that, in order to be backwards compatible, data packets are not required to carry any path selector, so loop-freeness is only assured if none of the possible combinations of the routes selected independently by different routers generate a cycle.

Stable under non-conflicting routing policies. ASes can choose their policies on their own, so the absence of conflicting policies in inter-domain routing cannot be guaranteed [20]. It has been shown that BGP is stable when transit and peering, the most common business relationships, are the only ones deployed [21]. Any multipath routing mechanism must assure that, at least, it is stable in the scenarios described in [21]. Note that this is not a trivial statement, as the work of Chau and Griffin [22] states that multipath can be unstable for configurations in which unipath is stable.

In this paper we present BGP eXtended Multipath, BGP-XM, a BGP extension that allows routers to use multiple paths across different ASes, including paths with different next ASes. BGP-XM defines a router architecture tailored to accommodate the information of multiple paths to the same network prefix into a single BGP update message. BGP-XM ensures that the resulting multipath updates preserve sufficient and meaningful information, so as to guarantee the routing policy intended by administrators is implemented. Regular tasks, such as policy-based route selection and loop detection are not altered significantly.

In particular, we argue that the benefits of selecting routes received from different ASes and advertising them using an aggregated attribute values such as AS_PATH attributes with standard AS_SET segment type [23], largely compensate for the drawbacks associated with the loss of detailed path description.

We base the design of BGP-XM on the analysis of the specific semantic requirements for intra-site and inter-site dissemination of inter-domain routing information, i.e. IBGP and EBGP modes of operation. From this analysis, we identify under which conditions, relaxing the tie-breaks of the BGP route selection increases path diversity without compromising the AS routing policy.

In addition, the modular BGP-XM architecture allows the definition of different multipath route selection procedures, which may suit different needs. We present K-BESTRO (K-Best Route Optimizer) as a possible implementation of the path selection procedure necessary in the BGP-XM architecture.

With these elements, the BGP-XM process architecture fulfills the requirements for an inter-domain multipath routing protocol that we stated above in this section. First, it allows selecting multiple paths which traverse different ASes, increasing the path diversity. It is backwards compatible with unipath BGP routers and enables router-level and AS-level incremental deployments. Moreover, loop detection is possible by inspecting the list of ASes in the generated AS_PATH. Since the semantic of the attributes is preserved, business relationships can still be configured by means of LOCAL_PREF and when combined with appropriate route selection mechanisms (such as K-BESTRO), most standard traffic engineering tools are supported. Also, stability is guaranteed under the conditions stated by Gao and Rexford [21] over BGP policies.

The paper is structured as follows: Section 2 analyzes how different routes can be merged into a single BGP advertisement with the lowest possible impact on the processing of incoming routes, route selection and processing of outgoing routes. To do this, we exhaustively discuss the effect of aggregating the most relevant BGP attributes from multiple routes. Section 3 presents the BGP-XM router architecture, which describes how a BGP-compatible update is generated, and the K-BESTRO route selection mechanism. Then, we provide an example to illustrate how route aggregation is performed by BGP-XM. In Section 5, we study in depth the stability properties of our inter-domain multipath framework, and we state one guideline for its safety, which fits with many configurations in use for unipath BGP. After this, the next section, Section 6, is devoted to present performance results for BGP-XM for an Internet-like topology, which show that high path diversity can be achieved with just a low number of ASes deploying this architecture. Related work is referenced in Section 7, and finally we present the conclusions.

Section snippets

Merging multiple paths into a single BGP update

In order to assure compatibility with BGP routers and to allow incremental deployment of inter-domain multipath, the BGP-XM process architecture is designed to generate BGP-compatible update messages.

The challenge in the design arises from the fixed BGP update scheme, which was defined to advertise one path per prefix. Although a BGP router may advertise multiple paths for a particular prefix, some of them even in the same BGP message, only the most recent is considered and it overrides any

BGP-XM architecture

As discussed before, there are a number of cases in which different routes can be selected at the same time without any impact in the operation of other BGP routers. BGP-XM defines an architecture that extends the BGP routing process to allow the use of these multiple routes. The design reuses most of the BGP protocol, e.g., session management, filtering and messages format. Fig. 4 shows the module diagram of the BGP-XM router architecture.

As for BGP, candidate paths for a prefix are retrieved

Example: an AS running BGP-XM

This example is referred to Fig. 5. The figure represents a transit AS (AS1) with three customers (AS3, AS4, AS6) and one provider (AS2). AS1 BGP-XM border routers are configured to comply with the filtering and preference rules usually defined to deploy business relationships [21]. All the routers at AS1 have ULMP = 1 (i.e., they select only paths with shortest AS_PATH length, and shortest AS_PATH length plus 1) and KBEST = 3 (i.e., they select up to three routes for a given destination). Routers

Stability analysis

The stability of BGP is well-known to depend on the preferences of the individual nodes over the available paths. In some cases, the preferences of all nodes cannot be fulfilled simultaneously and nodes may enter into conflicts, so-called dispute wheels [20]. Each router chooses a path according to its preferences. When several equally preferred paths are available, a BGP router selects just one of them applying a tie-break.

According to the results in [22], the stability of BGP depends on both

Performance evaluation

This section studies the performance of BGP-XM in a large-scale network. The experiments are carried out at the AS-level scope, i.e. each AS is represented as a single router. The Internet-like topology is extracted from the CAIDA Internet AS-Level Topology measurement project [31], with the AS business relationships dataset taken from [32]. The topology has a total number of 36,127 ASes, divided in 84.38% of stub ASes, 0.12% of transit ASes with large connectivity, and 15.5% of transit ASes

Related work

Several multipath inter-domain protocols have been proposed in the last years. Invariably, they encounter the problem of advertising multiple paths between domains that use BGP. In this section we collect and enumerate the different proposals according to the type of mechanism used to overcome the advertisement limitation introduced by BGP.

The novel Internet multipath architectures presented in [16], [15], [37] propose a loose version of source routing to influence the path followed by a

Conclusions

In this paper we have addressed the design of an inter-domain multipath routing framework compatible with unipath BGP operation, which is able to disclose high path diversity. The requirement of being backwards compatible with BGP has resulted in the use of the same route update message format as BGP, the adherence to the next-hop routing multipath model (as opposed to a source-routing multipath model), and an effort to preserve the complex BGP semantics resulting from the route processing,

Acknowledgements

The authors Francisco Valera, Marcelo Bagnulo and Jose M. Camacho are partially funded by the European Commission by means of the Trilogy project (ICT-216372) within the 7th Framework Programme and the Telefónica-UC3M chair in Future Internet for the Productivity. The work of Alberto García-Martínez has been partially supported by the eeCONTET Project (TEC2011-29688-C02-02) granted by the Spanish Science and Innovation Ministry. Acknowledgement to Lisardo Prieto, from University Carlos III de

Jose Manuel Camacho received a Telecommunication Engineering degree in 2008 and a Master of Science in Telematics in 2010, both from the University Carlos III of Madrid, Spain (UC3M). Since 2009, he holds a Ph.D. candidate position at the Telematics department of UC3M conducting research activities in the field of multi-path routing and traffic engineering.

References (46)

  • P. Oppenheimer

    Top-Down Network Design

    (2004)
  • D. Alderson et al.

    Understanding internet topology: principles, models, and validation

    IEEE/ACM Transactions on Networking

    (2005)
  • D. Meyer, University of oregon route views archive project, 2011....
  • A. Dhamdhere, C. Dovrolis, The internet is flat: modeling the transition from a transit hierarchy to a peering mesh,...
  • P. Merindol, V.V. den Schrieck, B. Donnet, O. Bonaventure, J.-J. Pansiot, Quantifying ases multiconnectivity using...
  • A. Akella et al.

    On the performance benefits of multihoming route control

    IEEE/ACM Transactions on Networking

    (2008)
  • R. Mahajan, D. Wetherall, T. Anderson, Towards coordinated interdomain traffic engineering, in: Proc. SIGCOMM Workshop...
  • S. Secci et al.

    Peering equilibrium multipath routing: a game theory framework for internet peering settlements

    IEEE/ACM Transactions on Networking

    (2011)
  • N. Feamster et al.

    Guidelines for interdomain traffic engineering

    ACM SIGCOMM Computer Communication Review

    (2003)
  • C. Villamizar, R. Chandra, R. Govindan, Rfc 2439: Bgp route flap damping, Internet Engineering Task Force,...
  • J. He et al.

    Toward internet-wide multipath routing

    IEEE Network

    (2008)
  • Y. Wang, M. Schapira, J. Rexford, Neighbor-specific BGP: more flexible routing policies while improving global...
  • W. Xu, J. Rexford, MIRO: multi-path interdomain routing, in: Proceedings of the 2006 Conference on Applications,...
  • D. Walton et al., Advertisement of Multiple Paths in BGP (draft-walton-bgp-add-paths-06. txt), Network Working Group...
  • M. Motiwala et al.

    Path splicing

    ACM SIGCOMM Computer Communication Review

    (2008)
  • X. Yang, D. Wetherall, Source selectable path diversity via routing deflections, in: SIGCOMM, ACM, 2006, pp....
  • Cisco, Bgp best path selection algorithm, 2012....
  • P. Faratin et al.

    The growing complexity of internet interconnection

    Communications & Strategies

    (2008)
  • M. Caesar et al.

    BGP routing policies in ISP networks

    IEEE Network

    (2005)
  • T. Griffin et al.

    The stable paths problem and interdomain routing

    IEEE/ACM Transactions on Networking (TON)

    (2002)
  • L. Gao et al.

    Stable Internet routing without global coordination

    IEEE/ACM Transactions on Networking (TON)

    (2001)
  • C. Chau, Policy-based routing with non-strict preferences, in: Proceedings of the 2006 Conference on Applications,...
  • Y. Rekhter, T. Li, S. Hares, RFC 4271: a Border Gateway Protocol 4 (BGP-4), Internet Engineering Task Force,...
  • Cited by (14)

    • RouteChain: Towards Blockchain-based secure and efficient BGP routing

      2022, Computer Networks
      Citation Excerpt :

      Moreover, it can be deployed in various inter-domain management applications, such as intrusion detection and failure analysis of routing. Camacho et al. [46] proposed BGP eXtended Multipath (BGP-XM) for transit Autonomous Systems. BGP-XM uses algorithms including K-Best Route Optimizer to strengthen BGP multi-path routing.

    • A survey on methods to provide interdomain multipath transmissions

      2016, Computer Networks
      Citation Excerpt :

      Furthermore, the capability of AMIR to introduce new paths through the negotiation process is considered as an important feature which is not present in other solutions like YAMR (Section 18) and Path Splicing (Section 10). The BGP extended multipath for transit domains (BGP-XM) is a routing mechanism which explores many redundant paths on the Internet [70]. This mechanism enables deployment of interdomain multipath and enables (flow) load balancing between different paths connecting distant sites (with many ASes on the way) on the Internet.

    • DIMR: Disjoint interdomain multipath routing

      2015, Computer Networks
      Citation Excerpt :

      Our deflection rule is different from the L-BGP, because it leverages the feature of the path pair. BGP-XM [13] enables a router to utilize multiple paths without coordination with others. In order to be compatible with standard BGP, it merges multiple paths in a single update.

    • It Takes Two to Tango: Cooperative Edge-to-Edge Routing

      2022, HotNets 2022 - Proceedings of the 2022 21st ACM Workshop on Hot Topics in Networks
    View all citing articles on Scopus

    Jose Manuel Camacho received a Telecommunication Engineering degree in 2008 and a Master of Science in Telematics in 2010, both from the University Carlos III of Madrid, Spain (UC3M). Since 2009, he holds a Ph.D. candidate position at the Telematics department of UC3M conducting research activities in the field of multi-path routing and traffic engineering.

    Alberto Garcia-Martinez received a telecommunication engineering degree in 1995 and a Ph.D. in telecommunications in 1999, both from the Polytechnic University of Madrid, Spain. In 1998 he joined the University Carlos III of Madrid (UC3M), where he has been an associate professor since 2001. His main interest areas are interdomain routing, layer-2 routing and security in IPv6. He has published more than 50 papers in technical journals, magazines, and conferences. He also participates in standardization efforts being carried out by the IETF.

    Marcelo Bagnulo received an electrical engineering degree in 1999 from the University of Uruguay, and a Ph.D. in telecommunications in 2005 from the University Carlos III of Madrid (UC3M). In 2000 he joined UC3M, where he has been an associate professor since 2006. He has published more than 50 papers in technical journals, magazines, and conferences, and he is co-author of many RFCs. He has served as member of the IAB, and currently he is the Director of the Telefonica Chair of Futute Internet At UC3M. His main interest areas are IPv6 and interdomain routing.

    Francisco Valera received the Telecommunication Engineering negree in 1998 from the Technical University of Madrid (UPM), and the Ph.D. in Telecommunications in 2002 from the Univ. Carlos III de Madrid (UC3M), where he is currently a tenured associate professor. He has published over 50 papers in the field of advanced communications in journals and conferences. He has also has participated in the scientific committee, organization and technical review in different national and international conferences.

    View full text