Decentralized media streaming infrastructure (DeMSI): An adaptive and high-performance peer-to-peer content delivery network

https://doi.org/10.1016/j.sysarc.2006.05.001Get rights and content

Abstract

Hosting an on-demand media content streaming service has been a challenging task mainly because of the outrageously enormous network and server bandwidth required to deliver large amount of content data to users simultaneously. We propose an infrastructure that helps online media content providers offload their server and network resources for media streaming. Using application level resource diversity together with the peer-to-peer resource-sharing model is a feasible approach to decentralize the content storage, server and network bandwidth. Each subscriber is responsible for only a small fraction of such resources. Most importantly, the cost of maintaining the service can also be shared amongst subscribers, especially when the subscriber base is large. As a result, subscribers can be benefit from lower subscription cost. There have been a few solutions out there that focused only on sharing the load of network bandwidth by division of a streaming task to be carried out by multiple sources. However, existing solutions require that the content to be replicated in full and stored in each source, which is impractical for a subscriber as the owner of the storage resource that is of consumer capacity. Our solution focuses on the division of responsibility on both the network bandwidth and content storage such that each subscriber is responsible for only a small portion of the content. We propose a light-weighted candidate peer selection strategy based on avoidance of network congestion and an adaptive re-scheduling algorithm in order to enhance smoothness of the aggregated streaming rate perceived at the consumer side. Experiments show that the performance of our peer-selection strategy out performs the traditional strategy based on end-to-end streaming bandwidth.

Introduction

Hosting an on-demand streaming service of persistent media content, such as video-on-demand, has been a challenging task mainly because of the outrageously enormous network and server bandwidth required to deliver, in real-time, large amount of video data to users simultaneously. In order to deliver a near DVD quality video stream while using as low the streaming rate as possible, a video compression technology such as MPEG-4 [5] is typically used. Informal studies [6] show that the streaming rate required for a near-DVD reproduction is at least 500 kbps. Maintaining a big network pipe enough to support the simultaneous video streams and a persistent 500 kbps bandwidth per stream for the duration of a movie (ranging from 1 to 3 h) is expensive. Therefore a pure single video server cluster to multiple consumers approach is quite a bad idea. The ability to scale is weak. A variant of that is to use multiple server clusters working like proxies in different regional locations or “edges” of the network to allow better scalability [8]. The consumer node is instructed to contact the proxy local to the consumer for streaming. Each proxy may act like the master that carries a replication of the contents, or caches a subset of contents most frequently requested by the local consumers [7]. Such distributed “edge architecture” helps reduce latency and number of hops before reaching the consumer, as the stream is scheduled to deliver from the proxy closest to the consumer. Hence the chance of encountering network congestion is lower. However, it does not mean that a local connection is free of congestion. As [9] suggests, packet loss (hence the congestion) in an end-to-end connection is usually caused by only a few hop-links in the path. Although there are more than one video server cluster to share the server and network loading, the system still suffers from single point of failure problem as the stream is still pushed from a single source over a single connection. Although the stream can be diverted through multiple paths of the network to avoid congestion [10], [11], [12], the technique is out of question as the routing is beyond the control of the content provider. In addition, since the cost of the servers and the network bandwidth for the streams belongs to the content provider, both the traditional single-server and the edge architecture suffer from under-utilized server and network resources problem during off-peak hours.

The existing problems of implementing a cost effective streaming service of persistent media content as mentioned above lead to the design of DeMSI – the Decentralized Media Streaming Infrastructure. The main objective of DeMSI is to ease the cost of content storage and workload of a video content distribution/delivery network (CDN), traditionally managed by the content provider, by offloading the streaming server, network and storage resources to subscriber workstations and their upstream internet bandwidth, without sacrificing video quality. Subscribers are not only the consumer of the service, but also a member of the content server. The fundamental idea is to allow multiple subscriber peers to serve streams of the same video content simultaneously to a consuming peer rather than the traditional single-server-to-client streaming model, while allowing each peer to store only a small portion of the content. It is anticipated that a subscriber peer can be a PC workstation, or simply a set top box with a few gigabytes of disk space to spare. Each peer has a broadband connection of at least 1.5 Mbps downstream and 256 kbps upstream to the internet. DeMSI is designed to be independent of the type of the media content. It is anticipated to work with both CBR/VBR video of any formats and bit rate, and it is not limited to serve video content, but any other media types that are stream-able.

Like other peer-to-peer applications, DeMSI has to face with the reliability issues of peer resources. Since peer resources are pretty much beyond control by the service owner, the domain of the reliability problems that DeMSI has to overcome includes:

  • 1.

    The unpredictability of dynamics in the condition of connection between the serving peers and the consuming peer. As a connection is made up of a path through hop-links, some links sharing traffic with other connections may be congested that result in delays and loss packets. Hence a varying end-to-end effective bandwidth;

  • 2.

    A peer may be turned off at any time. Even worse, a peer can be shut down abnormally such that one cannot expect a peer to notify another party of its unavailability;

  • 3.

    The integrity of contents is vulnerable as the content is stored at the peer end that is beyond control by the service owner.

The integrity of contents can be easily verified by employing a hash scheme, such as SHA-1, to the content data such that a tampered copy of the content can be detected upon deriving from the content a hash code different from the original. In DeMSI, the consumer may send the SHA-1 code of the content segment to the target peer along with the request for streaming. The peer then verify against the local copy and reply either by commencing the stream or a negative acknowledgement. This technique has been used in many P2P applications and we will not discuss this further here. Therefore, addressing problems 1 and 2 are the primary focus of this paper.

As the peers and their connections are unreliable, every P2P application have to deal with re-scheduling of streaming tasks and switching-over of peers when they become unavailable or the service level does not meet expectations. When a DeMSI consumer has a list of candidate subscriber peers discovered or previously contacted by others as consumers, it has to make a selection that achieves the following goals:

  • 1.

    To maximize the utilization of the network and peers.

  • 2.

    To minimize the number of peers to serve the content.

  • 3.

    To minimize the frequency of re-scheduling or emergency switching-over to other candidates over the course of streaming.

The goals are attributed to two important facts. Firstly, the need for fewer peers at a time in streaming implies fewer transitions over the course of a streaming session, and fewer peers are required to be online at any point in time. Secondly, the goals promote stability in aggregated streaming rate from the active serving peers. As a result, less buffering is required for received content prior to a playback. At first glance, peers with largest historic streaming rate should be selected first in order to achieve such a goal. However, this may not be true. Since the internet is made up of hop-links where they can share the traffic from multiple connections, DeMSI should expect there exist two or more candidates that have to send packets through the same hop-link(s). If one of those hop-links has tight bandwidth or filled with cross-traffic, while the previous selection of any one of those peers allows the peer to give 100% of its offered streaming rate, the selection of two or more of those peers may result in congestion such that those peers may only serve at a fraction of their offered rate. As our performance evaluation shows, this results in not only adding fluctuations to the aggregated streaming rate, but also the need for more peers in subsequent scheduling of streaming tasks. Moreover, re-scheduling becomes more frequent. DeMSI deals with this problem from two major directions:
  • 1.

    Proactive scheduling: Candidate peers with the largest historic end-to-end streaming bandwidth, smallest packet loss rate, and offer the largest portion of the content, while they share no or very few congested link(s) with the other actively serving peers, are selected first. The consumer constantly monitors and stores in its knowledgebase the above mentioned network metrics for each peer whenever it is actively serving. In addition, the consumer infers incrementally during the streaming session which peer connections are possibly sharing a congested link in the network, without contributing any additional overhead on the streams.

  • 2.

    Reactive scheduling: The underlying network characteristics of the peer-consumer connections and the availability of the peer itself change over time. Re-scheduling of streaming tasks and emergency switching-over of actively serving peers is unavoidable despite of how good the selection algorithm is. We design a sophisticated divide-and-conquer based scheduling/re-scheduling algorithm that is highly adaptive, flexible, aware of deadlines, and promotes smooth transitions.

As the storage of media content under DeMSI’s scenario is decentralized where no single peer contains the complete replication of the content, it is inherent that the consumer has to look for hundreds of peers, which means hundreds of transitions from one peer to another over the course of streaming. The scheduling and re-scheduling algorithms have to be light-weighted and perform in a timely fashion.

The remainder of this paper is organized as follows. Section 2 discusses related work. Section 3 provides the design details of DeMSI. Section 4 analyses the performance of our system in terms of the effectiveness of its peer selection strategy in the scheduling/re-scheduling processes, and the re-scheduling algorithm itself. Finally, the paper concludes with an outline of future work in Section 5.

Section snippets

Related work

A number of attempts have been made to decentralize CDNs. One of the popular approaches is to deploy several server clusters serving the same content in various regions of the world, or commonly known as the “edges” of the internet [8], [35]. Each server cluster serves the end-user population that is “local” to where the data center of that cluster node is located. The definition of local is a function of one or many parameters such as network topology, packet round-trip time, available

Architecture of DeMSI

This section presents an overview of our system and its functional components.

Performance evaluation

We evaluate DeMSI under a simplex (one-way) network as shown in Fig. 4-1 emulated by the NS-2 network simulator [25]. The network is made up of eight hop-links. Each cloud represents a combination of 3 Pareto traffic sources as cross-traffic. In particular, each of the clouds c1, c4, c5, c8 also contains 3 CBR traffic sources. Pareto sources are good approximation of the web traffic that is self-similar, whereas CBR sources are to approximate other long-lived streaming traffic. To simulate the

Conclusion and future work

This paper presents an infrastructural solution to address aggregated media streaming from a decentralized collection of unreliable subscriber resources, under the scenario where the media content is collectively stored at the subscriber ends. Unlike other P2P resource sharing solutions, each subscriber is responsible for only a small portion of the content rather than a complete replication of it. Our simulations demonstrate the effectiveness of the peer selection algorithm that employs a

References (39)

  • D. Rubenstein et al.

    Detecting shared congestion of flows via end-to-end measurement

    IEEE/ACM Transactions on Networking

    (2002)
  • O. Younis, S. Fahmy, On efficient on-line grouping of flows with shared bottlenecks at loaded servers, Technical Report...
  • M. Handley, S. Floyd, J. Padhye, J. Widmer, TCP Friendly Rate Control (TFRC) Protocol Specification – RFC 3448, January...
  • Onion Networks Inc., Java FEC Library v1.0.3. Available from:...
  • MPEG-4 Industry Forum FAQ. Available from:...
  • Dixon, Streaming Media: Trends and Formats, Manifest Technology,...
  • Bouras, Kapoulas, Konidaris, Sevasti, A dynamic distributed video on demand service, in: 20th IEEE International...
  • Akamai Technologies Inc. Available from:...
  • V.N. Padmanabhan, L. Qiu, H.J. Wang, Server-based inference of Internet link lossiness, in: Infocom 2003, IEEE,...
  • R. Teixeira, K. Marzullo, S. Savage, G.M. Voelker, In Search of Path Diversity in ISP Networks, IMC 03, ACM, October...
  • T. Nguyen, A. Zakhor, Path diversity with forward error correction (PDF) system for packet switched networks, in:...
  • J.G. Apostolopoulos, M.D. Trott, Path diversity for enhanced media streaming, in: IEEE Communications Magazine, IEEE,...
  • K. Calvert et al.

    Concast: design and implementation of an active network service

    IEEE Journal on Selected Area in Communications

    (2001)
  • T. Nguyen, A. ZakHor, Distributed video streaming with forward error correction, Packet Video Workshop 2002, Pittsburgh...
  • M. Hefeeda, A. Habib, D. Xu, B. Bhargava, B. Botev, CollectCast: a peer-to-peer service for media streaming, ACM...
  • M. Coates et al.

    Internet tomography

    IEEE Signal Processing Magazine

    (2002)
  • D. Katabi, C. Blake, Inferring congestion sharing and path characteristics for packet interarrival times,...
  • A. Rowstron, P. Druschel, Pastry: Scalable, distributed object location and routing for large-scale peer-to-peer...
  • S. Saroiu, P.K. Gummadi, S.D. Gribble, SProbe: a fast technique for measuring bottleneck bandwidth in uncooperative...
  • Cited by (8)

    • A vEB-tree-based architecture for interactive video on demand services in peer-to-peer networks

      2010, Journal of Network and Computer Applications
      Citation Excerpt :

      However, it needs to probe the real world network all the time, which is rarely available in practice. Wu et al. (2009) propose a system in which peers receive multiple forward error coding (FEC) packets from different parents to protect delivery content. P2VoD (Do et al., 2004) is used to handle the departure of peers without passing through the source by finding new parents in the upper layer.

    • Study and performance of a group-based Content Delivery Network

      2009, Journal of Network and Computer Applications
      Citation Excerpt :

      There many works that study the interconnection system between surrogates. Some of them are based on P2P systems (Jernberg et al., 2006; Yim and Buyya, 2006). There are also GRID-based proposals such as the one presented in reference (Di Stefano and Santoro, 2008).

    • A survey on dynamic data delivery through content delivery networks

      2018, International Journal of Engineering and Technology(UAE)
    • Impact of different content placement and delivery strategies on content delivery capacity of the wireless mesh networks

      2012, Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
    View all citing articles on Scopus
    View full text