Decentralized media streaming infrastructure (DeMSI): An adaptive and high-performance peer-to-peer content delivery network
Introduction
Hosting an on-demand streaming service of persistent media content, such as video-on-demand, has been a challenging task mainly because of the outrageously enormous network and server bandwidth required to deliver, in real-time, large amount of video data to users simultaneously. In order to deliver a near DVD quality video stream while using as low the streaming rate as possible, a video compression technology such as MPEG-4 [5] is typically used. Informal studies [6] show that the streaming rate required for a near-DVD reproduction is at least 500 kbps. Maintaining a big network pipe enough to support the simultaneous video streams and a persistent 500 kbps bandwidth per stream for the duration of a movie (ranging from 1 to 3 h) is expensive. Therefore a pure single video server cluster to multiple consumers approach is quite a bad idea. The ability to scale is weak. A variant of that is to use multiple server clusters working like proxies in different regional locations or “edges” of the network to allow better scalability [8]. The consumer node is instructed to contact the proxy local to the consumer for streaming. Each proxy may act like the master that carries a replication of the contents, or caches a subset of contents most frequently requested by the local consumers [7]. Such distributed “edge architecture” helps reduce latency and number of hops before reaching the consumer, as the stream is scheduled to deliver from the proxy closest to the consumer. Hence the chance of encountering network congestion is lower. However, it does not mean that a local connection is free of congestion. As [9] suggests, packet loss (hence the congestion) in an end-to-end connection is usually caused by only a few hop-links in the path. Although there are more than one video server cluster to share the server and network loading, the system still suffers from single point of failure problem as the stream is still pushed from a single source over a single connection. Although the stream can be diverted through multiple paths of the network to avoid congestion [10], [11], [12], the technique is out of question as the routing is beyond the control of the content provider. In addition, since the cost of the servers and the network bandwidth for the streams belongs to the content provider, both the traditional single-server and the edge architecture suffer from under-utilized server and network resources problem during off-peak hours.
The existing problems of implementing a cost effective streaming service of persistent media content as mentioned above lead to the design of DeMSI – the Decentralized Media Streaming Infrastructure. The main objective of DeMSI is to ease the cost of content storage and workload of a video content distribution/delivery network (CDN), traditionally managed by the content provider, by offloading the streaming server, network and storage resources to subscriber workstations and their upstream internet bandwidth, without sacrificing video quality. Subscribers are not only the consumer of the service, but also a member of the content server. The fundamental idea is to allow multiple subscriber peers to serve streams of the same video content simultaneously to a consuming peer rather than the traditional single-server-to-client streaming model, while allowing each peer to store only a small portion of the content. It is anticipated that a subscriber peer can be a PC workstation, or simply a set top box with a few gigabytes of disk space to spare. Each peer has a broadband connection of at least 1.5 Mbps downstream and 256 kbps upstream to the internet. DeMSI is designed to be independent of the type of the media content. It is anticipated to work with both CBR/VBR video of any formats and bit rate, and it is not limited to serve video content, but any other media types that are stream-able.
Like other peer-to-peer applications, DeMSI has to face with the reliability issues of peer resources. Since peer resources are pretty much beyond control by the service owner, the domain of the reliability problems that DeMSI has to overcome includes:
- 1.
The unpredictability of dynamics in the condition of connection between the serving peers and the consuming peer. As a connection is made up of a path through hop-links, some links sharing traffic with other connections may be congested that result in delays and loss packets. Hence a varying end-to-end effective bandwidth;
- 2.
A peer may be turned off at any time. Even worse, a peer can be shut down abnormally such that one cannot expect a peer to notify another party of its unavailability;
- 3.
The integrity of contents is vulnerable as the content is stored at the peer end that is beyond control by the service owner.
As the peers and their connections are unreliable, every P2P application have to deal with re-scheduling of streaming tasks and switching-over of peers when they become unavailable or the service level does not meet expectations. When a DeMSI consumer has a list of candidate subscriber peers discovered or previously contacted by others as consumers, it has to make a selection that achieves the following goals:
- 1.
To maximize the utilization of the network and peers.
- 2.
To minimize the number of peers to serve the content.
- 3.
To minimize the frequency of re-scheduling or emergency switching-over to other candidates over the course of streaming.
- 1.
Proactive scheduling: Candidate peers with the largest historic end-to-end streaming bandwidth, smallest packet loss rate, and offer the largest portion of the content, while they share no or very few congested link(s) with the other actively serving peers, are selected first. The consumer constantly monitors and stores in its knowledgebase the above mentioned network metrics for each peer whenever it is actively serving. In addition, the consumer infers incrementally during the streaming session which peer connections are possibly sharing a congested link in the network, without contributing any additional overhead on the streams.
- 2.
Reactive scheduling: The underlying network characteristics of the peer-consumer connections and the availability of the peer itself change over time. Re-scheduling of streaming tasks and emergency switching-over of actively serving peers is unavoidable despite of how good the selection algorithm is. We design a sophisticated divide-and-conquer based scheduling/re-scheduling algorithm that is highly adaptive, flexible, aware of deadlines, and promotes smooth transitions.
The remainder of this paper is organized as follows. Section 2 discusses related work. Section 3 provides the design details of DeMSI. Section 4 analyses the performance of our system in terms of the effectiveness of its peer selection strategy in the scheduling/re-scheduling processes, and the re-scheduling algorithm itself. Finally, the paper concludes with an outline of future work in Section 5.
Section snippets
Related work
A number of attempts have been made to decentralize CDNs. One of the popular approaches is to deploy several server clusters serving the same content in various regions of the world, or commonly known as the “edges” of the internet [8], [35]. Each server cluster serves the end-user population that is “local” to where the data center of that cluster node is located. The definition of local is a function of one or many parameters such as network topology, packet round-trip time, available
Architecture of DeMSI
This section presents an overview of our system and its functional components.
Performance evaluation
We evaluate DeMSI under a simplex (one-way) network as shown in Fig. 4-1 emulated by the NS-2 network simulator [25]. The network is made up of eight hop-links. Each cloud represents a combination of 3 Pareto traffic sources as cross-traffic. In particular, each of the clouds c1, c4, c5, c8 also contains 3 CBR traffic sources. Pareto sources are good approximation of the web traffic that is self-similar, whereas CBR sources are to approximate other long-lived streaming traffic. To simulate the
Conclusion and future work
This paper presents an infrastructural solution to address aggregated media streaming from a decentralized collection of unreliable subscriber resources, under the scenario where the media content is collectively stored at the subscriber ends. Unlike other P2P resource sharing solutions, each subscriber is responsible for only a small portion of the content rather than a complete replication of it. Our simulations demonstrate the effectiveness of the peer selection algorithm that employs a
References (39)
- et al.
Detecting shared congestion of flows via end-to-end measurement
IEEE/ACM Transactions on Networking
(2002) - O. Younis, S. Fahmy, On efficient on-line grouping of flows with shared bottlenecks at loaded servers, Technical Report...
- M. Handley, S. Floyd, J. Padhye, J. Widmer, TCP Friendly Rate Control (TFRC) Protocol Specification – RFC 3448, January...
- Onion Networks Inc., Java FEC Library v1.0.3. Available from:...
- MPEG-4 Industry Forum FAQ. Available from:...
- Dixon, Streaming Media: Trends and Formats, Manifest Technology,...
- Bouras, Kapoulas, Konidaris, Sevasti, A dynamic distributed video on demand service, in: 20th IEEE International...
- Akamai Technologies Inc. Available from:...
- V.N. Padmanabhan, L. Qiu, H.J. Wang, Server-based inference of Internet link lossiness, in: Infocom 2003, IEEE,...
- R. Teixeira, K. Marzullo, S. Savage, G.M. Voelker, In Search of Path Diversity in ISP Networks, IMC 03, ACM, October...
Concast: design and implementation of an active network service
IEEE Journal on Selected Area in Communications
Internet tomography
IEEE Signal Processing Magazine
Cited by (8)
A vEB-tree-based architecture for interactive video on demand services in peer-to-peer networks
2010, Journal of Network and Computer ApplicationsCitation Excerpt :However, it needs to probe the real world network all the time, which is rarely available in practice. Wu et al. (2009) propose a system in which peers receive multiple forward error coding (FEC) packets from different parents to protect delivery content. P2VoD (Do et al., 2004) is used to handle the departure of peers without passing through the source by finding new parents in the upper layer.
Study and performance of a group-based Content Delivery Network
2009, Journal of Network and Computer ApplicationsCitation Excerpt :There many works that study the interconnection system between surrogates. Some of them are based on P2P systems (Jernberg et al., 2006; Yim and Buyya, 2006). There are also GRID-based proposals such as the one presented in reference (Di Stefano and Santoro, 2008).
A survey on dynamic data delivery through content delivery networks
2018, International Journal of Engineering and Technology(UAE)Energy-Reduction Offloading Technique for Streaming Media Servers
2016, Mobile Information SystemsEfficient push-based packet scheduling for Peer-to-Peer live streaming
2013, Cluster ComputingImpact of different content placement and delivery strategies on content delivery capacity of the wireless mesh networks
2012, Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)