Abstract
We explore the use of compression methods to improve the middleware-based exchange of information in interactive or collaborative distributed applications. In such applications, good compression factors must be accompanied by compression speeds suitable for the data transfer rates sustainable across network links. Our approach combines methods that continuously monitor current network and processor resources and assess compression effectiveness, with techniques that automatically choose suitable compression techniques. By integrating these techniques into middleware, there is little need for end user involvement, other than expressing the target rates of data transmission. The resulting network- and user-aware compression methods are evaluated experimentally across a range of network links and application data, the former ranging from low end links to homes, to wide-area Internet links, to high end links in intranets, the latter including both scientific (binary molecular dynamics data) and commercial (XML) data sets. Results attained demonstrate substantial improvements of this adaptive technique for data compression over non-adaptive approaches, where better compression methods are used when CPU loads are low and/or network links are slow, and where less effective and typically, faster compression techniques are used in high end network infrastructures.
- DOE-TSI. Terascale supernova initiative, http://www.phy.ornl.gov/tsi.]]Google Scholar
- V. Oleson, K. Schwan, D. Amin, G. Eisenhauer, B. Plale, C. Pu, and P. Widener, Operational Information Systems, An Example from the Airline Industry, Workshop on Industrial Experiences with System Software (WIESS 2000), in conjunction with OSDI 2000, November 2000.]] Google ScholarDigital Library
- B. Plale, G. Eisenhauer, K. Schwan, J. Heiner, V. Martin, and J. Vetter. From interactive applications to distributed laboratories. IEEE Concurrency, 6(3), 1998.]] Google ScholarDigital Library
- M. Wolf, Z. Cai, W. Huang, and K. Schwan. Smart Pointers: Personalized Scientific Data Portals in Your Hand, In Proc. of Supercommputing 2002, Nov. 2002.]] Google ScholarDigital Library
- M. Beigl, MODBC - A Middleware for Accessing Databases from Mobile Computers. 3rd Cabernet Plenary Workshop, Rennes, France, 1997]]Google Scholar
- MMEV voice compression/decompression middleware, http://www.asahikasei.co.jp/vorero/en/onsei2/mmev1.html.]]Google Scholar
- INNOTECH Corporation, IMPress Version 2.0, Image compression and decompression middleware system for embedded systems. http://www.innotech.co.jp/english/news/contents/news021029e.html.]]Google Scholar
- Envivio Corporation, H.264 Live Solution for Cost-Effective Delivery of Broadcast-Quality Video Over Satellite Networks In Proc. of NAB 2003, Apr. 2003.]]Google Scholar
- E. Jeannot, B. Knutsson, M. Bjrkman, Adaptive Online Data Compression, IEEE High Performance Distributed Computing (HPDC'11), Edinburgh, Scotland, July, 2002.]] Google ScholarDigital Library
- M. Mathis. Web 100 and the End-to-End problem. http://www.web100.org/docs/jtech/.]]Google Scholar
- N. S. Rao, Y.-C. Bang, S. Radhakrishnan, Q. Wu, S. S. Iyengar, and H. Choo. NetLets: measurement-based.]]Google Scholar
- M. Jain and C. Dovrolis. End-to-end available Bandwidth: Measurement methodology, Dynamics, and Relation with TCP throughput. In Proceedings of ACM SIGCOMM, Aug. 2002.]] Google ScholarDigital Library
- M. Jain and C. Dovrolis. End-to-End Available Bandwidth: Measurement methodology, Dynamics, and Relation with TCP Throughput. IEEE/ACM Transactions in Networking, August, 2003.]] Google ScholarDigital Library
- Q. He and K. Schwan. IQ-RUDP: Coordinating Application Aaptation with Network Transport. In High Performance Distributed Computing, July 2002.]] Google ScholarDigital Library
- GriPhyN. The grid physics network. http://www.griphyn.org.]]Google Scholar
- Y. Chen, K. Schwan, and D. Zhou. Opportunistic channels: Mobility-aware event delivery. In Proceedings of the ACM/USENIX International Middleware Conference, 2003.]] Google ScholarDigital Library
- Huffman D. A method for the Construction of Minimum Redundancy Codes Proc. of the IRE 40, pp. 1098--1101 1952.]]Google Scholar
- Bookstein A., Klein S. T., Is Huffman coding dead?, Journal of Computing 50, pp. 279--296, 1993.]]Google ScholarCross Ref
- Wallace G. K.The JPEG Still Picture Compression Standard Communication of the ACM 34, pp. 3--44, 1991.]] Google ScholarDigital Library
- Information Technology Digital Compression and Coding of Continuous-Tone Still Images Requirements and Guidelines International Standard ISO/IEC 10918-1, 1993.]]Google Scholar
- Witten I. H., Neal R. M. and Cleary J. G. Arithmetic Coding for Data Compression, Communication of the ACM 30, pp. 520--540 1987.]] Google ScholarDigital Library
- Howard P. G. and Vitter J. S., Arithmetic Coding for Data Compression, Proceedings of the IEEE, 82(6), pp. 857--865, 1994.]]Google ScholarCross Ref
- Ziv J., Lempel A., A universal algorithm for sequential data compression, IEEE Transactions on Information Theory, IT-23, pp. 337--343, 1977.]]Google ScholarDigital Library
- Ziv J., Lempel A., Compression of individual sequences via variable-rate coding, IEEE Transactions Information Theory, IT-24, pp. 530--536, 1978.]]Google ScholarCross Ref
- WinZip, Nico Mak Computing, Inc., Mansfield, CT, USA 1998.]]Google Scholar
- gzip, Free Software Foundation, Inc., 675 Mass Ave, Cambridge, MA, USA 1991]]Google Scholar
- Brent. R. P., A linear algorithm for data compression. Australian Computer Journal, 19(2) pp. 64--68, May 1987.]]Google Scholar
- Burrows M. and Wheeler D. Block sorting Lossless Data Compression Algorithm, System research center, research report 124, Digital System research Center, Palo Alto, CA 1994.]]Google Scholar
- Nelson M. R. Data Compression with the Burrows Wheeler Transformation, Dr. Dobb's Journal, pp. 46--50 1996.]]Google Scholar
- SGI® IRIX® Freeware distribution, 1600 Amphitheatre Pkwy. Mountain View, CA, USA, Edition of February 2003.]]Google Scholar
- Klein S. T. and Wiseman Y. Parallel Huffman Decoding with Applications to JPEG Files, The Computer Journal, Swindon, UK, 2003.]]Google ScholarCross Ref
- Wiseman Y., Parallel Compression, Ph.D. Thesis, Bar-Ilan University, Ramat-Gan, Israel, 2000.]]Google Scholar
- Klein S. T. and Wiseman Y., Parallel Lempel Ziv Coding, The Journal of Discrete Applied Mathematics, 2003.]]Google Scholar
- Eisenhauer G. and Schwwan K., The ECho Event Delivery System, Technical Report GIT-CC-99-08, College of Computing, Georgia Institute of Technology, Atlanta, GA 30332-0280, 1999.]]Google Scholar
- Plale B., Eisenhauer G., Daley L. K., Widener P. and Schwan K., Fast Heterogenous Binary Data Interchange for Event-based Monitoring, Proceedings of the International Conference on Parallel and Distributed Computing Systems (PDCS2000), 2000.]]Google Scholar
- Cai Z., He Q., Eisenhauer G., Schwan K. and Wolf M., IQ-services: Network-Aware Middleware for Interactive Large-Data Application, submitted to Supercomputing (SC2003), May 2003.]]Google Scholar
Index Terms
- Efficient end to end data exchange using configurable compression
Recommendations
Efficient End to End Data Exchange Using Configurable Compression
ICDCS '04: Proceedings of the 24th International Conference on Distributed Computing Systems (ICDCS'04)We explore the use of compression methods to improve the middleware-based exchange of information in interactive or collaborative distributed applications. In such applications, good compression factors must be accompanied by compression speeds suitable ...
Lossless compression of VLSI layout image data
We present a novel lossless compression algorithm called Context Copy Combinatorial Code (C4), which integrates the advantages of two very disparate compression techniques: context-based modeling and Lempel-Ziv (LZ) style copying. While the algorithm ...
RETRACTED ARTICLE: Improved image compression using effective lossless compression technique
AbstractThe data or image transmission plays a very important role in current days. In general image can be transmitted in terms of data. The basic image can be converted or encoded into bits or chunks. This data can be transmitted in efficient form. ...
Comments