Anna Watson
ABSTRACT
This paper presents an experiment comparing the relative impact of different types of degradation on subjective quality ratings of interactive speech transmitted over packet-switched networks. The experiment was inspired by observations made during a large-scale, long-term field trial of multicast conferencing. We observed that user reports of unsatisfactory speech quality were rarely due to network effects such as packet loss and jitter. A subsequent analysis of conference recordings found that in most cases, the impairment was caused by end-system hardware, equipment setup or user behavior. The results from the experiment confirm that the effects of volume differences, echo and bad microphones are rated worse than the level of packet loss most users are likely to experience on the Internet today, provided that a simple repair mechanism is used. Consequently, anyone designing or deploying network speech applications and services ought to consider the addition of diagnostics and tutorials to ensure acceptable speech quality.
- 1.Bouch, A., Watson, A. and Sasse, M.A. QUASS: A tool for measuring the subjective quality of real-time multimedia audio and video. Poster presented at HCI '98 (Sheffield, England, September 1998).Google Scholar
- 2.Gruber, J.G. and Strawczynski, L. Subjective effects of variable delay and speech clipping in dynamically managed voice systems. IEEE Transactions on Cummunications, 1985, 33(8), 801-808.Google ScholarCross Ref
- 3.Hardman, V., Sasse, M.A, Handley, M.J. and Watson, A. Reliable audio for use over the Internet. Proceedings of 1NET '95 (Honolulu, Hawaii, June 1995), 171-178.Google Scholar
- 4.1TU-T P.800 Methods for subjective determination of transmission quality. Available from http://www.itu.int/publications/itu-t/iturec.htmGoogle Scholar
- 5.Jacobson, V. vat manual pages, Lawrence Berkeley Laboratory, USA. Software available from http://wwwnrg.ee.lbl.gov/vat/Google Scholar
- 6.Jayant, N.S. High-quality coding of telephone speech and wideband audio. IEEE Communications Magazine, Jan. 1990, 10-20.Google ScholarDigital Library
- 7.Jones, B.L. and McManus, P.R. Graphic scaling of qualitative terms. SMPTE Journal, November 1986, 1166- 1171.Google ScholarCross Ref
- 8.Kawalek, J. A user perspective for QoS management. Proceedings of 3 rd International Conference on Intelligence in Broadband Services and Network (IS & N '95, Crete, Greece).Google Scholar
- 9.Kitawaki, N. and Itoh, K. Pure delay effects on speech quality in telecommunications. IEEE Journal on Selected Areas in Telecommunication, 1991, 9(4), 586-593.Google ScholarDigital Library
- 10.Knoche, H., De Meer, H.G. and Kirsh, D. Utility curves: Mean opinion scores considered biased. Proceedings of IWQoS '99 (London, England, May 1999), 12-14.Google ScholarCross Ref
- 11.Preminger, J.E. and Van Tasell, D.J. Quantifying the relationship between speech quality and speech intelligibility. Journal of Speech and Hearing Research, 1995, 38, 714-725.Google ScholarCross Ref
- 12.PIPVIC-2 Project web site at http:// wwwmice.cs.ucl.ac.uk/multimedia/projects/pipvic2/Google Scholar
- 13.RAT (Robust Audio Tool). Available for download from http://www-mice.cs.ucl.ac.uk/multimedia/softwareGoogle Scholar
- 14.Recommended practices for enhancing digital audio compatibility in multimedia systems (version 3.00). Technical Report, Interactive Multimedia Association, Annapolis, MD, 1992.Google Scholar
- 15.Reeves, B. and Nass, C. The Media Equation. Cambridge University Press/CSLI Publications, 1996.Google Scholar
- 16.Sasse, M.A., Bilting, U., Schulz, C-D. and Turletti, T. Remote seminars through multimedia conferencings: Experiences from the MICE project. Proceedings of INET'94/JENCS. Google ScholarDigital Library
- 17.Teunissen, K. The validity of CCIR quality indicators along a graphical scale. SMPTE Journal, March 1996, 144-149.Google ScholarCross Ref
- 18.Watson, A. and Sasse, M.A. Measuring perceived quality of speech and video in multimedia conferencing applications. Proceedings of ACM Multimedia '98 (Bristol, England, September 1998), ACM Press, 55-60. Google ScholarDigital Library
- 19.Watson, A. and Sasse, M.A. Multimedia conferencing via multicast: Determining the quality of service required by the end user. Proceedings of AVSPN '97 (Aberdeen, Scotland, September 1997), 189-194.Google Scholar
- 20.Watson, A. and Sasse, M.A. Distance education via IP videoconferencing: Results from a national pilot project. Poster to be presented at CHI 2000 (The Hague, The Netherlands, April 2000). Google ScholarDigital Library
- 21.Wilson, G. & Sasse, M.A. Do users always know what's good for them? Utilising physiological responses to assess media quality. To be presented at HCI 2000, September 5th - 8th, Sunderland, UICGoogle Scholar
- 22.Wilson, G. & Sasse, M.A. Investigating the impact of audio degradations on users: Subjective vs. objective measurement methods. Submitted to OZCHI 2000. Available as UCL Computer Science research note RN/00/36.Google Scholar
- 23.Virtanen, M.T, Gleiss, N. and Goldstein, M. One the use of evaluative category scales in telecommunications. Proceedings of Human Factors in Telecommunications, 1995, 253-260.Google Scholar
- 24.Zhang, L., Deering, S., Estrin, D., Shenker, S. and Zappala, D. RSVP: A new resource ReSerVation Protocol, IEEE Network Magazine, 1995, 7(5), 8-18. Google ScholarDigital Library
Index Terms
- The good, the bad, and the muffled: the impact of different degradations on Internet speech
Recommendations
On the perception of "segmental intonation": F0 context effects on sibilant identification in German
In normal modally voiced utterances, voiceless fricatives like [s], [ź], [f], and [x] vary such that their aperiodic pitch impressions mirror the pitch level of the adjacent F0 contour. For instance, if the F0 contour creates a high or low pitch context,...
Video quality subjective assessment considering cognitive criteria and user preferences on video content
WebMedia '12: Proceedings of the 18th Brazilian symposium on Multimedia and the webThere are several subjective test methodologies for assessing video quality, being that each of these methods was developed by considering specific degradation factors in video quality. Therefore, for video streaming services that use TCP as transport ...
Subjective Multimedia Quality Assessment
The Video Quality Experts Group (VQEG) is preparing a programme of subjective multimedia quality tests. The results from these tests will be used to evaluate the performance of competing objective multimedia quality metrics. The reliability of the ...
Comments