Skip to main content
SpringerLink
Log in
Book cover

Pacific Rim Objective Measurement Symposium (PROMS) 2014 Conference Proceedings pp 65–85Cite as

A Rasch Measure of Test Takers’ Attitude Toward the Versant English Test

  • Conference paper
  • First Online:

  • 666 Accesses

Abstract

Despite the steadily increasing application of Rasch modeling in human sciences, few attempts have been made to use the Rasch measurement model to analyze Likert-type scale survey data in the field of language testing. This study investigated test takers’ attitude toward the Versant English Test (VET), a fully automated spoken English test using the Rating Scale Model (RSM) in Rasch measurement. Based on previous research, attitude in this study was conceptualized as a psychological construct consisting of three components: beliefs, opinions, and emotions. A 21-item questionnaire was designed to collect the data from 125 VET test takers. The collected data were then subjected to Rasch analysis including item statistics, reliability and separation indexes, category structure, Rasch factor analysis, and differential item functioning. The results indicated that the questionnaire was essentially unidimensional, tapping into a single attitudinal construct. The principle of measurement invariance held across the relevant subsamples. The findings of this study have implications for the VET provider in light of the further improvement and promotion of the test. Furthermore, this study also has methodological implications for researchers in the field of language testing.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD   109.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

References

  • Alderson, J. C., Clapham, C., & Wall, D. (1995). Language test construction and evaluation. Cambridge: Cambridge University Press.

    Google Scholar 

  • Allport, G. W. (1971). Attitudes. In K. Thomas (Ed.), Attitudes and behavior. Harmondsworth, UK: Penguin.

    Google Scholar 

  • Andrich, D. (1978). Rating formulation for ordered response categories. Psychometrika, 43(4), 561–573.

    Article  Google Scholar 

  • Bachman, L. F. (1990). Fundamental considerations in language testing. Oxford: Oxford University Press.

    Google Scholar 

  • Bachman, L. F. (2000). Modern language testing at the turn of the century: Assuring that what we count counts. Language Testing, 17(1), 1–42.

    Google Scholar 

  • Baker, C. (1988). Key issues in bilingualism and bilingual education. Clevedon, The UK: Multilingual Matters.

    Google Scholar 

  • Baker, C. (1992). Attitudes and language. Clevedon, The UK: Multilingual Matters.

    Google Scholar 

  • Bernstein, J., Van-Moere, A., & Cheng, J. (2010). Validating automated speaking tests. Language Testing, 27(3), 355–377.

    Article  Google Scholar 

  • Bond, T. G., & Fox, C. M. (2007). Applying the Rasch model: Fundamental measurement in the human sciences (2nd ed.). New Jersey, NJ: Lawrence Erlbaum Associates.

    Google Scholar 

  • Boone, W. J., Staver, J. R., & Yale, M. S. (2014). Rasch analysis in the human sciences. New York and London: Springer, Dordrecht Heidelberg.

    Book  Google Scholar 

  • Brown, A. (1993). The role of test-taker feedback in the test development process: test-takers’ reactions to a tape-mediated test of proficiency in spoken Japanese. Language Testing, 10(3), 277–301. doi:10.1177/026553229301000305.

    Article  Google Scholar 

  • Brown, D. H. (2000). Principles of language learning and teaching (4th ed.). New York: Longman.

    Google Scholar 

  • Cavanagh, R. F., & Waugh, R. F. (Eds.). (2011). Applications of rasch measurement in learning environments research. Rotterdam; Boston: Sense Publishers.

    Google Scholar 

  • Chun, C. W. (2006). An analysis of a language test for employment: The authenticity of the PhonePass test. Language Assessment Quarterly, 3(3), 295–306.

    Article  Google Scholar 

  • Council of Europe. (2001). Common European framework of reference for languages: Learning, teaching, assessment.

    Google Scholar 

  • Eagly, A. H., & Chaiken, S. (1993). The psychology of attitudes. New York: Handcourt Brace Jovanovich.

    Google Scholar 

  • Elder, C., Iwashita, N., & McNamara, T. (2002). Estimating the difficulty of oral proficiency tasks: What does the test taker have to offer? Language Testing, 19(4), 347–368.

    Article  Google Scholar 

  • Embreston, S. E., & Reise, S. P. (2000). Item response theory for psychologists. New Jersey, NJ: Lawrence Erlbaum.

    Google Scholar 

  • Fan, J. (2014). Chinese test takers’ attitudes towards the Versant English Test: A mixed-methods approach. Language Testing in Asia, 4(6), 1–17.

    Google Scholar 

  • Fan, J., & Ji, P. (2014). Test candidates’ attitudes and their test performance: The case of the Fudan English Test. University of Sydney Papers in TESOL, 9, 1–35.

    Google Scholar 

  • Farhady, H. (2008). Human operated, machine mediated, and automated tests of spoken English. Paper presented at the American Association of Applied Linguistics (AAAL) Conference, Washington, D. C.

    Google Scholar 

  • Gardner, R. (1985). Social psychology and second language learning: The role of attitude and motivation. London: Edward Arnold.

    Google Scholar 

  • Han, B., Dan, M., & Yang, L. (2004). Problems with college English test as emerged from a survey. Foreign Languages and Their Teaching, 179(2), 17–23.

    Google Scholar 

  • Ingram, E. (1977). Basic concepts in testing. In J. P. B. Allen & A. Davies (Eds.), Edinburgh course of applied linguistics (Vol. 4). Oxford: Oxford University Press.

    Google Scholar 

  • Jin, Y., & Cheng, L. (2013). The effects of psychological factors on the validity of high-stakes tests. Modern Foreign Languages, 36(1), 62–69.

    Google Scholar 

  • Karelitz, T. M. (2014). Using public opinion to inform the validation of test scores (Research report). Retrieved Novermber 17, 2014, from www.nite.org.il/files/reports/e387.pdf.

  • Ladegaard, H. J. (2000). Language attitudes and sociolinguistic behavior: Exploring attitude-behavior relations in language. Journal of Sociolinguistics, 4(2), 214–233.

    Article  Google Scholar 

  • Linacre, J. M. (2004). Optimal rating scale category effectiveness. In E. V. Smith, Jr. & R. M. Smith (Eds.), Introduction to Rasch measurement (pp. 258–278). Maple Grove, MN: JAM Press.

    Google Scholar 

  • Linacre, M. (2012). Winsteps Tutorial. Retrieved November 7, 2014, from http://www.winsteps.com/tutorials.htm.

  • Messick, S. (1989). Validity. In R. L. Linn (Ed.), Educational measurement (3rd ed., pp. 13–103). McMillan: American Council on Education.

    Google Scholar 

  • McNamara, T., & Knoch, U. (2012). The Rasch wars: The emergence of Rasch measurement in language testing. Language Testing, 29(4), 553–574.

    Article  Google Scholar 

  • Murray, J. C., Riazi, A. M., & Cross, J. L. (2012). Test candidates’ attitudes and their relationship to demographic and experiential variables: The case of overseas trained teachers in NSW Australia. Language Testing, 29(5), 577–595.

    Article  Google Scholar 

  • Oon, P. T., & Subramaniam, R. (2011). Rasch modelling of a scale that explores the take-up of physics among school students from the perspective of teachers. In R. F. Cavanaugh & R. F. Waugh (Eds.), Applications of Rasch measurement in learning environments research (pp. 119–139). Netherlands: Sense Publishers.

    Chapter  Google Scholar 

  • Pearson. (2008). Versant English test: Test description and validation summary. Retrieved November 7, 2014, from www.versanttest.co.uk/pdf/ValidationReport.pdf.

  • Present-Thomas, R., & Van-Moere, A. (2009). NRS classification consistency of two spoken English tests. Paper presented at the East Coast Organization of Language Testers (ECOLT) Conference, Washington, D. C.

    Google Scholar 

  • Rasch, G. (1960). Probalistic models for some intelligence and attainment tests. Copenhagen, Denmark: Paedagogiske Institut.

    Google Scholar 

  • Rasch, G. (1980). Probalistic models for some intelligence and attainment tests. Chicago: The University of Chicago Press.

    Google Scholar 

  • Rasti, I. (2009). Iranian candidates’ attitudes towards the IELTS. Asian EFL Journal, 11(3), 110–155.

    Google Scholar 

  • Reid, N. (2006). Thoughts on attitude measurement. Research in Science and Technological Education, 24(1), 3–27.

    Article  Google Scholar 

  • Shohamy, E. (2001). The power of tests: A critical perspective of the uses of language tests. London: Pearson Education.

    Google Scholar 

  • Stevenson, D. K. (1985). Authenticity, validity, and a tea party. Language Testing, 2(1), 41–47.

    Article  Google Scholar 

  • Wu, J. (2008). Views of Taiwanese students and teachers on English language testing. University of Cambridge ESOL Examinations Research Note, 34(2), 6–9.

    Google Scholar 

  • Zhao, J., & Cheng, L. (2010). Exploring the relationship between Chinese University students’ attitude towards the College English Test and their test performance. In L. Cheng & A. Curtis (Eds.), English language assessment and the Chinese leaner (pp. 190–218). New York and London: Routledge, Taylor & Francis Group.

    Google Scholar 

Download references

Acknowledgments

The preparation of this manuscript was supported by the National Social Sciences Fund of the People’s Republic of China under the project title “the Development and Validation of Standards in Language Testing” (Grant Nunber: 13CYY032) and a joint research project between the Language Testing Center, Fudan University and Pearson Knowledge Technologies. This study used the same data as reported in Fan (2014). We would like to thank Prof. Peiying Ji, Prof. Jiangbo Wan, and Ms. Diana Hon for their assistance in collecting the data for this study. Also, we would like to thank all the students who participated in this study.

Author information

Authors and Affiliations

  1. College of Foreign Languages and Literature, Fudan University, Shanghai, China

    Jinsong Fan

  2. College of Arts, Society and Education, James Cook University, Townsville, Australia

    Trevor Bond

Authors
  1. Jinsong Fan
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Trevor Bond
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jinsong Fan .

Editor information

Editors and Affiliations

  1. College of Foreign Studies, Jiaxing University, Jiaxing, Zhejiang, China

    Quan Zhang

  2. College of Foreign Studies, Jiaxing University, Jiaxing, Zhejiang, China

    Hong Yang

Rights and permissions

Reprints and permissions

Copyright information

© 2015 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Fan, J., Bond, T. (2015). A Rasch Measure of Test Takers’ Attitude Toward the Versant English Test. In: Zhang, Q., Yang, H. (eds) Pacific Rim Objective Measurement Symposium (PROMS) 2014 Conference Proceedings. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-662-47490-7_6

Download citation

Publish with us

Policies and ethics

Search

Navigation