Skip to main content
Search
Search
    Quick Search anywhere
    Quick search in Citations
Advanced Search
Skip main navigationClose Drawer MenuOpen Drawer Menu
Previous article
Next article
Original Article

Assessing Individual Differences in Basic Computer Skills

Psychometric Characteristics of an Interactive Performance Measure

Published Online:January 01, 2013https://doi.org/10.1027/1015-5759/a000153

Abstract

A definition of basic computer skills (BCS) is proposed and the psychometric properties of a newly developed BCS scale are investigated. BCS is defined as the ability and speed of performing basic actions in graphical user interfaces of computers to access, collect, and provide information. BCS is thus considered a basic component skill of the much broader construct of ICT literacy. Data from the German PISA 2009 field trial was used to determine the factor structure of the BCS scale as well as convergent and discriminant validity. The latent factor structure underlying the BCS scale was investigated by testing confirmatory factor analysis (CFA) models for response times and responses. CFA results suggest that there is one dimension of BCS speed and BCS ability, respectively. With respect to convergent validity, practical computer knowledge and skill in digital reading had strong associations with BCS speed and ability. With respect to discriminant validity, only moderate associations were found with lower level reading skills and self-reported computer skills. Differences between BCS speed and ability and further developments of the BCS scale are discussed.

References

  • Anderson, J. R. (1982). Acquisition of cognitive skill. Psychological Review, 89, 369–406. First citation in articleCrossref, Google Scholar

  • Anderson, J. R. , Matessa, M. , Lebiere, C. (1997). ACT-R: A theory of higher level cognition and its relation to visual attention. Human-Computer Interaction, 12, 439–462. First citation in articleCrossref, Google Scholar

  • Ballantine, J. A. , Larres, P. M. , Oyelere, P. (2007). Computer usage and the validity of self-assessed computer competence among first-year business students. Computers and Education, 49, 976–990. First citation in articleCrossref, Google Scholar

  • Balota, D. A. , Cortese, M. J. , Sergent-Marschall, S. D. , Spieler, D. H. , Yap, M. (2004). Visual word recognition of single-syllable words. Journal of Experimental Psychology: General, 133, 283–316. First citation in articleCrossref, Google Scholar

  • Bollen, K. A. (1989). Structural equations with latent variables. New York, NY: Wiley. First citation in articleCrossref, Google Scholar

  • Brand-Gruwel, S. , Wopereis, I. , Walraven, A. (2009). A descriptive model of information problem solving while using internet. Computers and Education, 53, 1207–1217. First citation in articleCrossref, Google Scholar

  • Carroll, J. B. (1993). Human cognitive abilities. New York, NY: Cambridge University Press. First citation in articleCrossref, Google Scholar

  • Colley, A. , Comber, C. (2003). Age and gender differences in computer use and attitudes among secondary school students: What has changed? Educational Research, 45, 155–165. First citation in articleCrossref, Google Scholar

  • Educational Testing Service . (2008). iSkills – Information and Communication Technology Literacy Test. Retrieved from www.ets.org/iskills/about First citation in articleGoogle Scholar

  • Forero, C. G. , & Maydeu-Olivares, A. (2009). Estimation of IRT graded models for rating data: Limited vs. full information methods. Psychological Methods, 14, 275–299. First citation in articleCrossref, Google Scholar

  • Goldhammer, F. , & Klein Entink, R. H. (2011). Speed of reasoning and its relation to reasoning ability. Intelligence, 39, 108–119. First citation in articleCrossref, Google Scholar

  • Hakkarainen, K. , Ilomäki, L. , Lipponen, L. , Muukkonen, H. , Rahikainen, M. , Tuominen, T. , ... Lehtinen, E. (2000). Students skills and practices of using ICT: Results of a national assessment in Finland. Computers and Education, 34, 103–117. First citation in articleCrossref, Google Scholar

  • Ilomäki, L. , & Rantanen, P. (2007). Intensive use of ICT in school: Developing differences in students’ ICT expertise. Computers and Education, 46, 119–136. First citation in articleCrossref, Google Scholar

  • Imhof, M. , Vollmeyer, R. , Beierlein, C. (2007). Computer use and the gender gap: The issue of access, use, motivation, and performance. Computers in Human Behavior, 23, 2823–2837. First citation in articleCrossref, Google Scholar

  • International ICT Literacy Panel . (2002). Digital transformation: A framework for ICT literacy (A report of the International ICT Literacy Panel). Princeton, NJ: Educational Testing Service. Retrieved from www.ets.org/Media/Research/pdf/ictreport.pdf First citation in articleGoogle Scholar

  • Katz, I. R. , Macklin, A. S. (2007). Information and communication technology (ICT) literacy: Integration and assessment in higher education. Journal of Systemics, Cybernetics and Informatics, 5, 50–55. First citation in articleGoogle Scholar

  • Keith, N. , Frese, M. (2005). Self-regulation in error management training: Emotion control and metacognition as mediators of performance effects. Journal of Applied Psychology, 90, 677–691. First citation in articleCrossref, Google Scholar

  • Keith, N. , Richter, T. , Naumann, J. (2010). Active/exploratory training promotes transfer even in learners with low motivation and cognitive ability. Applied Psychology: An International Review, 59, 97–123. First citation in articleCrossref, Google Scholar

  • Kim, J. H. , Jung, S. Y. , Lee, W. G. (2008). Design of contents for ICT literacy in-service training of teachers in Korea. Computers and Education, 51, 1683–1706. First citation in articleCrossref, Google Scholar

  • Kintsch, W. (1998). Comprehension: A paradigm for cognition. New York, NY: Cambridge University Press. First citation in articleGoogle Scholar

  • Klieme, E. , Hartig, J. , & Rauch, D. (2008). The concept of competence in educational contexts. In J. Hartig, E. Klieme, D. Leutner, (Eds.), Assessment of competencies in educational contexts (pp. 3–22). Göttingen: Hogrefe. First citation in articleGoogle Scholar

  • Kuhlemeier, H. , & Hemker, B. (2007). The impact of computer use at home on students’ internet skills. Computers and Education, 49, 460–480. First citation in articleCrossref, Google Scholar

  • Lennon, M. , Kirsch, I. , Von Davier, M. , Wagner, M. , Yamamoto, K. (2003). Feasibility study for the ICT Literacy Assessment: Report to network A. Retrieved from www.oecd.org/dataoecd/35/13/33699866.pdf First citation in articleGoogle Scholar

  • Leu, D. J. , Kinzer, C. K. , Coiro, J. L. , Cammack, D. W. (2004). Toward a theory of new literacies emerging from the internet and other information and communication technologies. In N. J. Unrau, R. B. Ruddell, (Eds.), Theoretical models and processes of reading (5th ed., pp. 1570–1613). Newark, NJ: International Reading Association. First citation in articleGoogle Scholar

  • Mabe, P. A. , West, S. W. (1982). Validity of self-evaluation of ability: Review and meta-analysis. Journal of Applied Psychology, 67, 280–296. First citation in articleCrossref, Google Scholar

  • Markauskaite, L. (2007). Exploring the structure of trainee teachers’ ICT literacy: The main components of, and relationships between, general cognitive and technical capabilities. Educational Technology Research and Development, 55, 547–572. First citation in articleCrossref, Google Scholar

  • Mayer, R. E. (2003). What causes individual differences in cognitive performance? In R. J. Sternberg, E. L. Grigorenko, (Eds.), The psychology of abilities, competencies, and expertise (pp. 263–74). New York, NY: Cambridge University Press. First citation in articleCrossref, Google Scholar

  • Metzger, M. J. , Flanagin, A. J. , Zwarun, L. (2003). College student web use, perceptions of information credibility, and verification behavior. Computers and Education, 41, 271–290. First citation in articleCrossref, Google Scholar

  • Muthén, B. O. (1998–2004). Mplus technical appendices. Los Angeles, CA: Muthén & Muthén. First citation in articleGoogle Scholar

  • Muthén, L. K. , & Muthén, B. (1998–2010). Mplus user’s guide. Version 6. Los Angeles, CA: Muthén & Muthén. First citation in articleGoogle Scholar

  • Naumann, J. , Richter, T. , Groeben, N. (2001). Validierung des Inventars zur Computerbildung (INCOBI) anhand eines Vergleichs von Anwendungsexperten und Anwendungsnovizen [Validation of the Computer Literacy Inventory (INCOBI) by a comparison of expert and novice computer users]. Zeitschrift für Pädagogische Psychologie/German Journal of Educational Psychology, 15, 219–232. First citation in articleLink, Google Scholar

  • OECD . (2005). Bildungspolitische Analyse 2004 [Education policy analysis 2004]. Paris: Author. First citation in articleGoogle Scholar

  • OECD . (2009). PISA 2009 assessment framework. Paris: Author. First citation in articleGoogle Scholar

  • OECD . (2011). PISA 2009 results Vol.VI. Students on line: Reading and using digital information. Paris: Author. First citation in articleCrossref, Google Scholar

  • Parshall, C. G. , Spray, J. A. , Kalohn, J. C. , Davey, T. (2002). Practical considerations in computer-based testing. New York, NY: Springer. First citation in articleCrossref, Google Scholar

  • Potosky, D. (2007). The Internet Knowledge (iKnow) measure. Computers in Human Behavior, 23, 2670–2777. First citation in articleCrossref, Google Scholar

  • Poynton, T. A. (2005). Computer literacy across the lifespan: A review with implications for educators. Computers in Human Behavior, 21, 861–872. First citation in articleCrossref, Google Scholar

  • R Development Core Team . (2009). R: A language and environment for statistical computing. Vienna, Austria: R Foundation for Statistical Computing. Available at www.R-project.org First citation in articleGoogle Scholar

  • Richter, T. , Naumann, J. , Groeben, N. (2001). Das Inventar zur Computerbildung (INCOBI): Ein Instrument zur Erfassung von Computer Literacy und computerbezogenen Einstellungen bei Studierenden der Geistes- und Sozialwissenschaften [The Computer Literacy Inventory: An instrument for the assessment of computer literacy and computer related attitudes in students of the humanities and social sciences]. Psychologie in Erziehung und Unterricht, 48, 1–13. First citation in articleGoogle Scholar

  • Richter, T. , Naumann, J. , Horz, H. (2010). Eine revidierte Fassung des Inventars zur Computerbildung (INCOBI-R) [A revised version of the Computer Literacy Inventory]. Zeitschrift für Pädagogische Psychologie/German Journal of Educational Psychology, 24, 23–37. First citation in articleLink, Google Scholar

  • Russell, M. , Goldberg, A. , O’Connor, K. (2003). Computer-based testing and validity: A look back into the future. Assessment in Education: Principles, Policy and Practice , 10, 279–293. First citation in articleCrossref, Google Scholar

  • Sandene, B. , Bennett, R. E. , Braswell, J. , Oranje, A. (2005). Online assessment in mathematics. In B. Sandene, N. Horkay, R. E. Bennett, N. Allen, J. Braswell, B. Kaplan, A. Oranje, (Eds.), Online assessment in mathematics and writing: Reports from the NAEP Technology-based Assessment Project (NCES 2005–457) (pp. 1–68). Washington, DC: US Department of Education, National Center for Education Statistics. First citation in articleGoogle Scholar

  • Schermelleh-Engel, K. , Moosbrugger, H. , Müller, H. (2003). Evaluating the fit of structural equation models: Test of significance and descriptive goodness-of-fit measures. Methods of Psychological Research – Online, 8, 23–74. Available at www.mpr-online.de/ First citation in articleGoogle Scholar

  • Schumacher, P. , & Morahan-Martin, J. (2001). Gender, internet and computer attitudes and experiences. Computers in Human Behavior, 17, 95–110. First citation in articleCrossref, Google Scholar

  • Schweizer, K. (2010). Some guidelines concerning the modeling of traits and abilities in test construction. European Journal of Psychological Assessment, 26, 1–2. First citation in articleLink, Google Scholar

  • Tsai, M.-J. (2002). Do male and female students often perform better than female students when learning computers? A study of Taiwanese eight graders’ computer education through strategic and cooperative learning. Journal of Educational and Computing Research, 26(1), 67–85. First citation in articleCrossref, Google Scholar

  • van der Linden, W. J. (2009). Conceptual issues in response-time modeling. Journal of Educational Measurement, 46, 247–272. First citation in articleCrossref, Google Scholar

  • Vekiri, I. , Chronaki, A. (2008). Gender issues in technology use: Perceived social support, computer self-efficacy and value beliefs, and computer use beyond school. Computers and Education, 51, 1392–1404. First citation in articleCrossref, Google Scholar

  • Wecker, C. , Kohnle, C. , & Fischer, F. (2007). Computer literacy and inquiry learning: When geeks learn less. Journal of Computer-Assisted learning, 23, 133–144. First citation in articleCrossref, Google Scholar

  • Weinert, S. , Artelt, C. , Prenzel, M. , Senkbeil, M. , Ehmke, T. , Carstensen, C. H. (2011). Development of competencies across the life span. Zeitschrift für Erziehungswissenschaft, 14, 67–86. First citation in articleCrossref, Google Scholar

  • Wu, M. L. , Adams, R. J. , Wilson, M. R. , Haldane, S. (2007). ACER ConQuest 2.0: General item response modeling software. Camberwell, Australia: Australian Council for Educational Research. First citation in articleGoogle Scholar