Skip to main content

Advertisement

SpringerLink
Log in

Authentic assessment in seafarer education: using literature review to investigate its validity and reliability through rubrics

  • IAMU Section Article
  • Published:
WMU Journal of Maritime Affairs Aims and scope Submit manuscript

Abstract

With the Standards of Training, Certification and Watchkeeping Convention 1995 (STCW’95) moving seafarer training towards outcome-based education (OBE), emphasis has shifted to assessment practices that will allow seafarer students to demonstrate their ability to perform workplace tasks at standards described in the STCW Code. This paper argues that authentic assessment comprising of performance-based tasks applied in real-world and meaningful contexts can provide a holistic approach to competence assessment for seafarers. But, authentic assessment can capture essential aspects of workplace tasks and result in consistency of student performance in different contexts only if they are valid and reliable. Rubrics as assessment tools are known to increase validity and reliability of assessments; however, it can do so only if different aspects of its own validity and reliability have been addressed. A literature review undertaken for this paper has uncovered an absence of academic investigation and empirical study on the different aspects of validity and reliability of authentic assessment through assessment rubrics. Moreover, there exists an even greater absence of global research on authentic assessment in the area of seafarer training. Through an investigation of authentic assessment, this research has uncovered the importance of using valid and reliable rubrics in order to improve not only the assessment process but also the tools and methods used to support the valid, reliable, and authentic assessment of outcomes achieved in the learning process. Future research aims to offer insights into improving the validity and reliability of rubrics and to empirically investigate how they can be used in authentic assessment within the confines of the STCW Code, in particular, to improve seafarer training practices, student engagement, resulting learning outcomes, and employer and regulator satisfaction with the attainment of the standards stipulated in the STCW Code to produce an evidence of competence.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Andrade H, Du Y (2005) Student perspectives on rubric-referenced assessment. Practical Assessment, Res & Eval 10:1–11

    Google Scholar 

  • Archbald DA (1991) Authentic assessment: principles, practices, and issues. Sch Psychol Q 6:279–293

    Article  Google Scholar 

  • Archbald DA, Newmann FM (1988) Beyond standardized testing: assessing authentic academic achievement in the secondary school. Reston, National Association of Secondary Principals

    Google Scholar 

  • Azim, S. & Khan, M. 2012. Authentic assessment: an instructional tool to enhance students learning. Academic Research International, 2

  • Bell, A. & Bell, M. 2003. Developing authentic assessment methods from a multiple intelligences perspective. Master of Arts Action Research Project, Saint Xavier University and SkyLight Professional Development Field-based Master’s Program.

  • Biggs, J. & Tang, C. 2007. Teaching for quality learning at university, Maidenhead Open University, McGraw Hill

  • Biggs J, Tang C (2011) Teaching for quality learning at university: what the student does. McGraw-Hill, England

    Google Scholar 

  • Blackburn, J. J. & Kelsey, K. D. 2013. Understanding authentic assessment in a secondary agricultural mechanics laboratory: an instrumental case study. Journal of Human Sciences and Extension, 1

  • Boarerpitchford, J. K. 2010. An examination of the assessment practices of community college instructors. PhD, Caella University

  • Brady D (1997) Assessment and the curriculum. In: Cullingford C (ed) Assessment versus evaluation. Cassell, Great Britain

    Google Scholar 

  • Brawley, N. 2009. Authentic assessment vs. traditional assessment: a comparative study. Bachelor of Science Honors, Coastal Carolina University

  • Brindley G (1998) Outcomes-based assessment and reporting in language learning programmes: a review of the issues. Lang Test 15:45–85

    Google Scholar 

  • Butler KG (1999) Dynamic and authentic assessment of spoken and written language disorders. In: Pinto MGLC, Veloso J, Maia B (eds) Plenary lecture 5th international society of applied psycholinguistics, Portugal, 1999, pp 47–57

  • Burke K (2011) From standards to rubrics in six steps: tools for assessing student learning. California, Corwin

    Google Scholar 

  • Burton LJ, Mazerollw SM (2011) Survey instrument validity part I: principles of survey instrument development and validation in athletic training education research. Athl Train Educ J 6:27–35

    Google Scholar 

  • Cassidy, K. E. 2009. Using authentic intellectual assessment to determine level of instructional quality of teacher practice of new elementary school teachers based on teacher preparation route. Doctor of Education PhD, The George Washington University

  • Chance BL (1997) Experiences with authentic assessment techniques in an introductory statistics course. J of Stat Educ 5

  • Cooper BS, Gargan A (2009) Rubrics in education: old term, new meanings. Phi Delta Kappan 91:54–55

    Article  Google Scholar 

  • Cooperstein SE, Kocevar-Weidinger E (2004) Beyond active learning: a constructivist approach to learning. Reference Services Review 32:141–148

    Article  Google Scholar 

  • Cumming JJ, Maxwell GS (1999) Contextualising authentic assessment. Assessment in Education: Principles, Policies and Practices 6:177–194

    Article  Google Scholar 

  • Darling-Hammond L, Snyder J (2000) Authentic assessment of teaching in context. Teach Teach Educ 16:523–545

  • Davis MH, Amin Z, Grande JP, O’Neill AE, Pawlina W, Viggiano TR, Zuberi R (2007) Case studies in outcome-based education. Med Teach 29:717–722

    Article  Google Scholar 

  • Dennison RD, Rosselli J, Dempsey A (2015) Evaluation beyond exams in nursing education: designing assignments and evaluating with rubrics. Springer Publishing Company, New York

    Google Scholar 

  • Dikli S (2003) Assessment at a distance: traditional vs alternative assessments. The Turkish Online J of Educ Technol 2:13–19

    Google Scholar 

  • Diller KR, Phelps SF (2008) Learning outcomes, portfolios, and rubrics, oh my! Authentic Assessment of an Information Literacy Program portal: Libraries and the Academy 8:75–89

    Google Scholar 

  • Driscoll A, Wood S (2007) Developing outcomes-based assessment for learner centered education: a faculty introduction. Stylus Publishing, Virginia

    Google Scholar 

  • Drost EA (2011) Validity and reliability in social science research. Educ Res and Perspect 38:105–123

    Google Scholar 

  • Emery, D. E. 2001. Authentic Assessment in High School Science A Classroom Perspective. In: Shepardson, D. P. (ed.) Assessment in science: a guide to professional development and classroom practice. Indiana: Springer-Science+Business Media Dordrecht

  • Fatonah S, Suyata P, Prasetyo ZK (2013) Developing an authentic assessment model in elementary school science teaching. J of Educ and Practice 4:50–61

    Google Scholar 

  • Finch AE (2002) Authentic assessment: implications for EFL performance testing in Korea. Secondary Education research 49:89–122

    Google Scholar 

  • Fook CY, Sidhu GK (2010) Authentic assessment and pedagogical strategies in higher education. J of Socl Sciences 6:153–161

    Article  Google Scholar 

  • Gulikers, J. T. M. 2006. Authenticity is in the eye of the beholder: beliefs and perceptions of authentic assessment and the influence on student learning. PhD, OpenUniversiteitNederland

  • Gulikers JTM, Bastiaens TJ, Kirschner PA (2004a) A five-dimensional framework for authentic assessment. ETR&D-Educational Technology Research and Development 52:67–86

    Article  Google Scholar 

  • Gulikers JTM, Bastiaens TJ, Kirschner PA (2004b) Perceptions of authentic assessment: five dimensions of authenticity. Second Biannual Joint Northumbria/EARLI SIG assessment conference, Bergen

    Google Scholar 

  • Gulikers JTM, Bastiaens TJ, Kirschner PA (2006) Authentic assessment, student and teacher perceptions: the practical value of the five dimensional framework. J of Vocational Educ and Training 58:337–357

    Article  Google Scholar 

  • Gulikers JTM, Bastiaens TJ, Kirschner PA, Kester L (2008) Authenticity is in the eye of the beholder: student and teacher perceptions of assessment authenticity. J of Vocational Education and Training 60:401–412

    Article  Google Scholar 

  • Hensel D, Stanley L (2014) Group simulation for “authentic” assessment in a maternal-child lecture course. Journal of the Scholarship of Teaching and Learning 14:61–70

    Article  Google Scholar 

  • Herrington J, Herrington A (1998) Authentic assessment and multimedia: how university students respond to a model of authentic assessment. Higher Education Research and Development Society of Australasia 17:305–322

    Article  Google Scholar 

  • HoepfL, M. 2000. Large-Scale Authentic Assessment. In: CUSTER, R. L. (ed.) Using authentic assessment in vocational education. Columbus: Center on Education and Training for Employment

  • Johnson, Y. L. 2007. The efficacy of authentic assessment versus pencil and paper testing in evaluating student achievement in a basic technology course. PhD, Walden University

  • Jonsson, A. 2008. Educative assessment for/of teacher competency. A study of assessment and learning in the “interactive examination” for student teachers. PhD Doctoral Thesis, Malmo University

  • Jonsson A, Svingby G (2007) The use of scoring rubrics: reliability, validity and educational consequences. Educational Research Review 2:130–144

    Article  Google Scholar 

  • Lang II, T. R. 2012. An examination of the relationship between elementary education teacher candidates’ authentic assessments and performance on the professional education subtests on the Florida Teacher Certification Exam (FTCE). Educational Specialist Graduate Theses and Dissertations, University of South Florida

  • Linn RL, Baker EL, Dunbar SB (1991) Complex, performance-based assessment: expectations and validation criteria. Educational Researcher 20:15–21

    Article  Google Scholar 

  • Lovorn MG, Rezaei AR (2011) Assessing the assessment: rubrics training for pre-service and new in-service teachers. Practical Assessment, Research & Evaluation 16:1–18

    Google Scholar 

  • Lynch, R. 2003. Authentic, performance-based assessment in ESL/EFL reading instruction. Asian EFL Journal, 1–28

  • Marzano, R. J., Pickering, D. & Mctighe, J. 1993. Assessing student outcomes: performance VA, Association for Supervision and Curriculum Development

  • Mccarthy G (2013) Authentic assessment—key to learning. In: Doyle E, Buckley P, Carroll C (eds) Innovative business school teaching: engaging the millennial generation. Routledge, United Kingdom

    Google Scholar 

  • Messick S (1994) The interplay of evidence and consequences in the validation of performance assessments. Educational Researcher 23:13–23

    Article  Google Scholar 

  • Messick S (1995) Validity of psychological assessment: validation of references from person’s responses and performances as scientific inquiry into score meaning. American Psychologist 50:741–749

    Article  Google Scholar 

  • Messick S (1996) Validity and washback in language testing. Language Testing 13:241–256

    Article  Google Scholar 

  • Meyer CA (1992) What’s the difference between “authentic” and “performance” assessment? Educational leadership 49:39–40

    Google Scholar 

  • Montgomery K (2002) Authentic tasks and rubrics: going beyond tradtional assessments in college teaching. College teaching 50:34–40

    Article  Google Scholar 

  • Moon TR, Brighton CM, Callahan CM, Robinson A (2005) Development of authentic assessments for the middle school classroom. The Journal of Secondary Gifted Education 16:119–133

    Google Scholar 

  • Morrison WGS (1997) Competent crews = safer ships: an aid to understanding STCW 95. Sweden, WMU Publications, Malmo

    Google Scholar 

  • Moskal BM (2000) Scoring rubrics: what, when and how? Practical Assessment, Research & Evaluation 7

  • Moskal BM, Leydens JA (2000) Scoring rubric development: validity and reliability. Practical Assessment, Research & Evaluation 7

  • Mueller, J. 2005. The authentic assessment toolbox: enhancing student learning through online faculty development. Journal of Online Learning and Teaching, 1

  • Neely P, Tucker J (2012) Using business simulations as authentic assessment tools. American Journal of Business Education 5:449–456

    Google Scholar 

  • Neo M, Neo KT-K, Tan HY-J (2012) Applying authentic learning strategies in a multimedia and web learning environment (MWLE) Malaysian students’ perspective. The Turkish Online Journal of Educational Technology 11:50–60

    Google Scholar 

  • Newmann FM, Marks HM, Gamoran A (1996) Authentic pedagogy and student performance. American Journal of Education 104:280–312

    Article  Google Scholar 

  • Olfos R, Zulantay H (2007) Reliability and validity of authentic assessment in a web based course. Educational Technology & Society 10:156–173

    Google Scholar 

  • Osborne R, Dunne E, Farrand P (2013) Integrating technologies into “authentic” assessment design: an affordances approach. Research in Learning Technology 21

  • Palm, T. 2008. Performance assessment and authentic assessment: a conceptual analysis of the literature. Practical Assessment, Research and Evaluation, 13.

  • Palomba CA, Banta TW (1999) Assessment essentials: planning, implementing, and improving assessment in higher education. Jossey-Bass, San Francisco

    Google Scholar 

  • Reddy MY (2007) Rubrics and the enhancement of student learning. Educ Comm Tech J 7:3–17

    Google Scholar 

  • Reeves, T. C. & Okey, J. R. 1996. Alternative assessment for constructivist learning environments, Englewood Cliffs, NJ, Educational Technology Publications

  • Rezaei AR, Lovorn M (2010) Reliability and validity of rubrics for assessment through writing. Assessing Writing 15:18–39

    Article  Google Scholar 

  • Rhodes TL, Finley A (2013) Using the VALUE rubrics for improvement of learning and authentic assessment. Association of American Colleges and Universities, United States of America

    Google Scholar 

  • Sadler R (2005) Interpretations of criteria-based assessment and grading in higher education. Assessment & Evaluation in Higher Education 30:175–194

    Article  Google Scholar 

  • Sampson, D. & Fytros, D. 2008. Competence models in technology-enhanced competence-based learning. In: Adelsberger, H. H., Kinshuk, Pawlowski, Jan Martin (ed.) Handbook on information technologies for education and training. Springer Berlin Heidelberg.

  • Scholtz A (2007) An analysis of the impact of authentic assessment strategy on student performance in a technology-mediated constructivist classroom. A study revisited International Journal of Education and Development using Information and Communication Technology (IJEDICT) 3:42–53

    Google Scholar 

  • Silvestri L, Oescher J (2006) Using rubrics to increase the reliability of assessment in health classes. International Electronic Journal of Health Education 9:25–30

    Google Scholar 

  • Spady, W. G. 1994. Outcome-based education: critical issues and answers, Arlington, VA, American Association of School Administrators.

  • Stellmack MA, Konheim-Kalkstein YL, Manor JE, Masset AR, Schmitz JAP (2009) An assessment of reliability and validity of a rubric for grading APA-style introductions. Teaching of Psychology 36:102–107

    Article  Google Scholar 

  • Stevens, P. 2013. An examination of a teacher’s use of authentic assessment in an urban middle school setting Doctor of Education PhD, Ohio University

  • Taylor, J. M. 2011. Interdisciplinary authentic assessment: cognitive expectations and student performance. Doctor of Education PhD, Pepperdine University

  • Todorov, K. R. & Brousseau, B. 1998. Authentic assessment of social studies. In: Education, M. D. O. (ed.). Lansing, Michigan

  • University OF Tasmania (UTAS) 2011. Guidelines for good assessment practice, Tasmania, Australia, UTAS.

  • Vendlinski, T., Underdahl, J., Simpson, E. & Stevens, R. Authentic assessment of student understanding in near-real time. NECC 2002: 23rd National Education Computing Conference Proceedings, 17–19 June 2002 San Antonio, Texas

  • Walvoord, B. E. 2004. Assessment clear and simple: a practical guide for institutions, departments and general education, San Francisco, Jossey-Bass

  • Wellington P, Thomas I, Powell I, Clarke B (2002) Authentic assessment applied to engineering and business undergraduate consulting teams. International Journal of Engineering education 18:168–179

    Google Scholar 

  • Wiggins G (1989) A true test: toward more authentic and equitable assessment. The Phi Delta Kappan 70:703–713

    Google Scholar 

  • Wiggins, G. 1990. The case for authentic assessment. Practical Assessment, Research and Evaluation, 2.

  • Wiggins G (1992) Creating tests worth taking. Educational Leadership 49:26–34

    Google Scholar 

  • Wiggins G (1993) Assessment: authenticity, context and validity. Phi Delta Kappa International 75(200–208):210–214

    Google Scholar 

  • Wiggins G (1998) Educative assessment: designing assessments to inform and improve student performance. Jossey-Bass Publishers, San Francisco

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Australian Maritime College (UTAS), Launceston, 7250, Australia

    Samrat Ghosh, Marcus Bowles, Dev Ranmuthugala & Ben Brooks

Authors
  1. Samrat Ghosh
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Marcus Bowles
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Dev Ranmuthugala
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Ben Brooks
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Samrat Ghosh.

Appendix

Appendix

Table 6 Classification of literature based on the extent of validity and reliability testing of rubrics when used as an authentic assessment instrument
Full size table

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Ghosh, S., Bowles, M., Ranmuthugala, D. et al. Authentic assessment in seafarer education: using literature review to investigate its validity and reliability through rubrics. WMU J Marit Affairs 15, 317–336 (2016). https://doi.org/10.1007/s13437-015-0094-0

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s13437-015-0094-0

Keywords

Search

Navigation