How do instructional designers manage learners’ cognitive load? An examination of awareness and application of strategies

Abstract

This study examined how practicing instructional designers manage cognitive load in a standardized scenario as they select and implement instructional strategies, message design, content sequencing, delivery medium, and technology within various domains with learners at different levels of expertise. The study employed a quasi-experimental, mixed methods design to gain insight into how practicing instructional designers perceive their awareness of strategies to manage cognitive load and implement those strategies within a standardized design scenario. The results of the study indicated that both novice and expert practitioners frequently used several strategies to manage extraneous load (worked examples, completion tasks, and dual modality) as prescribed by theory, as well as the simple-to-complex presentation strategy to manage intrinsic load. While participants frequently acknowledged differences in the levels of learner expertise within the instructional scenario, few employed strategies prescribed to address the expertise reversal effect as outlined by theory. Based on the results of this study, we present a framework to assist designers with managing for cognitive load in their everyday design practices.

Appendices

Appendix A: Cognitive load studies and prescribed strategies (well-structured domains)

Authors (Year)	Context	Domain	Cognitive load effects	Prescribed strategies
Sweller et al. (1983)	K-12 and higher education	Kinematics and geometry	Learners who studied with reduced goal specificity were more efficient	Goal-free tasks during acquisition rather than conventional problem solving
Sweller and Cooper (1985)	K-12 and higher education	Algebra	Learners who studies worked examples took less time and made fewer errors	Worked examples rather than solution generation as learners become familiar with subject
Tarmizi and Sweller (1988)	K-12 education	Geometry	Learners who used integrated diagrams and text took less time to solve and made fewer errors	Integrate multiple sources of related information into a single element
Chi et al. (1989)	Higher education	Physics	Students who generated explanations of solutions had higher problem-solving scores	Prompt learners to produce self-explanations while studying worked examples and completion tasks
Jelsma and van Merriënboer (1989)	Higher education	General problem solving	Participants who used a random practice schedule took less time and made fewer errors	Present series of random tasks containing high contextual interference
van Merriënboer (1990)	K-12 education	Computer programming	Learners who studied completion problems had higher completion rates and percentage of correct feature use	Have learners complete larger portions of a solution until they are prepared to generate solutions
Chandler and Sweller (1991)	K-12 and technical education	Engineering and biology	Shorter instruction time and higher test scores when students used integrated instructions	Eliminate redundant information if material can be understood from a single element
Paas (1992)	Technical education	Statistics	Lower mental effort ratings and time on task for students using completion problems	Use completion tasks to allow learner to finish partial problem solutions
Paas and van Merriënboer (1994)	Technical education	Geometry	Better test performance, lower perceived mental effort and time on task for learners studying examples with high variability	Present a series of tasks that differ in surface features as they would in realistic situations
Sweller and Chandler (1994)	K-12 and technical education	Computer software and electrical testing	Lower time for instruction and testing, higher test scores for learners who studied with only a manual rather than a manual and equipment	Examine material for number of interacting elements to determine complexity relative to learner expertise
Mousavi et al. (1995)	K-12 education	Geometry	Less time spent studying and solving problems and better performance for learners who used dual-modality worked examples	Supplement visual information with a second mode of delivery (audio explanations)
Kalyuga et al. (1998)	Technical education	Electrical circuits	Learners with less expertise had lower mental effort ratings and higher performance scores with integrated diagrams and text; reverse effect for learners with more expertise	Replace worked examples including fully integrated information with visual-only or text-only examples as learners develop expertise
Kalyuga et al. (2000)	Technical education	Manufacturing	Students with less expertise had lower task difficulty ratings and higher performance test scores when using diagrams with auditory text; reverse effect for learners with more expertise	Replace dual modality materials with visual-only materials (no supplemental audio information) as learners gain expertise
Pollock et al. (2002)	Technical education	Electrical circuits	Lower subjective mental load and higher performance scores for learners who used isolated task elements first and interacting elements second	Replace conventional problem solving tasks with a strategy of gradually moving from simple, isolated tasks to tasks of full complexity
Renkl et al. (2002)	Higher education	Probability	Learners who studied with faded worked examples had a lower number of errors and better performance in near transfer	Start learners with a larger amount of guidance and progressively fade guidance over time as they develop expertise (scaffolding)

Appendix B: Cognitive load studies and prescribed strategies (ill-structured domains)

Authors (year)	Context	Domain	Cognitive load effects	Prescribed strategies
Reisslein, Atkinson, Seeling, and Reisslein (2006)	Higher education	Engineering	Learners with low expertise had better performance scores when moving from examples to conventional problems	Fade instructional guidance over time as learners develop expertise
Schworm and Renkl (2006)	Higher education	Instructional design	Higher post-test scores for learners who used self-explanations	Prompt learners to produce self-explanations as they study worked examples and completion problems
Owens and Sweller (2008)	K-12 education	Music	More correct solutions during acquisition and higher post-test scores for learners using worked examples with spatial integration and simultaneous presentation	Integrate related information to reduce split attention
Rourke and Sweller (2009)	Higher education	Design history	Learners performed better after studying worked examples rather than problem solving	Use worked examples rather than conventional problem solving as learners become familiar with material
Oksa et al. (2010)	K-12 and adult education	Literary studies	Lower mental load ratings and better test performance for learners who studied worked examples; reverse effect for learners with more expertise	Fade instructional guidance over time as learners develop expertise
Stark et al. (2011)	Higher education	Medicine	Lower cognitive load scores and better performance for learners using worked examples with elaborated feedback	Prompt learners to produce self-explanations as they study worked examples and completion problems
Kyun et al. (2013)	Higher education	English literature	Lower mental effort ratings and higher performance for learners with less expertise studying worked examples	Move learners through a progression of tasks from worked examples to completion problems to solution generation
Nievelstein et al. (2013)	Higher education	Legal cases	Lower mental effort ratings and better learning outcomes for learners using worked examples	Use a high degree of task variability when presenting worked examples
Mulder et al. (2014)	K-12 education	Physics	Improved inquiry behavior and higher quality models during learning phase for students using worked examples	Gradually move the learner from tasks of low complexity to tasks of high complexity
Si et al. (2014)	Higher education	Computer programming	Higher efficiency for learners studying with adaptive instruction rather than fixed instruction	Have learners complete larger portions of a solution until they are prepared to generate solutions
Jung and Suzuki (2015)	Higher education	Japanese language learning	Better learning outcomes and higher student satisfaction for learners who used less comprehensive worked example templates	Use less detailed worked examples in instances where creative and independent thinking are intended outcomes
Margulieux and Catrambone (2016)	Higher education	Computer programming	Lower time on task and better performance for learners using worked examples with labeled sub-goals	Examine material for number of interacting elements to determine complexity relative to learner expertise

Appendix C: Instructional designer use of theory studies and findings

Authors (year)	Experience	Settings	Study goals	Findings on theory use
Rowland (1992)	7–20 years of experience (experts), one or no projects (novices)	Academic (2), business (1), government (1)	Problem understanding and solution generation	Significant differences in problem interpretation, representation, solution generation, solutions, use of internal resources, external resources, decision making; prescriptions without experience base for novices are not effective
Wedman and Tessmer (1993)	A few months to 25 years (mean of 6 years)	30 from same training and development group, 43 from business and government	Identify frequency of 11 ID activities and reasons why certain activities are excluded from projects	Design activities occur on an irregular basis in practice, almost no designers use all ID model design activities, various reasons for omitting design activities (decisions already made, lack of time, etc.); models need to be more practical
Winer and Vázquez-Abad (1995)	1–35 years of experience (mean of 13 years)	Business and industry	Identify frequency of 11 ID activities and reasons why certain activities are excluded from projects	Selecting instructional strategies and media formats seen as most necessary activities, emphasis on prototyping rather than classic ID life cycle (with extensive analysis) for projects; many contextual pressures, instructional methods may be considered inputs to the ID process rather than outcomes, use of heuristic knowledge rather than prescriptive models
Pieters and Bergman (1995)	2–6 years of professional experience	Education and training	Indicate characteristics of the design process in practice	Many deviations from general ISD models, practical context leads to less time for activities than needed, iterative processes are common; experts use contextual knowledge, social variables influence the process, evaluation and implementation activities do not get the time they deserve
Kirschner et al. (2002)	Expert designers (years not mentioned)	Industry and education	Identify important design principles and complete a design task through a series of actions to create a course	Designers in competency based education agree that design should be based on the needs of learners rather than the content, university designers feel alternative solutions are important and focus on the instructional blueprint, business designers are more client oriented and concerned about process buy-in
Christensen and Osguthorpe (2004)	Didn’t ask number of years of experience, 69% had master’s degree, 30% had doctorate	Industry, higher education, K12, military, adult education	Select frequency of their use of 12 ID theories and strategies; list useful ID and learning theories, rate information sources for learning about theories and strategies, list useful information sources	Little over half use theory when making strategy decisions, those who use theory use a variety of theories based on design situation, over 80% indicate interactions with others are most common means of making strategy decisions and learning about new theories; range of theories used such as Gagne and Merrill, as well as learning theories such as constructivism and information processing theory
Yanchar et al. (2010)	4 had master’s degrees in ID, 3 were informally trained with non-ID master’s degrees, 1 had a PhD in the natural sciences	1 in large design organization, 2 in small design organization, 1 in a university ID center, 2 as in-house designers in a laboratory organization, 1 as in-house designer in a technology corporation	Describe practical involvement in design process, uses and views of formal theories	Largely considered theory as potentially helpful for generating ideas of making sense of situations but didn’t endorse all aspects of a particular theory, situational limitations prevented them from implementing theory in practice more than they do, opposed to rigid use of theory without reflection on its application, importance of using theory to help shape and inform their intuition as they continue to gain experience within their craft
Honebein and Honebein (2014)	Didn’t ask number of years of experience; were seeking an ID certificate, master’s degree, or PhD degree	Corporations, consulting firms, colleges, K12	Indicate usefulness of methods for each content level (186 unique ratings) on a scale from 1 (least useful) to 5 (most useful)	Level of cognitive domain content had a statistically significant effect on judgments of IDs regarding the appropriateness of specific instructional methods, methods judged by practitioners aligned with those of experts; statistically significant effect of designer gender on judgments of methods, although effect size was small and didn’t support their hypothesis
Gray et al. (2015)	2–11 years of experience in ID	1 from in-house consultancy at a university, 7 from a commercial ID firm near the university	Practitioners conducted their everyday activities on various projects	IDs made an average of 35 judgments during observation (16 per h), wide variety of types of judgments regardless of design activity or project phase; judgments of practitioners tend to be clustered closely together based on the activity rather than occurring discretely, judgements depend on position, design firm culture, and type of client
Sugar and Luterbach (2016)	7–20 years of experience (experts), one or no projects (novices)	Academic (2), business (1), government (1)	Problem understanding and solution generation	6 effective practices (collaboration, providing resources, social presence, differentiated instruction, examples, creating instructional materials), 4 extraordinary (using theory in practice, managing complex projects, organizing content, matching methods to learners and content), 6 ineffective (dealing with inadequate technology, not collaborating, not using an instructional design process, selecting inappropriate strategies, not supporting interaction, mismatching methods to learners and content)

Appendix D: Instructional designer decision making questionnaire

Please complete this questionnaire to the best of your ability and respond to each of the questions as accurately as possible. The data gathered from the responses will be used to examine how instructional designers make decisions in practice.

The questionnaire will take approximately 15 min to complete. Please return it to jsent003@odu.edu prior to your appointment time for the instructional design scenario activity. You may use the same email address should you have any issues or questions. Thank you for your time.

Please answer the following questions about yourself.

For each of the following, please indicate the degree to which you agree with the statement:

For each of the following, please indicate how often you currently use the strategy in your instructional design work:

Appendix E: Instructional design scenario

Please review the instructional scenario below and design a solution that addresses the needs of the learners to the best of your ability. There is no correct or preferred approach to the scenario, so please do not worry about whether your approach is the “right” one. Please speak aloud as you are making your design decisions so that the researchers can follow the process you are using. The data gathered from the responses will be used to examine how instructional designers make decisions in practice.

The scenario will take approximately 30 min to complete. All of the information you will need is contained within the scenario, but please feel free to ask the researcher if you need clarification on any of the information. Thank you for your time.

Scenario

You are an instructional designer working for your current or most recent organization (K-12, higher education, industry, government, etc.), and you need to cover the creation of spreadsheets as part of the regular course of your instructional duties. The creation and use of spreadsheets is considered a basic competency within your area of practice that learners need to master in order to work with data in their particular settings. You have been asked to incorporate instruction related to the basic creation of spreadsheets within the Microsoft Excel software package as part of regular training/education activities for your learners.

Needs assessment

Your supervisor has asked you to create an instructional module on the creation of basic spreadsheets within Microsoft Excel that will enable all learners to establish a consistent level of competency inputting and manipulating data. The instruction needs to be basic enough that learners are able to complete it with only a fundamental understanding of the mathematical operations involved in creating a spreadsheet, and the instruction needs to be flexible enough to be used by learners independently at their own pace.

Learner analysis

Regardless of your particular practice setting, all learners are able to read English at an 8th grade level or higher, have basic proficiency in the use of computers and mathematical formulas, and are physically able to perform the tasks involved.

The majority of the learners (16 students in a class of 20) have little experience using Microsoft Excel and should be considered novices with respect to the creation and use of spreadsheet applications. Robert, shown below, is one of these learners:

Robert is a third-year undergraduate student who is majoring in studio art. He has used computers throughout his K-12 and college education but has not done much work with Microsoft Office applications other than basic word processing. He has experience viewing budget data in Excel spreadsheets during his time in the Art Club in high school, but he has not created a spreadsheet from scratch or manipulated the data in an existing spreadsheet. Robert has taken typical mathematics courses prior to enrolling in college, including 2 years of algebra. He is considering a minor in business due to his interest in starting his own art studio, so Robert is motivated to learn and apply the information from the unit to his area of study.

There are, however, a few learners (4 students in a class of 20) who have an intermediate understanding of Microsoft Excel and the creation of basic spreadsheets. Karen, shown below, is one of these learners:

Karen is a first-year undergraduate student who is majoring in business administration. She has used computers throughout her K-12 education and has some experience with each of the Microsoft Office applications. She has not taken any formal coursework in Excel, but she has a working knowledge of the basic functionality involved in creating a spreadsheet from tutorials within the program itself to put together simple spreadsheets for high school classes. Karen has taken business mathematics and algebra courses prior to enrolling in college. She anticipates taking a few accounting courses later in college as part of her major, so she is motivated to build upon her existing knowledge by learning the information. As with the other learners with more expertise, Karen is still required to take the instruction and meet the objectives.

Environmental analysis

The instruction may be delivered by any means of delivery deemed appropriate, provided that the learners are able to progress through the material at their own pace. The learners have access to computers in a lab at your organization/institution, and all computers are equipped with Microsoft Office and an Internet connection. Written materials can also be made available to the learners if you determine they are needed for the instruction. An instructor station and a projector are located at the front of the lab if you find a need to use those. Learners will be given access to the desks and computers during class/working hours as needed to complete the instruction.

Task analysis

A task analysis of basic spreadsheet creation revealed the following steps:

• Determine a practical need for a spreadsheet application.

• Sketch out the structure of the spreadsheet.

• Determine the calculations that will be needed to manipulate the data.

• Open microsoft excel.

• Create column headings appropriate to the application.

• Create row headings appropriate for the data.

• Input the data in the appropriate cells.

• Format cells as appropriate for the types of data included.

• Use basic math symbols (= , +, −, *,/) to create formulas as appropriate.

• Use a function to calculate totals (SUM) as appropriate.

• Use a function to calculate averages (AVERAGE) as appropriate.

• Use a function to find the highest value (MAX) in a range of numbers.

• Use a function to find the lowest value (MIN) in a range of numbers.

• Use a function to determine how many numbers (COUNT) are in a range of cells.

• Copy a function across multiple spreadsheet cells.

• Create a basic chart that displays the information graphically in a useful manner.

Instructional objectives

Upon completion of the instruction:

• 1. The learners will create a spreadsheet application that addresses a real-world problem of either personal or professional significance.

• 2. The learners will structure the spreadsheet in a logical manner that lends itself to solving the problem.

• 3. The learners will create column and row headings that sufficiently explain the data.

• 4. The learners will input data as appropriate for the spreadsheet structure created.

• 5. The learners will format the cells as appropriate for the type(s) of data involved.

• 6. The learners will use three or more math symbols to create formulas to manipulate the data.

• 7. The learners will use at least three functions to manipulate the data in the process of solving the problem.

• 8. The learners will create a basic chart that presents the data graphically in order to solve the practical problem they have identified.

You have approximately 30 min to design and explain your solution to this instructional scenario. Please describe aloud to the researcher the steps you are taking throughout the process and the reasons you are making those decisions. This study is primarily concerned with the decision making process you use and the reasons you are taking specific steps to design a solution.

Appendix F: Instructional design scenario observation sheet

Overall SOLO rating (circle):

0	Respondent did not apply any strategies to manage cognitive load. (pre-structural)
1	Respondent primarily considered a single source of cognitive load. (uni-structural)
2	Respondent considered multiple sources of cognitive load. (multi-structural)
3	Respondent considered the interaction of multiple sources of cognitive load. (relational)
4	Respondent considered cognitive load holistically and displayed a comprehensive understanding of its implications. (extended abstract)

Comments:

Appendix G: Debriefing interview protocol