Skip to main content
SpringerLink
Log in

Learning Analytics to Support Teachers’ Assessment of Problem Solving: A Novel Application for Machine Learning and Graph Algorithms

  • Chapter
  • First Online:
Utilizing Learning Analytics to Support Study Success

Abstract

In contrast to well-structured problems which have pre-defined, correct answers, complex real-world problems are often ill-structured problems (ISPs). The open-ended nature of ISPs creates considerable barriers to assess and guide students in forming better solutions, which results in low adoption levels of inquiry-based learning. Students can structure and represent their knowledge for an ISP in the form of knowledge maps or causal maps, which articulate relevant concepts and their causal relations (i.e., antecedents and consequents). Assessing such maps can involve a referent-free evaluation (e.g., to encourage the creation of maps with high density of concepts) or a comparison to an expert map used as reference. This chapter starts with a review of theories and tools to compare a student’s map to the expert map. Previous approaches often compared individual connections (e.g., scoring the number of connections that a student has/misses in contrast with the expert) or general map metrics (e.g., one map is denser than the other). In contrast, the problem of comparing two maps has been studied in network theory and graph theory for several decades, yielding categories of algorithms that are currently underutilized in educational research. This chapter reviews three categories of algorithms (i.e., graph kernel, graph editing distance, graph embedding) in light of their application to assessment and student success. We discuss an implementation of these algorithms through a new set of digital tools, designed to support a community of practice in problem-based instruction.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 129.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Hardcover Book
USD 169.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  • Arvind, V., & Jacobo, T. (2005). Isomorphism testing: Perspectives and open problems. Bulletin of the European Association for Theoretical Computer Science, 86, 66–84.

    Google Scholar 

  • Ausubel, D. G. (1963). Cognitive structure and the facilitation of meaningful verbal learning. Journal of Teacher Education, 14(2), 217–222.

    Article  Google Scholar 

  • Axelrod, R. (1974). Structure of decision: The cognitive maps of political elites. Princeton, NJ: Princeton University Press.

    Google Scholar 

  • Barrows, H. (1996). Problem-based learning in medicine and beyond: A brief overview. New Directions for Teaching and Learning, 1996(68), 3–12.

    Article  Google Scholar 

  • Bax, E. T. (1994). Algorithms to count paths and cycles. Information Processing Letters, 52(5), 249–252.

    Article  Google Scholar 

  • Bex, F., & Bench-Capon, T. J. (2014). Understanding narratives with argumentation. In COMMA (pp. 11–18).

    Google Scholar 

  • Budzynska, K., Janier, M., Kang, J., Reed, C., Saint-Dizier, P., Stede, M., & Yaskorska, O. (2014). Towards argument mining from dialogue. In Computational Models of Argument: Proceedings of COMMA 2014 (pp. 185–196).

    Google Scholar 

  • Carletti, V., Foggia, P., Saggese, A., & Vento, M. (2018). Challenging the time complexity of exact subgraph isomorphism for huge and dense graphs with VF3. IEEE Transactions on Pattern Analysis and Machine Learning, 40(4), 804–818.

    Article  Google Scholar 

  • Chang, K.-E., Sung, Y.-T., Chang, R.-B., & Lin, S.-C. (2005). A new assessment for computer-based concept mapping. Educational Technology & Society, 8(3), 138–148.

    Google Scholar 

  • Clariana, R. B. (2010). Deriving individual and group knowledge structure from network diagrams and from essays. In Computer-based diagnostics and systematic analysis of knowledge (pp. 117–130). Boston, MA: Springer.

    Chapter  Google Scholar 

  • Clariana, R. B., Engelmann, T., & Yu, W. (2013). Using centrality of concept maps as a measure of problem space states in computer-supported collaborative problem solving. Educational Technology Research & Development, 61(3), 423–442.

    Article  Google Scholar 

  • Dufresne, R. J., Gerace, W. J., Hardiman, P. T., & Mestre, J. P. (1992). Constraining novices to perform expert like problem analyses: Effects on schema acquisition. Journal of the Learning Sciences, 2(3), 307–331.

    Article  Google Scholar 

  • Ericsson, K. A. (2005). Recent advances in expertise research: A commentary on the contributions to the special issue. Applied Cognitive Psychology, 19(2), 233–241.

    Article  Google Scholar 

  • Ertmer, P. A., Stepich, D. A., York, C. S., Stickman, A., Wu, X. L., Zurek, S., & Goktas, Y. (2008). How instructional design experts use knowledge and experience to solve ill-structured problems. Performance Improvement Quarterly, 21(1), 17–42.

    Article  Google Scholar 

  • Eseryel, D., Ifenthaler, D., & Ge, X. (2013). Validation study of a method for assessing complex ill-structured problem solving by using causal representations. Educational Technology Research & Development, 61(3), 443–463.

    Article  Google Scholar 

  • Foggia, P., Percannella, G., & Vento, M. (2014). Graph matching and learning in pattern recognition in the last 10 years. International Journal of Pattern Recognition and Artificial Intelligence, 28(1), 1450001.

    Article  Google Scholar 

  • Ge, X., & Land, S. (2003). Scaffolding students’ problem-solving processes in an ill-structured task using question prompts and peer interactions. Educational Technology Research & Development, 51(1), 21–38.

    Article  Google Scholar 

  • Giabbanelli, P. J., & Baniukiewicz, M. (2018). Navigating complex systems for policymaking using simple software tools. In P. J. Giabbanelli, V. K. Mago, & E. I. Papageorgiou (Eds.), Advanced data analytics in health (pp. 21–40). Berlin, Germany: Springer.

    Chapter  Google Scholar 

  • Giabbanelli, P. J., & Crutzen, R. (2014). Creating groups with similar expected behavioural response in randomized controlled trials: A fuzzy cognitive map approach. BMC Medical Research Methodology, 14(1), 130.

    Article  Google Scholar 

  • Giabbanelli, P. J., Flarsheim, R. A., Vesuvala, C. X., & Drasic, L. (2016). Developing technology to support policymakers in taking a systems science approach to obesity and well-being. Obesity Reviews, 17, 194–195.

    Google Scholar 

  • Giabbanelli, P. J., & Tawfik, A. A. (2018). Overcoming the PBL assessment challenge: Design and development of the incremental thesaurus for assessing causal maps (ITACM). Technology, Knowledge and Learning. https://doi.org/10.1007/s10758-017-9338-8

  • Gray, S. A., Hilsberg, J., McFall, A., & Arlinghaus, R. (2015). The structure and function of angler mental models about fish population ecology. Journal of Outdoor Recreation and Tourism, 12, 1–13.

    Article  Google Scholar 

  • Grotzer, T. A., Kamarainen, A. M., Tutwiler, M. S., Metcalf, S., & Dede, C. (2013). Learning to reason about ecosystems dynamics over time: The challenges of an event-based causal focus. Bioscience, 63(4), 288–296.

    Article  Google Scholar 

  • Gupta, V. K., Giabbanelli, P. J., & Tawfik, A. A. (2018). An online environment to compare students’ and expert solutions to ill-structured problems. In Proceedings of the 2018 Human Computer Interactions (HCI) conference. To appear.

    Chapter  Google Scholar 

  • Hays, J. R., & Simon, H. A. (1974). Understanding written problem instructions. In L. W. Gregg (Ed.), Knowledge and cognition (pp. 167–200). Hillsdale, NJ: Erlbaum.

    Google Scholar 

  • Herrington, J., Reeves, T. C., & Oliver, R. (2014). Authentic learning environments. In J. M. Spector, M. D. Merrill, J. Elen, & M. J. Bishop (Eds.), Handbook of research on educational communications and technology (4th ed., pp. 453–464). New York, NY: Springer.

    Google Scholar 

  • Hjaltason, G. R., & Samet, H. (2003). Properties of embedding methods for similarity searching in metric spaces. IEEE Transactions on Pattern Analysis and Machine Intelligence, 25(5), 530–549.

    Article  Google Scholar 

  • Hmelo-Silver, C. (2013). Creating a learning space in problem-based learning. Interdisciplinary Journal of Problem-Based Learning, 7(1), 5. https://doi.org/10.7771/1541-5015.1334

    Article  Google Scholar 

  • Hmelo-Silver, C., & Barrows, H. (2006). Goals and strategies of a problem-based learning facilitator. Interdisciplinary Journal of Problem-Based Learning, 1(1), 4. https://doi.org/10.7771/1541-5015.1004

    Article  Google Scholar 

  • Hmelo-Silver, C., Marathe, S., & Liu, L. (2007). Fish swim, rocks sit, and lungs breathe: Expert-novice understanding of complex systems. Journal of the Learning Sciences, 16(3), 307–331.

    Article  Google Scholar 

  • Hmelo-Silver, C., & Pfeffer, M. G. (2004). Comparing expert and novice understanding of a complex system from the perspective of structures, behaviors, and functions. Cognitive Science, 28(1), 127–138.

    Article  Google Scholar 

  • Hong, Y.-C., & Choi, I. (2011). Three dimensions of reflective thinking in solving design problems: A conceptual model. Educational Technology Research & Development, 59(5), 687–710.

    Article  Google Scholar 

  • Horváth, T. (2005). Cyclic pattern kernels revisited. Lecture Notes in Computer Science, 3518, 791–801.

    Article  Google Scholar 

  • Hung, W. (2011). Theory to reality: A few issues in implementing problem-based learning. Educational Technology Research & Development, 59(4), 529–552.

    Article  Google Scholar 

  • Hung, W. (2015). Problem-based learning: Conception, practice, and future. In Y. Cho, I. S. Caleon, & M. Kapur (Eds.), Authentic problem solving and learning in the 21st century (pp. 75–92). Singapore: Springer.

    Chapter  Google Scholar 

  • Ifenthaler, D. (2010). Relational, structural, and semantic analysis of graphical representations and concept maps. Educational Technology Research & Development, 58(1), 81–97.

    Article  Google Scholar 

  • Ifenthaler, D. (2011). Bridging the gap between expert-novice differences: The model-based feedback approach. Journal of Research on Technology in Education, 43(2), 103–117.

    Article  Google Scholar 

  • Ifenthaler, D. (2012). Determining the effectiveness of prompts for self-regulated learning in problem-solving scenarios. Educational Technology & Society, 15(1), 38–52.

    Google Scholar 

  • Ifenthaler, D. (2014). AKOVIA: Automated knowledge visualization and assessment. Technology, Knowledge and Learning, 19(1–2), 241–248.

    Article  Google Scholar 

  • Ifenthaler, D., Masduki, I., & Seel, N. M. (2011). The mystery of cognitive structure and how we can detect it: Tracking the development of cognitive structures over time. Instructional Science, 39(1), 41–61.

    Article  Google Scholar 

  • Inselberg, A., & Dimsdale, B. (1990). Parallel coordinates: A tool for visualizing multi-dimensional geometry. In Proceedings of the First IEEE Conference on Visualization (pp. 361–378).

    Google Scholar 

  • Jacobson, M. J. (2001). Problem solving, cognition, and complex systems: Differences between experts and novices. Complexity, 6(3), 41–49.

    Article  Google Scholar 

  • Jeong, A. (2014). Sequentially analyzing and modeling causal mapping processes that support causal understanding and systems thinking. In D. Ifenthaler & R. Hanewald (Eds.), Digital knowledge maps in education: Technology-enhanced support for teachers and learners. Chapter 13 (pp. 239–251). New York, NY: Springer.

    Google Scholar 

  • Jonassen, D. H. (1991). Objectivism versus constructivism: Do we need a new philosophical paradigm? Educational Technology Research & Development, 39(3), 5–14.

    Article  Google Scholar 

  • Jonassen, D. H. (1997). Instructional design models for well-structured and ill-structured problem-solving learning outcomes. Educational Technology Research and Development, 45(1), 65–94.

    Article  Google Scholar 

  • Jonassen, D. H. (2011). Learning to solve problems: A handbook for designing problem-solving learning environments (1st ed.). London, England: Routledge.

    Google Scholar 

  • Ju, H., & Choi, I. (2017). The role of argumentation in hypothetico-deductive reasoning during problem-based learning in medical education: A conceptual framework. Interdisciplinary Journal of Problem-Based Learning, 12(1), 4.

    Article  Google Scholar 

  • Juszczyszyn, K., Kazienko, P., & Musiał, K. (2008). Local topology of social network based on motif analysis. Lecture Notes in Computer Science, 5178, 97–105.

    Article  Google Scholar 

  • Kim, K., & Clariana, R. B. (2015). Knowledge structure measures of reader’s situation models across languages: Translation engenders richer structure. Technology, Knowledge and Learning, 20(2), 249–268.

    Article  Google Scholar 

  • Kim, N. J., Belland, B. R., & Walker, A. E. (2017). Effectiveness of computer-based scaffolding in the context of problem-based learning for STEM education: Bayesian meta-analysis. Educational Psychology Review, 30(2), 397–429.

    Article  Google Scholar 

  • Kotovsky, K., Hayes, J. R., & Simon, H. A. (1985). Why are some problems hard? Evidence from Tower of Hanoi. Cognitive Psychology, 17(2), 248–294.

    Article  Google Scholar 

  • Krabbe, H. (2014). Digital concept mapping for formative assessment. In D. Ifenthaler & R. Hanewald (Eds.), Digital knowledge maps in education: Technology-enhanced support for teachers and learners. Chapter 15 (pp. 275–297). New York, NY: Springer.

    Google Scholar 

  • Lavin, E. A., Giabbanelli, P. J., Stefanik, A. T., Gray, S. A., & Arlinghaus, R. (2018). Should we simulate mental models to assess whether they agree? In Proceedings of the 2018 Spring Simulation Multi-Conference, Annual Simulation Symposium (SpringSim-ANSS).

    Google Scholar 

  • Lazonder, A., & Harmsen, R. (2016). Meta-analysis of inquiry-based learning: Effects of guidance. Review of Educational Research, 87(4), 1–38.

    Google Scholar 

  • Loyens, S., & Rikers, R. (2011). Instruction based on inquiry. In Handbook of research on learning and instruction (pp. 361–381). New York, NY: Routledge.

    Google Scholar 

  • Mahe, P., & Vert, J.-P. (2009). Graph kernels based on tree patterns for molecules. Machine Learning, 75(1), 3–35.

    Article  Google Scholar 

  • Malhi, L., Karanfil, O., Merth, T., Acheson, M., Palmer, A., & Finegood, D. T. (2009). Places to intervene to make complex food systems more healthy, green, fair, and affordable. Journal of Hunger & Environmental Nutrition, 4(3–4), 466–476.

    Article  Google Scholar 

  • Meadows, D. H. (2008). Thinking in systems: A primer. Hartford, VT: Chelsea Green Publishing.

    Google Scholar 

  • Mueller, L. A. J., Dehmer, M., & Emmert-Streib, F. (2013). Comparing biological networks: A survey on graph classifying techniques. In A. Prokop & B. Csukás (Eds.), Systems biology. Dordrecht, The Netherlands: Springer.

    Google Scholar 

  • Nariman, N., & Chrispeels, J. (2015). PBL in the era of reform standards: Challenges and benefits perceived by teachers in one elementary school. Interdisciplinary Journal of Problem-Based Learning, 10(1), 5.

    Article  Google Scholar 

  • Olney, A. M., Graesser, A. C., & Person, N. K. (2012). Question generation from concept maps. Dialogue & Discourse, 3(2), 75–99.

    Article  Google Scholar 

  • Passmore, G. J. (2004). Extending the power of the concept map. Alberta Journal of Educational Research, 50(4), 370–390.

    Google Scholar 

  • Perera, D., Kay, J., Koprinska, I., Yacef, K., & Zaiane, O. R. (2008). Clustering and sequential pattern mining of online collaborative learning data. IEEE Transactions on Knowledge and Data Engineering, 21(6), 759–772.

    Article  Google Scholar 

  • Riesen, K. (2015). Structural pattern recognition with graph edit distance. In Advances in computer vision and pattern recognition. Berlin, Germany: Springer.

    Google Scholar 

  • Riesen, K., Emmenegger, S., & Bunke, H. (2013). A novel software toolkit for graph edit distance computation. Lecture Notes in Computer Science, 7877, 142–151.

    Article  Google Scholar 

  • Riesen, K., Fischer, A., & Bunke, F. (2014). Combining bipartite graph matching and beam search for graph edit distance approximation. Lecture Notes in Computer Science, 8774, 117–128.

    Article  Google Scholar 

  • Ruiz-Primo, M. (2000). On the use of concept maps as an assessment tool in science: What we have learned so far. Revista Electronica de Investigacion Educativa, 2(1), 29–52.

    Google Scholar 

  • Sanz, J., Navarro, J., Arbues, A., Martin, C., Marijuan, P. C., & Moreno, Y. (2011). The transcriptional regulatory network of Mycobacterium tuberculosis. PLoS One, 6(7), e22178.

    Article  Google Scholar 

  • Savery, J. (2006). Overview of problem-based learning: Definitions and distinctions. Interdisciplinary Journal of Problem-Based Learning, 1(1), 3.

    Article  Google Scholar 

  • Schmidt, H. G., Rotgans, J. I., & Yew, E. (2011). The process of problem-based learning: What works and why. Medical Education, 45(8), 792–806.

    Article  Google Scholar 

  • Sedki, K. (2018). Formalizing arguments from cause-effect rules. In International conference on industrial, engineering and other applications of applied intelligent systems (pp. 279–285). Basel, Switzerland: Springer.

    Google Scholar 

  • Sen, S., Li, T. J.-J., Lesicko, M., Weiland, A., Gold, R., Li, Y., … Hecht, B. (2014). WikiBrain: Democratizing computation on Wikipedia. In Proceedings of the International Symposium on Open Collaboration (OpenSym).

    Google Scholar 

  • Shervashidze, A., Vishwanathan, S. V. N., Petri, T., Melhorn, K., Borgwardt, K. (2009). Efficient graphlet kernels for large graph comparison. In Proceedings of the Twelfth International Conference on Artificial Intelligence and Statistics, PMLR 5 (pp. 488–495).

    Google Scholar 

  • Simon, H. A., & Newell, A. (1971). Human problem solving: The state of the theory in 1970. The American Psychologist, 26(2), 145–159.

    Article  Google Scholar 

  • Skyttner, L. (2006). General systems theory. River Edge, NJ: World Scientific.

    Book  Google Scholar 

  • Sole-Ribalta, A., Serratosa, F., & Sanfeliu, A. (2012). On the graph edit distance cost: Properties and applications. International Journal of Pattern Recognition and Artificial Intelligence, 26(5), 1260004.

    Article  Google Scholar 

  • Tamim, S., & Grant, M. (2013). Definitions and uses: Case study of teachers implementing project-based learning. Interdisciplinary Journal of Problem-Based Learning, 7(2), 3. https://doi.org/10.7771/1541-5015.1323

    Article  Google Scholar 

  • Tarjan, R. (1973). Enumeration of the elementary circuits of a directed graph. SIAM Journal on Computing, 2(3), 211–216.

    Article  Google Scholar 

  • Tawfik, A. A., Gill, A., Hogan, M., York, C. S., & Keene, C. W. (2018a). How novices use expert case libraries for problem solving. Technology, Knowledge and Learning, 1–18. https://doi.org/10.1007/s10758-017-9324-1

  • Tawfik, A. A., Law, V., Ge, X., Xing, W., & Kim, K. (2018b). The effect of sustained vs. faded scaffolding on students’ argumentation in ill-structured problem solving. Computers in Human Behavior, 87, 436–449.

    Article  Google Scholar 

  • Tawfik, A. A., Rong, H., & Choi, I. (2015). Failing to learn: Towards a unified design approach for failure-based learning. Educational Technology Research and Development, 63(6), 975–994.

    Article  Google Scholar 

  • Trumpower, D. L., Filiz, M., & Sarwar, G. S. (2014). Assessment for learning using digital knowledge maps. In D. Ifenthaler & R. Hanewald (Eds.), Digital knowledge maps in education: Technology-enhanced support for teachers and learners. Chapter 12 (pp. 221–237). New York, NY: Springer.

    Google Scholar 

  • Trumpower, D. L., & Goldsmith, T. E. (2004). Structural enhancement of learning. Contemporary Educational Psychology, 29, 426–446.

    Article  Google Scholar 

  • Vento, M. (2015). A long trip in the charming world of graphs for pattern recognition. Pattern Recognition, 48, 291–301.

    Article  Google Scholar 

  • Walker, A., & Leary, H. (2009). A problem based learning meta analysis: Differences across problem types, implementation types, disciplines, and assessment levels. Interdisciplinary Journal of Problem-Based Learning, 3(1), 6.

    Article  Google Scholar 

  • Weinerth, K., Koenig, V., Brunner, M., & Martin, R. (2014). Concept maps: A useful and usable tool for computer-based knowledge assessment? A literature review with a focus on usability. Computers & Education, 78, 201–209.

    Article  Google Scholar 

  • Wijnen, M., Loyens, S. M. M., Smeets, G., Kroeze, M. J., & Van der Molen, H. T. (2017). Students’ and teachers’ experiences with the implementation of problem-based learning at a university law school. Interdisciplinary Journal of Problem-Based Learning, 11(2), 5.

    Article  Google Scholar 

Download references

Acknowledgments

Vishrant K. Gupta acknowledges funding support from the College of Liberal Arts and Sciences at Northern Illinois University. The authors thank Kaspar Riesen for sharing his implementation of beam search.

Author Contributions

PJG designed the project and supervised VKG. PJG and AAT wrote and revised the manuscript. VKG wrote the software.

Author information

Authors and Affiliations

  1. Computer Science Department, Furman University, Greenville, SC, USA

    Philippe J. Giabbanelli

  2. Department of Instruction and Curriculum Leadership, University of Memphis, Memphis, TN, USA

    Andrew A. Tawfik

  3. Computer Science Department, Northern Illinois University, DeKalb, IL, USA

    Vishrant K. Gupta

Authors
  1. Philippe J. Giabbanelli
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Andrew A. Tawfik
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Vishrant K. Gupta
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Philippe J. Giabbanelli .

Editor information

Editors and Affiliations

  1. University of Mannheim, Mannheim, BW, Germany

    Dirk Ifenthaler , Dana-Kristin Mah  & Jane Yin-Kim Yau ,  & 

  2. Curtin University, Perth, WA, Australia

    Dirk Ifenthaler

Rights and permissions

Reprints and permissions

Copyright information

© 2019 Springer Nature Switzerland AG

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Giabbanelli, P.J., Tawfik, A.A., Gupta, V.K. (2019). Learning Analytics to Support Teachers’ Assessment of Problem Solving: A Novel Application for Machine Learning and Graph Algorithms. In: Ifenthaler, D., Mah, DK., Yau, J.YK. (eds) Utilizing Learning Analytics to Support Study Success. Springer, Cham. https://doi.org/10.1007/978-3-319-64792-0_11

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-64792-0_11

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-64791-3

  • Online ISBN: 978-3-319-64792-0

  • eBook Packages: EducationEducation (R0)

Publish with us

Policies and ethics

Search

Navigation