Evaluating the efficiency of social learning networks: Perspectives for harnessing learning analytics to improve discussions

https://doi.org/10.1016/j.compedu.2021.104124Get rights and content

Highlights

  • Implements a SLN optimizer that calculates efficiency as the ratio of optimized to observed network connections.

  • The algorithm operationalizes SLN as a function of knowledge disseminating and seeking behaviors.

  • We employ the algorithm to assess the SLN in video lecture based massive open online course (MOOC).

  • We observe low learning benefits overall in the discussion forum studied, yet the optimizer finds efficiencies.

  • We validate the optimized network by invoking a measure of the informativeness of the overall network.

Abstract

This study evaluates the validity of an algorithm for measuring the efficiency of social learning networks in discussion forums accompanying MOOCs of conventional format, which consist of video lectures and problem assignments. The algorithm models social learning networks as a function of users knowledge seeking and knowledge disseminating tendencies across course topics and offers a means to optimize social learning networks by connecting users seeking and disseminating information on specific topics. We use the algorithm to analyze the social learning network manifest in the discussion format of a MOOC forum incorporating video lectures and problem assignments. As a measure of the degree that knowledge seekers and knowledge disseminators are connected in the network, we observe a very sparse network with few discussion participants and a limited range of topics. Hence, only small gains are available through optimization, since for a very sparse network, few connections can be made. The development of a metric for the analysis of social learning networks would provide instructors and researchers with a means to optimize online learning environments for empowering social learning. Finally, we discuss our findings with respect to the potential of self-optimizing discussion forums for supporting social learning online.

Introduction

Social learning networks are ubiquitous in our contemporary hyper-connected reality. They are built into our online networks, as forums, faqs, mailing lists, news feeds and social feeds, etc. The internet has been celebrated as a means to leverage the power of the crowd and cheap processing power to bring mass education online by facilitating access and dissemination of information. The promise is to empower human ingenuity through collective creation. MOOCs represent the course model with the largest reach worldwide. MOOCs follow a format that includes video lecture, worked problems and quizzes, and the occasional case study and project assignments. However, some MOOCs do follow alternate designs and successfully incorporate discussion and collaborative learning.

MOOCs are offered regularly for credit but often the materials are made freely available for self-paced study as well. Thus, they can be considered multi-modal social learning networks. There are two primary modes to consume the course, actively, in interaction with others, or passively, on one's own time. Indeed, “passive” online learning has received less attention though statistics demonstrate that most users consume the internet passively rather than actively contribute to content creation; a phenomenon known as participation inequality (Nielsen, 2006). Yet “passive” does not mean that learning does not occur. In fact, passive learning is too general a concept to be useful, it masks other distinctions, concerning for instance, the goal directedness of the behavior. One form of online learning that can be considered “passive” concerns the use of knowledge sharing forums and question and answer sites such as Stack Overflow in the programming community. Whether one actively contributes through forming questions and posting answers, a user can still extract a learning benefit from having their own personal inquiries satisfied by searching and reading posts. Thus, a social learning network affords multiple learning modes, as producer and consumer of knowledge.

Perhaps more than the transmission of knowledge, the potential of social learning networks lies in its powerful extension of vicarious learning (Bandura, 1986) to global dimensions. Through online learning, we can learn by pooling our experience together and collectively build on earlier successes through the co-construction of common knowledge (Vygotsky, 1978). Discussion forums are constructivist educational technology (Lemay & Doleck, 2020a), or technology forms and informs learning as the medium that creates the affordances for learning. Yet the use of live, seminar-style discussion forums remains limited (Dennen & Wieland, 2007) perhaps because they require a good deal of planning to successfully orchestrate and are generally short-lived and contextually-bound (Sun & Chen, 2016). As a digital artifact, a discussion forum can continue to inform a social learning network.

Research to help improve discussion forums online can help contribute to the success of MOOCs and empower social learning network effects. Measuring social learning network efficiency would provide feedback to optimize knowledge sharing and cultivate vicarious online learning. Researchers have begun to analyze social learning networks using tools developed in engineering fields for network and signal analysis. Brinton et al. (2018) have proposed an algorithm for evaluating and optimizing social learning networks. In the present study, we implement their algorithm and use it to evaluate the efficiency of a MOOC developed using a video lecture and worked problem instructional format.

We asked “Can Brinton et al.'s (2018) algorithm for measuring social learning efficiency be used to evaluate the efficiency of a social learning network manifest in a MOOC employing video lectures and worked problem assignments?”

We shall briefly review social learning theory and discuss related studies of discussion forums in online learning before presenting Brinton et al.'s (2018) social learning network optimization algorithm.

Section snippets

Social learning

Social learning theories (Bandura, 1977; Wenger, 1999) emphasize the role of collective behavior, especially in regards to learning in a social context. In social learning, knowledge is constructed and shared among members of the group. Indeed, one of the key requirements for a process to be considered social learning is that it must occur through social interactions (Reed et al., 2010). The construct of social learning reflects the importance of discussion as a means of building and supporting

Research design

The study is confirmatory in nature as it seeks to assess the validity of a theoretical model of social learning to evaluate the efficiency of social learning networks as manifested in a MOOC developed for the EdX course platform.

Context

The data are taken from an EdX course offered in the summer of 2015 entitled Big Data and Education. The course had over 10,432 registered users but only 519 active forum users. Learners were distributed globally and had a range of backgrounds. Many were professionals

Weighted adjacency network

Of the 10,432 registered users, only 519 were actively engaged in the forum. These numbers suggest that this MOOC's discussion forum did not serve an important role in student learning. Thus, the social learning network was not expected to be very efficient in this particular iteration of the MOOC format.

Topic inference

Topics were inferred using latent Dirichlet allocation (LDA; Blei et al., 2003) to estimate the probability of topic associations based on patterns of word occurrences. From the LDA analysis,

Discussion

With an average learning benefit of 1.48, the social learning network was manifestly inefficient prior to optimization. However, even following optimization, at only 20.73, the network would still be inefficient in its potential learning gain. How do we interpret these results? It is important to remember that this network only comprised 1/20 of the overall course registration and the topics were heavily weighted toward administrative topics like personal introductions and course information

Declaration of competing interest

None.

Acknowledgments

We wish to thank Dr. Ryan Baker and his Learning Analytics Lab for this collaboration and providing the performance data from his Big Data and Education MOOC from the University of Pennsylvania. David John Lemay was awarded a postdoctoral fellowship (56-2019-0651) from the Social Sciences and Humanities Research Council, Canada.

References (70)

  • M.W. Aulls et al.

    Inquiry in education

  • A. Bandura
    (1977)
  • A. Bandura

    Social foundations of thought and action: A social cognitive theory

    (1986)
  • Y. Bergner et al.

    Methodological challenges in the analysis of MOOC data for exploring the relationship between discussion forum views and learning outcomes

  • D.M. Blei et al.

    Latent dirichlet allocation

    Journal of Machine Learning Research

    (2003)
  • S. Boyd et al.

    Convex optimization

    (2004)
  • C. Brinton et al.

    On the efficiency of online social learning networks

    IEEE/ACM Transactions on Networking

    (2018)
  • C. Brinton et al.

    Learning about social learning in MOOCs: From statistical analysis to generative model

    IEEE Transactions On Learning Technologies

    (2014)
  • S. Buckingham Shum et al.

    Social learning analytics

    Educational Technology & Society

    (2012)
  • S. Butler et al.

    Spectral graph theory

  • T.K. Chiu et al.

    Factors influencing peer learning and performance in MOOC asynchronous online discussion forum

    Australasian Journal of Educational Technology

    (2018)
  • V. Dennen et al.

    From Interaction to Intersubjectivity: Facilitating online group discourse processes

    Distance Education

    (2007)
  • N.M. Dowell et al.

    Modeling learners’ social centrality and performance through language and discourse

  • A. Ezen-Can et al.

    March). Unsupervised modeling for understanding MOOC discussion forums: A learning analytics approach

  • Y. Feng et al.

    The impact of students and TAs' participation on students' academic performance in mooc

  • J. Gardner et al.

    Student success prediction in MOOCs

    User Modeling and User-Adapted Interaction

    (2018)
  • P. Gorsky et al.

    Dialogue: A theoretical framework for distance education instructional systems

    British Journal of Educational Technology

    (2005)
  • M. Hammond

    A review of recent papers on online discussion in teaching and learning in higher education

    Journal of Asynchronous Learning Networks

    (2005)
  • N. Hara et al.

    Content analysis of online discussion in an applied educational psychology course

    Instructional Science

    (2000)
  • J. Hill et al.

    Social learning theory and web-based learning environments: A review of research and discussion of implications

    American Journal of Distance Education

    (2009)
  • S.L. Houston et al.

    Pass the idea please: The relationship between network position, direct engagement, and course performance in MOOCs

  • S. Jiang et al.

    Social positioning and performance in MOOCs

  • S. Jiang et al.

    Predicting MOOC performance with week 1 behavior

  • S. Joksimović et al.

    April). Translating network position into performance: Importance of centrality in different network configurations

  • S. Joksimović et al.

    How do we model learning at scale? A systematic review of research on MOOCs

    Review of Educational Research

    (2017)
  • Cited by (0)

    View full text