Learner analytics: Hindsight evaluation at course-level

Authors

DOI:

https://doi.org/10.7203/realia.31.25526

Keywords:

Learner analytics, student engagement, attainment, retention, higher education

Abstract

The concept of student engagement is a contentious construct. The task of learner analytics (LA) to meaningfully measure student engagement is therefore complicated both by a lack of agreement over what is being measured and a discomfort or lack of confidence around what collated data might believably indicate. This challenge is made harder by availability, accuracy and reliability of data feeds. The aim of LA would be to collate and share early measures of engagement that can be used predictively to support learners’ experience and outcomes. However, most HEI LA are descriptive and therefore of limited utility. Where the LA available are descriptive, this paper explores how credible such LA might be when used at course level.   This study supports an analysis of comprehensive data gathered within and beyond LA for a level 4 cohort in one programme across the 2019-20 academic year. It also draws on data relating to study completion, with the benefit of hindsight giving further insights to the utility of LA data available earlier in students’ journeys. Given the actual outcomes for these 2019 starters, the study cohort’s understanding of ‘engagement’ is then applied to support insights to their own measurable indicators of interaction and actions that might best enable constructive engagement. Meaningful correlations were noted between use of E-resources and student outcomes and the most significant indicators of risk were found to be extensions, fails and non-submissions for assignments in the first semester of level 4 and average grades <39% by the end of level 4. Study recommendations include supporting better and more confident access to literature content and targeting timely interventions at students flagged by the most significant indicators of risk.

Downloads

Download data is not yet available.

Author Biography

Rachel Cliodhna Bassett-Dubsky, University of Northampton

Senior Lecturer - Childhood, Youth and Families Student Experience Lead - Faculty of Health, Education and Society

References

Agudo-Peregrina, A., Iglesias-Pradas, S., Conde-Gonzalez, M., & Hernandez-Garcia, A. (2014). Can we predict success from log data in VLEs? Classification of interactions for learning analytics and their relation with performance in VLE-supported F2F and online learning. Computers in Human Behavior, 31, 542–550. https://doi.org/10.1016/ j.chb.2013.05.031


Bassett-Dubsky, R. (2020). Student engagements project blog. [Blog post]. Retrieved from https://mypad.northampton.ac.uk/rdubsk/


Bassett-Dubsky, R. (2021). How has Covid-19 shifted how we support, recognise and measure student engagement? In S. Studente, S. Ellis, & B. Desai (Eds.), The Impact of COVID-19 on teaching and learning in Higher Education (pp. 151–173). Nova Science Publishers.


Beetham, H., Collier, A., Czerniewicz, L., Lamb, B., Lin, Y., Ross, J., … Wilson, A. (2022).    Surveillance Practices, Risks and Responses in the Post Pandemic University. Digital Culture & Education, 14(1), 16–37. Retrieved from https:// www.digitalcultureandeducation.com/volume-141-papers/beetham-2022


Blundell, R., Costa-Dias, M., Cribb, J., Joyce, R., Waters, T., Wernham, T., & Xu, X. (2022). Inequality and the Covid-19 crisis in the United Kingdom. Annual review of Economics, 14, 607–636. https://doi.org/10.1146/annurev-economics-051520-030252


Bond, M., Buntins, K., Bedenlier, S., Zawacki-Richter, O., & Kerres, M. (2020). Mapping research in student engagement and educational technology in higher education: a systematic evidence map. International Journal of Educational Technology in Higher Education, 17 (2). https://doi.org/10.1186/s41239-019-0176-8


Braun, V., & Clarke, V. (2022). Thematic Analysis: A practical guide. London: Sage.

 

Broughan, C., & Prinsloo, P. (2020). ‘(Re)centring students in learning analytics: in conversation with Paulo Freire. Assessment & Evaluation in Higher Education, 45(4), 617–628. https://doi.org/10.1080/02602938.2019.1679716


Bunce, L., King, N., Saran, S., & Talib, N. (2021). Experiences of black and minority ethnic (BME) students in higher education: applying self-determination theory to understand the BME attainment gap. Studies in Higher Education, 46(3), 534–547. https://doi.org/ 10.1080/03075079.2019.1643305


De Freitas, S., Gibson, D., Plessis, C. D., Halloran, P., Williams, E., Ambrose, M., … Arnab, S. (2015). Foundations of dynamic learning analytics: Using university student data to increase retention. British Journal of Educational Technology, 46(6), 1175–1188. https:// doi.org/10.1111/bjet.12212


Dommett, E., Gardner, B., & Van Tilburg, W. (2019). Staff and student views of lecture capture: a qualitative study. International Journal of Educational Technology in Higher Education, 16(23), 16–16. https://doi.org/10.1186/s41239-019-0153-2


Dyment, J., Stone, C., & Milthorpe, N.    (2020).    Beyond busy work: rethinking the measurement of online student engagement.Higher Education Research & Development, 39(7), 1440–1453. https://doi.org/10.1080/07294360.2020.1732879

Gravett, K., Kinchin, I., & Winstone, N. (2020). Frailty in transition? Troubling the norms, boundaries and limitations of transition theory and practice. Higher Education Research & Development, 39(6), 1169–1185.https://doi.org/10.1080/07294360.2020.1721442


GSU. (2019). Leading with predictive analytics . Retrieved from https://success.gsu.edu/ approach/


Hanover-Research. (2016, November). Learning Analytics for tracking student progress.Retrieved February 2023, 22, from https://www.imperial.edu/docs/research-planning/ 7932-learning-analytics-for-tracking-student-progress/file


Herodotou, C., Rienties, B., Boroowa, A., & Zdráhal, Z. (2019). A large-scale implementation of predictive learning analytics in higher education: the teachers’ role and perspective. Educational Technology Research and Development, 67 (2). http://dx.doi.org/10.1007/ s11423-019-09685-0


HESA. (2023, 01 31). Who’s studying in HE?: Personal characteristics . Retrieved from https:// www.hesa.ac.uk/data-and-analysis/students/whos-in-he/characteristics


Kika, C., Duan, Y., & Cao, G. (2016). Understanding the use and impact of learner analytics on Student Experience management in the UK Higher Education sector. In and others (Ed.), PACIS 2016 proceedings (pp. 54–54). Pacific Asia Conference on Information Systems. Retrieved from http://hdl.handle.net/10547/624450


Korhonen, V. (2012). Towards Inclusive Higher Education? - Outlining a Student-centred Counselling Framework for Strengthening Student Engagement. In S. Stolz & P. Gonon (Eds.), Challenges and Reforms in Vocational Education - Aspects of Inclusion and Exclusion (pp. 297–320). Bern: Peter Lang.


Korhonen, V., Mattsson, M., Inkinen, M., & Toom, A.    (2019).    Understanding the multidimensional nature of Student Engagement during the first year of Higher Education. Frontiers in Psychology, 10(1056), 1–15. https://doi.org/10.3389/fpsyg.2019.01056


Mathrani, A., Susnjak, T., Ramaswami, G., & Barczak, A. (2021). Perspectives on the challenges of generalisability, transparency and ethics in predictive learning analytics. Computers and Education Open, 2. https://doi.org/10.1016/j.caeo.2021.100060


Morgan, M. (2019, November). Presentation to SEDA conference.


Ofs. (2022). OfS sets news expectations for Student Outcomes . Retrieved from https:// www.officeforstudents.org.uk/news-blog-and-events/press-and-media/ofs-sets-new-expectations-for-student-outcomes/


Oldfield, J., Rodwell, J., Curry, L., & Marks, G. (2018). Psychological and demographic predictors of undergraduate non-attendance at university lectures and seminars. Journal of Further and Higher Education, 42(4), 509–523. https://doi.org/10.1080/ 0309877X.2017.1301404


Parkes, S., Benkwitz, A., Bardy, H., Myler, K., & Peters, J. (2020). Being more human: rooting learning analytics through resistance and reconnection with the values of higher education. Higher Education Research and Development, 39(1), 113–126. https://doi.org/10.1080/07294360.2019.1677569


Robertshaw, M. B., & Asher, A. (2019). Unethical Numbers? A Meta-Analysis of Library Learning Analytics Studies. Library Trends, 68(1), 76–101. https://doi.org/10.1353/ lib.2019.0031


Summers, R., Higson, H., & Moores, E. (2020). Measures of engagement in the first three weeks of higher education predict subsequent activity and attainment in first year undergraduate students: a UK case study. Assessment & Evaluation in Higher Education, 46(5), 821–836. https://doi.org/10.1080/02602938.2020.1822282


Susnjak, T., Suganya-Ramaswami, G., & Mathrani, A. (2022). Learning analytics dashboard: a tool for providing actionable insights to learners. International Journal of Educational Technology in Higher Education, 19(12). https://doi.org/10.1186/s41239-021-00313-7


Thorpe, A., Lukes, R., Bever, D., & He, Y.  (2016). The Impact of the Academic Library on Student Success: Connecting the Dots. Portal: Libraries and the Academy, 16(2), 373– 392. https://doi.org/10.1353/pla.2016.0027


Tobbell, J., Burton, R., Gaynor, A., Golding, B., Greenhough, K., Rhodes, C., & White, S. (2021). Inclusion in higher education: an exploration of the subjective experiences of students. Journal of Further and Higher Education, 45(2), 284–295. https://doi.org/ 10.1080/0309877X.2020.1753180


Venn, E., Park, J., Palle-Anderson, L., & Hejmadi, M. (2020). How do learning technologies impact on undergraduates’ emotional and cognitive engagement with their learning? Teaching in Higher Education, 28(4), 822–839. https://doi.org/10.1080/13562517.2020.1863349


Viberg, O., Hatakka, M., Bälter, O., & Mavroudi, A. (2018). The current landscape of learning analytics in higher education. Computers in Human Behavior, 89, 98–110. https:// doi.org/10.1016/j.chb.2018.07.027


Zepke, N. (2018). Student engagement in neo-liberal times: what is missing? Higher Education Research & Development, 37 (2), 433–446. https://doi.org/10.1080/07294360.2017.1370440

Downloads

Published

2023-07-26

How to Cite

Bassett-Dubsky, R. C. (2023). Learner analytics: Hindsight evaluation at course-level. Research in Education and Learning Innovation Archives, (31), 1–16. https://doi.org/10.7203/realia.31.25526
Metrics
Views/Downloads
  • Abstract
    962
  • PDF
    467
  • EPUB
    283
  • HTML
    539

Issue

No. 31 (2023)

Section

ARTICLES

License

Authors who publish with this journal agree to the following terms:

  1. The Universitat de València is the editor of Research in Education and Learning Innovation Archives (REALIA) and retains the copyright of all that is published in the journal, while it allows and favours reuse under a copyleft license.
  2. The originals published in this journal are – unless otherwise specified – under a Creative Commons License: Attribution-NonComercial-ShareAlike 4.0 International (CC BY-NC-SA 4.0).

Research in Education and Learning Innovation Archives (REALIA) encourages authors to disseminate and give visibility to their research published in this journal. For this, REALIA wishes to inform that:

  1. Authors retain copyright and grant the journal right of first publication.
  2. The work is simultaneously licensed under a Creative Commons Attribution License: Attribution-NonComercial-ShareAlike 4.0 International (CC BY-NC-SA 4.0).
  3. Self-archive: Authors are permitted and encouraged to post their work online (e.g. in institutional repositories or on their website) to promote greater visibility and citation, with an acknowledgement of its first publication in this journal.

Metrics

Similar Articles

<< < 1 2 3 4 5 6 7 8 > >> 

You may also start an advanced similarity search for this article.