The Implementation Process of Learning Analytics
DOI:
https://doi.org/10.5944/ried.23.2.26283Keywords:
learning analytics, educational data mining, methodology, educational technologies, data-driven education, data science.Abstract
With the popularity takeoff of the learning analytics area during the last decade, numerous research studies have emerged and public opinion has echoed this trend as well. However, the fact is that the impact the field has had in practice has been quite limited, and there has been little transfer to educational institutions. One of the possible causes is the high complexity of the field, and that there are no clear implementation processes; therefore, in this work, we propose a pragmatic implementation process of learning analytics in five stages: 1) learning environments, 2) raw data capture, 3) data tidying and feature engineering, 4) analysis and modelling and 5) educational application. In addition, we also review a series of transverse factors that affect this implementation, like technology, learning sciences, privacy, institutions, and educational policies. The detailed process can be helpful for researchers, educational data analysts, teachers and educational institutions that are looking to start working in this area. Achieving the true potential of learning analytics will require close collaboration and conversation between all the actors involved in their development, which might eventually lead to the desired systematic and productive implementation.
Downloads
References
Aldowah, H., Al-Samarraie, H., & Fauzy, W. M. (2019). Educational data mining and learning analytics for 21st century higher education: A review and synthesis. Telematics and Informatics, 37(January), 13–49. https://doi.org/10.1016/j.tele.2019.01.007
Arnold, K. E., Lynch, G., Huston, D., Wong, L., Jorn, L., & Olsen, C. W. (2014). Building institutional capacities and competencies for systemic learning analytics initiatives. In Proceedings of the Fourth International Conference on Learning Analytics And Knowledge (pp. 257–260).
Berg, A., Scheffel, M., Drachsler, H., Ternier, S., & Specht, M. (2016). The dutch xAPI experience. In Proceedings of the Sixth International Conference on Learning Analytics & Knowledge (pp. 544–545).
Blikstein, P., & Worsley, M. (2016). Multimodal learning analytics and education data mining : using computational technologies to measure complex learning tasks. Journal of Learning Analytics, 3(2), 220–238. https://doi.org/http://dx.doi.org/10.18608/jla.2016.32.11
Bollen, J., Van de Sompel, H., Hagberg, A., Bettencourt, L., Chute, R., Rodriguez, M. A., & Balakireva, L. (2009). Clickstream data yields high-resolution Maps of science. PLoS ONE, 4(3). https://doi.org/10.1371/journal.pone.0004803
Breslow, L., Pritchard, D. E., DeBoer, J., Stump, G. S., Ho, A. D., & Seaton, D. T. (2013). Studying Learning in the Worldwide Classroom: Research into edX’s First MOOC. Research and Practice in Assessment, 8, 13–25.
Campbell, J. P., & Oblinger, D. G. (2007). Academic Analytics. Educause Review, (October), 1–20. Retrieved from https://er.educause.edu/articles/2007/7/academic-analytics-a-new-tool-for-a-new-era
Chatti, M. A., Dyckhoff, A. L., Schroeder, U., & Thüs, H. (2012). A reference model for learning analytics. International Journal of Technology Enhanced Learning, 4(5/6), 318. https://doi.org/10.1504/IJTEL.2012.051815
Chen, C. M., Lee, H. M., & Chen, Y. H. (2005). Personalized e-learning system using Item Response Theory. Computers and Education, 44(3), 237–255. https://doi.org/10.1016/j.compedu.2004.01.006
Clow, D. (2012). The learning analytics cycle: closing the loop effectively. In Proceedings of the 2nd International Conference on Learning Analytics and Knowledge (pp. 134–138). https://doi.org/10.1145/2330601.2330636
Daniel, J. (2012). Making Sense of MOOCs: Musings in a Maze of Myth, Paradox and Possibility. Journal of Interactive Media in Education, Perspectiv. https://doi.org/10.1145/2316936.2316953
de Laat, M., & Schreurs, B. (2013). Visualizing Informal Professional Development Networks: Building a Case for Learning Analytics in the Workplace. American Behavioral Scientist, 57(10), 1421–1438. https://doi.org/10.1177/0002764213479364
Demchenko, Y., Laat, C. De, Membreel A, & Al., E. (2014). Defining architecture components of the Big Data Ecosystem. In Collaboration Technologies and Systems (CTS). In International Conference on IEE (p. 104:112). Retrieved from http://www.uazone.org/demch/papers/bddac2014-bd-ecosystem-archi-v05.pdf
Drachsler, H., & Kalz, M. (2016). The MOOC and learning analytics innovation cycle (MOLAC): A reflective summary of ongoing research and its challenges. Journal of Computer Assisted Learning, 32(3), 281–290. https://doi.org/10.1111/jcal.12135
Drachsler, Hendrik, & Greller, W. (2016). Privacy and analytics - it’s a DELICATE issue a checklist for trusted learning analytics. In Proceedings of the sixth international conference on learning analytics & knowledge (pp. 89–98). ACM. https://doi.org/10.1145/2883851.2883893
Ferguson, R. (2012). Learning analytics: drivers, developments and challenges. International Journal of Technology Enhanced Learning, 4(5–6), 304–317. https://doi.org/10.1504/IJTEL.2012.051816
Freire, M., Serrano-laguna, Á., Iglesias, B. M., Martínez-ortiz, I., Moreno-ger, P., & Fernández-manjón, B. (2016). Game Learning Analytics: Learning Analytics for Serious Games. In Learning, Design, and Technology: An International Compendium of Theory, Research, Practice, and Policy (pp. 1–29). Springer. https://doi.org/10.1007/978-3-319-17727-4
Friend Wise, A., & Williamson Schaffer, D. (2015). Why theory matters more than ever in the age of big data. Journal of Learning Analytics, 2(2), 5–13. https://doi.org/10.18608/jla.2015.22.2
Gasevic, D., Dawson, S., Mirriahi, N., & Long, P. D. (2015). Learning Analytics – A Growing Field and Community Engagement. Journal of Learning Analytics, 2(1), 1–6. https://doi.org/10.18608/jla.2015.21.1
Gašević, D., Dawson, S., & Siemens, G. (2015). Let’s not forget : Learning analytics are about learning. TechTrends, 59(1), 64–71. https://doi.org/10.1007/s11528-014-0822-x
Greller, W., & Drachsler, H. (2012). Translating learning into numbers: A generic framework for learning analytics. Educational Technology and Society, 15(3), 42–57.
Hoel, T., & Chen, W. (2014). Learning Analytics Interoperability - Looking for low-hanging fruits. In Workshop Proceedings of the 22nd International Conference on Computers in Education, ICCE 2014 (pp. 253–263).
Hsiao, I.-H., Huang, P.-K., & Murphy, H. (2017). Integrating programming learning analytics across physical and digital space. IEEE Transactions on Emerging Topics in Computing.
Jaques, N., Conati, C., Harley, J., & Azevedo, R. (2014). Predicting Affect from Gaze Data during Interaction with an Intelligent Tutoring System. In Proceedings 12th International Conference, Intelligent Tutoring Systems 2014, Honolulu, HI, USA. (pp. 29–38). Springer International Publishing.
https://doi.org/10.1007/978-3-319-07221-0_4
Jobs, S. (2005). You’ve got to find what you love. Stanford News. Retrieved from http://news.stanford.edu/news/2005/june15/jobs-061505.html (last accessed December 29, 2019)
Kaul, A., Maheshwary, S., & Pudi, V. (2017). Autolearn—Automated feature generation and selection. In 2017 IEEE International Conference on Data Mining (ICDM) (pp. 217–226).
Kizilcec, R. F., & Cohen, G. L. (2017). Eight-minute self-regulation intervention raises educational attainment at scale in individualist but not collectivist cultures. Proceedings of the National Academy of Sciences, 114(17), 4348–4353. https://doi.org/10.1073/pnas.1611898114
Koedinger, K. R., Kim, J., Jia, J. Z., McLaughlin, E. A., & Bier, N. L. (2015). Learning is not a spectator sport: doing is better than watching for learning from a MOOC. In Proceedings of the Second ACM Conference on Learning@Scale (pp. 111–120). https://doi.org/10.1145/2724660.2724681
Leitner, P., Khalil, M., & Ebner, M. (2017). Learning Analytics: Fundaments, Applications, and Trends. Learning Analytics: Fundaments, Applications, and Trends, Studies in Systems, Decision and Control, 94(February), 1–23. https://doi.org/10.1007/978-3-319-52977-6
Lopez, G., Seaton, D. T., Ang, A., Tingley, D., & Chuang, I. (2017). Google BigQuery for education: Framework for parsing and analyzing edX MOOC data. In L@S 2017 - Proceedings of the 4th (2017) ACM Conference on Learning at Scale (pp. 181–184). https://doi.org/10.1145/3051457.3053980
Macarini, L. A., Ochoa, X., Cechinel, C., Rodés, V., Dos Santos, H. L.,
Alonso, G. E., … Díaz, P. (2019). Challenges on implementing learning analytics over countrywide K-12 data. ACM International Conference Proceeding Series, 441–445. https://doi.org/10.1145/3303772.3303819
Mangaroska, K., Vesin, B., & Giannakos, M. (2019). Cross-platform analytics: A step towards personalization and adaptation in education. In 9th International Learning Analytics & Knowledge Conference (LAK19) (pp. 71–75). https://doi.org/10.1145/3303772.3303825
Milligan, C., Littlejohn, A., & Margaryan, A. (2013). Patterns of engagement in connectivist MOOCs. MERLOT Journal of Online Learning and Teaching, 9(2), 149–159.
Monroy, C., Snodgrass Rangel, V., & Whitaker, R. (2014). A Strategy for Incorporating Learning Analytics into the Design and Evaluation of a K-12 Science Curriculum. Journal of Learning Analytics, 1(2), 94–125. https://doi.org/10.18608/jla.2014.12.6
Paquette, L., Carvalho, A. M. J. a De, Baker, R., & Ocumpaugh, J. (2014). Reengineering the Feature Distillation Process : A Case Study in the Detection of Gaming the System. Proceedings of the 7th International Conference on Educational Data Mining (EDM), 284–287.
Pardo, A., & Siemens, G. (2014). Ethical and privacy principles for learning analytics. British Journal of Educational Technology, 45(3), 438–450. https://doi.org/10.1111/bjet.12152
Park, Y., & Jo, I. (2015). Development of the Learning Analytics Dashboard to Support Students ’ Learning Performance Learning Analytics Dashboards ( LADs ). Journal of Universal Computer Science, 21(1), 110–133.
Prinsloo, P., Archer, E., Barnes, G., Chetty, Y., & Zyl, D. Van. (2019). Big(ger) Data as Better Data in Open Distance Learning Paul. International Review of Research in Open and Distributed Learning, 16(1), 284–306.
R, S., V.Vaidhehi, V. V., & Easwaran Iyer, N. (2015). Survey of Learning Analytics based on Purpose and Techniques for Improving Student Performance. International Journal of Computer Applications, 111(1), 22–26. https://doi.org/10.5120/19502-1097
Ramasubramanian, K., & Singh, A. (2017). Feature Engineering. In Machine Learning Using R (pp. 181–217). Springer.
Redondo, D., Muñoz-Merino, P. J., Ruipérez-Valiente, J. A., Delgado Kloos, C., Pijeira Díaz, H. J., & Santofimia Ruiz, J. (2015). Combining Learning Analytics and the Flipped Classroom in a MOOC of maths. In International Workshop on Applied and Practical Learning Analytics (pp. 71–79). Retrieved from http://ceur-ws.org/Vol-1599/9WAPLA_2015.pdf
Reich, J., & Ruipérez-Valiente, J. A. (2019). The MOOC pivot. Science, 363(6423), 130–131. https://doi.org/10.1126/science.aav7958
Rezaei, M. S., & Yaraghtalaie, M. (2019). Next learning topic prediction for learner’s guidance in informal learning environment. International Journal of Technology Enhanced Learning, 11(1), 62–70. https://doi.org/10.1504/IJTEL.2019.096738
Romero, C., Ventura, S., & García, E. (2008). Data mining in course management systems: Moodle case study and tutorial. Computers & Education, 51(1), 368–384. https://doi.org/10.1016/j.compedu.2007.05.016
Ros, S., Hernández, R., Robles-Gomez, A., Caminero, A. C., Tobarra, L., & Ruíz, E. S. (2013). Open service-oriented platforms for personal learning environments. IEEE Internet Computing, 17(4), 26–31.
Ruiperez-Valiente, J.A., Munoz-Merino, P. J., Gascon-Pinedo, J. A., & Kloos, C. D. (2016). Scaling to Massiveness With ANALYSE: A Learning Analytics Tool for Open edX. IEEE Transactions on Human-Machine Systems. https://doi.org/10.1109/THMS.2016.2630420
Ruipérez-Valiente, J.A., Muñoz-Merino, P. J., Leony, D., & Delgado Kloos, C. (2015). ALAS-KA: A learning analytics extension for better understanding the learning process in the Khan Academy platform. Computers in Human Behavior, 47. https://doi.org/10.1016/j.chb.2014.07.002
Ruiperez-Valiente, Jose A., Cobos, R., Muñoz-Merino, P. J., Andújar, Á., & Delgado Kloos, C. (2017). Early Prediction and Variable Importance of Certificate Accomplishment in a MOOC. In Fifth European MOOCs Stakeholders Summit. Madrid, Spain.
Ruipérez-Valiente, José A, Muñoz-Merino, P. J., & Delgado Kloos, C. (2017). Detecting and Clustering Students by their Gamification Behavior with Badges: A Case Study in Engineering Education. International Journal of Engineering Education, 33(2-B), 816–830.
Ruipérez-Valiente, José A, Muñoz-Merino, P. J., Gascón-Pinedo, J. A., &
Delgado Kloos, C. (2016). Scaling to Massiveness with ANALYSE: A Learning Analytics Tool for Open edX. IEEE Transactions on Human-Machine Systems, 47(6), 909–914.
https://doi.org/10.1109/THMS.2016.2630420
Serrano-Laguna, Á., Martínez-Ortiz, I., Haag, J., Regan, D., Johnson, A., & Fernández-Manjón, B. (2017). Applying standards to systematize learning analytics in serious games. Computer Standards & Interfaces, 50, 116–123.
Shute, V., & Zapata-Rivera, D. (2012). Adaptive educational systems. In Adaptive Technologies for Training and Education (pp. 7–27). Retrieved from http://books.google.com/books?hl=en&lr=&id=mOlPSl6iJaIC&oi=fnd&pg=PA7&dq=Adaptive+Educational+Systems&ots=i9wDCR_zkc&sig=Sb-l4gFHsItZup0ZpSuUYZRCBKc
Siemens, G., & Baker, R. S. J. d. (2012). Learning Analytics and Educational Data Mining: Torwards Communication and Collaboration. Proceedings of the 2nd International Conference on Learning Analytics and Knowledge - LAK ’12, 252–254. https://doi.org/10.1145/2330601.2330661
Slade, S., & Prinsloo, P. (2013). Learning Analytics: Ethical Issues and Dilemmas. American Behavioral Scientist, 57(10), 1510–1529. https://doi.org/10.1177/0002764213479366
Thomas, S. (2016). Future Ready Learning. Reimagining the Role of Technology in Education. Office of Educational Technology, US Department of Education. ERIC.
Tsai, Y. S., & Gasevic, D. (2017). Learning analytics in higher education - Challenges and policies: A review of eight learning analytics policies. In Seventh international learning analytics & knowledge conference (pp. 233–242). https://doi.org/10.1145/3027385.3027400
van der Zee, T.,& Reich, J. (2018). Open Education Science. AERA Open, 4(3), 2332858418787466.Veeramachaneni, K., Reilly, U. O., & Taylor, C. (2014). Towards feature engineering at scale for data from massive open online courses. ArXiv Preprint ArXiv:1407.5238.
Viberg, O., Hatakka, M., Bälter, O., & Mavroudi, A. (2018). The current landscape of learning analytics in higher education. Computers in Human Behavior, 89(October 2017), 98–110. https://doi.org/10.1016/j.chb.2018.07.027
Wang, S., & Wu, C. (2011). Application of context-aware and personalized recommendation to implement an adaptive ubiquitous learning system. Expert Systems with Applications, 38(9), 10831–10838. https://doi.org/10.1016/j.eswa.2011.02.083
Wolff, A., Moore, J., Zdrahal, Z., Hlosta, M., & Kuzilek, J. (2016). Data literacy for learning analytics. In Proceedings of the Sixth International Conference on Learning Analytics & Knowledge (pp. 500–501). ACM. https://doi.org/10.1145/2883851.2883864
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2020 RIED. Revista Iberoamericana de Educación a Distancia
This work is licensed under a Creative Commons Attribution 4.0 International License.
The articles that are published in this journal are subject to the following terms:
1. The authors grant the exploitation rights of the work accepted for publication to RIED, guarantee to the journal the right to be the first publication of research understaken and permit the journal to distribute the work published under the license indicated in point 2.
2. The articles are published in the electronic edition of the journal under a Creative Commons Attribution 4.0 International (CC BY 4.0) license. You can copy and redistribute the material in any medium or format, adapt, remix, transform, and build upon the material for any purpose, even commercially. You must give appropriate credit, provide a link to the license, and indicate if changes were made. You may do so in any reasonable manner, but not in any way that suggests the licensor endorses you or your use.
3. Conditions for self-archiving. Authors are encouraged to disseminate electronically the OnlineFirst version (assessed version and accepted for publication) of its articles before publication, always with reference to its publication by RIED, favoring its circulation and dissemination earlier and with this a possible increase in its citation and reach among the academic community.
