Data-driven educational algorithms pedagogical framing
DOI:
https://doi.org/10.5944/ried.23.2.26470Palavras-chave:
teaching practice, learning conditions, sciences of education, experimental education, educational research, electronic data processingResumo
Data from students and learning practices are essential for feeding the artificial intelligence systems used in education. Recurrent data trains the algorithms so that they can be adapted to new situations, either to optimize coursework or to manage repetitive tasks. As the algorithms spread in different learning contexts and the actions which they perform expand, pedagogical interpretative frameworks are required to use them properly. Based on case analyses and a literature review, the paper analyses the limits of learning practices based on the massive use of data from a pedagogical approach. The focus is on data capture, biases associated with datasets, and human intervention both in the training of artificial intelligence algorithms and in the design of machine learning pipelines. In order to facilitate the adequate use of data-driven learning practices, it is proposed to frame appropriate heuristics to determine the pedagogical suitability of artificial intelligence systems and also their evaluation both in terms of accountability and of the quality of the teaching-learning process. Thus, finally, a set of top-down proposed rules that can contribute to fill the identified gaps to improve the educational use of data-driven educational algorithms is discussed.
Downloads
Referências
Alsuwaiket, M., Blasi, A.H., & Al-Msie’deen, R.A. (2019). Formulating module assessment for improved academic performance predictability in higher education. Engineering, Technology & Applied Science Research, 9(3), 4287–4291. Retrieved from https://www.etasr.com/index.php/ETASR/article/view/2794/pdf
Amo, D., Fonseca, D., Alier, M., García-Peñalvo, F. J., Casañ, M. J., & Alsina, M. (2019). Personal data broker: A solution to assure data privacy in EdTech. In P. Zaphiris & A. Ioannou (Eds.), Learning and collaboration technologies. Design, experiences. 6th International Conference, LCT 2019, Held as Part of the 21st HCI International Conference, HCII 2019, Orlando, FL, USA, July 26–31, 2019. Proceedings, Part I (pp. 3–14). Cham, Switzerland: Springer Nature.
Ball, S. J., Bowe, R., & Gewirtz, S. (1995). Circuits of schooling: a sociological exploration of parental choice of school in social class contexts. The Sociological Review, 43(1), 52–78.
Bodily, R., & Verbert, K. (2017). Review of research on student-facing learning analytics dashboards and educational recommender systems. IEEE Transactions on Learning Technologies, 10(4), 405–418. Retrieved from https://dblp.org/rec/journals/tlt/BodilyV17
Boyd, D., & Crawford, K. (2012). Critical questions for big data. Provocations for a cultural, technological, and scholarly phenomenon. Information, Communication & Society, 15(5), 662–679. https://doi.org/10.1080/1369118X.2012.678878
Brady, H.E. (2019). The challenge of big data and data science. Annual Review of Political Science, 22, 297–323. https://doi.org/10.1146/annurev-polisci-090216-023229
Breslow, L., Pritchard, D.E., DeBoer, J., Stump, G.S., Ho, A.D., & Seaton, D.T. (2013). Studying learning in the worldwide classroom: Research into edX’s first MOOC. Research & Practice in Assessment, 8(1), 13–25. Retrieved from http://www.rpajournal.com/dev/wp-content/uploads/2013/05/SF2.pdf
Buckingham Shum, S., & Ferguson, R. (2012). Social Learning Analytics. Journal of Educational Technology & Society, 15(3), 3–26. Retrieved from https://drive.google.com/file/d/1fu8JL6t8pwfGSkAnktZ4AEWChPjRnbdI/view
Bulger, M. (2016). Personalized learning: The conversations we’re not having. Retrieved from Data & Society Research Institute website: https://datasociety.net/pubs/ecl/PersonalizedLearning_primer_2016.pdf
Bunker, R.P., & Thabtah, F. (2019). A machine learning framework for sport result prediction. Applied Computing and Informatics, 15(1), 27–33. https://doi.org/10.1016/j.aci.2017.09.005
Caplan, R., Donovan, J., Hanson, L., & Matthews, J. (2018). Algorithmic accountability: A primer. Retrieved from Data & Society Research Institute website: https://datasociety.net/pubs/alg_accountability.pdf
Coleman, J.S. (1966). Equality of educational opportunity. U.S. Dept. of Health, Education, and Welfare, Office of Education.
Crawford, K. (2016). Can an algorithm be agonistic? Ten scenes from life in calculated publics. Science, Technology & Human Values, 41(1), 77–92. https://doi.org/10.1177/0162243915589635
Diakopoulos, N., Friedler, S., Arenas, M., Barocas, S., Hay, M., Howe, B., H. V. Jagadish, H.V., Unsworth, K., Sahuguet, A., Tech, C., Venkatasubramanian, S., Wilson, C., Yu, C., & Zevenbergen, B. (2017). Principles for accountable algorithms and a social impact statement for algorithms. FAT/ML. Retrieved from https://www.fatml.org/resources/principles-for-accountable-algorithms
Domínguez, D. (2018). Big Data, educación basada en datos y analítica del aprendizaje. In A. Sacristán (Coord.), Sociedad digital, tecnología y educación (pp. 299–329). Madrid, Spain: UNED.
Domínguez, D., Álvarez, J.F., & Gil-Jaurena, I. (2016). Learning Analytics and Big Data: Heuristics as Interpretive Frameworks. DILEMATA, International Journal of Applied Ethics, 22, 87–103. Retrieved from https://www.dilemata.net/revista/index.php/dilemata/article/view/412000042
Farrow, R. (2016). A Framework for the ethics of open education. Open Praxis, 8(2), pp. 93–109. http://dx.doi.org/10.5944/openpraxis.8.2.291
Floridi, L., & Cowls, J. (2019). A unified framework of five principles for AI in society. Harvard Data Science Review, 1(1). https://doi.org/10.1162/99608f92.8cd550d1
Geddes, B. (1990). How the cases you choose affect the answers you get: Selection bias in comparative politics. Political Analysis, 2(1), 131–150. https://doi.org/10.1093/pan/2.1.131
Gigerenzer, G., & Selten, R. (Eds.). (2002). Bounded rationality: The adaptive toolbox. MIT press.
Gitelman, L. (Ed.)(2013). Raw data is an oxymoron. MIT Press.
Goksel, N., & Bozkurt, A. (2019). Artificial intelligence in education: Current insights and future perspectives. In S. Sisman-Ugur, & G. Kurubacak (Eds.), Handbook of Research on Learning in the Age of Transhumanism (pp. 224–236). Hershey, IGI Global.
González-Bailón, S., Wang, N., Rivero, A., Borge-Holthoefer, J., &
Moreno, Y. (2012). Assessing the bias in communication networks sampled from twitter. https://arxiv.org/abs/1212.1684
Greller, W., & Drachsler, H. (2012). Translating Learning into Numbers: A Generic Framework for Learning Analytics. Educational Technology & Society, 15(3), 42–57. Retrieved from https://drive.google.com/file/d/1R84FXoT3W3X6C2JV1BBXha3tCoOQiQ7l/view
Hansen, J.D., & Reich, J. (2015). Democratizing education? Examining access and usage patterns in massive open online courses. Science, 350(6265), 1245–1248. https://doi.org/10.1126/science.aab3782
Hew, K.F., Qiao, C., & Tang, Y. (2018). Understanding student engagement in large-scale open online courses: A machine learning facilitated analysis of student’s reflections in 18 highly rated MOOCs. International Review of Research in Open and Distributed Learning, 19(3). https://doi.org/10.19173/irrodl.v19i3.3596
Hussain, M., Zhu, W., Zhang, W., Abidi, S. M. R., & Ali, S. (2019). Using machine learning to predict student difficulties from learning session data. Artificial Intelligence Review, 52(1), 381–407. https://doi.org/10.1007/s10462-018-9620-8
Houlden, S., & Veletsianos, G. (2019). A posthumanist critique of flexible online learning and its “anytime anyplace” claims. British Journal of Educational Technology, 50(3), 1005–1018. https://doi.org/10.1111/bjet.12779
Jobin, A., Ienca, M., & Vayena, E. (2019). The global landscape of AI ethics guidelines. Nature Machine Intelligence, 1(9), 389–399. https://doi.org/10.1038/s42256-019-0088-2
Johnson, C., & Newett, E. (2015). From idea to execution: Spotify’s discover weekly. Retrieved from https://www.slideshare.net/MrChrisJohnson/from-idea-to-execution-spotifys-discover-weekly/
Kitchin, R. (2014). Big data, new epistemologies and paradigm shifts. Big Data & Society, 1(1), 1–12. https://doi.org/10.1177/2053951714528481
Kurilovas, E. (2019). Advanced machine learning approaches to personalise learning: learning analytics and decision making. Behaviour & Information Technology, 38(4), 410–421. https://doi.org/10.1080/0144929X.2018.1539517
Luckin, R., & Cukurova, M. (2019). Designing educational technologies in the age of AI: A learning sciences‐driven approach. British Journal of Educational Technology, 50(6), 2824–2838. https://doi.org/10.1111/bjet.12861
Metcalf, J., Keller, E.F., & boyd, d. (2016). Perspectives on big data, ethics, and society. Retrieved from The Council for Big Data, Ethics, and Society website: https://bdes.datasociety.net/council-output/perspectives-on-big-data-ethics-and-society/
Michael, K., & Miller, K.W. (2013). Big data: New opportunities and new challenges. Computer, 46(6), 22–24. https://doi.ieeecomputersociety.org/10.1109/MC.2013.196
Monarrez, T. (2018). Segregated neighborhoods, segregated schools? Methodology. Washington, DC: Urban Institute. Retrieved from https://www.urban.org/sites/default/files/segregated_neighborhoods_methodology.pdf
Mousavi, S., & Gigerenzer, G. (2014). Risk, uncertainty, and heuristics. Journal of Business Research, 67(8), 1671-1678. https://doi.org/10.1016/j.jbusres.2014.02.013
Muller, J. Z. (2018). The tyranny of metrics. Princeton University Press.
Onnela, J. P., Saramäki, J., Hyvönen, J., Szabó, G., Lazer, D., Kaski, K., Kertész, J., et al. (2007). Structure and tie strengths in mobile communication networks. Proceedings of the National Academy of Sciences, 104(18), 7332–7336. https://doi.org/10.1073/pnas.0610245104
Orfield, G., & Lee, C. (2005). Why segregation matters: Poverty and educational inequality. Retrieved from https://civilrightsproject.ucla.edu/research/k-12-education/integration-and-diversity/why-segregation-matters-poverty-and-educational-inequality/orfield-why-segregation-matters-2005.pdf
Pasick, A. (2015, December 21). The magic that makes Spotify’s Discover Weekly playlists so damn good. Quartz. Retrieved from https://qz.com/571007/
Perrotta, C., & Selwyn, N. (2019). Deep learning goes to school: toward a relational understanding of AI in education. Learning, Media and Technology. https://doi.org/10.1080/17439884.2020.1686017
Pfeffer, J., Mayer, K., & Morstatter, F. (2018). Tampering with twitter’s sample API. EPJ Data Science, 7(50). https://doi.org/10.1140/epjds/s13688-018-0178-0
Pitcan, M. (2016, July 13). Student Data Privacy: An Overview [Blog post]. Retrieved from https://medium.com/enabling-connected-learning/student-data-privacy-an-overview-ea41ebd99095#.8jv3n83w2
Poel, M., Meyer, E.T., & Schroeder, R. (2018). Big data for policymaking: Great expectations, but with limited progress? Policy & Internet, 10(3), 347–367. https://doi.org/10.1002/poi3.176
Prabhakar, S., Spanakis, G., & Zaiane, O. (2017). Reciprocal recommender system for learners in massive open online courses (MOOCs). In H. Xie, E. Popescu, G. Hancke & B.F. Manjón (Eds.)(2017), Advances in Web-Based Learning–ICWL 2017 (pp. 157–167). Cham: Springer. https://doi.org/10.1007/978-3-319-66733-1_17
Reich, J. (2014, March 30). Big data MOOC research breakthrough: Learning activities lead to achievement [Blog post]. Retrieved from http://blogs.edweek.org/edweek/edtechresearcher/2014/03/big_data_mooc_research_breakthrough_learning_activities_lead_to_achievement.html
Reif, J.H. (n.d.). Rules for algorithm design [Lecture notes]. Retrieved from https://users.cs.duke.edu/~reif/courses/alglectures/skiena.lectures/lecture6.2.pdf
Romero, C., & Ventura, S. (2017). Educational data science in massive open online courses. Data Mining and Knowledge Discovery, 7(1). https://doi.org/10.1002/widm.1187
Ruipérez-Valiente, J.A., Halawa, S., Slama, R., & Reich, J. (2019). Using multi-platform learning analytics to compare regional and global MOOC learning in the Arab world. Computers & Education, 146. https://doi.org/10.1016/j.compedu.2019.103776
Ruths, D., & Pfeffer, J. (2014). Social media for large studies of behavior. Science, 346(6213), 1063–1064. https://doi.org/10.1126/science.346.6213.1063
Saurwein, F., Just, N., & Latzer, M. (2015). Governance of algorithms: options and limitations. Info, 17(6), 35–49. https://doi.org/10.1108/info-05-2015-0025
Sharma, R.C., Kawachi, P., & Bozkurt, A. (2019). The Landscape of Artificial Intelligence in Open, Online and Distance Education: Promises and Concerns [Editorial]. Asian Journal of Distance Education, 14(2). Retrieved from http://asianjde.org/ojs/index.php/AsianJDE/article/view/432
Siemens, G., Gasevic, D., Haythornthwaite, C., Dawson, S., Buckingham, S., Ferguson, R., Duval, E., Verbert, K., & Baker, R.S.J.d. (2011). Open Learning Analytics: an integrated & modularized platform. Retrieved from Society for Learning Analytics Research website: https://solaresearch.org/wp-content/uploads/2011/12/OpenLearningAnalytics.pdf
Sinders, C. (2019a, November 12). Reimagining privacy online through a spectrum of intimacy [Blog post]. Retrieved from https://www.are.na/blog/reimagining-privacy-online-through-gradients-of-intimacy
Sinders, C. (2019b). Making critical ethical software. In L. Bogers, & L. Chiappini (Eds.), The Critical Makers Reader:(Un) learning Technology
(pp. 86–94). Amsterdam: Institute of Network Cultures.
Sinders, C. (2019c). Data ingredients: A provocation towards making algorithms human readable. Retrieved from https://privacy.shorensteincenter.org/data-ingredients
Sirin, S.R. (2005). Socioeconomic status and academic achievement: A meta-analytic review of research. Review of Educational Research, 75(3), 417–453. https://doi.org/10.3102/00346543075003417
Sloane, M., & Moss, E. (2019). AI’s social sciences deficit. Nature Machine Intelligence, 1(8), 330–331. https://doi.org/10.1038/s42256-019-0084-6
Smith, C.S. (2019, December 18). The machines are learning, and so are the students. The New York Times. Retrieved from https://www.nytimes.com/2019/12/18/education/artificial-intelligence-tutors-teachers.html
Sundar, S., & Singh, A. (2013). New heuristic approaches for the dominating tree problem. Applied Soft Computing, 13(12), 4695–4703. https://doi.org/10.1016/j.asoc.2013.07.014
Thille, C., Schneider, E., Kizilcec, R.F., Piech, C., Halawa, S.A., & Greene, D.K. (2014). The future of data-enriched assessment. Research & Practice in Assessment, 9(2), 5–16. Retrieved from http://www.rpajournal.com/dev/wp-content/uploads/2014/10/A1.pdf
Tufekci, Z. (2013). Big data: Pitfalls, methods and concepts for an emergent field. http://dx.doi.org/10.2139/ssrn.2229952
UNESCO. (2019, February 12). How can artificial intelligence enhance education? Retrieved from https://en.unesco.org/news/how-can-artificial-intelligence-enhance-education
US-ACM. (2017). Statement on algorithmic transparency and accountability. Retrieved from https://www.acm.org/binaries/content/assets/public-policy/2017_usacm_statement_algorithms.pdf
Uskov, V.L., Bakken, J.P., Byerly, A., & Shah, A. (2019). Machine learning-based predictive analytics of student academic performance in STEM education. In A.K. Ashmawy & S. Schreiter, Proceedings of 2019 IEEE Global Engineering Education Conference (pp. 1370–1376). Piscataway, NJ: IEEE.
Warnakulasooriya, R., & Black, A. (Eds.) (2018). Beyond the Hype of Big Data in Education. Practical lessons and illustrative examples of how to derive reliable insights in learning analytics. MacMillan Learning. Retrieved from http://prod-cat-files.macmillan.cloud/MediaResources/instructorcatalog/legacy/BFWCatalog/uploadedFiles/Beyond-the-Hype-of-Big-Data-in-Education.pdf
Zawacki-Richter, O., Marín, V.I., Bond, M, & Gouverneur, F. (2019). Systematic review of research on artificial intelligence applications in higher education – where are the educators? International Journal of Educational Technology in Higher Education, 16. https://doi.org/10.1186/s41239-019-0171-0
Zhou, L. (2018, September 5). How to Build a Better Machine Learning Pipeline. Datanami. Retrieved from https://www.datanami.com/2018/09/05/how-to-build-a-better-machine-learning-pipeline/
Zwitter, A. (2014). Big data ethics. Big Data & Society, July-December, 1–6. https://doi.org/10.1177/2053951714559253
Publicado
Como Citar
Edição
Secção
Licença
Direitos de Autor (c) 2020 RIED. Revista Iberoamericana de Educación a Distancia
Este trabalho encontra-se publicado com a Licença Internacional Creative Commons Atribuição 4.0.
As obras que são publicadas neste revista estão sujeitas ao seguintes termos:
1. Os autores cedem de forma não exclusiva os direitos de exploração dos trabalhos aceitos para sua publicação a "RIED. Revista Iberoamericana de Educação a Distância", garantem à revista o direito de ser a primeira publicação do trabalho e permitem que a revista distribua os trabalhos publicados sob a licença de indicada no ponto 2.
2. As obras são publicadas na edição eletrônica da revista sob uma licença Creative Commons Atribuição 4.0 Internacional (CC BY 4.0). Podem copiar e redistribuir o material em qualquer suporte ou formato, adaptar, remixar, transformar, e criar a partir do material para qualquer fim, mesmo que comercial. Você deve atribuir o devido crédito, fornecer um link para a licença, e indicar se foram feitas alterações. Você pode fazê-lo de qualquer forma razoável, mas não de uma forma que sugira que o licenciante o apoia ou aprova o seu uso.
3. Condições de auto-arquivo. Permite-se e incentava-se aos autores difundir eletronicamente a versõ OnlineFirst (versão avaliada e aceita para publicação) de su obra antes de sua publicação, sempre com referência a sua publicação na RIED, já que favorece sua circulação e difusão mais cedo e com isso um possível aumento em sua citação e alcance entre a comunidade acadêmica. Color RoMEO: verde.
