The integration of technology in education holds promise for revolutionizing learning experiences by improving efficiency, enriching content, and empowering educators and learners alike. Learning analytics (LA) emerges as a crucial tool, utilizing data within virtual learning environments to scrutinize and refine the teaching/ learning process. However, despite the abundance of data, educators often lack the expertise to effectively utilize this information, impeding its potential to enhance teaching methods and student outcomes. Furthermore, the ethical and practical boundaries of employing learning analytics remain undefined.
This article endeavours to explore the foundational prerequisites for devising an integrated Learning analytics and Action research (LAAR) model. This model seeks to equip educators in general education institutions with the tools to monitor learners and teaching/ learning dynamics in digital settings, facilitating data-driven decision-making for pedagogical improvements. Employing the Design science research methodology, particularly focusing on cognitive artifacts like human-computer interfaces, this research aims to address existing methodological limitations and propose novel solutions. By aligning with the socio-technical nature of learning analytics systems, involving diverse stakeholders and technology, this approach aims to foster the interpretation and application of analytics data in educational contexts.
____________________
Journal of Digital Pedagogy – ISSN 3008 – 2021
2023, Vol. 2, No. 1, pp. 8-13
https://doi.org/10.61071/JDP.2311
HTML | PDF
____________________
Introduction
Learning in digital environments makes sense when the application of technology strengthens and makes the learning process more efficient, enriches the learning content, helps the teacher to apply learning methods more effectively, and the learner to monitor the learning process and change learning behaviour in a timely manner in order to achieve better learning results.
Monitoring and analysing the teaching/ learning process is supported by learning analytics, which can be used to analyse the data generated by learners in the virtual learning environment, with the aim that the learning and teaching processes allow the teacher to perform interventions more effectively, to advise or consult learners in a timely manner in order to increase their academic success (Misiulienė, 2021; Volungevičienė, 2021).
In digital learning environments, there is enough data provided by learning analytics about learner behaviour and results, but teachers do not have enough competencies to use this data to improve the learning and teaching process, there is a lack of recommendations on how to evaluate the impact of learning analytics (Tsai, Gasevic, 2017). Furthermore, it is important to define the limits to the application, effectiveness, and legitimacy of learning analytics. Scientific evidence is needed to address these issues to serve educational organizations using virtual learning environments. This is also an international problem (Hernández-de-Menéndez, Morales-Menendez, Escobar, et al. (2022); Kew, Tasir, 2022; Ong, Singh, 2021), which is becoming more and more relevant every year.
There is a huge potential of learning analytics in monitoring and improving the teaching and learning process, noting a gap in teacher competencies to effectively use the data.
The aim of the article is to discuss the preconditions for the development of an integrated learning analytics (LA) and action research (AR) model (LAAR), which would enable teachers of general education schools to monitor learners and the teaching/ learning process in digital learning environments and to make data-based decisions to improve pedagogical practice.
The research will use the Design science research methodology (Hevner et al., 2004) applied to cognitive artefacts, e.g. human-computer interface, categories (Dolgopolov, 2018). The suitability of the methodology for learning analytics research is based on the following statements: LA is a socio-technical type of system; LA includes participants (teachers, students, educational institutions, etc.), EdTech, teaching/ learning design methods, teaching/ learning tools; LA systems are artifacts that manifest as support systems for people who need to be able to interpret analytics data and act on analytics results in learning/ learning contexts (Ferguson et al., 2016; Wilson et al., 2017). Scientific design research is applied to finding new alternatives for solving problems, in this respect this methodology is close to the methodology of activity research (Ketokivi, Hameri, 2009; Carstensen, Bernhard, 2019; Mertler, 2021).
1. Justification of the problem’s relevance
Modern research on learning analytics as well as practice in general education schools prove its importance in solving issues related to monitoring learners and the learning process (Mayer-Schönberger, Cukier, 2014; Rupšienė et al., 2021): determining students’ learning skills (Williamson, 2016; Mangaroska et al. ., 2019), collecting data on student progress (Ifenthaler et al., 2019), personalizing and adapting the learning process (Peng et al., 2019; Ifenthaler et al., 2020), student assessment (Sclater, Mullan, 2017), quality of teaching/ learning activities (Maseleno et al., 2018), identification of students from risk groups (Mangaroska and Giannakos, 2018; Williamson, 2016), reduction of exclusion (Kurvinen et al., 2020), etc.
Main beneficiary groups of learning analytics are teachers.
One of the main beneficiary groups of learning analytics is teachers (Khine, 2018), on the other hand, it is the willingness of teachers to use data to improve pedagogical activities that connects the essence of learning analytics with the needs of educational theory and practice, educational content and understanding of how students learn (Ferguson et al., 2016; Khine, 2018; Mandinach, Gummer, 2016; Wilson et al., 2017). Research (van Leeuwen et al., 2021) highlights the need to help teachers master Learning analytics technologies to monitor learners and improve the teaching/ learning process. Other studies (Zhu, Urhahne, 2018) emphasize the need to develop teachers’ competencies to effectively use learning analytics data for pedagogical decisions. Research in Lithuanian educational institutions has actualized the necessity of teachers’ competences for working with data – starting from “reading” and understanding learning analytics reports (data summaries), ending with their correct interpretation and making data-based pedagogical decisions (Volungevičienė et al., 2019; Rupšienė et al., 2021).
To improve pedagogical decisions, teachers are traditionally offered the action research (AR) method (Altrichter et al., 2005; Hinchey, 2008), which according to modern research data (Nunes, McPherson, 2019; Mazza, Dimitrova, 2004) is particularly suitable for learning and teaching monitoring in digital environments. The purpose of action research is not only the summarization of data and its interpretation, but also the application of results to improve practical activities (Hinchey, 2008). Several models of pedagogic action research have been found in the literature, such as e.g. EMAR model (Nunes, McPherson, 2004), teacher inquiry model (Mertler, 2021). However, these action research models were not combined with learning analytics capabilities.
Learning analytics can provide teachers with powerful tools to help them explore teaching and learning practices by automatically collecting, analysing, and visualizing relevant data (Dyckhoff, 2012).
However, by providing a summary of data, LA does not provide information about how the learning process is going, how the curriculum, content, etc. should change. (Volungevičienė et al., 2019). The teacher must interpret these indicators, reflect on them and decide how these data should be used and analyzed for monitoring learners, improving the teaching/ learning process and developing the curriculum, i.e. integrating LA into the AR cycle (Berg, 2001; Dyckhoff et al., 2013; Mertler, 2021). Integrating LA with AR emphasizes the aspect of human decision-making: perceiving information, making a decision based on data, and implementing a specific action/ intervention based on it (Dyckhoff et al., 2013). i.e. promotes a human-centred learning analytics approach (Buckingham Shum et al., 2020).
The model is seen as a practical tool to enhance pedagogical practices and can potentially benefit various stakeholders, including learners, school leaders, and IT administrators.
There are no LA and AR models for general education teachers in the literature. The research-based integrated LA and AR model could become a practical tool that will help teachers to observe learners and the learning process in digital learning environments, to reflect, understand and improve pedagogical practice, developed teachers’ data analysis and application of its results in practice. The LAAR model could also be useful for other target groups (e.g. learners), interested parties (e.g. school leaders, IT administrators, etc.).
2. Research design
The artifact being developed – an integrated LAAR model – should become a practical tool enabling teachers of general education schools to monitor learners and the teaching/ learning process in digital learning environments and make data-based pedagogical decisions. Qualitative methods will be used to substantiate the scientific applied problem and create a prototype: analysis of sources (in terms of political, legal, strategic documents, scientific articles, research data, etc. in LA and AR education), focus group interviews of target groups and various interested parties (1. teachers, learners and their parents, school leaders; 2. IT specialists, educational policy makers – a total of about 20 focus group participants are planned). The created prototype will be tested and evaluated by Lithuanian school teachers (at least 10) using the advanced learning platforms Eduten Playground and Learnlab in the educational process. The test will be focused on 3 main research questions: 1. Analysis and understanding of student and learning process monitoring data, 2. Data-based reflection of pedagogical practice, 3. Implementation potential of action research. Qualitative methods (e.g. semi-structured interviews with teachers piloting the prototype, observation, case narratives) will be used to collect research data. Data analysis will be based on grounded theory. The research results will be continuously presented and discussed at international conferences (at least 3 reports), scientific journals (at least 3 scientific publications) related to LA in education research areas.
The LAAR prototype would consist of a structural model (5 interacting areas: target groups and stakeholders (e.g. teachers); context (e.g. technological, pedagogical, etc.); goals and objectives (e.g. monitoring of learners, personalization of learning, etc. ); digital learning environments and the data they generate (qualitative and quantitative, their summaries, aspects of data protection, confidentiality and ethics, etc.); methods (statistics, information visualization, etc.), 6 stages of the pedagogical activity research cycle ( identify problems, formulate questions, use data, transform data into information, turn information into a solution/ pedagogical intervention, evaluate results), about 60 questions related to the above-mentioned areas, factors and limitations affecting LAAR) and a process model (integration of LA, AR activities into a continuous cycle).
The result of the research is the didactic justification of the prototype, testing and methodological recommendations for teachers, but the created tool can be digitized in the future, and could also be integrated into educational technologies.
3. Selection of distance learning platforms for research
Two digital learning environments (regarded globally as the most technologically and pedagogically advanced) are proposed for the study. The learning analytics integrated in these learning platforms are of all types: 1) summative and descriptive, 2) formative, 3) predictive and prescriptive, but based on different methodological approaches. Eduten Playground is an AI-powered gamified math learning platform designed to develop well-defined math skills. This platform is characterized by high-quality and informative quantitative learning analytics tools. Its content is translated into Lithuanian and harmonized with the general programs of Lithuania. The LearnLab platform offers teachers and students mind mapping, book creation and other digital tools for learning about 21st century learning. the most important concepts and concepts for a person by integrating the knowledge of various educational subjects and applying the principles of “deep” learning. Learning analytics on this platform is focused on formative assessment and providing feedback to each student by collecting qualitative data on the overall development of students. The content of the platform is combined with general education programs.
The choice of such platforms is best suited for analyzing the possibilities of learning analytics for monitoring learners and the learning process.
- Comparing the methodological approaches of these two platforms would be very important for designing future scenarios.
- These platforms promote inclusive and engaging learning(s).
- These technologies promote self-directed, adaptive, personalized learning models.
- 2022 – 2023 according to data, 73 Lithuanian general education schools use the Eduten Playground platform and 8 schools use the Learnlab platform.
It is also an important fact that there are already experienced teachers in Lithuanian general education schools and teachers who will just start using these platforms in the educational process (in the course of the EdTech project, more and more schools are acquiring licenses). This is relevant for the study, as it will allow differentiation of the target group of the study.
Conclusions
The integration of technology in learning environments has the potential to enhance the efficiency of learning processes, enrich educational content, and empower educators to employ more effective teaching methodologies while enabling learners to monitor and adapt their learning strategies for improved outcomes.
Learning analytics serves as a pivotal tool in monitoring and analysing the teaching and learning process, utilizing data generated within virtual learning environments.
However, despite the wealth of data available, educators often lack the expertise to leverage this information effectively, hindering its potential to enhance teaching practices and student success. Moreover, there remains a pressing need to establish boundaries for the application, efficacy, and ethical use of learning analytics.
This article aims to explore the foundational requirements for developing an integrated Learning Analytics and Action Research (LAAR) model. This model seeks to equip educators in general education schools with the means to monitor learners and the teaching/ learning process in digital environments, enabling data-driven decision-making to enhance pedagogical practices.
Adopting the Design science research methodology, particularly concerning cognitive artifacts such as human-computer interfaces and categories, the research will seek to address the limitations of existing methodologies and propose new solutions. This approach aligns with the socio-technical nature of learning analytics systems, involving diverse participants, educational technology, teaching/ learning design methods, and support systems for interpreting and acting on analytics data in educational contexts.
Ultimately, this research aims to contribute to the field of education by providing new insights into the development of a comprehensive LAAR model, offering practical strategies for educators to harness the potential of learning analytics and improve the teaching and learning experience in digital environments.
References
Altrichter, H., Posch, P., Somekh B. (2005). Teachers Investigate Their Work: An Introduction to the Methods of Action Research. 6th Edition. London: Routledge.
Berg, B. L. (2001). Action Research. In Qualitative Research Methods for the Social Science, 4th Edition.
Buckingham Shum, S., Ferguson, R., Martinez-Maldonado, R. (2019). Humancentered learning analytics. Journal of Learning Analytics, 6(2), 1-9. https://doi.org/10.18608/jla.2019.62.1
Carstensen, A. K., & Bernhard, J. (2019). Design science research–a powerful tool for improving methods in engineering education research. European Journal of Engineering Education, 44(1-2), 85-102.
Dyckhoff, A. L., Lukarov, V., Muslim, A., Chatti M.A., Schroeder, U. (2013). Supporting Action Research with Learning Analytics. Proc. of the 3rd International Conf. on Learning Analytics and Knowledge (LAK’13), 220–229. New York, New York, USA: ACM Press. https://doi.org/10.1145/2460296.2460340
Dolgopolovas, V., Dagienė, V., Jevsikova, T. (2020). Methodological guidelines for the design and integration of software learning objects for scientific programming education. Scientific programming. vol. 2020, art. no. 6807515, p. [1-19]. London: Hindawi. https://doi.org/10.1155/2020/6807515
Ferguson, R. et alii (2016). Research evidence on the use of learning analytics: Implications for education policy. Joint Research Centre, Seville, Spain https://doi.org/10.2791/955210
Hevner, A. R., Salvatore T. March, J.P., Ram S. (2004). Design Science in Information Systems Research. MIS Quarterly 28 (1): 75–105.
Hinchey, P. H. (2008). Action Research Primer. New York: Peter Lang Publishing,
Ifenthaler, D., Gibson, D., Prasse, D., Shimada, A., Yamada, M. (2020) Putting learning back into learning analytics: actions for policy makers, researchers, and practitioners. Education Tech Research Dev. https://doi.org/10.1007/s11423-020-09909-8
Ifenthaler, D., D. Mah and J. Yau (2019), Utilising learning analytics for study success. Reflections on current empirical findings, in Ifenthaler, D., J. Yau and D. Mah (eds.), Utilizing learning analytics to support study success, Springer, Cham.
Ketokivi, M., Hameri, A. P. (2009). Bridging practice and theory: A design science approach. Decision sciences, 40(1), 65-87.
Khine, M. (2018). Learning Analytics for Student Success: Future of Education in Digital Era. The European Conference on Education 2018.
Kurvinen, E., Kaila, E., Laakso, M.-J., Salakoski, T. (2020). Long Term Effects on Technology Enhanced Learning: The Use of Weekly Digital Lessons in Mathematics. Informatics in Education. 19. 51-75. Vilnius, Vilniaus universitetas. https://doi.org/10.15388/infedu.2020.04
Mayer-Schönberger, V., Cukier, K. (2014). Big Data: A Revolution That Will Transform How We Live, Work, and Think. Boston: Houghton Mifflin Harcourt
Mandinach, E. B., Gummer, E. S. (2016). What does it mean for teachers to be data literate: Laying out the skills, knowledge, and dispositions. Teaching and Teacher Education, 60, 366-376. https://doi.org/10.1016/j.tate.2016.07.011
Mangaroska, K., Giannakos, M. (2018). Learning analytics for learning design: A systematic literature review of analytics-driven design to enhance learning. IEEE Transactions on Learning Technologies 12 (4), 516-534.
Mangaroska, K., Vesin, B., Giannakos, M. (2019). Cross-platform analytics: A step towards personalization and adaptation in education. Proceedings of the 9th international conference, 2019. https://ntnuopen.ntnu.no/ntnu-xmlui/bitstream/handle/11250/2648295/2019-LAK-Cross-Platform-Analytics.pdf
Mazza, R., Dimitrova, V. (2004). Visualising student tracking data to support instructors in web-based distance education. Proceedings of the 13th international World Wide Web conference on Alternate track papers & posters (pp. 154-161).
Mertler, C. A. (2021) Action Research as Teacher Inquiry: A Viable Strategy for Resolving Problems of Practice. Practical Assessment, Research, and Evaluation: Vol. 26, Article 19. DOI: https://doi.org/10.7275/22014442
Nunes, M., McPherson, M. (2019). An Action Research Model for the Management of Change in Continuing Professional Distance Education. Journal of ITALICS. https://eprints.whiterose.ac.uk/702/
Rupšienė ir kt. (2021). Mokymosi analitika ir dirbtinis intelektas mokykloje: ateitis prasideda šiandien. Klaipėda : Klaipėdos universiteto leidykla, 2021. ISBN 9786094811326
Sclater, N. and J. Mullan (2017), Learning analytics and student success – assessing the evidence, JISC, Bristol. https://repository.jisc.ac.uk/6560/1/learning-analytics_and_student_success.pdf
van Leeuwen, A., Knoop-van Campen, C. A., Molenaar, I., Rummel, N. (2021). How teacher characteristics relate to how teachers use dashboards: Results from two case studies in K-12. Journal of Learning Analytics, 8(2), 6-21. https://doi.org/10.18608/jla.2021.7325
Volungeviciene, A., Duart, J., Naujokaitienė, J. Tamoliune, G., Misiulienė, R. (2019). Learning Analytics: Learning to Think and Make Decisions. The Journal of Educators Online, 16. https://10.9743/jeo.2019.12.2.13
Williamson, B. (2016). Digital education governance: data visualization, predictive analytics, and ‘real-time’policy instruments. Journal of education policy, 31(2), 123-141.
Wilson, A., Watson, C., Thompson, T. L., Drew, V., & Doyle, S. (2017). Learning analytics: Challenges and limitations. Teaching in Higher Education, 22(8), 991-1007.
Zhu, M., Urhahne, D., Rubie-Davies, C. M. (2018). The longitudinal effects of teacher judgement and different teacher treatment on students’ academic outcomes. Educational Psychology, 38(5), 648–668.
____________________
Authors:
Aleksandra Batuchina
Klaipeda University, Department of Pedagogy
aleksandra.batuchina@ku.lt
https://orcid.org/0000-0003-0012-9421
Evelina Brazauskienė
PhD student, Klaipeda University, Department of Pedagogy
https://orcid.org/0009-0008-9127-896X
Kamilė Kesylė
PhD student, Klaipeda University, Department of Pedagogy
https://orcid.org/0009-0009-1399-6082
Julija Melnikova
senior researcher, Klaipeda University, Department of Pedagogy
https://orcid.org/0000-0001-6078-6963
Received: 15.11.2023. Accepted: 24.11.2023
© Aleksandra Batuchina, Evelina Brazauskienė, Kamilė Kesylė, Julija Melnikova, 2023. This open access article is distributed under the terms of the Creative Commons Attribution Licence CC BY, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited:
Citation:
Batuchina, A., Brazauskienė, E., Kesylė, K., Melnikova, J. (2023). Preconditions for the development of the integrated learning analytics and action research model for Lithuanian general education schools: theoretical insights and research design. Journal of Digital Pedagogy, 2(1) 8-13. Bucharest: Institute for Education. https://doi.org/10.61071/JDP.2311