- Jaan Aru, Head of the AI Leap research team and Associate Professor at the Institute of Computer Science of the University of Tartu
- Kristjan-Julius Laak, Junior Research Fellow in Artificial Intelligence at the Institute of Computer Science of the University of Tartu
- Grete Arro, Senior Research Fellow in Educational Psychology at the School of Educational Sciences of Tallinn University
In January, the AI Leap foundation started distributing its AI learning app to students in the 10th and 11th grades in Estonian schools. How does it differ from the regular ChatGPT? And more importantly, how do we know that it’s actually helping the students learn?
Imagine a stadium where a student is having to run laps to improve their endurance. Current AI models are like an overly supportive friend who calls out to them from the sidelines, “Take five, I’ll run the next lap for you!” Sure, the lap might be completed in record time and the results might be impressive, but the kid’s fitness won’t improve one jot.
This is exactly what happens when students let the regular ChatGPT do their schoolwork for them. By its very nature, learning is a slow process that requires you to put in the hard yards. You have to rack your brains and come up with ideas and solutions. It can be an uncomfortable experience, but it’s an unavoidable one if you want to make any progress. Our aim is to create technology that doesn’t take the effort out of the equation, but encourages consistent thinking in students.
A different approach to the usual ChatGPT
In cooperation with Estonian researchers and OpenAI, the AI Leap team has been working on a custom-made AI learning app for students in Estonia for almost a year. It resembles the normal ChatGPT, but works the other way round to common language models in that it:
- doesn’t offer ready-made answers, but guides the learner to make an effort themselves,
- helps the learner make their task meaningful to them personally,
- helps activate what the learner already knows about the subject,
- supports self-reflection,
- adapts the level of difficulty of explanations to the student’s own level, and
- always answers in natural Estonian.
For example, if a student asks an ordinary AI app how cells divide in an organism, the machine gives a quick answer and outlines the stages. The student copies that answer and almost immediately forgets the details. The learning app designed for Estonian schools, on the other hand, guides the learner rather than directly answering them: “Let’s brainstorm! Why might cell division be necessary for organisms in the first place?”
This simple change in the dynamics of the conversation is crucial, as it shifts the focus from passively obtaining a response to triggering the learner to think.
Support for learning, not short-term performance
Those who study learning and the brain agree that gaining new knowledge requires a sustained effort on the part of learners – which is something that can be fostered. Working with education researchers from the University of Tartu and Tallinn University, we’ve identified four key attributes a learning app must have in order to get students to make such an effort.
The first is autonomous motivation. This means that students are prepared to make an effort if they feel they’re learning for themselves and that the subject means something to them. Fear of failure or punishment doesn’t motivate them to learn things properly.
The second attribute is belief in your own progress. No doubt we’ve all heard someone claim they’re “no good at maths”. The learning app supports the belief that abilities can be fostered, and that seemingly difficult tasks and making mistakes are a natural part of the learning process.
The third attribute is learning strategies that promote learning. Students often use ineffective techniques like passively reading through a text. Instead, the app guides them to use proven methods like calling to mind what they’ve already learned and hammering away at a complicated problem before checking the solution.
The fourth and final attribute is support for the development of the self-regulation skills needed for lifelong learning. These are skills that help you manage your own learning. A self-regulated learner sets themselves goals, consciously monitors their learning process and assesses their progress. They also adapt their learning strategies and reflect on their results.
Research has shown that these four learning attributes directly determine a person’s academic progress and well-being. As such, the aim of the AI Leap learning app isn’t to speed up the completion of tasks in school, but the opposite: to normalise slower learning that takes more effort and to support the development of the competences needed for lifelong learning.
Taking the fight to the quick and easy option
Studies to date have shown that the vast majority of secondary students already use AI (mostly free versions of commercial apps) to get their schoolwork done more quickly. Our goal isn’t to encourage students to make more use of AI, but to offer them a learning-friendly alternative to widely used AI apps that inhibit critical thinking and the development of learning skills.
The AI Leap development team monitors the performance of the app on an ongoing basis using anonymous and automated assessment tools that analyse the learning app’s responses. For example, developers check to make sure that it’s answering in Estonian and not doing the student’s work for them.
We’ve also enabled learners themselves to give us feedback on the learning app’s behaviour. The process of gathering feedback and making changes based on it will continue for the duration of the AI Leap programme so as to improve the learning app.
Does it really work?
To answer the question of whether the learning app actually helps students learn more effectively, the Ministry of Education and Research has asked the University of Tartu to conduct a wide-ranging impact study. Our aim is to find out how and under what conditions the learning app is influencing learners’ development.
We will use classic questionnaires and tests to determine the app’s impact on the four attributes that support learning mentioned above (e.g. learning beliefs). Only students who have given their consent participate in the study.
In order to understand what exactly is causing a change, we will use other data related to the student’s learning environment (e.g. the number of AI Leap professional learning communities that take place in a school) and statistics on the use of the app. We will analyse the students’ data in a way that does not allow them to be linked to specific individuals, i.e. by pseudonymising the data.
We will also look more generally at whether a learner’s progress can be assessed through conversations and how learning with AI differs from learning with a teacher. To that end, the automated system will exclude from analysis any conversations that are not related to learning and that it suspects involve personal data or private matters. We will only analyse anonymous conversations pertaining to learning. Conclusions won’t be drawn from individual conversations or about individuals.
This will make it possible in the future to quickly and effectively identify the impact of the app on learning and to further direct the learning app’s behaviour so that it is more supportive of learners.
The automated assessment tools developed in cooperation with the University of Tartu, Stanford University and OpenAI will analyse conversations collectively and identify general patterns. For example, we will try to find out whether using the learning app helps increase learners’ ability to make a sustained effort.
Research helping to fill a gap in knowledge
The results of the study will provide a more solid foundation for Estonian schools and parents alike, helping them to better understand how to learn with AI.
We’ll soon be able to say more about what’s needed for learners to progress by using AI. The results of the study will enable researchers and developers to improve the learning app and, in the future, to evaluate the ability of any other AI app to support our learners.
We will be conducting our first survey among students in February. The results of the study will be published in an international scientific journal and on the AI Leap’s web channels later this year. The schools taking part in the study will receive generalised feedback for their school on how to better target their use of AI.
The study has been coordinated with the Research Ethics Committee of the Estonian Research Council and the National Institute for Health Development. The Data Protection Inspectorate was also consulted in its preparation.