Join host John Kleeman in an insightful conversation with Carla Aerts, a trailblazer in the reinvention of education. Carla, with an unconventional journey from the performing arts to education technology, shares her accidental foray into the sector and delves into the importance of making impactful contributions to teaching and learning.
Carla explains what “perfect EdTech” looks like to her and critiques traditional exam-style assessments, advocating for a shift towards formative assessments and diverse models that prioritize skills, collaboration, and social engagement.
The discussion extends to the role of AI in education and Carla’s predictions of how it will impact the future. The episode concludes with three key takeaways: the significance of social learning, the need for skills development in reflection and metacognitive thinking, and the promotion of learner agency for effective lifelong learning.
Tune in for a thought-provoking exploration of education, technology, and assessments, guided by Carla Aerts’ forward-thinking perspectives.
Full Transcript
John Kleeman:
Hello everyone and welcome to Unlocking the Potential of Assessments, the show that delves into creating, delivering, reporting on fair and reliable assessments. In each episode, we chat with assessment luminaries, influencers, subject matter experts, and customers to discover and examine the latest in best practice guidance for all things assessment. I’m your host, John Kleeman, founder of Question Mark and EVP of Industry Relations and Business Development at Learnosity, the assessment technology company. Today, really pleased to welcome Carla Aerts. Carla is a really inspiring thinker and doer in the reinvention of education. She’s worked with a wide range of educational publishers, including being digital director at Cambridge University Press. And she was director of futures at the Institute of Education in London. She currently consults for a number of EdTech organizations, universities and works for a science of learning network. And in her time, she has engaged and worked with over 100 EdTech startups and continues to mentor. Hello Carla, really pleased to have you here.
Carla Aerts:
Hi John. Nice to see you.
John Kleeman:
So, the question I ask everybody is how did you get into learning and assessment?
Carla Aerts:
Okay, John, I have to own up here. This was purely accidental. I always swore that I would never work in education and technology because I felt education wasn’t really moving or moving fast enough. So to put it in a bit of a nutshell, I started my career in the performing arts. I was an assistant director in the Opera world and a stage manager and also come from a musical background. But then I kind of encountered an Apple Macintosh and I thought, oh, this is quite interesting. And I sort of pivoted into the digital space. And I’d done a few years of digital work with agencies in a variety of contexts. And I got this offer from an international education publisher who needed help with a whole program of transformation of education publishing, moving from the textbook and print to online and mainly blended learning at that point because we were still doing a number of CD-ROMs as well. But the internet was there and we were moving online.
So they wanted me to manage that program across a number of operating companies in Europe. The publisher was called Wolters Kluwer Education and then became Infinitas Learning. And that’s where I really tumbled into education. I saw, oh my God, this is far more interesting than I thought because I was working internationally and I could see all these kinds of contexts and the sort of politics of education, the importance of context. And what I also noticed straight away was that actually what we were doing with technology for education was pretty appalling. So I thought we could probably do a bit of a better job of it and I just got completely sucked in.
John Kleeman:
That’s great to hear. And it’s amazing how many of the people I interview didn’t plan to start going into assessments or learning, but ended up by accident. So you are by no means the only person. And so look, amazingly you’ve worked with 100 EdTech startups. Tell me about that and what advice would you give to a current EdTech startup?
Carla Aerts:
So the work with EdTech startups was a little bit twofold, if you like. So when I was at the Institute of Education, I set up the Educate project or program with Professor Luckin who was the kind of founder, and sort of it was her baby and her brainchild if you like. And I helped her set it up with a whole team of people sort of designing what the participant experience would look like and how we were going to engage with these EdTech companies. Now, the mission and the vision of this program was really to get evidencing of impact into the mindset of EdTech entrepreneurs and startups if you like, to make EdTech better and to kind of really focus on what works. And through that program by itself, I kind of engaged with over 100 startups because the program was serving 260 of them over two and a half years. But before that, I’d already engaged with quite a lot of startups.
And during my time at Cambridge as well, I started looking at the EdTech space beyond the UK and worked very internationally as I was kind of global director for the education division there. So sort my radar with EdTech was covering quite a lot of territories, if you like, and engaging with startups in those territories. So that’s sort of where it all started. And increasingly, startups contacted me to say, “Can we get you to advise us? Can we get you to be a mentor? Can we have you on our advisory board?” And so I did that with a lot of startups. I also felt I had to give something back. So for a long time I did this purely pro bono and because I also wanted to learn from them and engage with them. And that’s sort of where it started. And I still do, I still mentor startups, probably a bit less than I did in the past because other things kind of started taking over but that’s basically it.
John Kleeman:
And if it’s possible to generalize, what would be your key advice to somebody who’s thinking about setting up an EdTech or assessment or a learning business?
Carla Aerts:
I think you have to really, really clue in on sort of your context. We all know that education is really context and context is a wide principle or a wide concept if you like, because politics influence education, actually the learning environment, the culture, the language. It’s a big beast, but you really need to kind of come up I think with a very, very clear sort of problem statement because especially in the early days when I was talking to a lot of startups, they would all come to me and say, “Carla, are we going to solve the education problem?” Now, there is not the education problem. There are a lot of education challenges perhaps, but there’s not a single education problem. And I think that’s where it kind of also went a bit wrong because people try to solve the problem, not a facet of education that could be augmented by technology.
So what I would recommend to startups is be very clear about the challenge or the augmentation that you want to bring, to make an impact on how learning in whatever environment you are targeting can be improved. Whether that’s to better teaching, whether that’s to better access to resources, whether that’s through better collaboration with peers, but what is the problem or the challenge that you’re really trying to address? And then really honing onto that challenge. Don’t come up with too many ideas around that challenge because too many ideas actually don’t help you crystallize. You need to be very, very clear about what it is you want to solve. And then very clear about the context or the market in which you’re trying to solve this.
We know for instance that K-12 is a really, really difficult market in a lot of geographies because access to schools is very problematic for EdTech. So is that then the best route for you to get your startup up and running? So it’s those kinds of questions that you need to ask yourself. And then also around your proposition and really, really bottom that out. And then the last thing I would say is whatever you do, make sure that you actually make an impact. The impact can be very small and that’s fine, but don’t do it because you think it’s a good idea, do it because you’re trying to make teaching and learning or learning in itself better.
John Kleeman:
So a slightly related question, but what would be perfect EdTech?
Carla Aerts:
Perfect EdTech for me would be EdTech that also engages socially. One of the things that it hasn’t done very well because of the emphasis on personalization, on metrics, on measuring on individual learning progress or teaching practice, we’ve kind of lost a sense of actually technology can do things socially really, really well. And COVID told us how little social interaction there was in education, and we are still kind of feeling that aftermath even in university students. So we talk constantly about learning loss and a lot of EdTech companies are trying to solve that problem, but actually that is maybe not the problem. The problem is much more to do with actually how do we create social fabric in teaching and learning using EdTech and enhance teaching and learning with that. So for me, that social element is really important. That doesn’t mean that every EdTech company has to focus on that social element, but I think there’s so much room to bring that into it.
So that’s definitely one thing that I think would make EdTech better. And then also the second thing I would say is don’t try to retrofit into kind of existing practices. Try to augment them and trying to change them because every good technology that’s successful is actually changing our behavior or changing what we do or augmenting what we do. And in education I don’t think that has really happened, but it’s changing and I’m very optimistic about it.
John Kleeman:
What about assessment? So, you told me you’re a big fan if I understand right of the way a lot of assessment happens. Tell me about that and what your thoughts are on assessment.
Carla Aerts:
Yes, John, you did get that right. I have got a bit of a problem with especially the kind of exam style and how stakes type of assessment that sort of measure what a student or a learner does at a particular point in time. That particular moment in time can really define someone’s life based on a single exam that went well or that didn’t go so well. Now we also know that a lot of learners find doing assessment really hard because it’s a lot of pressure and stress. But the real gripe I have with it is that we really get learners to cram knowledge and more knowledge and more knowledge for that particular moment in time or those moments in time, and then have to spew it out in an assessment.
And then the assessment or the exam’s finished and the next day they’ve forgotten 99% of it. So that for me doesn’t make good use of education because also how are they going to apply the knowledge that they’ve acquired So from that perspective, I really, really struggle with a lot of assessment models. But I also am in favor of other types of assessment models.
John Kleeman:
So is it about trying to find ways that people can gain knowledge and skill and retain it, and that the risk is that a big exam doesn’t encourage you to do that?
Carla Aerts:
I don’t think it does because it’s too focused on you have to really cram in all that knowledge to get to that point. And in many education systems is that single point, that single exam point that can be twice a year, that can be yearly or that can be at after two years or three years is such a definitive factor, I think. And that’s where I’ve got a problem. And it also causes a lot of stress. We know that for instance, GCSE and A-level students in the UK experience huge amounts of anxiety and stress. And given that we also face or are in a mental health crisis in a lot of youngsters, do we really want to exacerbate that because this sort of system feels that doing an exam is the best way to measure how learners are doing. And what that means and defining your access to higher education, further education, all these kinds of things are so hooked onto that assessment moment or exam moment that I have a real problem with that.
John Kleeman:
So, would you think that it’d be better to have a series of either coursework or else a whole series of assessments each one less significant, and perhaps this ties into some of the stuff happening in the US with these big college admissions exams as well?
Carla Aerts:
I think. For instance, I’m quite excited about formative assessment and types of formative assessment because what that also helps is… And if that’s scaffolded well in a pedagogical framework, that can also help with learning transfer. But I also feel that we shouldn’t just focus in our assessments on the knowledge acquisition. We should also look at actually what skills are our students using? How are they engaging socially? How are they problem solving together? What are they focusing on? How are they progressing? Is there something that they’re really not getting? With the means of teacher intervention and use of augmentative technology, how can we help the students unravel a concept that they really, really don’t get? And actually that with formative assessment and the right interventions can really help a student progressing.
John Kleeman:
And would you say this applies both in sort of school and in the university and in the workplace, or more in some than others?
Carla Aerts:
I think it definitely applies in school, I think in university as well. There’s definitely room to do different types of assessment also based on coursework and based on project-based learning for instance. We know that that tends to work really, really well in certain contexts and that you kind of mix it up. You have formats of assessment that you mix up so that some learners will thrive in one format and maybe not do so well in another format. So you give them actually an kind of inclusive type of assessment and measuring if you like, that also gives them a lot of relevant feedback that will aid them through that process as well. And technology can do a lot of stuff there.
John Kleeman:
What about test anxiety and people being anxious about it, do you think there’s anything we can do to make people less stressed and less anxious around tests and exams?
Carla Aerts:
Well, I would eradicate exams and that level of assessment. But I would replace it with a sort of more continuous engagement and assessment formats that will help also the learning process. But to your question which is more related to that kind of exam or high stakes type assessment, I think there is potentially something we can do and that is maybe also spread it out a little bit more so that it’s less about cramming everything. And also underpinning it with the kind of more regular formative assessment at certain times are maybe more focused not on doing the mock exam or anything like that, but actually helping students sort of encounter assessment an format that they might not be necessarily that used to but that they can get their head around it. Like for instance, we know that AQA which is a UK exam board is looking at doing online assessment now as a format going forward.
There’s been a lot of anxiety around online assessment because suddenly sitting in front of a screen may scare of quite a lot of people. Well, you can’t do online assessment unless you have prepared students or learners for it.
John Kleeman:
Yeah, I think it might be more concern around the administrators rather than the learners, because I think learners find it very irritating that they have to answer questions on pen and paper or write stuff when they never have to write stuff in the real world and things. So I am not sure how much student concern there is about online assessments. It may be more people worried about the technology or new things.
Carla Aerts:
Yeah, there’s definitely that and there’s some parental worry as well. And then it also depends on the age group of the student of course. If you’re looking at smaller children, doing online assessment is not always straightforward.
John Kleeman:
No, no, no, no, for sure. So let’s move on perhaps to AI… Or perhaps before we go to AI, what about perfect assessment. We talked about perfect EdTech, what would perfect assessment be?
Carla Aerts:
For me, perfect assessment would be giving learners and their tutors or teachers an opportunity to engage in an assessment exercise, that could be peer to peer, that could be collaborative, that could be individual, could be personalized, could be all sorts of flavors of assessment. But also that it’s scaffolded and embedded in something that also gives something back to that learner in terms of really good feedback and a sense of achievement too. Even if the learner hasn’t done very well, there has to be a sense of achievement. I’ve done this assessment, I know I can improve. I know I need to look at X, Y and Z but I’ve done this. And for me also one form of assessment which is not really that common and is probably quite experimental, is also focusing on helping learners ask questions rather than just answer them. So they bring a level of critical thinking.
And this could be in a social context, John. This can be again in collaborative social context or individual, but actually helping learners frame the questions they need to ask. And this will also come back when we talk about AI because we are also increasingly living in a world that has got a lot of questions. And we also need to be able to ask those questions to identify our approach to a solution. And currently in education maybe we don’t ask enough questions, we are too focused on the knowledge acquisition itself. And asking the question is for me a missing element. For me that would be bringing a perfection in assessments.
John Kleeman:
So essentially I think you would say that maybe we would give people some stimulus or whatever and ask them to tell us some questions about the stimulus, and we’d then measure how relevant or exciting or useful the questions were?
Carla Aerts:
Yes, I would certainly kind of put that in there. And then the other thing I think I would like to see in assessment is also how people… And that’s harder to do, is sort of measuring how they work together, how they solve problems together, how they look at inquiries together or individually. But those kinds of approaches rather than just give me the equation for X, Y, Z or solve this equation.
John Kleeman:
And so how would we do that fairly because I suppose you’re in a group with people then you get assessed, you obviously would depend on the other people in the group and other groups might be different. So how would you do that fairly?
Carla Aerts:
I think it also depends a little bit on the type of assessment and what the outcome of the assessment is. If you did that at several times in a year, you could also mix up the groups. And for instance, we know from learning science and science of learning that actually if you put mixed abilities together, the outcome for the mix of abilities will be positives on all fronts. So typically we often have a tendency to say, ‘Oh, let’s put the kids who learn faster…” Which is a bit of a misnomer. “Then the others who might be a bit slower, let’s separate them and put them in like-minded or like-skilled groups.” That actually is not a very good idea. So if you mix and match then you can introduce a level of fairness. Getting kids or the students to work in the same group on a single project or a number of projects for the whole duration of an academic year or whatever the period is you’re trying to assess, without mixing that up over various assessments I think would be terribly unfair. But I think there are mechanisms to kind of mitigate that.
John Kleeman:
Well, that makes sense. And just while we’re touching on the science of learning, so if you want learning to stick and be retained what are some of the good ways of making that happen?
Carla Aerts:
So there’s obviously a lot happening around memory and how you memorize, and your working memory and then sort of transferring to kind of long-term memory and then you’ve got procedural memory and different flavors of memory. But actually one thing that we do know is that learning transfer happens only if you can learn something built on something that you’ve already learned. So there’s kind of building blocks in learning that you need to progress through. But the other thing we also know, and I’m not an expert in this field at all but I’m very interested in it, is that actually there’s something called learning transfer. And learning transfer relies on repeated practice but actually spaced practice. And that’s I think also where formative assessment has a really, really good role to play. It’s actually how do we space and also interleave the practice so you don’t just focus on one thing, but you start using the building blocks to interleave the learning to aid that learning transfer.
And then if you actually couple that with where it’s applicable because it might not work for every subject or every subtopic in a subject, where it’s applicable to a sort of application or a problem solving exercise, then you’re really going to aid the learning transfer and also mitigate the forgetting.
John Kleeman:
Yeah. No, that’s certainly what I’ve heard as well, that’s the three things that really help, one is distributing out your learning over time. One is interleaving it, so you jump around on different things. And the other is giving people retrieval practice so that they can practice retrieving, and tests and quizzes are a great way of giving retrieval practice. I think I’ve kind of just said a little bit of what you said in a different way, if that makes sense.
Carla Aerts:
Yes. One other thing that I think is quite interesting to add to that as well is actually where you bring the peer element in. And that is for instance getting… We know that if you can explain something to someone, you’ve actually really learned it or you will learn it even more during that explanation. So there might be ways in kind of assessment where you kind of set up exercises like that or use assessment tools to get learners to explain it to each other. Obviously that’s not an exam or a high stakes type assessment, but it could be very, very relevant in a sort of formative setting.
John Kleeman:
That sounds very interesting. So what do you think AI will mean for education, big change, small change?
Carla Aerts:
I think it depends. It will ultimately result in a big change, but it might take some time because we know that education is also quite slow and it also depends which area of education you are kind of focused on. AI for the workplace and for lifelong learning I think will go quite fast because you’re also dealing with an informal sector. AI at universities, I think universities are grappling with it but I think they’re engaging with it big time now. Their first reaction when ChatGPT launched in November last year was like, ‘Oh my God, all our students are going to cheat.” Which I found actually an outrageous reaction and I got really cross about it because what does that tell you about the trust that the university puts in its own students, and the mechanisms that it’s proposing to their students? So if the first thing you’re going to say is, “Oh, you’re going to cheat on your essay or your assignment.” That’s a pretty terrible thing to say. And no doubt some people will cheat, but that’s not the point.
The point is actually what do you do with this new tool to kind of help students learn better? And how do you deploy that tool in terms of, okay, you might need to write an essay, but we know that you might go to chat GPT to write your essay and none of the tools are really good at spotting that. So there are many ways in which you can flip that model and you say, “Okay, write an essay using ChatGPT, but now you are going to critique that essay.” You may be critiquing it first round with ChatGPT itself, and then you critique it as bringing your own perspective to that critiquing. Now, that as a learning exercise and an assessment exercise I think would be pretty amazing because suddenly the deeper thinking that’s involved in doing an exercise like that is pretty amazing.
And it still gives universities a sense of what their students are doing and how they’re engaging with these tools, and with this new form of what we call intelligence which has to evolve into a joint form of working together with human intelligence and machine intelligence towards a new type of collective intelligence, if you like. And I think definitely you can see signs of that in universities and students are using it. Many lecturers are still struggling with it perhaps, but it also depends on from university to universities and some are really embracing it big time. So at that level I can see it move reasonably fast. Obviously in the school sector it’s a totally different beast.
John Kleeman:
So essentially the future is maybe humans and AI working together and we need to prepare people for doing that?
Carla Aerts:
I think so, absolutely. And because AI is going to permeate the work place much more, we also know that with automation that actually highly skilled workers are probably going to be more affected. Well, the predictions are that they will be more affected than possibly some of the lowest skilled workers through the automation and through AI. So the problem will be is the people who haven’t connected or haven’t been trained or educated in the use of AI, will be the ones that are disadvantaged because they won’t have the skills to work in a workplace that does use AI or is very heavily influenced by AI. And it’s the same argument with some teachers who kind of say, “Well, this AI, I really don’t want to use this because it’s going to take my job.” Actually, the teachers who don’t use AI are far more likely to lose their jobs than the teachers who will engage with it.
John Kleeman:
So I think that’s very true. So let’s try and wrap this up with maybe three takeaways. But I think you’ve said probably about 16 things which are interesting here, I might be under counting there. So what would you like people to take away? One of them probably ought to be social learning because I remember when I first met you at an EdTechX in London, you were talking about how social learning was so important and was missing in a lot of EdTech.
Carla Aerts:
Yeah, so that’s definitely one thing. The other thing I think, and we haven’t really touched on that hugely, John, but I sort of touched on it, the other skills that you need, and one of the skills that I think is very important especially in the kind of AI context as well, is actually how we develop reflection and metacognitive thinking that we need to kind of introduce more into learning. And with the use of ChatGPT, there are plenty of really good models in which you can do that. And then the third thing is that we need to use the technologies including assessment to promote and develop learner agency because they will need it for lifelong learning. They will need it in the workplace, they will need it in the future, that sense of agency.
And that you’re also in control of what do I need to learn now because there’s a change happening in the workplace, or a change happening in AI or in technology or the societal force of technology. How do I engage with this? And that’s kind of the skillset that we need to focus in not only in education and in assessment, but also definitely in EdTech.
John Kleeman:
And so essentially learning to be able to learn effectively?
Carla Aerts:
Yes, and then also do that in a social context and be able to work in interdisciplinary teams and engage in that process more effectively with your peers, who may come from a completely different discipline than yours.
John Kleeman:
Thank you, Carla. I really wanted to interview you because you’ve got such interesting ideas and I hope people enjoy listening to them. So thank you. And thank you also to our audience for listening to us today. We appreciate your support. And don’t forget, if you’ve enjoyed this podcast why not follow us through your favorite listening platform? Also, please reach out to me directly at john.kleeman@learnosity.com with any questions, comments, or if you’d like to keep the conversation going. You can also visit the Questionmark website at www.questionmark.com to register for our best practice webinars that we host monthly. Thank you and thank you, Carla.
Carla Aerts:
Thank you, John.
John Kleeman:
And please tune in for another exciting podcast discussion we’ll be releasing shortly.