How would you explain to non-experts what augmented reality is and for what purposes it is used?
You see, it's a little bit like the magic that we read in our childhood novels about. Magicking something up. Making something appear. Creating a ghost. Moving objects with your fingertips. That is all possible with Augmented Reality, which is defined as the digital overlay over the real world, presenting audio-visual information registered in 3D space. AR is enabling us to see and hear an alternate reality, changed or even significantly enhanced, but still anchored in the real world surrounding us.
There are a lot of different technologies available today to experience Augmented Reality. Some of them are still very research-oriented, only accessible in labs, not ready for a wider consumer market yet. The majority of Augmented Reality is using either smart glasses for delivery or see-through mobile devices, predominantly presenting audio-visual information.
I should mention that there are delivery systems for other senses under development, for example, for providing haptic feedback. Haptic gloves can help you feel with your palm or your fingertips. Electro-muscular stimulation pads can be worn directly on the skin, allowing you to directly feel the impact of a bullet in a video game on your body. Some of these experimental technologies are really far distant still, like a cocktail experience where you have to put your tongue onto electrodes on a specially prepared glass, activating your sensory area for taste on your tongue with electric current to simulate the experience of actually drinking something sweet, sour, or salty, even when you are actually only drinking water. There are many more sensory delivery technologies that we can see in research that aren't there yet and aren't available for a mass market. But some way down the line they could, and nevertheless what is readily available already today allows us to create really very magical illusions at scale, ready for the use by everyone. No doubt, that over the next 5 or 10 years, we will increasingly see how Augmented Reality is penetrating into the mainstream.
Can you tell us something about challenges and risks related to implementation of new technologies in education?
Generally, the biggest risk to education is that we are spending time on something that does not pay off, that does not provide a learning gain, does not lead to meaningful engagement. This is a general risk to any technology that is used in support of learning and teaching, not just AR. Regarding AR, however, from the perspective of educators, over the last few years we seem to have entered a space, the realm of Augmented Reality education, where it is safe to use this technology. There are plenty of examples of what you can do with AR in specific subject areas or for general purpose learning, and there is quite a lot of guidance available as well on how to integrate it into the curriculum.
Educational technology is nothing without its link to the bigger plan of what we want to learn, even if that plan is driven by learners themselves and not so much by educational offers. We want to make sure that learning technologies are used efficiently, in support of intended learning outcomes, and in an engaging way. This is to mitigate the risk that we are wasting away our precious learning time on the wrong activity. There is ample evidence available that education with AR can work really well, if it is done well. But in general, about challenges and risks of the medium, I think that the same question as with the book: is The Book bad or is it good? I don't know. It really depends on the specific book you are reading, as is the case with any technology. Of course, there are risks involved that are particular to the classroom. That means if as an educator you're not familiar with this new AR technology, which is traditionally now delivered on mobile phones or on iPads, then you may struggle with keeping your class or course on track. But it is not any different to other mobile learning activities that we routinely use now for the past decade or so. If you as an educator enter the space well informed, then AR can give you a low-risk, high-reward activity that can positively influence the way people memorize, that can positively influence engagement. It is a fascinating, extremely engaging, very lively technology, if applied well, and there is plenty evidence that it can improve learning gains. But same as with the book: if it is a bad book, put it to the site, and get a better one.
How do you see the role of augmented reality evolving in education and training in the next 5-10 years?
I think we currently see a bit of a paradigm change in education. As such, many of the traditional formats of education are increasingly challenged, like, for example, the essay as the traditional form of assessment is being challenged by the facilities available through artificial intelligence. Interestingly, this challenge comes at the same time as advances in Augmented Reality are being made. AR is the medium for practice-based education, where you need to show that you can do something rather than just talk about it. I think in that sense the rise of AR in education is timely and allows us to up the game for systematic education and training. We can now suddenly put instruction directly into the workplace. We can put instruction into where the action happens. And we can do that in a systematic way.
We can leverage this element of apprenticeship of looking over someone's shoulder, which traditionally is limited to two shoulders. To take a close look, there are only very few people who can peek over your shoulder from behind and imagine what it would be like to be you, step in your footsteps, and do what exactly what you are handling and doing. With Augmented Reality, that ‘two shoulder’ limitation is gone. We can do this element of looking over someone’s shoulder better, closer, and in a way more exciting way, but at scale for millions of people. If we understand a specific area of knowledge and its practical application, we can build the system and the content surrounding that.
I think that this will bring an entirely new market opportunity along for education and training that previously was not directly accessible. We will see AR arise over the next 5 to 10 years in this market of ‘guided learning’, maybe even in a collaborative way, as it is a good steppingstone in the wider endeavour of systematising education and training in especially technical skills. It is reasonable to think that from there it cross-pollinates well to many other areas in education and training, bringing AR to these areas as well.
Over the last 10 years already, we could see amazing AR examples in almost any subject area. The tricky bit is how to get from there to the next level, where we have not just a single example showing off how great AR can be, but many, a multitude of experiences. Think about it: A student typically goes through one single learning activity in a few minutes, possibly a few times during a lesson. There are a several lessons per day, and several study days per week. That is a lot of learning activity, and, thus, requires a lot of content that we would need to support at the moment. Even if the area of AR learning shows explosive growth in recent years, it still is very nascent, and availability is comparatively low if you compare it with other types of activities available out there. But I'm very convinced that with innovation in the field that we have made possible, we can achieve the goal of using augmented reality-based education at a very large scale.
You are the winner of the CEN-CENELEC innovation award in 2022 for the ARETE project, could you provide some background information on this project and what it aims to achieve?
To say it with Confuzius:
“Just tell me and I will forget.
Show me and I may remember.
Involve me and I will understand.”
ARETE centres around the idea of learning by doing, of kinaesthetic learning, of, as we sometimes say, learning by experience. Engagement is the key to learning and in ARETE, we tried to pick up on this advantage of the new medium of Augmented Reality and develop a blueprint for the next generation of applications that use AR for the purpose of education.
We methodically developed the technology and approach for using Augmented Reality in education at scale in various subject areas, and then evaluated it in four large pilot studies, carefully analysing the findings from lots of teachers and students, feeding insights back into research and development, following a participatory, human-centric design approach. We looked at the next generation of Augmented Reality applications for learning from a technological lens, developing the content, content model, and pertinent standards.
We were looking at how we need to modify AR applications, how we need to evolve the techno ecosystem surrounding it, what usable authoring tools look like, investigate what guidance material is required for adoption, how we can lower the entry barriers for content creation, etc.
We evaluated thoroughly, originally with three, then with four pilots. The first one focused on AR for English literacy, especially for students with English as a foreign language background. We ran another pilot on AR for STEM learning, particularly in math and geography. And we looked at an area that has more to do with learning culture in schools, AR for positive behavioural intervention and supports, PBIS.
Halfway through the project, we realized that we need to add an additional pilot that focuses on teacher education, evaluating how we can support teachers in building Augmented Reality learning experiences with an authoring toolkit that we developed.
The four pilots were large. We tested with over four hundred teachers and over three thousand students across 27 countries. Plus some additional outreach activity in India and in 10 countries in South America. We found positive evidence of use of the technologies, validating what they can do in the right hands, verifying our technological approach. This included building an authoring tool and player called MirageXR, which is freely available for anyone to use, also available as open source (search for “MirageXR” on Google Play or Apple App Store, or on GitHub for the source code). Technically, we have implemented the cloud infrastructure behind the app as part of the Moodle learning management system, so it is easy to integrate it with a local school or university learning environment.
How do you see the ARETE project impacting the field of education and training?
I mentioned earlier, there are plenty examples of good use of augmented reality in in education and training. They are, however, often just incidental. 10 years ago, major corporations, schools, educational institutions were all messing around with Augmented Reality already, on a piloting level trying out what these new technologies can bring. When we wrote the grant proposal, we realized that the biggest blocker on the way to large scale success is that it is not sufficiently understood how you scale up from these high-intensity/high-cost initiatives to something that can be put into the hands of teachers routinely, something that can directly be used by independent learners as well. This was a key focus for us.
Looking into easier ways of creating Augmented Reality contents where you don't need to hire a software developer, a graphics designer, a story-boarder, a director, you name it.
All user groups have their own challenges. Students are not necessarily the most easy to satisfy. They have seen Augmented Reality apps quite likely already, in the gaming world or other use at home. Therefore their expectations are often very high regarding how things should be done and built, expecting the same visual quality they already know. This engagement from which is quite different from the challenges that teachers face. Typically, their challenge is more about keeping it all together, making sure you are aware who in the classroom is following and who is lost, even more so if it is not in the classroom but a home learning exercise. And making sure the AR activity fits to the rest. They care a lot about engagement tracking.
Could you provide some examples of how standards have influenced or shaped innovations in your field?
I've been active in learning technologies standardization for some years now, and I've seen several initiatives being seen through to completion. For example, at the time when Open Educational Resources became a movement, we have done quite a bit of work around search and retrieval of learning objects, and I believe that helped understand how will you need to describe and mark up learning resources so that others can find them and reutilize, maybe even in different contexts. I have also seen quite some work around learning design, understanding the process of creating and managing learning content better.
All these initiatives share that they have helped roll out new markets. Previously, any existing market potential or potential market was too fragmented. If you are a commercial vendor or educational provider, the entry cost was too high. The advantage of establishing interoperability, of ‘going digital’ and being able to share and exchange digital learning resources, is that the overhead costs of, for example, findability and delivery, become lower.
For me, standards are enablers that can create markets. This is the case for the most successful interoperability standards that I have followed, and this is also the rationale behind what we tried to do with the workshop agreement we proposed – CWA 18006:2023, eXtended Reality for Learning and Performance Augmentation – Methodology, techniques, and data formats (and the standards that we developed for that): help establish a market where previously there is none, so that nothing can hold back the development and uptake of Augmented Reality in education and training.
How important is it for innovators to stay informed and engaged with standards development organizations?
I think it depends on where you are in the innovation cycle. If you're looking at exploratory research, or the kind of blue sky thinking, that is not the right point in time to engage with standards development. The point in time where to engage is when you can positively impact on the lives of many, where you are on to something with utility for everyone.
In the exploratory phase of research, outcomes may be failure. Standardisation becomes more relevant when the research is becoming more applied, and research outcomes can likely be put to use, and where a pathway becomes visible to shape how many could benefit from specific research associated with it. It is the right time to engage with standard development, when there is potential impact.
Standards are a vehicle of knowledge exchange as well. If you are not staying up to date with the standards in development and the standards already published, as an innovator you risk reinventing the wheel. Standards are an important knowledge exchange vehicle. You need to connect. Research cannot happen in isolation, it always happens in a research frontier.
In science, sometimes PhD students dream of this situation, like hundreds of years ago, that there is this genius inventing something and writing that up in a book, and then nobody realizes for two hundred years until someone rediscovers that work. This is not how research works these days anymore, if ever. Research, if it is disconnected from the community, if it is disconnected from the research frontier, then it is at risk of isolation and of being forgotten, more than ever. It might be that today someone invents the wheel, but if nobody cares, nobody reads about it, it will simply be forgotten. Standards development organizations are both safeguard and curator, making sure that what is important is preserved and amplified, patiently engaging with innovation and research.
Finally, what advice would you give to innovators who want to ensure their work aligns with relevant standards while still pushing the boundaries of innovation?
The main advice I have is do not underestimate the power, but also the duration of standards development. Hereby, timing is key. You cannot come to a standards development organization with a ready-made specification and say here it is, please put your quality stamp on it and accept this standard that now should be executed across the world or across Europe. That is not how it works. Standards development is also a consensus building process. Standard development organizations are guardians of the process to ensure due diligence, preventing bias, no unfair taking advantage of. Multiple perspectives are heard and seen, and such a process takes time. The average duration for development of a fully standard is somewhere in the range of 36 months. That means if you're planning for a major contribution or having your say in the making of a standard, that means planning for contributing already when you are still in the process of researching it. With the right project management methodology and process, you can plan that it will take you there. But I am also very thankful for the instrument of the ‘workshop agreements’. They are form of pre-standardization agreements for European Research projects, which typically already have a broad consortium and a consensus building process built into the methodology, talking to all relevant stakeholders. This fast track to pre- standardisation is fantastic.
My main word of advice is do not underestimate the time it takes establish a standard. The safeguards of the process are here for a reason, and if you come with of something ready-made outside of this process, then it is not going to fly. This is not how standardisation works.
Prepared by: Jovana Korićanac
