How immersive technologies (AR/VR) will shape our future | Dinesh Punni | TEDxTUBerlinSalon

Transcriber: Hallie Lawson Reviewer: Ido Dekkers This is the first augmented reality headset developed in 1968 by Ivan Sutherland. And it’s called The Sword of Damocles. It’s called this because it literally can kill you if it falls on your head.

So I have a question for you. Would you wear this one in public? Well, I guess it’s a bit dangerous. This is an early prototype of the Oculus Rift developed back in 2012, and we see already it is a lot smaller and most likely won’t kill you

If it falls on your head. Same question. Would you wear this one on your head? Not too sure yet. Let’s see what’s next. This is how AR and VR technology will look in a few years from now. By the way, these are just normal glasses.

But here, in fact, we see a few companies already with their early concepts. On the left side is Nreal, who have already launched a product in Europe a couple of days ago. On the right side, we see Intel with their latest AR glasses or some concept of those.

And on the bottom, we see already some early concepts of the Apple AR glasses, which are about to launch in the next couple of years. So I see a trend here. Many years ago, computers could fill whole rooms, and with their weight and size, they definitely could kill you as well.

Those were bulky, heavy, expensive machines that could only do one thing, so they were not really efficient. Today, most of us can have a phone inside of our pockets. And these things are much more powerful than these big machines. And they’re also less expensive and a lot smaller.

The same is going on with immersive technologies. Immersive technologies, so AR/VR, are past the hype now. They are productively being used in many industries outside of just gaming and entertainment. Many people and companies see these benefits already, and in a few years from now,

We will have to completely reset our mind on how we interact with digital information. My name is Dinesh Punni, and over the past six years, I have researched, developed and shipped many AR and VR projects. Today, my goal is to inspire and educate people all around the world

So they can create amazing AR and VR apps. I do this by teaching them how to code and design for these technologies. With all these words going on right now: AR, VR, MR, XR, immersive technologies. Let’s define them now so we are all on common ground. Let’s start with virtual reality.

Have you ever wondered what it feels like to be in space, or a time or place you have never been before? You could be on Mars, you could be in the past hanging out with Albert Einstein, or you could be in an alternative universe where cats rule the world.

With VR, virtually anything is possible. VR replaces your existing world with digital content. Imagine being the size of a tiny human cell inside of the body to study it in comparison to reading about it in a book or watching a video on the internet.

Imagine you don’t have to travel 17 hours on a plane just to meet with someone who probably has a sweaty handshake. And then on the next day, fly all the way back, feeling jet lagged and exhausted. Instead, you can just meet virtually from the comfort of your home. It will feel quite similar.

And also, it is a lot less expensive, it saves money, it saves the environment, and, really important today, it’s pandemic proof. VR enables a whole new universe of possibilities. Now, think of instead of replacing our existing world, we want to enhance our existing world with digital content.

This is called augmented reality. With AR we start to use our environment as a window to our digital world. Imagine we can use our existing world. We can interact naturally with content. And all of this is just much faster than typing it on a screen

Because we can just speak to it. We can feel it just more naturally. Think of buying furniture for now. This can be quite stressful. You have to buy it, and before you even buy it, you have to do a ton of research.

Then you have to buy it. Then you have to wait a couple of weeks. Then it comes to your home. You have to build it up just to see that you maybe don’t even like the end result, or even worse, it doesn’t quite fit your apartment.

With the HoloLens, for example, you can previsualize anything before you buy. And this really feels like it’s actually in your room with these holograms, 3D holograms. And the coolest thing, if you don’t like anything, you can switch it in a second and then just try out new things.

This also will save a lot of time, money, stress and probably many relationships. Imagine surgeons have all the vision, all the information inside of their vision already, hands-free, so they can really focus on what matters, which is the surgery. All of these examples are being used today already.

Now we talked about AR and VR, but what the hell is going on with XR and immersive technologies? Well, XR stands for extended realities and covers all these realities: AR, VR, and also MR, which stands for mixed reality, but for now, we can just use it as a subset of AR.

We don’t have to get too technical here. So it’s an umbrella term covering all of these technologies. Same with immersive technologies. It’s a whole one word for all of these. I believe that at some point in the future AR and VR technology will merge together

Because we see already that with many AR headsets, if we have the full view and we have it on full brightness, it kind of feels like VR already. And with VR headsets where you can enable the cameras to see the outside world, they’re also acting like AR cameras.

So there we see that these two, at some point, will merge together, which is why I prefer the term immersive technologies. Now, you may think, well, this all sounds cool and great, but I’m not a surgeon. I’m also not Iron Man, so… why does it even apply to me?

And isn’t it too far in the future anyways? I want to share two examples. The first is a personal story of mine, and the second is a live demo of how I believe AR and VR will shape our education system. So many years ago, back in 2017,

When I was still working as a student here at Technical University Berlin, we used VR on a research project to treat stroke patients. So imagine you had a stroke and you can’t move your left arm anymore. The conventional therapy is called mirror therapy, and it places a mirror in front of you,

And you have to heavily focus on the mirrored image so that you trick your brain that it’s actually the paralyzed arm. And if you do it over a long period of time and really repetitive, then this actually makes you regain the ability to move your paralyzed arm again.

We built a system in VR, of course. So using a VR headset, but also hand tracking sensors, eye tracking sensors so we knew exactly what the patient was looking at because the eye tracking sensors were inside of the headset, and a machine-learning algorithm that helped us to determine how well

The patient is doing with the exercises. So we can adjust the levels automatically. And by a click of a button, we can just change the arm from left to right or vice versa. And this enables us to get the patient really focused on the test

Because the patient is not being distracted by anything that is going on in the real world. Because one of the beauties with VR is that we can perfectly create the environment the patient is seeing. And this is why VR is being used so heavily for medical training and exposure therapy.

But also many companies use VR for immersive training and education. So let me ask you a question. How do you learn most efficiently? Well, there’s no one size fits all answer to this, but here we see the Learning Pyramid of Edgar Dale.

And generally, we remember 20 percent of what we read and hear, 40 percent of what we see and discuss, and about 70 percent of what we practically do and teach. So kind of obvious… we learn best when we actively do things. Unfortunately, most of learning today is still very passive.

So what could the future of education look like? Well, let me show you. So this is a holographic computer called the HoloLens 2. And this device empowers me to see holograms inside of my real space. So let me just open it real quick. There we go.

And now this device is equipped with a bunch of sensors. We have eye tracking, so it knows where I’m looking at, voice recognition, hand tracking, and really important, spatial understanding. So it really understands the room we’re in. Let me just start.

Imagine the following scenario: you want to learn about the human anatomy. Of course, there are many books on this already. But the problem is these books teach in 2D on a concept that is 3D. We are three dimensional creatures. We think in 3D. Our world is in 3D.

And also we best learn in 3D. There are studies that show this. Now I’m seeing a skeleton right in front of me, and this looks scarily real already. And we see already I can get a spatial feeling of all these parts, where these are.

And it’s not like just watching it in a video or looking at in a book. It really is like literally it could just be here. The cool thing is this application can also be interactive. So if I want to see more, I can just switch to the muscle system.

And quick disclaimer: you are still seeing a 2D representation of something that is 3D. You are not watching with 3D lenses. So it’s kind of hard to exactly project the feeling that I have right now. But to me, there could be like a real human, while with the skin off

And stuff like this, but still. And I can see the muscles. I can really see where they start and how they are aligned. And I’m not a biology student, but it helps me really to understand all the details. I can watch inside and just get a much, much, much better view

Of anything in any book or video. This concept is not only applied in biology, this could also be used in chemistry, in math, science, physics, history, everywhere. So this is the future of education. Fortunately, it’s not only the future. Let me just turn it off real quick. There we go.

Fortunately, it’s not only the future, it’s already being used at present. So, let me give you one example. Walmart, you probably have heard of it, just bought in 2018 17,000 VR headsets to train over one million of their employees. So what do they do with this?

They use VR training for soft skills training, customer training, and also highly dangerous training simulations like prepping for Black Friday. And there are many, many more use cases, of course. But you see already where this is going. So now we’ve talked about the past, the origins of AR and VR technologies.

We’ve talked about the use cases today. So what about the future? Well, we now have enough computing power to create immersive worlds that our brains perceive as real. What does it mean? Think of science fiction movies like Iron Man, Ready Player One or Star Wars.

At some point, I believe our physical world is merging together with our digital world. So, what does it mean if more and more people start wearing these glasses on a regular basis? Think about what artists can do. Think about what educators could do. Think about what you could do.

I believe that immersive technologies will shape our lives just as much as the internet, smartphones, and social media. VR technology allows us to live in a new dimension, and this is the reality I’m excited to live in. Thank you.