Welcome to episode 40 of Lost in Immersion, your weekly 45-minute stream about innovation, as VR and AR veterans will discuss the latest news of the immersive industry. It's the 40th episode already, and about our sound problem, we made some corrections, we'll see if it stays. Thank you for your patience. So, Fabien. Yeah, hello, thanks. So today I want to talk about a very interesting conference that was released on YouTube by Douglas Lallemonde. So he's a senior director at Reality Labs, Meta, and he oversees the VR headset visual part, so visual development and what's to come. And he has this very interesting concept is the visual Turing test. So we know the Turing test is famous in AI is like, at which point do we, the interaction with an AI is not distinct from interacting with a human. And his concept of the visual Turing test is more like, at which point, wearing a headset will become like as if we were looking at the reality. And so he goes through a bunch of concepts that Meta is working on to go there. Obviously, we are not there, but there are a few things that we already talked about that he is making updates on, and new things as well in this presentation. So he has, he's talking about two challenges. The first one is the comfort of the VR headset. Obviously, currently, most of the VR headsets are quite heavy, they are large, the comfort of the eyes as well, and the sensation of the brain is not as good as it could be, at least not to have a perfect pass through. And, of course, the quality, the realism of the pass through videos and the cameras. So, again, he goes through different concepts. The first one being butterscotch, so we talked about it. It's a concept where the focal is variable, depending on where your eyes are looking at in virtual reality. And I actually had the chance to test it a couple weeks back. And I have to say it's pretty good. I was kind of impressed by how fast the change of focal is happening on that. I was expecting a lot more delays when switching the point of view. And the calibration as well was pretty fast. I just like to look at a few points that we are moving, like 20 seconds, and then the calibration was done. So, of course, all of the concepts that he's showing in this video, and the butterscotch is one, are not ready for production. It's just like, what could a VR headset become? But, yeah, it was pretty interesting to see and to test. Another concept is, so, butterscotch is a hardware solution. Here, what he's showcasing is a software solution to, so, you know, that one of the biggest problems now in the pass-through devices, it's the distortion that you have. Because the cameras are not exactly at the position of the eyes. And this is something that most devices have. And the AI model that this concept is using is trying to undistort the field of view and the cameras with AI models. So, yeah, that's another way of improving the quality of the pass-through. Another one is something that we talked about is to adjust the way that the reality is captured. So, using this kind of multiple focal points, multiple cameras here, multiple lenses, sorry, in this prototype. So, to avoid this distortion completely. And finally, something that is very interesting is he's showing a paper from 2021, which is called Reverse Pass-Through VR. And I checked it, it's a legit paper, it was actually published in 2021, which is basically the same concept as the Apple Vision Pro, which is basically to reproject the eyes on the outside screens. So, the goal here is for the Turing test to be for you, of course, if you are in a VR headset, but also for the other people to think that it's really your eyes. So, that was a very interesting conclusion for him. So, of course, we can say, oh, maybe Apple kind of did a copy of this for the Vision Pro. I don't think so. I think it's more like a coincidence, but it's very interesting to see the different concepts that they are currently working on. So, if this kind of technology is interesting to you, I recommend watching that video. We'll put the link in the description, which is a very interesting video. So, yeah, that's in a very short way, my summary of this. And I'm curious to know what you guys think. Maybe start with you, Seb. Yeah, it's interesting to see that he's considering that right now the version that are coming soon of the headset, like one or two years, will be acceptable for the Turing test, or maybe later for the variable focal. But yeah, do you feel the same having tested this device? Do you feel that it mocks the reality and you can feel that everything that you see in the headset is real? Or does it still lack also some graphic performances to really render heavy scene? Yeah, I think for the butterscotch, like the weight of the device, the fact that you can feel the engines moving when the small motors, they are now moving for the focal, so you can feel it. And the field of view, so the resolution is pretty good. I think it's very close to the retina, but the field of view is quite narrow. And I think he's saying in his video, like, we are not there. We still need like many, many years and iterations to go there. Yeah, it's like small steps. Right. So is it just a guess or does he has like a laboratory or a lot of documents to prove his point where it has been tested and confirmed that when it has this quality or this resolution or with this technology of varifocal? I think it's a guess from him. Yeah. Yeah, because I don't think until we have like real user testing, we won't be able to really know. And like testing during a long time, you know, so yeah. Right. Okay, thank you. Yeah, first of all, I'm glad that we've talked about all the solutions that he's presenting. So we are up to date to this. And well, I guess one headset that is missing is the latest Vario XR4 that we've talked about, because they are claiming that we can't make the difference between real and virtual anymore. So it would be very interesting to have his feedback about this. But I guess due to the pricing tag of this, and especially the availability of this headset, we won't have that much review of it, which is kind of a bummer, because I guess we would have a lot of things to learn about this headset. But when it's around 10k, I don't think that we would be able to have lots of YouTubers review or. So we'll see. But yeah, I get this headset might might change something at some point, especially if what they are claiming is right. But yeah, very interesting to see the procedure and all the different steps to classify all those new headsets. And about the reflection of the eye or the display of the eye. You said that maybe Apple or we could say that Apple copied this, but we know that there have been a lot of works about projecting what the user is doing inside the headset. I guess it was one of our first episodes when we had those googly eyes, like very cartoonish in front of the headset. And we made fun of this because it was very weird to see those googly eyes that were just representing where the user were looking. So there were lots of lots of works about this. So it's just not one paper. It's one among many. Yeah, yeah, I agree. It's it's kind of an easy joke to to make. There was there were many other people working on that. Yeah, I think he doesn't talk about the Vario because so he's from Meta. So he will only talk about the Meta headset. But yeah, especially something I think we already talked about it, but something I'm really curious to see on the Vario is the mixed reality cameras that focus where you are looking. I think it can be a real game changer for mixed reality. So hopefully at some point we will be able to test it. I guess Seb would be the one that is the closest to them. So maybe you you will see if you can catch it or or simply see one of them on on a trade show. Yeah, that's what I was going to say. I will see if they plan to be there. Okay, so yeah, we'll see still how many years until the visual Turing test is passed. Until then, yeah. Okay, so Seb now. Yeah, so on my side, I wanted to talk about this new device that has been new. Glasses that has been announced by a new DCM. Sorry, it's a new X2, which is a kind of, from what I understand, an upgrade from a Google Glass, where there is a release of some tracking is using the Qualcomm XR2, so quite a powerful processor. They are not linked to anything, there is no cable, you wear it and the processor is inside the glasses. So it's quite a small form factor for this one. And they've got several cameras embedded here in the, in the center of the, of the, of the device. So one cameras and a depth sensor from what I understand. And they are talking about having multi-language dialogue translation in real time, smart navigation with SLAM, so I guess they will have a good enough camera to do that. I hope so. They are not announcing the quality of the webcam they have embedded in the device. Seems like they have a nice screen, but like we can see here, the position seems to be quite centered, so it will be a narrow field of view. So information that keeps in the center of your field of vision. And, and yeah, they are mainly focusing apparently, from what I can see, on having like a mobile phone on your, on your head, and being able to call a person, listen to music, I guess, taking video or photo. But not watching VR or augmented reality full, full experience with 3D displayed around you. So I don't know what is your thought on that, if you saw the news, if you have any information that I missed. Yeah, I think, to me, there are a lot of devices currently that have similar goals, I think. So basically, you have, the goal is to have a very large screen in front of you, without the need of a screen. So like the NREOL glasses or the Xiaomi glasses as well. But on this one, what I'm worried about is, as you said, Seb, the screen seems very small. So I'm really curious to see how, like the, if there is really a possibility for us to watch a movie, or if it's just more small information. Like, as you say, the assisted reality kind of glasses, instead of like glasses to play a game or, or watch a movie. So, so yeah, seeing the size of the screen, maybe it's more assisted reality. Yeah, yeah, this kind of thing. Yeah. Yeah, I agree. I think that's the main focus on this one, looking at the glasses and the way they can display stuff, even the pictures they're sharing. It's really focusing on the small thing displayed in the center to help you in your daily life. Any thoughts Guillaume? Yeah, my first thought is that, compared to other manufacturers of this kind of glasses, they integrated captors, unlike the Ray-Ban glasses, for example, which are just smart glasses in a way that is just displaying is not displaying is just an interface to your smartphone. So it's more like you said, the supposed to be released at some point, the Xiaomi glasses, which were supposed to have some six degrees of freedom tracking as well. But we can't see any release date now. So I don't know if the project is dead at this point. You highlighted the autonomy, so I'm glad you found the information, which is about one hour. It's very, very short, even if you are taking your glasses for a full ride with your bike. I wonder why they chose the standalone device version, because most of the other manufacturers are using your smartphone as the computer power. I guess they should get better autonomy by doing that. So maybe they don't have any partnership with Google or other manufacturers for them to have this kind of possibility. And my last thoughts about this is I found it very funny that all the glasses have basically the same design. Well, the glasses frame is like always a Ray-Ban like glasses. However, I don't think we are all wearing the same kind of glasses. So it's kind of gimmicky at this point to see that if you want smart glasses, it should look like Ray-Ban's glasses. These were the kind of glasses I was wearing when I was a kid. So it's very funny for me to see that. But I guess this is the trend, so they should do all the same design. But yeah, very interesting to see. I didn't hear about this, so I think their communication is not perfect at this point. And as always, when we are seeing these kind of devices and the promises they are giving us, the main question is, is that real? And indeed, how about the field of view? Is it usable or is just one first version or prototype of what could be the real deal at some point? Yeah, I agree. They are not announcing any pricing yet. They say it's coming soon. So I guess they're still trying to find who they will sell it to and what will be the use case for those. If it's only navigation information, basically replacing your phone, but without the ability to watch movies or pictures or dig into a website through the glasses, then if it's only small information that you could have when you take your phone and look at it, and it's a heavy price, I guess, for this kind of glasses. So I doubt they have a really clear vision on who they will target. I guess they will target, after all, industries, because this could be useful in a warehouse or doing stuff with instruction being displayed in the glasses. That's still a good use case. And if they last three hours, then it could be usable for some use case in the industry. Apart from that, I don't see someone spending a lot of money for this kind of small information being displayed in front of you. And yeah, I think that's it for me on that subject. Okay. So for me, I have several news or rumors and dramas. So the first one I would like to talk about is the release of Gemini last week by Google. It's been a long time that we've spoken about AI, but now there are those main players that are releasing their own version. So there were Google, there were Meta, and I guess Elon Musk slash Gork. But I didn't see much news about this, despite the fact that there are some rumors and so on. So I won't speak much about Gork, but my main interest is about Gemini and the kind of drama associated with it. So I guess every one of you saw the release video on all the communication associated with. There were like three, four different videos. One mainly displaying and communicating that the model that is supposed to be better than any other AI model on the market, especially through their benchmarks and so on. And there were one video I saw about how their AI can help you develop new piece of code and how it is better generating code. And the last one, which is the main subject of this conversation, is their real-time-ish presentation of the power of their visual ability, the visual capability of their AI. And we can see someone interacting with the AI through a webcam and presenting different objects and actions, and the AI is interacting with the person very smoothly and very efficiently. And it's very fun because when I've seen the video with other colleagues, everyone was like, hmm, it's very powerful at this point. But the way it was presented was very clean and sleek, and people were saying, yeah, is it really real? We can ask the question at this point. And a few hours after that, we've seen the official communication about Google saying that, yeah, it's not really how it's working. We made some editing to make it look like it's powerful and it's not the real deal about the AI. So they cheat at some point here to make them look better than it is really, actually, and they communicate about how the actions are done for real. And it's very disappointing at this point because, well, at some point, it's better than OpenAI and ChatGPT4, and at some point, it's really not. So, yeah, I want to have your feedback about this. What do you think? Do you think that they made a mistake about this by trying to make it better than it is real, really? And we know that there were some consequences on the market the same day with the action of Alphabet just launched at some point. Now it's back up. But, yeah, some people were very disappointed about this. Fabien? Yeah. So, yeah, the video in itself is edited. So, like, the loading times are shorter, and they say that the prompts and the responses have been edited as well. Yep. So, yeah, we are not sure of the quality. But I would say, like, all videos, all, like, marketing videos are edited. I'm sure, like, the samples from OpenAI and ChatGPT are selected as well when they release something. What is a bit more strange is, like, they are not releasing anything yet. They are making announcements, but the API is, like, coming soon. I didn't find a link to register for any, like, beta program or any access to any API. So, like, yeah, is it coming? Or is it completely real or fake? Would it come at some point? Or are they more targeting a different audience than ChatGPT? Like, they already have BARD. And I don't know if they will upgrade BARD. Or if they are more targeting, like, the API side, business side of people who will use an AI through APIs. Yeah. Yeah. But if it works as well, like, I mean, multi-modal AI are very impressive. So... Yeah, it could be great to have a making of the video. Because, yeah, it seems too good to be true. It's quite amazing. But it's like having an AI person to talk to that really reacts to what it sees. Yeah. That's kind of amazing. But, yeah, well, the thing I would like to add to this conversation is about we know that Meta released also, like, Imagine. Imagine.meta.com, which is basically a mid-journey by Meta. Nothing very impressive about what they've shown or released. But the thing I would like to talk about is that we are seeing some giant tech company, like Google and Meta, releasing those AI. And we are kind of seeing that they are late on the market compared to OpenAI. And you said it, Fabien, they made the announcement, but nothing is released. When we are seeing OpenAI conferences, usually they are showcasing their new technologies. And just one hour or a few hours later, we can access it. So are they in panic mode? Meaning that they have to release something to show that they are not that late, but they really are. And we've already discussed about it. We knew that Google was very, they had some wake-up call at some point that they were very on the late train of AI and they are trying to catch up. But by showcasing this, making some fake-ish video, it's really not reassuring about their real ability to do AI. And we are seeing Microsoft, well, completely destroying the market right now. So yeah, I know that we had some discussion at my work about the position right now of Microsoft, which is like the one they had in the 80s, 90s, when they had some trouble with justice because they were getting the whole market. And they are basically in the same situation. They are a bit smarter about this just by making OpenAI on the side. It's not Microsoft for real. So they are playing on this, but they are, especially with Copilot, which is now, it will be released everywhere. So they have such power and they are really in advance compared to the others. Maybe we'll be relieving this era where Microsoft is completely powerful and almighty at some point. Yeah, that might be something that is also a bit scary is that more and more open source models, like free-to-use models are released. I think this week or a few weeks ago, the French company Mistral.ai, they released a model that seems to outperform GPT-3.5. And it's completely open source to use. So there is also like, I'm a bit on the side, on the safety side of the AI drama. So I'm a bit worried of the use of these open source models for spam, fake, all this kind of usage. GPT-3.5 was pretty good at spam. So if these kind of models are outperforming such OpenAI models, then there's some worries to have on that as well. Yeah, I can relate to that. I'm working right now with stable diffusion, seeing what it's capable of and for the realistic rendering of people. Because I did some research about deepfake and face swap. I talked about this and maybe I'll be presenting this to you. But yeah, when you are seeing the results of an open source model like stable diffusion and the ability to have your own personal computation station. You can generate any kind of face and create new identities for sure with this. So yeah, it's very intriguing and bothering at this point that it could be used by anyone. It's very easy to use now. You have some web AI embedded. So if you are reading some articles and doing some tutorials, you can generate very accurate faces that you can use for not so good actions. Yeah, I agree on that part. That's kind of amazing what you can do with this technology, but also scary at the same time. And yeah, very interesting to see the communities that are gathering to create new models as well. We know that for some, I see that there is an anime community. They are just gathering a lot of input data and they are trying to find and buy GPUs to create this because it's taking them weeks or months to train. So there is this side of, I would say, enthusiastic that are trying to improve their models and bring them to the open source models that are available. But yeah, the main thing that is slowing down this evolution is the hardware and the price of it. So they are doing this, those patreons and so on, and people are giving up giving money for them to build their farming. Yeah, I don't know if you can see that. But it's a rack of GPUs basically for them to be able to do this faster. And this is what is the main difference between those huge players and individuals. And this is the only way. Well, this is the only thing that is slowing them down. It's a GPU power and the time for them to train those models. Okay, so next news. I saw that Apple is trying to release the Apple Vision Pro on Android. And I'm not sure if you can see that. I don't know if it's because they are focusing on the second version and they would like to release the first one as quick as possible for them to iterate. I saw that Apple is trying to release the Apple Vision Pro on Android. They are trying to make it faster. I don't know if it's because they are focusing on the second version and they would like to release the first one as quick as possible for them to iterate. Maybe at the end of 2024 or at the beginning of 2025. It could be a strategy at this point. I don't know if those Apple stores are ready for release. Because we know that there are a lot of organizations for them to have, especially on the medical prescription. Because everything should be in the Apple store for you. If you are wearing glasses, you should be able to have your prescription and specified lenses to work with the headset. I don't know if you went to an Apple store recently to know if something is up there. But on my side, there is nothing showing in the Apple store here in Quebec. I didn't see anything yet. I saw the same information as you. I saw as well that Apple is supposed to target 400,000 shipments at launch. Which seems very high to me. I don't know if it's accurate or if it's just an Apple effect of people in the industry betting on the device. Thinking that the end user will buy it. I don't know. It's very hard to predict. Is this release planned worldwide or only in the US? Because they were thinking of a deployment in the US at first. I guess in the first place it will be in the US only. As usual, I would say. Just one question about this. Would you buy it if it comes in January? It could be some kind of placement. We know that the first iteration of Apple products could be worth some money in the next 10 years. So we should not open it? Keep the box closed. That's a very good question. We didn't decide yet. I think it all depends on our clients as well. If they are open to this kind of technology and this kind of price. Maybe. The last news. Maybe you saw that there was a main event about video games last week. It's a game awards. There was a VR section. The game that won is Resident Evil Village. The discussion was started by Tom Fiske. He has the immersive wire. Shout out to him. One of his main reflections is that during this kind of event, only the PS5 games were showcased. He finds that maybe the video games community is not aware or is not willing to share this environment with the other VR community. Which is a very specific one. He has a very good question about why some main media like Upload VR or Road to VR are not invited to this kind of event for them to be able to vote. And showcase the other side of the VR community and VR games. Which is not specifically PS5 games. I find this reflection very interesting. Do you think that the video games industry is not interested or is protecting their share and trying to bring back VR towards their field? And not showcasing that much what is going on in the VR. Maybe more independent VR games community. What do you think? It's very interesting. Because on the Quest, there are a lot of very nice games. As in Screed, which may be available on the PSVR. I don't know. It's quite nicely done. It's very possible that there is a... I don't know if it's a protection or... What can I say? By default, putting aside the device that are not super performant. Or that don't have the processing power of PS5. I don't know. Maybe it's reversed. It's to showcase more this kind of game to sell more VR headsets just before Christmas. Because I don't think the numbers of PSVR devices is very high. I don't think there is a lot of community on PS5 right now. Or at least I didn't see a lot of review or video of people using it. The numbers are lower than expected. But I don't know how much lower. You're right. It's maybe just a communication campaign for them to sell some PSVR. It could be that. It looks awesome. Buy a PSVR and experience it yourself at home. Yeah, it makes sense. Fabien? Just very quickly on that topic. I saw that during Black Friday, the Quest 2 sold twice as much as the Quest 3. The pricing was really different. I think maybe it was a strategy to get Quest 2 out of the way. But yeah, it's interesting to see either way. That the Quest 2 outsold the Quest 3. Yeah, I think it would be the Christmas present if there is something VR. Under your tree, it would be a Quest 2. Because people are not that naive or whatever. They saw that the Quest 2 will have some major updates in the upcoming months. And the device itself is very interesting and very performing. I guess it was the smart choice during Black Friday to buy a Quest 2. Okay, anything more to add? Okay, so it's a wrap up for today. Thank you guys for your interventions and topics. And we'll see you next week.
