Matthieu Labeau: “How Gaming Clears the Path for Cinematic VR”

Justine:                      Good morning. It is our last day in Cannes, it’s Saturday Cannes XR. We are here and my guest this morning is Matthew Labeau. He’s from Starbreeze welcome. You have something to tell us about what Starbreeze is doing at Cannes XR.

Matthieu:                 Thank you. Justine thank you for having me. It’s really a pleasure being here in Cannes again. yeah, I can explain to you a little bit what I’m doing here and what Starbreeze in general is doing and what my team in Brussels is doing with vrs. So I’ll explain where I’m coming from maybe. And so I can explain how we develop this project and the whole, we are now belonging to Starbreeze?

Justine:                      Yeah.

Matthieu:                 So basically I started in VR five years ago. we started this project called ‘Presence’, which is a technology for volumetric movies. So it’s a volumetric movie format.

Justine:                      Right.

Matthieu:                 And so we created a startup around this technology from our studio that is also based in Brussels, which is Nozon. There’s only hours to do. So we are also creating content for VR.

Justine:                      What kind of content are you mainly doing?

Matthieu:                 We created two movies until now. That is linked to interspace experience that you can still experience in Stockholm I think. And another one is ‘Construct’. It’s actually a movie with the robots fighting. It’s a, it’s a very, very interesting movie. It’s a, it’s an action movie in VR. So I believe it’s, it’s actually the first action movie in VR and it’s the first real high-end job that we distributed it in our format, in our presence format.

Justine:                      Is this the one with robots like jump out of a helicopter and or something like that?

Matthieu:                 No, it’s not that one. So I guess robbers are using VR quite much. It’s quite much so Construct is distributed in France, will be soon distributed in China. We are partnering in general are with Iconinc Engine.

Justine:                      Well tell us more about Construct in your partnership with Iconinc Engine.

Matthieu:                 Basically Construct is is a movie about robots fighting on the construction sites. That’s a movie that we finished last year. it got a two year awards, so including one from the audience. So like the most appreciated experience for by the audience the public award. so yeah, it’s a very important over to get and we showed the movie on the Iconinc Engine seat and platform at Cinema Con NAB this year. So that’s a very very cool partnership that we have there. we are excited to see the possibilities of their mix platform. Basically, you’re start watching the we adapted the movie to, for their platform. So you start watching the movie on their panoramic screen and then you put the headset on and you are inside the movie a new switch to VR basically. that’s the adaptation that we that we made for their for their platform. And we are going to continue distributing Construct as the first example of what we are able to do with our technology presence and with our team in Brussels, our creative team in Brussels. Basically the original IP is not from us. It’s showing also how you can adapt a story and material 3D materials that had been created for standard cinema. So it was, first of all, it was made for 2D flat experience. And we took the assets, we took all the material, we really did everything we rendered in our format. We recreated the experience to make it like a watchable in an enjoyable and great in VR. So that, that’s what we did. And it shows also the possibility of doing that with other contents. That’s what our technology enables.

Justine:                      What kind of content?

Matthieu:                 We basically, just to take an example so the tools that we are providing to content creators, they are the plugin on a standard visual effects and 3D animation pipeline. So let’s say that someone developed a, created a blockbuster. These blockbuster is full of 3D. you have a scene let’s take an example. We are not working with them, so I can take like, let’s say we have Game of Thrones, you know, like it’s like or, or such kind of movie or Marvel movie, you know, and suddenly you can, with our technology, you can rerender the 3D assets directly in VR with six degrees of freedom. So we’d be able to have like a gigantic dragon or monster, exactly the same than in the movie, right in front of you with the same quality. And you’ll be able to actually…

Justine:                      Walk around?

Matthieu:                 yes, walk around depending on, on the experience degree, but at least like get closers from different perspective. And I have really a feeling to be inside of the movie. That’s really our ambition and that’s what’s our technology enables and that’s what we are providing to content creators. So Construct was a first example of that possibility. And a I think it succeeded quite well and people really enjoyed it, but we want to go further. We want to to do it with with more famous ips with even more qualitative contents more ambitious projects. That’s really what we, what we would like to, to achieve.

Justine:                      Well, that’s interesting, but let’s go back a little bit because I had the pleasure of hearing a little bit about your history before we went on the mic. And Yeah, I’m curious how you went because you have a fascinating career first as an academic and in a very, very specialized subject matter. And I want to know how you give us, you know, tell us a little bit about that and how it led or did not lead you to VR, but how you got there.

Matthieu:                 didn’t really lead me to VR. but yeah, I actually started my career as an academic teaching literature and art history in Budapest, in Brazil. I specialize in African literature and sub Syrah in the African literature in French. I published a book about the, the new generation of African writers. That’s my first career I would say. And so I’ve been always fascinated by history, by storytelling. And and that’s why when I met Tristan and the team at Nozon and seeing their ambition of, of putting people inside a story, it’s actually something that fascinated me.

Justine:                      Well I can see the link with that with, you know if you’re fascinated by literature, that story for sure. And Art History is certainly looking at, you know, the aesthetics and understanding the meaning of the aesthetics. I mean, that’s a very good and strong basis I think to come into VR.

Matthieu:                 I think it’s also like a child dream in a way, you know to, to be able to jump in, signed your, you know, when you open a book basically you are kind of jumping inside the story and you brain is working as increasing the images, it’s creating the images and basically you would love to be there with the, the heroes of the book are always the characters and being able to do that, to actually achieve it in the virtual world and being able to put the headsets and, and being teleported inside a movie. It’s something that is really fascinating.

Justine:                      Well, and being fascinated by another diaspora. So the, you know, sub Saharan African diaspora. Yeah, that’s another world. And so, you know, at least with VR, we can visit these other worlds and much more…

Matthieu:                 That’s something a little bit different. But I’ve been always interested by, by the relationship between being an artist and the world that is around you. So engagement and like, Oh, do you like what, what do you talk about as an artists, you know, especially when you are coming from a developing country or from a country that is experiencing troubles. So that’s also something that VR can help in terms of storytelling and in terms of making people understand what it is to, to live in a certain area, to live in under certain conditions. so, so we are as a whole is a fascinating medium to make people understand a very different way of living that, that what they are actually experiencing on their, on their everyday life.

Justine:                      So, your previous careers still does match up to your current course?

Matthieu:                 Of course. And that’s why I’m enjoying it as much as I I do for the moment, you know.

Justine:                      Tell me what happened. So you had an experience, you’re telling that what made you take the leap into the world of headsets

Matthieu:                 It’s basically meeting the, you know, it’s basically life, you know, you meet some people at a certain moment and you, and they are bringing you to their, to their dream and you just like, okay, I will join you there. It’s not basically, that was not my initiative at first to say like, I’m gonna develop something in VR. I just met some, some great people at Nozon that had that dream. and they needed someone to to develop the business, to develop the project, to be there alongside with them, you know, and, and that’s what we have done now for five years, you know, building this project together. Going from a startup to being acquired by starbreeze that is based in Stockholm. It’s actually a game developer and a publisher.

Justine:                      Okay.

Matthieu:                 That invested a lot. from 2014 in VR starting, I mean, acquiring a a company, developing a headset, they rebranded it, named the star VR. So that’s a project that that is on hold for the moment. But the, a, that was basically helping defining the standards for visual quality that you need to have for headsets to, you know, to be successful in the markets in the future. So it was very high resolution headset with a very wide field of view and it was the first of its kind and now it’s defining, defines, you know, what, what headsets will be in the future.

Justine:                      Well actually we hope are lighter and more enjoyable.

Matthieu:                 Yes. Yeah. But that’s basically the, it’s common. It’s coming, it’s coming, but higher, higher resolution as well, you know, and wide field of view. It’s something that helps the immersion and that’s what the Star VR project developed. So we’ve been working with them in 2016, 2017 and 2019 mostly showcasing what we believe a high end cinematic should be. I mean, Star VR project is on the wall, but we are pursuing our, our project ourselves with presence, trying to push new project that have this better quality like and really we are trying to achieve very, very high end visual quality and very comfortable immersion and bring a storytelling that makes sense for people to enjoy that kind of movie. So we I can tell you maybe a little bit more about what we’re going to have in the future. Yeah, so we have one, one project now that is under development. So we are raising money for, it’s looking for investors. I don’t know if you know, Minuscule, no? Minuscule basically is a series of short format, very short format for television, like two minutes. and telling stories about insects, having small stories and small interactions with each other. And the main character is a beetle and decreed it like two feature films afterwards. The latest one was released the beginning 2019. And now we want to make a VR movie with this IP and in this world. So I think it’s going to be very, very very fun because the old world is about playing with the scale of the insects being at their size or being like a giant for them or, you know, you can play a lot with that. And it’s also a very funny world. Those insects are actually a, it’s a cartoon, you know, so it’s gonna, it’s gonna be for a very broad audience. so that’s one project that I can talk about because we are openly looking for coproducers. Another, I mean, my projects is that I really want to to see what’s going to be the distribution for cinematic content VR cinematic content. So that’s why we partnership with Iconinc Engine because I think we have a very ambitious projects and their way of developing a platform that is mixing screens and VR is very interesting and I’m also looking at at very different ways of developing of distributing, sorry, VR cinematic content, a multi viewer, you know, having like a lot of people experiencing it at the same time. How do you build this kind of a, do you need interaction? Do you need people just to see each other? just to, to see the reaction that they have in front of the contents. You know, what, what it is that makes, makes us want to go to the, to the theater or to the same enjoying and enjoying a movie together. You know, that’s what is missing in VR for the moment as well, you know, because you are just there alone.

Justine:                      The social aspect.

Matthieu:                 Yes. The social aspects for cinematic VR is kind of missing for a moment. And that’s why, you know, a multiplayer games are working so well in VR now because you are going there playing those games with your friends or with people that you don’t know about it. Yeah, exactly. So you’re going there, you’re enjoying a game together. Then you talk about it after having a coffee or a drink or a burger or whatever. And and that’s what makes it so successful and we need to have that we cinematic content as well, you know, so, so I think cinematic content will be very successful in certain circumstances. Like you want to relax, you’re going home, you just put your headset on and you just relaxed in front of, have a very relaxing, you know, inside of a very relaxing experience. So that’s one thing. But if you want to develop entertainment further, I think you’d need to develop the social aspect of the experience.

Justine:                      Oh, fun. What I’ve heard and seen, I think a lot of companies are, you know, figure that out and are looking at different ways. You know, it could be in Germany and watching your headset altogether at one time. It could be more AR experiences that you can do together and nobody’s in the headset. So it’s good. I mean, it’s good to see the industry push along those lines and say we need to make it less solitary and also a little more public.

Matthieu:                 Yeah. Obviously we are not, and I’m not the only one thinking about this

Justine:                      It sounds like you’re on the right track, I’m just trying to validate that.

Matthieu:                 Yeah, of course. Of course. And, and that’s why I think it’s very important because we see everyone trying to solve this problem. And you know, for the moment the industry is sometimes taking a shortcut, like, okay, Games are providing this, the social aspects, so let’s go for games, let’s distribute games. You know, but I strongly believe in the, in VR cinematic and the power of storytelling. You know, you don’t want to always to, to impersonates a character and play a game. Sometimes you just want to observe, to observe, you know, and to, and to be told the great story. Yeah.

Justine:                      I think that is one thing that I mean there is usually, or there should be some kind of story in games. At least there is some story. Maybe not in Beat Saber, I mean, but yeah, but usually there is, and I think story, what we all forget is, you know, that’s gone back since, you know, we are prehistoric, drawing on caves. Stories been in our lives. It’s hardwired into our brain. We can’t, you can’t get around it. So of course it’s more powerful. And of course we should look at ways to enhance and improve the technology. So we’re better storytellers. And you’re right. I think games are a shortcut. I mean, they’re important and games are important. They’ve helped advanced the technology, they have helped familiarize people with the technology. So it’s good stuff. But it’s good that you’re also coming back to thinking about what do we do with the story, the one that is, you know, a primal need.

Matthieu:               I totally agree with you. I mean, it’s, it’s really important to to be able to tell stories in a very powerful way. we’ve been doing that for thousands of years. and, and I think VR is a powerful medium to tell stories. So, so we should really explore it to the end, you know, or explore to what it can really bring and solve those little issues that makes people maybe like, or make the industry less profitable for a moment. Because this is the big thing, you know, it’s like, it’s the business around it. It’s like, how can we make this easy to distribute profitable for everyone? Because that’s, that’s how we’re gonna achieve.

Justine:                      All right. Well tell me a little bit about volumetric capturing and how, what it is specifically about volumetric capturing that you see helps move the story forward, like, okay.

Matthieu:                 Yeah. So that’s what I’m going to talk about later today.

Justine:                      Ahaa.

Matthieu:                 We have a Stereopsia here and XR4all is organizing a talk around the volumetric capturing and volumetric movies and volumetric storytelling and how it helps Hollywood moving forwards in, in different aspects. So I’m going to talk about volumetric. How do you capture humans for a volumetric experience of six degrees of freedom experience. For us basically we are not developing those technologies. It was a, it’s more like a need for our own experiences. So to roll back a little bit to what we are doing, we are rendering images from 3D content the same way you do for blockbusters. So we can render transform or robots or the apes from planet of the Apes. you know, so this quality of, of CG, we can achieve it in VR. That’s what we are providing. But we needed to have humans in those experiences, in those stories. You know. So I mean, CG content is great, but if we really want to build the industry further, you need to have humans in those stories.

Justine:                      I kinda like to think of myself as not being forced identify with an ape or a car.

Matthieu:                 Yeah, exactly. So, so at some points you need to have realistic humans in those experiences because we can have great CG. That’s, that’s the fact we proved it. So, but how do you bring humans in those experiences? Humans that look very realistic, that have a high resolution that don’t look weird because the did the problem now is either you get a poor quality or you get an uncanny experience.

Justine:                      They almost look animated to me. Like you know, or depends what in drawing or something.

Matthieu:                 It depends how, I mean not to enter too much into details, but basically there are two main ways of, of, of capturing or bringing people into a volumetric environment. It’s either you do a video capture either with cameras that are around yeah. Around, yeah. Around the character. Like at stage. Yeah. It’s like a stage full of cameras all around the character and you do videogrammetry and then you get like a full video of, of the person. but the resolution usually is quite low. You have a lot of artifacts, so you need to clean a lot. so it’s not perfect. And you have also the light field technology, we create a lot of content. And for the moments you have a problem of the minimal distance between the cameras. And the actor. So what makes, what makes VR amazing is this proximity and this presence, that’s what we call our technology presence. And it’s, it’s the ability for you to be at 20 centimeters of your favorite actor. You know, that’s, that’s basically what we want to achieve in VR. And I have this actor looks real and not uncanny at all and not animate it. So the other way of doing it, it’s actually capturing a fixed model of that person and reanimating it with motion capture all with by hand, you know, but that looks, uncanny, it doesn’t, it’s not fully working for a moment. You still have the feeling that it’s not a real person in front of you. So it gives a very, very good quality. But it’s still uncanny. I mean our brain is, and we perceive the fact that we have another human in front of us is very complex. It’s, we have been building that for thousands of years because that’s how we, that’s how we interpret the what the other person wants so we are able to to understand a lot of expressions from this person if this person is feeling good, is feeling annoyed, is feeling excited,

Justine:                      There’s a sixth sense that we’ve developed it because we have to be able to ascertain and, and milliseconds this person a threat or not.

Matthieu:                 So we still have, you know, our ancestral psychology. Exactly, exactly. And it’s coming from so many little, small expressions on your feed that are very, very hard to reproduce. It’s extremely complex. So we are not there yet. If we do reanimation, if we do reanimation a, it’s, it’s what you say, you know, they look like animated character and that that’s going to be solved. But it’s extremely complex to, to achieve that. So, so yeah. So bringing humans in inside volumetric environments and make volumetric movies or movies with six degrees of freedom where you can actually move freely in is extremely complex.

Justine:                      Last question. Matthew, what’s next for you?

Matthieu:                 What’s next for me? I want to, I want to continue that because I think we are not there yet. We are not there. I mean my vision of what a, a good VR movie is, I haven’t seen it yet. So I still want to, to push it further. And for that we, we need the distribution to be in place so that we can get the finance, we can get like the money for the production. It’s an emerging market technology world. So, you can not expected to be able, like to build like a 10 millions movies easily. You know, it was at the beginning it was possible for a few movies to get that kind of money. But that was because everybody was excited. Now it’s a real, it’s getting real. So it’s, it’s a real business. You need to develop it as a real business. And for that we need the distribution to be in place. We need people to, to actually have access to the headset because when people have access to the headset, they enjoy it. Everybody enjoys it. It’s like I haven’t seen anyone who said like, oh, that’s crap. When, when it’s a good experience, everybody enjoys it and people are ready to pay for it. So we need to put the distribution in place and that get money for making high quality VR cinematics. So that’s basically the future is linked to business. You know, you can have like big dreams, but, but we need to build a real real business around it. And that’s why I’m trying to push with my team and with all the people I meet here in Cannes. And I think I think we are going to achieve it.

Justine:                      That’s a great note to end on. We like positivity by the industry. Thank you so much for joining us this morning and keep going.

Matthieu:                 Yes, we will.