Danilo Moura: “Experimenting with the strengths of 360 video and volumetric”

Justine:                      Today I am with Danilo Moura and he has made seven miracles, which is part of or made possible by the HTV vive studios, is that correct?

Danilo:                       HTC vive studios. Yes.

Justine:                      Thank you so much. So seven miracles. First of all, I can’t remember what they are. Do you want to go over them? What we’re going to be talking about? It’s a film, I take it Jesus is, is the star.

Danilo:                       Yes, yes. Yeah.

Justine:                      And, and she uses, okay. I nailed one thing correctly.

Danilo:                       The seven miracles of Jesus are based on the book of Joan. Okay. Where is the famous stories in the Bible? You know, healing the sick by term for transforming water into wine. The multiplication of foods, the fishes in the bread, the fish is in the bread. Okay. Also known as the feeding the 5,000. Okay. You have the blind man, Lazarus and Jesus walking on water.

Justine:                      Okay. So there’s a lot

Danilo:                       A lot of miracles, a lot of visual effects, which is my sales, my background. Yeah. It’s like you have a background in and visual effects.

Danilo:                       Yes. Actually, I started with computer graphics and you know, really got into movies through the visual effects pipeline and I’m happy building VR applications for the last four years.

Justine:                      And so how did this come to be? What, what, what ex, you know, incited you to do a story about Jesus? So

Danilo:                       I received a call from Rodrigo from Panograma, which is the production company behind, you know, the, the production for HTC vives studios. He’s an amazing movie director and he came to me saying, hey, Danilo, I want to shoot this movie in volumetric. How do we do it? Cause you know, he has already made a movie. He was in 360 video and I was like, well it depends to know everything depends on budget and time and how can we put things together. One thing to know about the difference between 360 video and volumetric video is the three 60 video. You put a camera and all the actors are around you. You usually have to remove at least the camera tripod, the camera shadow, and probably some lights that are needed for the scene to exist. So it’s like every scene of a 360 movie has some visual effects post production to clean it up. Those elements like the camera. And uh, in some scenes we needed the director to be there, so he also had to paint out the director. But the good thing about 360 video is that it’s a streamable. If you want to watch that movie, you click it and starts playing it right away. The downside of it is that you get stuck in a position, meaning not just in the scene but also the height. So some people, when they, they watch that scene, we decided to make it a seated experience. Because doesn’t matter how tall you are, we are looking each other in the eye. Because when you see that most people are pretty much the same height. So we chose the, the camera heights to be that one. Okay. But sometimes if you’re trying it out and you’re standing up, you feel like, oh, I’m too short. Or you know, if it’s a kid’s like, oh, I’m too tall. If you put the camera too high it feels like, oh I’m flying. That’s because where you put the camera in the scene defines how you’re going to see it cause you can’t move, right. You can look around but you can’t move. So this called three degrees of freedom. You can just rotate your head left right up and down. When you’re talking about the difference between 360 video and volumetric capture or a volumetric scene the volumemetric scene everything is digital. The environment digital you, the live action is captured and created as either point cloud or mashes with textures on top of it. But it’s, it’s fully three d but don’t get confused of stereoscopic. We talking about real 3D models that are basically be animated 30 or 60 frames per second. So it’s kind of like, I don’t want to scare people away with too much of the technology, but just believe that you can walk around the scene. That’s the main difference. But the way you construct that, you capture the actress separate from the environment. And then you need to have a good experience for the visual effects pipeline to make sure that they match later that you can relight. And the downside, that’s the good thing about volumetric video. Have the freedom to walk around. Do you have your own height? You have the freedom to get closer to the actors to see the scene from any other angle. It’s much easier to make an interactive mini if you want just to look back at me in the eye as you can. And we decided to, not in this one, but you know, the technology allows you to be able to interact with the actors and even like decide where the story is going to be unfolding. But the downside of it is that the file sizes are pretty big. So if you want to watch a three minute scene, it’s probably gonna take like about like nine minutes to download. So there’s a lot of improvements in the technology to make sure that this is going to be streamable, but the file sizes are to pretty long. So you need to consider that, oh, I want to watch this, this better as like as a bonus scene. Okay. And then you download in the background and then you later go and sit down and watch. But you don’t have that immediate access like, you know, Netflix or any other streaming service that I want to watch this and it starts in truthful seconds. Um, but you know, the advantages of the technology as you know, just trying to give them a little bit of an overview is that you really feel present for us. That’s what we call true VR. 360 video is an immersive technology that feels like you are there, but you don’t really have the parallax. Like if you move her head to the left into the right, that doesn’t translate as motioning side of the video. So some people immediately get discomfort, you know, the resolution needed to actually keep the image really, really sharp. We need, you know, like eight k, 12 k resolution, which is really not what, you know, the devices can actually play right now. But there’s a lot of challenges on that side. And what we ended up is a mix of telling the story of, for example, of the water to wine. So you have, you know, the Canaa wedding and he, Jesus arrive, you tell the whole story, Mary comes to Jesus, Hey, you’re running out of wine. And then just said like, oh, what I have to do this, you know? And then when he’s actually going to do the miracles, that’s when it was switched to the fully volumetric scene that you have the agency to walk around and see that transformation of water to wine from any angle you want. Then after that, it comes back to the 360 video. So I’m gonna try to say the three 60 video is bad audits volumetrics the answer, but I’m just trying to, just to try to play with the strengths of each one of those technologies

Justine:                      solves different problems and depending on you and what you as a creative went to accomplish. Exactly. It’s worth exploring. Well, you’ve done that. It’s, it’s kind of debuted here. Is it? Is it, can we say it debuted here at Cannes is that the right word?

Danilo:                       It was actually launched at our Good Friday just a few weeks ago. Uh, and where was this strategically, strategically religious. Do these like the week of Good Friday. Yes. That makes sense for that is available. Yeah. That is available on the Vive Port platform. Okay. Is also available in most Android phones and iOS phones as well. So if you just look for seven miracles VR or you go to www.sevenmiraclesvr.com, you will find the links to all the other platforms.

Justine:                      Well, terrific. And what’s the, what’s the hope here for being at Cannes XR.

Danilo:                       Really to talk about the production of this, this project and it really started discussing the next ones. You know, we do have, um, a lot of stories that we want to tell and here’s the place to meet everybody and discuss, you know, how to start working on the next projects.

Justine:                      Yeah. It’s a very dynamic and interesting in, in terms of who it can attract and to, have you met anyone or been inspired by anything that you’ve seen here at this time?

Danilo:                       Believe it or not, I just came out of an agency meeting with another religious project. I, by the way, I also made a museum about a religious event that happens in my hometown, which is very unique. Only my town, 2 million people follow the image of Mary between two Churches for seven hours.

Justine:                      Okay. And where is, where is this town?

Danilo:                       It’s belaying [inaudible] para in the north of Brazil.

Justine:                      Okay.

Danilo:                       And I briefly mentioned that in my speaking engagement here at Cannes XR yesterday and one person or the arrows just clicked because you know that person has also a religious project else related to Mary. And we just had just had, I don’t know how much I can talk about it cause it’s just so brand new, but those are the kinds of people that you want to meet, you know, say hey, I actually not just working on the Jesus seven miracles movie, I also made a museum to talk about that event that happens in my hometown every year. In October 2nd Sunday of October, I travel with two 60 gear, laser scanners, photogrammetry, everything that I can I take to my hometown. And then I capture that actually 12 events. There are 12 per sessions. This statue of Mary that is being carried from two shirts. Actually they do that in multiple events. You can follow the image of the, the, the, the statue of Mary by boat, by car, by motorcycle, by feet. So it’s, it’s amazing. Like it’s, it’s, it’s so strong, the religion in my hometown and after leaving in San Paulo after many years and living in the US for 12 years now, I just feel like this is very unique to me was, was common and I just thought so, you know, do you know about this event? And no one knew about it. So it’s like, hey, you know, as, as a gratitude because you know, I can see they’re being very successful in everything that I wanted to do. The doors just open. So I just did a project actually as a gratitude and it looks like, now I’m going to jump into another amazing big project. You know, also in that religious realm, which is exciting to me.

Justine:                      Well, that’s a great story. I mean, coming all the way to Cannes and finding that kind of collaboration.

Danilo:                       Yeah. He’s like two people just like, oh my God. It’s very, very rare.

Justine:                      I, I would say, wow. Even it could be a miracle. Yeah. It could be.

Justine:                      Thanks Danilo talking with us today and we look forward to seeing what you do next.

Danilo:                       Awesome. My pleasure. Thank you so much.