Story as a Protective Device: Skip Rizzo on VR Storytelling

Justine:          Welcome back. Justine Harcourt de Tourville here Editor-in-Chief of VRTL and my next guest, Skip Rizzo is a legend in virtual reality, not necessarily in cinematic, but you could argue he does have a cinematic quality about him, like a distant cousin of the dude a larger than life. I suppose that’s because when Skip talks about Vr, he seen the technology evolve. His office even boasts an ancient astronauts headset but mostly Skip’s got this science-based excitement. He’s knowledgeable, willing to share the facts, but he’s not trying to convince you about VR and some kind of smarmy salesman way. Rather it’s because he knows firsthand about the good he sees with virtual reality and the impact of story. In this episode I’m talking with Skip who first, who himself has the Ayatollah of VR, but by day he’s a research scientist and director of medical virtual reality at the institute for creative technologies at the University of Southern California. Have a listen

Justine:         If you have any kind of anecdotes or thoughts specific to narrative and storytelling. I know that’s not what you do, but I’m wondering if you have,

Skip:         I’ve got a couple of projects. So one in 2014, we built six full on episodes called stress resilience in virtual environments. I can describe, I can show you the link online. Basically immersive narratives to prepare service members for what they might face and cutting huddle in a real, no one they, before they even go this was designed to put ourselves out of a job on a backend treating PTSD by doing a better job in preparing people. So you put them through a seven minute band of brothers like episode where they’re in a VR headset and they’re with a squad on a mission. At the end of the episode, Shit hits the Fan, a kid dies or your get blown up in a hum V, one of your squad members gets killed. And up to that point, and these are all modelled after the things that people with PTSD talk about being things that haunt them. So at the end, at that point, it’s like an emotional obstacle course. You know, it’s like this the shit you’re going to face and we want you to experience it in advance. But we don’t just stop there at that moment. At the moment outwalks a virtual mentor through the smoke and rubble or whatever debris and uh, he’ll guide you through a whole range of coping strategies for dealing with stress. So the idea was whether it’s teaching acute stress reduction by deep breathing or higher level conceptualization of moral injury and dealing with death and dying. In the context of this hellish thing, you’re learning the tactics. And think of it, think of this for me, and you might be different, but a 19 year old whose biggest stress or up to that point was Mary Sue breaking up with them before the junior prom. Now they’re seeing a kid die, a civilian kid that they couldn’t prevent or one of their own or whatever. They’re not prepared for it. So you give them an advance look, but then give them the skills to deal with it.

Justine:         You’re saying that virtual reality headset can help familiarize and help them anticipate what’s going to happen.

Skip:         What I’m doing, what I’m saying here is that by immersing somebody in a VR context, but also having story that you’re leveraging two of the most powerful ways that people learn, learn by experience and learn through narrative, learn through storytelling. You want to rack that up against a reading something out of a book or sitting through a death by PowerPoint presentation or getting a lecture just to play in verbal lecture. You’re going to remember the experience and if it has a story, it’s going to stick.

Justine:         So it’s a one, two punch.

Skip:         Yeah, and that’s what the, the, the center that we’re at right now, the institute for Creative Technologies, we were founded on that premise that the military wanted a research institute in LA to leverage the Hollywood talent pool in special effects, gaming, stuff like that, graphics. But in narrative, so the army, believe it or not, had foresight in 1999 to see that narrative was an important component for boosting potentially the training effects of what goes on in a simulation. So that has always been, the vision here. And in some ways because of that in this building, we’re sort of like the unholy alliance between Hollywood, the military and academia three completely different cultures that somehow get along. And that’s, that’s part of it. So that’s one example where we basically built the simulations of the combat environments from the same 3D graphic content that we had developed for treating PTSD on the backend by helping people to go back and confront and process the difficult emotional memories that they went through. But doing it in a safe place with a clinician, when you first hear it, it sounds like torture why would you do it? And admittedly, it’s hard. It’s hard medicine for hard problem, but it works because the biggest problem with PTSD is people just do everything they can to avoid thinking about it. Going to places that remind him about it, telling people about it. Nobody understands them. So here you put them in a VR environment that in some is like a narrative therapy. It’s bringing them back to the scene of the crime at a pace that can handle, a gradual pace, but they’re actively narrating and telling her story. Like they’re going through it right then and there and the clinician is adjusting the time of day and the number of people in the scenario or the bomb that went off 100 yards in that direction. And then the one that went off right next to the vehicle clinician as a wizard of Oz control panel. And in that way you can mimic the patient’s story in real time and get them to dig down beyond the cognitive sanitized description of the event. But beyond that to pull up the emotions that come with it. This is why I like to say that VR is an emotionally evocative technology. You can either put people in stories that if you design the simulation right, that is very emotionally evocative or you can put them in store in content that evokes in her own story for a therapeutic purpose. So that in a nutshell is a couple of examples of how narrow narrative fits in to mental health in VR,

Justine:          Can you tell or do you have any data or any, have you seen results in terms of how people process stories in headsets specifically or things that people that are doing regular cinematic productions the Hollywood versions or the Playa Vista versions? Where they can learn from that?

Skip:         Yeah. We haven’t done the kinds of research that’s specific to the impact of story other than can we activate someone by putting them in these simulations. And we’ve documented that certainly that, with heart rate variability going through the predeployment resilience training, when the crap hits the fan at the end, you see a real change in heart rate variability, which is indicative of stress. So we know that the content is emotionally activating.

Justine:         Good, then we have proof that it is not just hearsay, that people are putting on a headset and having an emotional response that you can and you know…

Skip:         Right! There were, so there is data and that’s the first data point you need to collect because if you’re just building something that’s boring and somebody’s sitting like the other sitting through an HR training program for how not to be a sexual harasser and people were falling asleep while they’re doing it and

Justine:         Then you’re not effective.

Skip:         That’s the key thing. What are the best metrics to gauge that? Now, we had done work back in 2004 with spherical video for journalism. This was, well before the next generation renaissance of spherical video. We had a very primitive camera, but we did a news story down in Skid Row and we were right there in the middle of Skid Row homeless people, crowds walk in and go and by people with mental health conditions walking up to the camera. We had a news anchor telling the story, a 3 minute story and the user could eat. You could either look at just a news anchor and single frame and here’s the story. Or You could be in a VR headset and look around while she’s narrating the story over there and looking around at the place and we studied memory. What do people remember from the actual content of the news story? Uh we were able to break up the three minute story into separable bits of information that were conveyed. And what we found was immediately after the event, and you said, okay, tell me what you saw and you had two groups, the VR group and the single frame group. It was pretty much equivalent. People remembered the same amount. It was like short term, very short term memory, remember the same amount of content. But when we called them up a week later and asked him, do you remember that story that you saw when you came to our lab that people that had the VR experience remembered more of the objective content? We published out, I think 2005 or 2006, somewhere in there. But the interesting thing with that is that if you look at the sheer amount of information someone’s exposed to in that immersive narrative where they’re in the scene, while the story’s being told, there’s a heck of a lot more information that a person is being deluged with because they’re hearing a story, but they’re seeing a guy pushing a shopping cart over here or somebody with a mental health condition hallucinating over there. And so you could make the case. That’s a more distracting setting to extract content from a narrative. But it wasn’t the case. And I think what it is is that all that context provided a framework so that the oral narrative of the news reporter had some hooks to hang on to and it stuck. And so I’d love to redo that now with much better spherical video. We have another one we did for a museum exhibit recently the hero’s hall veterans museum down in Orange County. And this museum is set up as part of the Orange County fairgrounds. It’s a real big fair that they have. And the idea was they wanted us, they want to do a whole exhibit on PTSD. And I said, well to do the clinical thing you need, you need somebody there to operate the equipment. You’re not going to get a lot of throughput. You know, you get a lot of people come. And so we came up with the idea of taking a very low cost headset, like Samsung gear VR and also the Oculus Go. And we built a six minute narrative called the soldier’s tale. Basically what it was is a, a guy who had been in Afghanistan spending time just telling a story about his experience but not a once upon a time story. It was more like existential musings about things that he did and events. And I should’ve paid attention over here cause I didn’t see that guy with the cell phone and, and ended up ignited a bond. Well, so you have that narrative going on, that narration, but you also have, 3D graphics that are following the narrative. Then we took from our PTSD stuff. So it starts off, he’s walking down a market place in Afghanistan and he’s talking about just everyday kinds of things about the patrol. And then a bomb goes off and the next scene, it kind of gets old jaggedy and a scene It opens up. He’s in a helicopter, medic helicopter being flown away with blast injury so he didn’t have any gouges or cuts because we couldn’t show up any blood in this thing because kids could, wouldn’t have to go through it too. And there’s a narration at that point. Then it lands and he’s getting wheeled into Bagua him and he’s talking about, I I thought I had my dome rattled, but I didn’t realize it was going to take me a month and a half before they clear me to return to get back to my tribe ? You see him getting wheeled into the into the, the emergency area, and then it fades to black as he says that. And then it pops back up and now he’s in a turret and a vehicle going down a desert road way. And he starts talking about it was great to be back. I really miss my guys. I didn’t know what to do with myself. I thought I was much better telling that story. And it, all of a sudden it’s another attack. And he starts talking about the kinds of things that soldiers talk about when they’re going through this kind of stuff and the aftermath. It finally ends after that attack scene, which goes on for about a minute and a half. It fades out and fades up and he’s on a hillside, kind of an idyllic setting, looking out over an Afghan village sun sunset kind of thing. Really Nice clouds. And, and he starts talking like this is my last day here and I’ll probably never be back here. And it’s almost like he’s, he’s, yeah, he’s wanting to stay in some way but it’s like an emotionally evocative experience that basically 20000 visitors to the museum in the span of 21 days in the Orange County fair got to experience, we had 10 setups set up with a year ago, you slap it on your head automatically starts and after this, after they went through the exhibit and saw all kinds of photos and videos of patients talking and they finish it off by putting the headset on. And I think, I thought it was pretty, pretty cool because we couldn’t set up the therapy system, but we could set up a narrative that gave people the sense of somebody that actually went through this stuff and then recreate it in VR while they’re talking.

Justine:         That’s impressive feat and do that in six minutes to have people really experienced something so heavy and deep.

Skip:         Yeah. And it was delicate because as a museum exhibit, it wasn’t a training component. So yeah. Yeah. You had to be, you had to get across the impact of the experience without freaking people out.

Justine:         Yeah. So how do you work? Do you, how do you collaborate? How do is a process like this work? are you writing the scripts yourself?

Skip:         The way this one went was that the veteran who went through it. He’s also works for, he set up a veteran support agency called Crusades 22. we’ve been kicking around the idea of doing it and then the museum came up. They wanted to do it and they had very little money, but enough we could build this thing out without making call of duty heavy, 3D graphics cinematics, but I think there’s pretty good, or you can try it if you want before you go. I have a headset here. He wrote down some rough thoughts and then I went through it with our team here and it was like, okay, do we have the content to mirror the story? And we had a modify a couple elements because we don’t have it’s complete library of every kind of a place. So we were able to massage the story to be able to fit it into a way that we could represent it in the 3D graphics that we had available to us. And went back and forth with him and he was totally cool. You know, he loved the process and integrate voice. We used him for the narration. You really should try as you can get the idea,

Justine:         Well it sounds awesome, the authenticity.

Skip:         A real story to it . And uh, and then we got some software that allowed us to take our 3D graphic content and create a spherical video, if you will, on rails. But instead of it being taken from a camera, it’s now made from the 3D graphics that we’ve created. And the guys spent about a month, hammered on it back and forth. We’d look at it, point things out ,and I would like to actually add a couple elements to it and clean up some of it and submit it to some of these film festivals where you can submit the immersive narratives like this because again, it’s not you’re not talking about Spielberg level special effects and graphics, but the graphics are enough to set the stage so that you focus on emotional content of the story.

Justine:         So that’s really good.

Skip:         Yeah. And I liked the, that we were able to get 20,000 people over, about a thousand people a day during the, during the fair time. The exhibit was up for nine months there. And one of the most compelling immersive narratives I’ve ever been in was at a studio called real effects in Dallas. And the flight school group there has built a Dan Carlin, the hardcore history podcast guy. He did five, five or six episodes on world war one there about four hours a piece and really graphic descriptions of trench warfare. Well, I went through the trench warfare experience in a full on Vive headset being tracked and I’ve, I don’t know if I’ve ever had as provocative and fulfilling emotional experience and that is playing at being revealed. And it was revealed at Tribeca and I think it’s going to be in the Smithsonian. And it really showed the potential power for historical narrative to be delivered in a VR component. So sort of going into the void. Now you were in an event and you get to build, hopefully see the empathy for the people that are being ordered to climb out of a trench into a meat grinder and just getting shot instantly. And all of a sudden it’s your turn to climb out and you’re looking at the guy like, maybe I should get over here and shoot rather than climbing out and being a turd in the mustard gas gas, they had the gas flow into, had those old tanks going up. They had the bomb craters filled with a toxic stew of bodies and everything. It was emotionally evocative.

Justine:         So as somebody in the psychology, psychiatry realm, is there anything else specific to VR that you think is overlooked by mainstream or in terms of it’s great at kind of passing on emotion or conveying emotion and empathy. Is there something else that I do? It sounds like it’s good at learning or teaching.

Skip:         How to respond and cope with stressful events. And we did it in a military context, but maybe police in the police academy. My benefit from it maybe first responders, firefighters, anybody that has to deal with stress. I mean, I think police, it’s so underestimated the amount of stress that is in that job, that it’s amazing that it’s, it’s not even worse for them in terms of mental health, divorce, drinking, which are all higher than most other occupations because they’re, they’re dealing every day with, you think about your worst day. Well, they’re dealing with 10 other people’s worst days in an average in a day.

Justine:         There’s a lot of decision making that they have to do under stress.

Skip:         Yeah. Yeah. So I see, I see the power of the technology in being able to put people in a context that I either activates an emotion that helps them to learn how to deal with, events in the future or helps them to reprocess emotions about things that happened in their past so that they can move forward rather than being paralyzed by those things like post traumatic stress. But even for smaller things, whether it’s, you know anxiety disorders with phobias, fear of flying for public speaking. These are all things that VR has been documented scientifically to be a real value, less narrative and more experiential. but there’s bunches of things. I mean, engaging people when they’re undergoing painful, acutely painful medical procedures, engaging them in an immersive narrative or giving them an experience of being somewhere other than where they are going through this procedure. That’s another word. Vr has shown a significant benefits and is being widely adopted now as a real tool for distraction. you know, physical therapy, cognitive rehabilitation therapy. After a stroke or brain injury. you know, the problem we see there is that people have a hard time doing sufficient practice, drill and practice trials to recover their function well. We put it in a game like context. We tracked their body movement or their actions, put them in a VR simulation that makes the very boring, repetitive and frustrating activities of Rehab, maybe more fun and engaging. Adding in a narrative element to it I think would, would, would be real positive, especially if you had a game that you could split up into chapters where you do your physical therapy one day and it’s one game and you get to a point where you’ve done sufficient number of trials and you’ve moved along in a story and now it’s a cliffhanger. And it’s like you’d be looking forward to doing your therapy the next day just to find out what happened and you know, and as result, the cliff, when you binge out at three in the morning, it’s like, all right, maybe one more before we go to bed.

Justine:         One more rehabilitation exercise please

Skip:         Yeah. So there’s motivational I mean, we want to engage people, we want to motivate them, want to give them opportunities to do things that, they might not do in the real world.

Justine:         That’s true. Right. Do you have any last project? Do you want to share anything that you’re working on? That’s super cool. Super Neat.

Skip:         The big passion project right now, of course his or her work with autism, I mean, we’re continuing of course on all the fronts of disgust particularly the PTSD and moving that towards civilian use. But the work we’re doing, autism I think is really a positive because I’m seeing positive benefits. And in one project, this would be one that we’ll be talking about at SXSW, is about using augmented reality to help people in the high end of the autism spectrum practice or job interviewing skills. And we’ve done it in VR. We’ve done it with a big TV and basically what it is, is think of the population, this population that oftentimes is very talented and very intellectually capable. They can do the damn job, but they choke in social interaction of an interview that’s, that’s the part of the the condition or the challenges that they face. So we created it in VR initially. a system where you can pick from six different characters, different age, gender, ethnic background, and you can put them into different job context. So you have six context, a restaurant, job and office job, a warehouse, things like that. And then you can adjust each character’s personality from soft touch interviewer to neutral interviewer to son of a cranky and cranky interviewers there. And so you’ve got all these ways to modify the challenge level and it does two things. It gives them an opportunity and they’re more willing to talk to a piece of software and a 3D graphic character than role playing with a real person.

Justine:         Yes, because they’re not being judged?

Skip:         Yeah, exactly. Exactly. And so they can go through the nice character, soft touch interviewer, easy questions. But then you can make that character cranky or you can go to another character. And it’s not like practicing with a familiar person like revoke counselor where you’re comfortable when you go on a job interview, you’re going to be getting interviewed by a stranger. So you got to have these, these different characters to train with. And by doing that repeatedly, you’re doing like, kind of like exposure therapy where you’re helping people confront and process fears. But we also give them a chance to practice how are you going to represent yourself? And that roleplay stuff works that matters no matter what the population is. So we can all benefit. And that’s the beauty. You know, this is not, this is not our most dramatic intellectual achievement to build this system. You know, you know, it’s on rails and a lot of ways it’s not gigantic, but it has tremendous generalize ability. So now we’re using it with veterans that are having a hard time. Change up the questions a bit how does your mill, how did your military experience for Perrier to be able to do the tasks of this job? You know, you would think people would prepare for that, but a lot of times people get caught out of the blue and now you’ve got to start thinking about that. Or what about the ass that says two Oh, you were in Afghanistan. Do you ever kill anybody? You know, or you, you’re a veteran, you don’t have PTSD do, you’re not going to go postal on us. These were inappropriate questions, but they get asked. So you prepare veterans for those challenges? The third group, now we’re applying it with is juveniles that are incarcerated that are about to be released.

Justine:         Wow.

Skip:         Yeah. So they’re going to be released at 18. Some of them have been in, in residential programs since there were 14. They’ve never done a job interview. And now they got to, they got to get out and not just do a job interview, but they’ve got to explain you know, I did. I got a GED because I didn’t finish high school because I had this.

Justine:         Because I was in juvenile detention

Skip:         Yeah, exactly. So all of a sudden now, a very simple, basic, kind of an interactive simulation can have generalized ability to everybody. What about maybe I don’t care about making money on the pain of people on the spectrum or veterans or juveniles, but I will make money on a parent that wants to buy that system. We’re licensed use of it to help their kid practice for college entrance interview or whatever, whatever. The next thing that we tackle, I mean, so there’s, there’s a lot of ways to build and test VR simulations that are designed for helping people with health conditions or whatever. But eventually you could make your money. you know, translating everything. You’ll learn to do that commercial. Yeah. Yeah.

Justine:         Well, I hope you do.

Skip:         Thanks

Justine:         Scott, thanks so much. I really appreciate you spending your afternoon here when us.

Skip:         Certainly it’s been great

Justine:         I look forward to speaking to you again.

Skip:         Yeah, thank you.