Bridge 4: Immersive Holocaust Education
Proposal | Project – Click to Navigate
^ Click and drag to look around
© MidnightCoffee, Inc. 2020 – All rights reserved
Proposal: – Or click to view in Google Docs for full formatting
As someone who grew up going to Yeshivas (schools where Judaic studies are taught in addition to the regular elementary and high school curriculum), I was exposed to stories of the Holocaust from an early age. I’ve spoken to many survivors on school trips, at museums, and with family, and was always eager to hear what they had to say. Unfortunately, now there are very few living survivors left, and that number is only going to diminish as the years go on, and I believe that something needs to be done to preserve their stories as soon as possible, especially considering the frightening statistic that, according to NPR, “41 percent of Americans and 66 percent of millennials say they don’t know about the Auschwitz death camp where more than a million Jews and others, including Poiles, Roma people, and gays were executed.”
Fortunately, many institutions have recorded these brave men and women speaking about their experiences and are available to view online or at many Holocaust museums around the world, some even using artificial intelligence to allow the viewer to speak directly to a hologram of the person. However, I believe that just listening to their stories and looking at photographs, while extremely powerful, are lacking the experiential element that makes a memorial so successful, in my opinion.
Various films, both documentary and fiction, have portrayed the Holocaust visually, with varying degrees of accuracy, but I propose that using virtual reality (VR), we can allow people to get up close and understand the Holocaust from the perspective of a survivor in unprecedented ways, as well as memorializing the six million Jews who perished in the Holocaust on the building’s exterior. This location-based VR experience (LBE) will take the user(s) through the locations and scenes from different survivors’ stories, paired with their narration being played all throughout the experience.
After extensive research, I learned that there are a few similar educational exhibitions in existence, but none as advanced and immersive as what I want to create. Unlike the VR applications that most people have tried, this will be an experience using six degrees of freedom, hand tracking, and 3D environments mapped to real-world objects.
This means that as opposed to just turning your head to look around a 360-degree photo or video like in many museums, this will allow you to physically walk through 3D space, getting up close and personal with the environments created. Additionally, as opposed to holding bulky, tracked controllers, this experience will utilize full hand tracking, eliminating the need for extra hardware.
Most importantly, though, is the way the experience will be physically built. Other examples of full 3D VR have the virtual environment overlaid on what is essentially an empty room, leaving the user able to walk through geometry with no physical feedback, breaking immersion. This experience will have the VR environment mapped directly onto pre-setup objects, meaning that, when paired with hand tracking, the user will be able to touch and interact with the world physically, as opposed to only virtually. Because the visuals are completely rendered digitally, the physical space can be reused for any number of stories that the user can select, even allowing multiple users with VR headsets to explore different virtual spaces while being in the same physical space for added efficiency.
Additionally, this plan is not intended to be placed permanently in a museum, like some of the other existing VR applications are. Because the physical spaces being mapped to can be any number of simple objects (tables, chairs, walls/dividers, etc.) this memorial can be easily packed up and brought to any number of locations all over the world, and translated to any language needed (albeit losing the authenticity of hearing the survivors’ actual voices).
Every element of the VR experience will be housed in a temporary, pop-up building, so it can be easily moved around. This building, from the outside, will be a cube, with a single entrance at the front. The sides of this cube will be stylized to look like glossy Jerusalem stone, similar to that of the Kotel, or “Western Wall,” in Jerusalem.
All along the sides of the cube will be quotes from survivors about their experience in Hebrew, Yiddish, and German, in a handwriting script typeface, with the phrase “Never Again,” appended at the end in standard block font. Below each quote will be that survivor’s name in the language associated with where they were born.
The glossy nature of this material will allow guests to see their reflected faces, superimposed with the text. The goal with the design of the exterior is to prepare the visitors for the intensity of what they will experience on the inside. The intention is not to scare them, but merely educate visitors on the horrors of the Holocaust and show visitors that it is their responsibility to ensure that similar events will never happen again.
This LBE is based upon technology established by various startups a few years ago, but updated with today’s technology, making the experience much more accessible. The most notable example is Utah-based company, The Void, which has created different experiences on view around the country. The quality of these experiences is unparalleled because of their partnership with ILMxLabs, Lucasfilm’s immersive technology division.
Because this experience is based around true events and stories, as opposed to fictional IPs like what The Void does, in addition to hand modeling 3D assets, I plan on using high-quality photogrammetry scans of the real locations from the survivors’ stories, giving us unmatched accuracy and detail in what the user sees.
From a technical perspective, this LBE will utilize off the shelf hardware that is readily available (an Oculus Quest 2 VR headset, in this case), with custom software built to run on the headset, taking advantage of its all-in-one nature, removing the need for a backpack-mounted computer like past LBEs. Despite the use of standard VR hardware, because of the environment mapping, this is not something users can download and try at home–it is required that they travel to see the memorial in person.
I believe that the entirety of this VR experience, including the exterior can be created for under $30,000, not including the cost of traveling with this memorial. In terms of placement, I think in order to reach the most amount of people, it would need to start out on a museum’s property, and gain word of mouth from there and continue traveling around the country. The temporary nature of this memorial will hopefully encourage people to go see it when it is nearby, knowing that it will soon be moved somewhat far away. Additionally, due to the technology and hardware involved with this memorial, it’s not something that people can try at home: they have to actually visit the memorial in person.
I don’t want this to be an “attraction” in the ways other “gamified” LBEs have been. This is a serious, educational experience, utilising state of the art immersive technology. My goal is not to entertain–this is not even remotely like any VR game out there–but to find a way of reaching people using tech that would otherwise not be interested or even care about the genocide that happened many years ago. My goal is to make sure that no one forgets.
Bibliography
Lebovic, Matt. “The ‘Virtual’ Future of Holocaust Education Is Already Here.” www.timesofisrael.com. Times of Israel, January 10, 2020. https://www.timesofisrael.com/the-virtual-future-of-holocaust-education-is-already-here/.
Martindale, Jon. “The Void Will Use Reality to Transport You to a Virtual World.” Digital Trends. Digital Trends, May 24, 2015. https://www.digitaltrends.com/computing/how-the-void-plans-to-put-reality-back-in-virtual-reality/.
McKeand, Kirk. “Videogames’ Portrayal of the Holocaust Does a Disservice to Both Players and Victims.” PCGamesN. PCGamesN, January 18, 2018. https://www.pcgamesn.com/jewish-opinions-on-nazis-in-videogames.
Simon, Scott. “NPR Choice Page.” Npr.org, April 14, 2018. https://www.npr.org/2018/04/14/602443782/the-startling-statistics-about-peoples-holocaust-knowledge.
Welle, Deutsche. “3-D Auschwitz Model to Help in Last Nazi Trials in Germany | DW | 02.10.2016.” DW.COM, February 10, 2016. https://www.dw.com/en/3-d-auschwitz-model-to-help-in-last-nazi-trials-in-germany/a-35943451.
Project:
Due to time constraints, I will not be able to construct a prototype of the physically-mapped environment. Additionally, would I do this, it wouldn’t be scalable because I don’t have the physical resources, and would essentially be mapping out my basement, which can’t be taken anywhere else, like my memorial dictates. Therefore, I will be creating a “lite” version to show the potential use of the technology for Holocaust education. It will still be a room-scale, six-degrees-of-freedom VR experience, but without the feedback of being able to touch real-world objects.
The process for creating this was extremely powerful, difficult, and educational.
To find the right survivor’s story, I watched countless interviews about being in the Nazi death camps. This wasn’t an easy process, simply because of how horrific these stories are. I eventually found a survivor’s story that I thought would work for this project. I then edited the interview so it was just David Wisnia’s words, omitting the interviewer and any fluff that didn’t pertain to the main story he was telling. In the future I would liked to use full interviews, but for this project, I limited myself to around 5 minutes.
Then there was the process of creating the actual 3D VR experience. I used Unity3D, the game engine I’m most familiar with, along with the SteamVR plugin. This allows the application with full tracking to be used on almost any VR headset on the market. When actually laying out the 3D environment, I did my best to use high-resolution photogrammetry scans of real locations and ruins from WW2 ara Germany, as well as a few generic models that I re-textured, such as flags, planes, etc. Because many of these models were real photoscans, the geometry had way too many polygons, which made the application stutter (even when run on an ‘RTX 2070 Super’ graphics card). So I used Blender to “decimate” these FBX files so they were usable, while still using the realistic textures. The end result of this is completely inperceptable to the viewer, it just increases performance.
I also implemented spatial audio, when run on a PC or natively on a headset. This means that you can hear sounds coming from different directions, as opposed to just having generic mono audio. I used this to change where the narration is coming from to direct the player to turn their head to what’s being spoken about or animated. Unfortunately this doesn’t work when run in a browser, and is defaulted to mono audio.
Because I know most people won’t have the hardware for a high-fidelity, fully tracked VR application, I’ve converted it into a 360 video that can be watched in any browser, or simple “Cardboard” mobile phone headset. Keep in mind that because it’s compressed quite a bit, there is no spatial audio, the rendered lighting isn’t as realistic, and overall there might be some pixely artifacts. The content is unchanged, but it’s not the full six-degrees-of-freedom experience that is best.
The completed 360 video can be watched at tinyurl.com/JudahHolocaustEducation or embedded at the top of this post.
3D Model Credit:
Sketchfab Users:
WalterSwinney
emijar
Kyle May
KIFIR
Urban 3DScans
D.U.H
cmbassnet
CommonSpense
Austin Beaulier
danieljorge435
suramis
Petro.Stepaniuk
Вячеслав
bez_glaza
mega1912
MarySince910
theoslan
olof Moleman
helija
CG Duck
