{"id":7870,"date":"2018-09-14T08:19:13","date_gmt":"2018-09-14T12:19:13","guid":{"rendered":"https:\/\/dev.elearninginside.com\/?p=7870"},"modified":"2018-09-14T09:26:36","modified_gmt":"2018-09-14T13:26:36","slug":"simon-che-de-boer-virtual-reality-virtual-environment","status":"publish","type":"post","link":"https:\/\/news.elearninginside.com\/simon-che-de-boer-virtual-reality-virtual-environment\/","title":{"rendered":"Capturing the Past in Virtual Reality: An Interview with Simon Che de Boer"},"content":{"rendered":"

Earlier this summer, eLearning Inside <\/i>reported on a virtual reality experience<\/a> that would go on to top charts on Steam and become a huge success for CuriosityStream, the platform on which it was offered. Nefertari: Journey to Eternity <\/i>was created in large part by Simon Che de Boer, a Kiwi who describes himself as a jack-of-all-trades. Over the past five years, he, along with his team at Reality Virtual, have been pushing the edge of what is possible in virtual reality and virtual environment creation. They hope to empower photographers around the world to capture and preserve environments of cultural and historical significance. And for Che de Boer, the process started with tragedy. <\/span><\/p>\n

\u201cAbout five years ago, I was in a house fire,\u201d he said via teleconference. All of his personal belongings were destroyed. \u201cI got into this space where I needed to find a way to cope with this traumatic experience. And I thought I\u2019d essentially deal with it.\u201d<\/span><\/p>\n

\u201cI thought, \u2018Could I use my old family photographs of my place to recreate my old house so my daughter and I could go home?\u2019 I didn\u2019t know anything about virtual reality, I just knew how to put things together. So I spent many years working on a pipeline of how to do things. Along the way, I discovered photogrammetry. So when I was getting into it, photogrammetry was very new, no one was doing environments.\u201d<\/span><\/p>\n

Translating Image to Virtual Environment with Photogrammetry<\/h1>\n

The process would lead to breakthroughs in virtual environment creation, involve machine learning algorithms dealing with vast data sets, and developments that will, with luck, become indispensable for purposes of cultural preservation and education around the world.<\/span><\/p>\n

An early step was the use of photogrammetry. The process involves taking hundreds to thousands of photographs of objects or environments from set positions, moving incrementally with each new image. Upwards of 4,000 images went in to the virtual rendering of Nefertari\u2019s tomb.<\/span><\/p>\n

\"people
Outside Nefertari’s tomb.<\/figcaption><\/figure>\n

Che de Boer then used software to compile those images into a virtual space. \u201cThe first thing we achieved was getting billions of points to work in real time,\u201d he said. We\u2019ve achieved many other new things since then. We can get the computer to help fill in the gaps. It understands what a wall is, what a chair is, all that stuff. We can use that to fill in areas that might be considered incomplete.\u201d<\/span><\/p>\n

From there, Che de Boer and his team began to manipulate the images en masse in order to create a more stylized version of the virtual environment made up of raw images. <\/span>That process has recently developed into an application that Che de Boer believes will become instrumental across media.<\/span><\/p>\n

“Unbaking”<\/h1>\n

\u201cThe problem with photogrammetry currently is what we call baked lighting,\u201d he said. In other words, when a photographer snaps an image, a whole slew of qualities are also encapsulated\u2014or baked\u2014into it. These include the angle of lighting, tones of color, hue and saturation, whether a surface appears shiny or matte, etc. <\/span><\/p>\n

\u201cWe are able to remove that using a custom photograph rig that we have. We\u2019ve actually managed to train a network to look at a normal photograph and remove all the lighting qualities in it. So it can use it as just a base texture. And that\u2019s where base rendering comes in.\u00a0<\/span>Under other circumstances, it takes hundreds of graphics artists hours upon days to prepare these textures.\u201d<\/span><\/p>\n

The implications are groundbreaking. Using Che de Boer\u2019s technique, a film studio need not wait for the weather to comply for an outdoor shot. They could merely capture the environment and then shoot all the action in a studio.<\/span><\/p>\n

For purposes of education and cultural preservation, it has even greater potential. <\/span><\/p>\n

\u201cIt means that average people with their cameras can go out to historical sites and capture them, but make them such a high quality that they can create a virtual environment,\u201d Che de Boer said.<\/span><\/p>\n