Creating a better future with XR

Recipe

 

Cross-Reality (XR) is a technology that comes from a fusion of three terms: Virtual Reality (VR), Augmented Reality (AR) and Mixed Reality (MR)[1]. It has the advantage that it can create a virtual environment. This means that it is an interface which can show something that does not exist or is difficult to recreate in the real world. The XR experience can also teach people how to act in certain situations.

While some may argue that this technology can lead humanity to a dystopian future, I feel that with conscious effort and the right direction, these technologies can be exploited to increase the human potential to act.

Consider critical topics like climate change. Presently, people are unable to understand the impact climate change mostly because they do not see any visible changes around them and think it will happen in the future. What if we built XR experiences around creating those environments for people to experience how the world would look like when it is in climate distress?

Now, think about natural disasters. Situations like these demand proper techniques for disaster recovery. So, hurricanes, floods, wildfires, and other disaster environments can be created in XR to train the people how to deal with such situations and teach the fleet how to do an emergency evacuation.

Inciting action through educational is a setting where I see XR flourishing because it has the potential to develop a deeper connection with the person since the technology is built around enhancing the real-world experiences.

 

Recipe Summary

This recipe is a summarized version of how I see XR getting implemented in the world so that it is seen in a positive light.

1. Research
   1. Pick a topic for the experience
   2. Pick the XR medium (VR/ AR/ MR)
   3. Pick the delivery platform (smartphones, web, museums, schools, exhibitions)
   4. Pick the method of communication (game, interactive experience)
   5. Research the physical environment
   6. Decide the realism level
2. Develop
   1. Create a storyboard
   2. Create a prototype
   3. Test the prototype and update the storyboard
   4. Design the scene in a game engine
   5. Add interactions in the scene
   6. Export the experience as an application
   7. Prepare a guiding tutorial
3. Update
   1. Install/ share the final product
   2. Collect feedback
   3. Update the experience based on feedback

 

Recipe Detail

Here’s the detailed recipe for creating a new form of awareness through XR:

Research

1. Pick a topic you want to create an experience around
   1. These are some categories to choose from:
      1. Climate change
      2. Natural disaster
      3. Medical operation
      4. Vehicle assembly
      5. Flight simulation

2. Pick the XR medium by researching the location of your target audience
   1. If it is in the world around them, create an AR/ MR experience because the audience can experience first-hand what is happening around them. For instance, our immediate environment can be overlaid with the harsh visuals of the future impacted by climate change so that people can understand what it would be like to live in that world – for them and their future generations
   2. If it is somewhere far away from where they cannot be physically present, create a VR experience because they can digitally transport to that space and experience it. For instance, we can create a VR experience showing the melting ice caps because people cannot go there physically to experience it.
3. Pick the platform for the placement of the experience
   1. This matters more for VR because it usually requires hardware for its implementation. So, museums, schools, conferences, and exhibitions might be better platforms for the placement of this experience because people may be unable to invest in setting up this technology.
   2. AR and MR are more accessible because smartphones today have AR capabilities. So, developers can build apps for the respective app stores.
4. Pick the method of communicating the message
   1. Game
      1. Games are engaging and recreational. So, the interaction formed using games creates a connection with the player and enables passive learning of concepts. They are a preferred media for VR because players usually immerse themselves in a different environment when they play games. So, the transition from reality is smoother for VR experiences. While working on the concept of a game, plan the following:
         1. Define the winning and losing conditions of the game
         2. Decide if it is a one-level or a multi-level game
         3. Decide the interactions that would take place in the game
      2. Interactive Experience
         1. Interaction can be physical, digital or a mix of the two. Bodysuits, tactile feedback gloves, and other hardware can be connected to the body to get a more immersive experience. For an interactive experience, plan the following:
            1. Decide the interactions that would take place in the experience
            2. Whether the experience would be from a first-person perspective or a third-person perspective
         2. Research the environment and the experience which needs to be recreated
            1. Since the experience must feel real, in-depth research of the surroundings that should be created in XR is required.
         3. Decide how realistic the environment should look like
            1. The choice of the experience would depend on the target audience of the experience and the type of content to be communicated. The prominent styles are:
               1. Low poly or low polygon style
                  1. The low polygon style means using fewer polygons to create an environment. This is helpful when the content of the experience is too graphic to be shown as is. For instance, you may adopt a low poly style for children when creating an experience which shows deaths during a natural disaster as the visuals can get too intense to be viewed by children
               2. Actual
                  1. Volumetric scanning techniques can be used to render real-world environment. This style is used when the realism of the experience is of primary importance for communicating the message

Develop

7. Create a storyboard of the interaction
   1. Create a visual narrative using post-its to depict how the entire interaction will take place
8. Create a prototype of the experience
   1. Create a draft version of the experience based on the storyboard to give the player an understanding of how the final version will work
9. Test the prototype with a section of your target audience to get feedback on the process
   1. Testing the prototype is critical because it helps to understand whether the experience that is yet to be built for the audience is reaching its true potential, and what improvements you can make to reach your goal.
   2. Once the testing is complete, incorporate the feedback received during the initial trial to update the storyboard
10. Design the scene in an XR engine
    1. Begin designing the game/ interactive experience.
    2. Unity is an open-source tool which is an excellent engine for XR development and is powered by an online community that can guide you if you are stuck anywhere
11. Add interaction to the scene(s) using scripts and animations
    1. While creating interactions, it is essential to think of how the users would interact in a scene. XR has long been seen through a dystopian lens. So, we must focus on creating something that engages users to do more than be inside the experience. For instance, with Pokemon Go, Niantic made the objective of the game much more than just catching Pokemon and encouraged users to explore their cities and meet new people.
12. Export your XR app to the respective interactive environment you had decided: web, mobile, VR headset, or other devices
13. Prepare a reference tutorial of the interaction to guide people on how to interact
    1. This step is critical to the entire process because it indicates how you want the users to interact inside your experience. The guidelines and the tutorials help them understand the interaction more. So, use visuals (preferably inside the XR experience) to guide the users.

Update

14. Share the final game/ interactive experience with the target audience
    1. It is essential for a small audience to test the experience before it is finally shared with the entire audience. Their feedback would initiate the first set of improvements in the experience.
    2. Once that is done, sharing with the target audience would involve thinking about modes of sharing and the mechanism of follow-up
15. Collect feedback from the users as they continue using the application
    1. Feedback and improvement is a constant cycle that must take place after development is complete. This allows a systematic improvement in the product based on the needs of the users.
    2. It is critical to ensure that the modes of collecting feedback for the experience are simplified so that the users can quickly engage with it. If it is an app on the App Store or Play Store, then it is simpler for users to give feedback. But for other experiences, there must be a dedicated interface for giving feedback.
16. Update the XR experience based on the feedback received over time
    1. Once the user gives their feedback, an acknowledgment of receipt of that feedback and work done on that feedback is essential. This ensures that the person developing the experience is interested in improving it.

 

Following the above-stated recipe will allow users to create relatable experiences with the audience. XR is evolving, and it needs a direction. This recipe is the starting point for that.

 

Enclosure: XR Interaction Recipe Checklist

Recipe with Enclosure – Creating a Better Future with XR – Akshansh Chaudhary

 

Notes

[1] At the time of this writing, VR, AR, and MR were the three primary technologies that were being used