The Metaverse has been firmly established in the world of science fiction for decades now. With blockbuster titles like Neuromancer, The Matrix, Ready Player One, and other works of popular media, it is a concept that has long captured the public imagination. With the rapid advances of artificial intelligence and computer imaging technology, we appear to be on the cusp of making it a reality.

In the Metaverse, you do not simply view content, you simultaneously are present in it and create it. You see other people and places, and interact with them. As the technology behind virtual and augmented reality improves, notably the ever-shrinking size of headsets required, this embodied internet, a component of Web3 where users are able to control and own their creations and online content, will become increasingly more realistic and prevalent.

Instead of tapping a few buttons on your phone to order a cup of coffee, you instead put on your headset and are transported to a virtual coffee shop. You scan the menu above the counter, give your order to the barista who is also appearing as an avatar, and then can walk down the street to pick it up.

Or perhaps you are standing at the end of a long conference table giving a presentation to colleagues, each of whom is sitting in a different city around the world. The Metaverse is also likely to be huge in the world of entertainment. You may be more likely to be fighting off a horde of zombies with some friends or watching the latest blockbuster from Hollywood than chatting about work. With the fully immersive experience virtual reality offers, large screens are no longer necessary to experience the full range of cinematic effects.

To create a truly seamless transition from the real world to a virtual one, both the computer vision models that are used for a better understanding of images and videos and the hardware these Metaverse environments run on need to improve. The headgear needs to not only project images for your eyes to see, it also needs to know in which direction your head is turned.

The amount of data required for simultaneous localization and mapping (SLAM), gaze estimation, and the creation of a realistic, full body avatar is tremendous. Even if a sufficiently large dataset could be collected, the time consuming and laborious process of labeling the data would be required before models could be trained on it. It is simply not practical, and perhaps not even possible, to deploy the amount of real data required for augmented and virtual reality, and thus the Metaverse.

Synthetic data, on the other hand, can be created in a fraction of the time and is not only orders of magnitude cheaper to produce, it is often better as well. The annotations are pixel perfect and do not contain any human errors that can creep in with manual labeling.

With perfect labeling comes the ability to render images of 3D scenes with ease. SLAM is made painless with labels built into the dataset, including segmentation, depth maps, and more. From a computer imaging perspective, the human eyeball is not very large or complicated. Generative adversarial networks were able to make more realistic images fairly early on by training on synthetic data, and datasets specifically for augmented reality are now available. 

Virtual avatars must be able to capture the poses, gestures, emotions, and motions of the humans they represent. This can be done by observing an actual human model, but machine learning models are now generating synthetic data of such a high quality that brands are producing virtual influencers to promote their products on social media. In the near future, we expect to see even more realistic avatars gracing screens of all sizes.

The technology has not yet progressed to the levels imagined in The Matrix, or even Ready Player One, but it is coming faster than many realize. Synthetic data is a key element to making Metaverses, what was not that long ago the stuff of fantasy, a scientific fact.

Learn more about how to use synthetic data to build metaverses. Read part 2, our deeper dive into synthetic data and the metaverse.