How One Startup Will Change Shopping with Virtual Reality

We envision a future where a person can travel beyond the four walls of their home and into their favorite stores without moving a toe. We imagine a reality where shopping is as easy as curling up into your couch and exploring and buying Nike Air Jordans in 3D. A future where smart home is equivalent with infinite space. Gatsby is the first app to allow users to put themselves in retail stores and to get all the benefits of shopping in-person right from their abode. Although AR and VR have been used for gaming and certain furniture applications in retail, no-one has yet ventured into the complete virtual retail experience Gatsby is offering - Gatsby is the first to provide virtual shopping for the stores we all recognize.

Today, VR in retail mainly consists of testing store designs and in-store product layouts. Buying something in retail has been flooded mainly with AR looking to augment the shopping experience. Most software in this area looks to increase the information accessible to buyers. The only realized intersection between VR and the buyer’s side of retail comes in the form of “try it before you buy it” software, typically offered in-store. Gatsby is creating specialized virtual “habitats” for each store where products of any size can be spun and rotated mid-air with the swipe of your finger, and easily brought to checkout.

Mobile VR

Gatsby makes choices with the future in mind. Of all the mainstream market options available in virtual reality, Google Cardboard is by far the most accessible and portable, helping with Gatsby’s goal of mainstreaming VR use. While other options like HTC Vive and Oculus Rift provide better quality and computational power, they lack the portability and accessibility of Google Cardboard. HTC Vive and the Oculus Rift have steep starting prices at $800 and $500 respectively -  a whole different world compared to the $15 price tag of the Google Cardboard. This alone makes Google Cardboard accessible to a far larger audience, and means that retailers can also easily distribute and sponsor Google Cardboard headsets, promoting their brand’s widespread adoption of VR simultaneously.

The difference in computational power between Oculus/Vive and Google Cardboard comes mostly from their power source, as Oculus/Vive require a PC connection, which sacrifices portability for added power. Google Cardboard requires only a smartphone - meaning that the cost in computational power comes with the benefit of additional portability and accessibility, as a large portion of the general public has access to smartphones and uses them as their primary technology. With Moore’s law still in play, the computational power of cell phones will only increase, helping to increase the quality of Google Cardboard.

When choosing between Google Cardboard and the newer mobile VR choices coming out today, such as the Samsung Gear, the decision gets a little tougher. The  Samsung Gear has much better resolution,  and does not cost nearly as much as Oculus or Vive - this category of headset is thus more competitive with the likes of Google Cardboard, though a steep price difference remains. For now we’re betting on Google Cardboard due to its low cost and the fact that it works for nearly any device. We will be expanding to other headsets and watching out for progress that makes VR cross platform application easier for both users and developers.

Photo Acquisition and Rendering

Currently, we are using CGI to generate the models for the objects that populate the stores. Rendering these models is difficult to do on mobile devices as overheating and timing out become issues. Our current solution to these problems is to not render the CGI objects directly onto the scene on the phone but instead pre-construct the scenes with the object already in them and download the scene onto the phone. Using this strategy, we are currently limited on the amount of object models per scene, as even after compression the scenes are pretty large files. We might also have to require the user to download additional scenes as they go, which could pose a usability issue.

We are moving away from CGI and towards photogrammetry. When comparing photogrammetry to other model acquisition options, photogrammetry is the clear winner. Laser scanning has a leg up on photogrammetry in quality, but this difference is not enough to justify the steep cost difference. Photogrammetry provides photorealistic models with small error margins. Photogrammetry only struggles with large objects with difficult-to-identify edges (as would be a completely white empty room). Furthermore, lower resolution yet still realistic objects might actually be an advantage, particularly when working with mobile technology as it is currently limited in both space and computational power.

Moving towards photogrammetry does bring a series of hurdles to consider. While photogrammetry would bring us to a resolution and display that would be suitable for the final release of the App, it would also increase the size and rendering difficulty of the created models. We are currently in testing to see exactly how models obtained through photogrammetry compare to those we are currently using. This could lead us to a set maximum number of objects per scene for the time being, but this of course depends on the size and complexity of those objects. We are still looking for a solution on one of two fronts: is there an efficient way to render more objects per scene on a mobile phone without sacrificing quality? Or could the retail experience be modified, perhaps enhanced, by only displaying a limited amount of objects per scene? With new applications of VR, tech advancements, experimentation and novel ways of thinking on our side, we plan to find the sweet spot.

Further features that the app envisions will include being able to communicate with a virtual employee who can answer your questions in real-time, sharing objects and scenes you are interested in purchasing with friends and family, and pushing the functions of VR into the social sphere (think your virtual twin) - something that has also been lost with online shopping. The tools that VR provide for us allow for a more human-centric, intuitive online shopping experience, much of which was lost in 2D online shopping. Beyond this, VR offers many more possibilities beyond imitation of real-life shopping - brick and mortar walls need not limit shopping spaces, working hours need not delineate when we can experience our favorite brands. This is just the beginning of the possibilities of VR shopping - as the technology is pushed further and Gatsby continues to develop in these areas, the possibility for market transformation comes into closer vision.