Virtual Reality has a staggering number of potential applications from the obvious gaming and entertainment angle to education and communication (and, of course, porn). But so far, existing devices have been limited by the need for a powerful graphics card to render high-quality environments that the user can be immersed in. Enter Microsoft Research’s FlashBack, an approach that could bring high-quality VR to even the most underpowered devices.
The Oculus Rift or Vive VR headsets offer high-fidelity, low-latency virtual environments but they also require a constant tether to a powerful PC, that can push the large number of pixels necessary to create those environments. These first-gen devices are also expensive in themselves and require an even more expensive machine alongside.
Meanwhile, Google Cardboard or Samsung’s Gear VR are much more affordable alternatives, but are very limited in the graphic quality that they can deliver. Plus, while the headsets themselves can be quite cheap, the high-end smartphones needed to power the devices may be out of reach for many.
Microsoft Research has seemingly fixed all of these issues thanks to a new algorithmic approach that can deliver high-quality virtual environments on low-end smartphones and PCs. The system, dubbed FlashBack, shows huge improvements on mobile phones as compared to traditional approaches including an 8x better framerate, 97x less energy consumed by the device and 15x latency reduction.
FlashBack essentially works by eliminating the need for a device to render real-time frames for the virtual environment. In other words, Flashback almost takes the GPU out of the equation, instead relying on cached pre-rendered frames that get displayed based on the user’s actions. In terms of what your device is doing it’s somewhat similar to watching a 360-degree video on YouTube.
Of course, that’s the oversimplified explanation, because the FlashBack system can be used in such a way that even dynamic objects, let’s say a moving car in VR, can be fully pre-rendered in so-called “mega-frames” and only the correct frames get displayed based on the user’s position.
Flashback also compresses and stores said “mega-frames” either in the GPU’s VRAM, or in the RAM or on the SSD/Flash memory of the device. The frames only get decompressed in the GPU’s memory when they’re most likely to be displayed. This allows even huge VR environments to be cached efficiently without using up your smartphone’s entire storage. As an example, a decoded 4K texture uses up to 8MB of memory, but compressed on the flash storage or RAM it only needs 100 KB.
Such compression rates, which could perhaps be optimized further, would allow for a VR app to have its entire environment pre-loaded on the device. Or perhaps the needed VR frames could be downloaded straight from the Internet, or at the very least be pre-rendered straight on the device, albeit that would take a long time.
The researchers show off the improvements that FlashBack offers on a low-end HP Pavilion Mini and the Oculus Rift DK2 headset. You can see the full effects for yourselves in the video above.
At this point FlashBack is only a prototype and it has a number of limitations, mostly related to displaying a large number of dynamic objects at the same time. However, given the huge improvement in performance, not to mention battery usage, that it could provide, we might see an implementation of FlashBack show up in a commercial product in the near future.
Update: Microsoft Research reached out to us announcing they had posted a better presentation video for FlashBack, with a short explanation of what it can do. That video has been embedded above.