The human eye sees reduced detail in its peripheral vision. Foveated rendering, pioneered by Microsoft (SIGGRAPH 2012) and Nvidia (SIGGRAPH 2016), is a technique that mirrors the way the eye works to improve quality in the fovea area and reduce render cost. As display densities increase, we can take advantage of this to create a visually life-like experience.

‍

At SIGGRAPH 2018, we presented the ideal use case of foveated rendering using the StarVR One with built in Tobii eye tracking. More details on the project and event can be found in our project page.

‍

Rendering a highly detailed scene on the 1,830 x 2,928 display at 90fps is already a challenge, but to create a true-to-life experience, we have to reduce pixel aliasing artefacts as much as possible. As a result, the actual render resolution is much larger: when using multi-sampled antialiasing, we render an image at 4x the display resolution.

Even though modern GPUs are very powerful, there are finite rendering resources available when rendering at 90fps with less than 20ms latency. To avoid having to decrease the image quality, foveated rendering greatly reduces the cost of the 210-degree field of view (FOV) by only rendering the area their eye is focused on in high detail.

The solution created requires the scene to be rendered an additional two times: firstly, to the lower-resolution, wide-FOV target for each eye, and, secondly, to a focused, high-resolution render target for the fovea area. The image above shows the two rendering configurations we support: 4 views for high-resolution wide FOV and 4 views for foveated rendering. The location of the high-detailed render area is calculated using the data returned by the integrated Tobii eye tracking. The image below shows the area that is rendered in higher detail - look at the difference between the ‘S' and the ‘V'!

The results speak for themselves. Using Intel GPA, the two captures below show the performance difference when rendering the 4 viewports with equal pixel density split over the wide FOV (~14ms) and then with the foveated viewports (~11ms).

‍

We discovered that, due to the small latency in eye tracking, the blend between the foveated area and the lower resolution background had to cover a larger area to prevent the user from seeing flickering as the pixel density changed. This adds an additional pixel-processing cost but ensures the user doesn't experience any visual artefacts as they move their eyes around the scene.

Foveated rendering will become more common place as the second generation of VR headsets become more widespread and greater resolutions are required for true-to-life immersion.

For the latest tech news, follow our ZeroLight Tech Twitter page and our #ZLTech hashtag.

‍