Neural Super-Resolution in Real-Time Rendering Using Auxiliary Feature Enhancement

Introduction

The GPU was invented to meet the demands of massively parallel computation tasks from computer graphics, so a mutual relationship exists between the development of GPU performance and computer graphics technology. Display devices have developed in recent years, including 4K/8K high-resolution monitors and virtual reality headsets, which demand higher resolutions and refresh rates than before. The exponential growth in resolution places a tremendous burden on the GPU, while the GPU computing power can only grow linearly. Because modern hardware is far from meeting the actual requirements, it is inevitable to keep rendering at low resolution and obtain high resolution through some means.

While superresolution is a relatively new topic in computer graphics, there are other methods of rendering at a resolution different from the target. In fact, when dealing with the aliasing problem, an approach called Super-Sampling Anti-Aliasing (SSAA) solves it by sampling at a higher resolution and downsampling to the original resolution. It can be seen that antialiasing and superresolution are both reducible to undersampling, the same basic graphical problem. Aliasing results from insufficient sampling, whereas superresolution is expected to reconstruct the result with a much lower resolution. As a result, advanced antialiasing methods can be modified to solve superresolution problems.

The G-buffers are visual data structures that are packed with various scene information and can be easily accessed in real-time rendering applications. In most cases, buffers are produced in just a few milliseconds per frame in a single draw call. Using G-buffers effectively can help the network predict high-resolution results better.