NVIDIA Canvas is an AI-powered tool that turns your sketches into photorealistic scenes

In March 2019, NVIDIA Research showed an AI-driven project, GauGAN, that used a deep learning model to convert simple doodles into photorealistic images. Over two years later, the technology is ready for showtime as NVIDIA Canvas. NVIDIA writes, ‘Use AI to turn simple brushstrokes into realistic landscape images. Create backgrounds quickly, or speed up your concept exploration so you can spend more time visualizing ideas.’

There is a slight caveat, though. Not anyone can download and use the NVIDIA Canvas beta. You must be running Windows 10 and use an NVIDIA GeForce RTX, NVIDIA RTX, Quadra RTX or TITAN RTX GPU. NVIDIA Canvas is part of NVIDIA Studio, a program that delivers hardware and software tools to artists.

In the two years since GauGAN was shown off, NVIDIA has made considerable progress with its AI technology. When users draw simple shapes and lines from a pool of 15 tools, including things like mountains or weather effects, Canvas’s AI turns your sketches into photorealistic scenes nearly instantaneously.

Canvas allows users to create backgrounds quickly or speed up concept artwork. Within seconds, you can go from a messy, simplistic sketch that barely resembles a water and mountain scene to a photorealistic piece of concept art. NVIDIA writes, ‘Paint simple shapes and lines with a palette of real-world materials, like grass or clouds. Then, in real-time, our revolutionary AI model fills the screen with show-stopping results.’ It’s a lofty promise, but Canvas looks good in action.

Credit: NVIDIA

After Canvas has done its thing, you can quickly export from Canvas to Photoshop for further refinements or to combine your image with other files. It’s not necessarily designed to create final, marketing-ready results, but it aims to save concept artists considerable time during the design and preliminary phases. That said, the results are impressive.

Canvas is built upon generative adversarial networks, GANs. These AI networks not only generate an appropriate image for your selected materials, lines and shapes, the AI also works to integrate new changes with the existing image, making sure that everything matches and blends together realistically.

In NVIDIA Canvas, you can choose from 15 materials and 10 styles to create a photorealistic scene from simple lines and shapes. Credit: NVIDIA

The AI was trained using millions of images of real-world environments. In 2019, NVIDIA’s VP of applied deep learning research, Bryan Cantanzaro, said, ‘This technology is not just stitching together pieces of other images, or cutting and pasting textures. It’s actually synthesizing new images, very similar to how an artist would draw something.’ The technology has come a long way in two years and appears to be delivering on its initial promise.

If you have the required GPU and would like to try NVIDIA Canvas for yourself, click here to download a beta version.