Matte Painting an Environment in VR

Matte painting (literally: ‘background painting’) is a technique widely used in the world of cinema, which makes it possible to create, probably without great difficulty, non-existent scenarios that are difficult to reach or are too expensive to build.

Originally, when cinema was still made by film impressions, a matte painting was made by painting the various elements of the backdrops on a sheet of glass so that it could easily be combined with the negatives.

Today, this technique has been completely adapted for the digital age: through image editing software (such as Photoshop) and compositing software (such as Nuke or After Effects), the backgrounds are first composed and subsequently integrated into the shot. Not only that, but the agility and potential of this technique have also meant that, over the years, more and more video games have included it in their production workflow.

We too have decided to use matte painting to create the backgrounds of our experience. Since we only needed to reconstruct what was visible in the distance and through the windows, we resorted to this technique, which was capable of guaranteeing a photorealistic and optimized result, both in terms of polygonal count and workflow.

So instead of using the normal workflow of modeling first and then texturing, a process that would have involved two different professional figures for many days, I started to work.

The gathering of the references

I’ll never tire of saying it: the search for references is a fundamental operation when it comes to the visual design of any type of element.

In order to give historical coherence to the narrative – but with the due artistic licenses – we identified the position of the house near the artificial basin, taking inspiration from small swarms of houses present before the disaster. We used books from that time to have a more precise idea of the location of these houses, for the most part – although not in our case – used as seasonal housing for farmers.

Afterwards, I concentrated on studying the mountains in the Vajont area, understanding both their shape and the flora; I reconstructed, using images from that time, what was the size of the lake before the disaster

the lake of Vajont before and after the disaster

Some of the references I collected. Left and center: the lake of Vajont, before the disaster. Right: the profile of the mountains and the current size of the lake.

I used the measurement tools integrated in Google Maps to get a clear idea of the size of the basin and the distance between the various elements.

distance from the dam to the lake of Vajont Google Maps

The distance from the dam (considered as ‘point 0’) to the end of the lake, measured by Google Maps. This area was home to the Vajont reservoir before the disaster.

From block-out to the visual concept

In Maya I positioned the point of view near the windows, framing from inside to outside with a camera angle that was consistent with that of the VR headset (depending on the model used, the field-of-view of each headset is between 100° and 110°).

After that, I placed some general shapes that could give me a general idea of the volumes to be implemented at a later stage. I positioned the dam so that it was far enough away from the viewpoint, while maintaining its ‘presence’ within the field of view.

example block-out of window render 3d maya

The block-out of the window next to the dam, as rendered in Maya.

I then imported the render into Photoshop, with the aim of creating a real 2D concept of each of the two views I was interested in defining, to be used as a ‘visual guide’ for the next steps.

I added the mountains on different layers, reconstructing a point of view that was consistent with the collected visual references; I added trees and fog to give a clearer idea of what I would recreate in VR.

3d concept lake and mountains maya for virtual reality

The various stages of the concept.

Texture projections and the modeling of the mountains in Maya

From Photoshop I exported the different levels of mountains into as many .png files without a background; after that, I went back to work in Maya.

Inside Maya’s Hypershade, I put a projection node inside a common material. This node generates a real ‘image projector‘, which acts when it is applied to a mesh. I then duplicated this material for the number of mountains in the scene, assigning a different .png to each of them: my goal was to make each ‘projector’ recreate the profiles of one, and only one, mountain.

projection node Maya Hypershade

The projection node as realized in the Hypershade

I created a number of planes equal to the number of mountains to be modeled, connecting each of them to a specific projector. The choice to use planes – and not complex geometric shapes – has saved time and a number of polygons on stage, to create an optimized project from the early stages of development.

I arranged the planes by simulating a different distance from the house, making sure that there was no emptiness between them. Then, using Maya’s modeling tools, I gave each plane the appearance of a mountain: an essential step, so that the light in Unity could hit the meshes in a realistic way.

3d z-axis profiles of mountains in Maya for VR

The profiles of the mountains as recreated in Maya, arranged at different distances on the z-axis.

Obviously, the closer the mountain is to the point of view, the greater the number of polygons and details.

3d mountains in Maya for VR topology of mesh

The different mountains, taken from a different angle. You can see the topology of the selected mesh.

I repeated these steps for the houses on the stage, the water, and the dam.

In Unity, I applied a delight algorithm to eliminate the light information on the different textures, so as not to overlap – in an inconsistent way – the general lighting of the environment.

3d render lake mountains vajont for virtual reality

The final result

In addition to the matte-painting of the Vajont experience, I also realized the visual design of the interiors.


Related Articles

Environment development

Texturing the environment – Interiors and props

Once I received the 3D assets created by Yuri and the concepts carried out by Thomas, I set to work on the interior texturing. Although I had a fairly precise idea of ​​how to...

Posted on by Irene Zappon
Environment development

Modeling of the environment – Scenography and props

When modeling the internal environment of Vajont, I had to keep in mind three fundamental requests: The size of the room and the resulting arrangement of the interactive objects:...

Posted on by Yuri Giordani