Compositing Fundamentals

This tutorial will cover the fundamentals of compositing, regardless of the program you use.

Before Starting

This tutorial assumes that you’ve read the Glossary page on Compositing so that you have a basic understanding of what Compositing actually is.

It also assumes that you are familiar with working with Nodes. If not, read the Glossary page on Nodes

For the sake of simplicity I will be using Blender’s compositor in this Fundamentals guide, since that’s the one most people will be familiar with the layout of, but the theory will be applicable to all node-based compositing software.

This will not teach you how to make a specific effect or something like that, rather it will cover a bunch of concepts that are important to know when compositing your renders.

The information in some of these sections can be very dense. If you find yourself not understanding something, don’t worry. Take your time when reading and try to just go sentence by sentence. Look stuff up if you need to, or just move on and try to learn by doing later on. You can always come back to this.

Render Passes

Render Passes are more commonly known as AOVs in the VFX industry, and Blender sometimes refers to them as such. AOV stands for Arbitrary Output Variable.

While we can absolutely do compositing on just a .jpg that we get out of Blender, it's important to note that Blender does not only spit out an image. We get a lot more data than that for every single pixel, as long as we've enabled Render Passes. Here we can enable a bunch of things, most of which are extremely useful. They're separated into a few sections:

  • Data
    Things like the Depth pass (how far is a pixel from the camera), the Position pass (where in 3D space is the pixel) and the UV pass (what are the UVs of the mesh at the pixel) can be found here. These don't directly affect the look of the images but we can use them to drive other effects when compositing such as Depth of Field or Fog.
  • Light
    These are the middle-points of how Blender calculates the scene's light. For example if you multiply Diffuse Direct with Diffuse Color, the result will be what the scene would look like with only Direct (no light bounces) Diffuse (rough, non-specular) lighting.
    We can use all this to for example change how shiny something appears, or changing the underlying color of an object but keeping the color of the light shining onto it the same.
    Of particular use is the Emission pass which only shows the things you gave an Emission shader. This makes it perfect for adding a glow to particles and things like that.
  • Cryptomatte
    First off, Cryptomattes to not have anything to do with cryptocurrency, so don't worry about that. They are extremely useful tools for automatically getting a perfect mask of an object, by just letting you sample for example your character's hair and then having a mask you can use to edit only the hair.
    You can do this per object, material, and asset (which is objects that are parented to each other, and is entirely optional if you have object cryptomattes enabled, they're just a convenience)
  • Shader AOV
    You can add a node called AOV Output to materials and plug whatever you want into either a color or float output. For example if you want to plug a material's metallic map into the output and be able to view that when compositing, you can!
  • Light Groups
    These are incredibly useful. Any object (including lights) can be given a Light Group, which can then be added here. When compositing you then see the scene as if it was only lit by the things in that light group. You can for example add every light to a different Light Group, and be able to control the strength and color of each light after you've already rendered the scene.

EXR Files and 32-bit Images

Another thing Blender gives us that a regular .jpg doesn’t is 32-bit image data. Normally if a pixel is brigher than 255 (which is the highest you can count with 8 bits, which is what most image files use and what your monitor can display) then it just clips to 255, anything beyond that is discarded. What we get out of blender can go mugh higher though. This means we have a lot more details in highlights and colors than the eye can see, but we can bring that out when compositing.
This example shows how a render exported as a JPG does not retain any information in the bright parts of the image when darkened. But if exported as an EXR file we can count to 2.14 billion as opposed to just 255, and all the brightness information stays!

Does this mean that you should start exporting your final renders as EXR files? No, we still want the final image to be 8-bit like a .jpg since that’s what our monitors can display, but we want to work in 32-bit. If you’re planning on working directly in Blender then you don’t technically need to export it as an EXR ever, since data there will already be 32-bit. But it can still be nice to save your render so you can close and re-open Blender without needing to re-render.

There’s another very important benefit to EXR files over the usual formats: They can store multiple images (or layers) in one file.
This means that you can pack your entire render, render passes included, into one file per frame. When imported you can then extract each layer again.

If you just did this though the file size would be enormous, up in the gigabytes. For that reason it’s also good to know that we can compress some data while still keeping it high-quailty.
An EXR file can be compressed in many ways but the most important ones are DWAA and ZIP (yes, just like zip files). ZIP is lossless, meaning no data is lost at all. This creates big file sizes. DWAA is lossy but mostly perceptually lossless, meaning it throws away some data our eyes can’t really percieve anyways. This can greatly reduce file sizes.
When exporting it’s a good idea to export most of your render passes as DWAA using something like 85% quality and set to Float (Half), and only save data passes like Cryptomattes as ZIP in Float (Full).

In Blender you can choose what gets exported where by setting up File Output nodes in the Compositor window. When the compositor runs, those files are saved.

Cryptomattes

Cryptomattes allow you to select a part of your render and get a mask of it. It works on individual objects, materials, or assets (groupings of parented objects). Cryptomattes HAVE to be losslessly compressed in order to work once exported to an EXR file.

They can't mask everything perfectly especially if it's an object in fog or similar. If you can see the object but are unable to sample it with the eyedropper tool, you can still enter the name of the object/material manually.

Merging

Merging is how we add one image on top of another, or blend them together in various ways. In Blender there are two nodes for this (whereas there's usually only one in other compositing software), Alpha Over and Mix.

Alpha Over simply places a foreground image over a background image, based on the Alpha channel of the foreground. It's the same as just dragging one image on top of another in a layer-based program. Mix lets you select various blend modes like Screen, Add, Multiply, Color, etc.

In the Mix node, the B input is the foreground and the A input is the background. This is the opposite in Nuke where B is your main input, and Fusion just labels the inputs with colors.

Alpha

An image usually consists of four channels: Red, Green, Blue and Alpha. Alpha is transparency. Where Alpha is 1 the image is fully opaque and where it is 0 the image is fully transparent. But even if the Alpha is 0 that doesn't mean what's in the RGB channels goes away.

Blender will by default "Premultiply" or "Apply" your Alpha channel, so that you see what the image looks like when the Alpha is used as intended. The alternative is that your Alpha is "Straight".

You won't need to think about this a lot in Blender specifically, but if you do compositing in other software it becomes very important to understand what effect your Alpha channel has on your image.

Math

Images are just numbers, and we can do math on numbers. This can be incredibly useful once you start making the connection and learning which math operations are useful where.
Say for example you have two masks and only want an effect to happen where they overlap. In that case you can just multiply one mask by the other. In fact every operation you do in compositing is just various amounts of math, some of which is very simple to replicate. Complete desaturation for example can be done with this formula: 0.3 * R + 0.59 * G + 0.11 * B (The different color channels are added with different weights based on how brightly our eyes percieve them)
You won’t always use this but it’s very good to know that you can, because it helps you realize that some things are possible that you wouldn’t otherwise think of.

Important to know when dealing with numbers in compositing is that we consider images to normally be in the range of 0-1. You might be used to 0-255, but that is not the case here, for various reasons and with various benefits.

Color Spaces & Tone-mapping

Working Color-space is what your math happens in

Authors

This site uses Just the Docs, a documentation theme for Jekyll.