Breakdown: The Science Of Lighting

Introduction

Hey everyone, my name is Arthur Tasquin and I'm a Lead Unreal Artist working in the VFX industry. I'm passionate about world-building and environment design, and in my free time, I like to create visuals in Unreal Engine.

I also started to write about visual theory in movies and video games to get a better sense of what makes a visual interesting. Today, I'll dive into the making of my last project and my understanding of the science behind lighting.

The Project: Look Up Doodles

References

This project started with a few photographs I took on my way to work. I always liked to look at the roofs in the morning light. I loved how the sun shaped the shadows and I knew I wanted to picture that in my work.

Using Quixel Assets, I started to assemble a little scene and I realized it would be a great opportunity for me to improve my lighting by creating several lighting scenarios.

From my photographs, I still needed to find moods to reproduce. The first one was this amazing overcast render by Marcin Martynowski. I also came upon a few lighting studies that really captured what I was aiming for: The Relight of the Seaside Town by Lucas Uny or the work of Ted Mebratu.

Throughout my research, I decided to go for 7 different moods: Morning, Sunset, Overcast, Blue Hour, Horror, Neon, and Foggy. For the first 4 scenarios, I aimed for a realistic approach that sparked the desire to learn more about the physics of lighting. The last ones, stylized and more dramatic, were just a fun exercise.

Set Dressing

The aim of the project was first and foremost lighting so I didn't want to spend time creating any asset from scratch. To achieve this, I decided to stick to a specific framing: most of the shots are low angle and only focus on the upper part of the buildings. By hiding the streets, I made the whole project very easy to set up.

Using Megascans and a few props from the Modular Seaside Town I got from the Marketplace, I started to dress the scene. Level Instances were very helpful in that regard as I could work on one building and replicate it on demand without overcrowding the outliner. It's a very clean workflow and it's iterative: any time you want to make a change to your Level Instance, it replicates it on every other instance in your scene.

Structure

Dealing with seven lighting cases can lead to chaos if you don't stick to the right workflow. I could have used World Partition and the data layer system but as the scene was relatively small and I didn't plan to work on the same level with someone else, I used the good old sub-level workflow.

If you don’t know about it, it's just a way to compartmentalize your scene into a set of different levels: the main one is called persistent and its children are called sub-level. The benefit of this technique is that you have a specific window only for your lighting scenarios and you can play with their visibility through Blueprints or the Sequencer.

As every lighting scenario was supposed to feature the same environment (with a few tweaks), all the main assets were contained in the persistent level. Each lighting scenario had its own sub-level that I could turn on and off. 

Even when the end result is an image, I usually play with Level Sequencers to iterate on the framing or the position of the lights. It helps me compare several ideas and prevents me from losing them. When working on a shot-based project, you always want to refine details depending on the scenario: windows placement, light blockers, light position, materials, you name it.

Another great feature of the Level Sequencer is the Level Visibility Track. For each Level Sequencer, you can choose the levels you want to hide and the ones you want to load. It is essential for us to automate this process to save some time during the rendering phase. You don't want to manually edit this kind of stuff, especially if you have a lot of variations.

One recurrent issue I always come across with Level Instances in Sequencer is the inability to make them invisible when rendering the frame. Sometimes, I don't like the shadow's shape of a Level Instance and I want it disabled for a specific shot but the only solution for me was to remove completely the link to the level in the Level Instance detail panel.

Lighting

I already knew from the start that the sky atmosphere would not be enough for what I wanted to achieve in this project. The only time I used it was through the Ultra Dynamic Sky for the sunset scenario.

The rest of the project relied mostly on the HDR Backdrop, Screen Space Fog, light reflectors, Lumen, and regular lights. While searching for HDRs, I came across this website that offers a lot of free and various HDRs. For the lighting itself, I'm a big fan of Rect Lights and it's almost the only type of light I used in this project except for the Directional Light in the sunset and morning scenarios.

I also extensively used Lumen and similarly to the way we shoot movies, I used light reflectors to regulate it. If you don't know about it, light reflectors are pieces of fabric that we use to reflect light (if it's white) or absorb it (if it's black).

In our case, it's very useful to tune down or boost Lumen in a localized space. I often use it to darken an out-of-focus foreground and put the emphasis on the subject of the frame. In the pictures below, you can see a white reflector bouncing light and a black reflector, also called a flag, reducing the light on the subject, from the Ultimate Guide to Cinematic Lighting by Studio Binder.

Another way I used simple primitives out of the frame was to shape the shadows of the sun. This proved to be really powerful in the morning mood to get closer to my photographs.

The fog usually plays a huge part in my work. I’m a big fan of Blade Runner 2049's Las Vegas look so when I heard about the Screen Space Fog Scattering plug-in that creates a believable deep fog effect, I rushed to test it and I can't do without it anymore. You can tweak it with a few console commands but it usually works right out of the box.

Here's the same scene with SSFS on and off:

Regarding the tech, I tried to use raytracing shadows but it wasn't that interesting. The quality of the shadows wasn't that substantial for the performance costs it introduced. A common issue you have when combining raytraced shadows and Nanite is weird shadow artifacts due to the way Ray Tracing uses the preview mesh from Nanite to cast the shadows. You have this mismatch between mesh and shadows that you can get rid of by using the following console command: r.RayTracing.Nanite.Mode 1

Be careful though because this console command can lead to performance issues.

Rendering

Like every project, it's a matter of iteration. You try something, it doesn't work, so you try another thing and refine it until it does. For Look Up Doodles, I made plenty of variations from the framing to the lighting itself that allowed me to experiment as much as possible until I was happy with the result.

The Movie Render Queue proved to be invaluable when you need to render multiple versions of the same scene. Through the use of presets, I managed to create a plug-and-play solution to quickly render the right mood with all the good settings.

The first step was to create presets of the settings based on the tech used: switching on and off raytracing shadows and reflections. The second step was to create a queue preset linked to all the right sequencers and their specific preset settings with a few tweaks. This was very powerful as I only had to load one preset and then select the scenario I wanted to render.

I noticed that Unreal's Depth of Field can introduce artifacts when you have an out-of-focus foreground. I tried a few console commands but I quickly switched to Path Tracing which improved substantially the result. However, I only used it for the Neon scenario.

The Science Of Lighting

Introduction

Lighting can be a vast subject, especially if you take the science behind it. The following section is an attempt to make the matter understandable to any digital artist. I strongly believe that knowing how things operate under the hood can elevate the outcome. 

Keep in mind that I'm not a scientist, this part will only be relevant to digital artists who want to grasp the basics of lighting science. For the sake of simplicity, I took a few shortcuts and approximations that won't impact the result in the engine. However, if anyone has a deeper understanding and wants to add something, I would be glad to chat!

Why Use Physically Accurate Data in the First Place?

In a digital camera, the exposure is defined by 3 parameters: the ISO (sensor sensibility to light), the aperture (how big the diameter that lets the light in is), and the shutter speed (length of time at which the sensor is exposed to light). The combination of those is called EV (exposure value) and it's your job, as a visual maker, to match the right value to the lighting of your scene.

This chart comes from The Exposure Triangle, a free book by Studio Binder:

A bright sunlit afternoon would need a lower exposure to avoid being dazzled and a cold night scene would require a higher exposure value. In a practical way, it means that if you double the light intensity in your scene and divide the exposure by 2, you'll get the same result.

Unreal Engine works the same way: the exposure dictates the sensibility to the light of your level. As long as you're careful with your exposure, you can totally manage without using a single physically accurate value. So why do we even bother? The answer is consistency: to create a pleasing balance between your lights.

The scale of light intensity is huge, numbers can go from 0.001 to 100.000 depending on the source (candle vs sun). By using arbitrary values, there's a good chance you won’t respect the right ratio between the lights in your scene.

This is especially useful for beginners who try like me to learn how lights work. It can ground you into reality. Over the years, you get an eye for that sort of thing and you can trust your guts on what works or not.

Working with real values will also display the camera's limitations. Similarly to a real camera, if you calibrate your shot for an interior scene, your windows will be over-exposed. This is something a lot of cinematographers have to deal with. Without any references, you would be tempted to reduce what's over-exposed and crank up what's under-exposed but that's not how cameras work. For a specific EV number, they have an ideal range of illuminance. Everything above and unreal that range is clamped. In Unreal, you can play with Auto Exposure to get dynamic exposure.

Where to Stop?

Using physically accurate data is a good start but it'll never replace your artistic eye. Your ideas, moodboards, and references prevail over the technicality. In the end, the only thing that matters is the result, not how you got there. Relying only on science will lead you nowhere. If you spent the last hour trying to force 2 values to fit together, it means you lost yourself in the details. The database I've linked to this article should only be used as a scale reference but I'll address this in the values section.

Breaking the rules of physics is not wrong if it's intentional. Movies and video games are smoke and mirrors. They use tons of tricks to elevate the visuals. Moonlit scenes are the perfect example: the moon is not bright enough for camera sensors so we always need to use additional cinema lighting that we place out of the frame. We also got used to the blue tint of night time even if it’s not the case in real life.

To sum up: don't let physical values stop you from achieving a specific look you're after, a mood, or an idea. Use it only as a starting point and tweak it as much as you want.

Units

When talking about lighting, there are 4 major units that you should know: the candela (cd), the lumen (lm), the nits (cd/m2), and the lux (lx). The following definitions are from Wikipedia:

• The candela (symbol: cd) measures luminous power per unit solid angle emitted by a light source in a particular direction;
• The lumen (symbol: lm) is the unit of luminous flux, a measure of the perceived power of visible light emitted by a source;
• The nits or Luminance (symbol: cd/m²) is a photometric measure of the luminous intensity per unit area of light traveling in a given direction. It describes the amount of light that passes through, is emitted from, or is reflected from a particular area, and falls within a given solid angle;
• The lux or Illuminance (symbol: lx=lm/m²) is the total luminous flux incident on a surface, per unit area. Similarly, luminous emittance is the luminous flux per unit area emitted from a surface.

If you are lost, don't worry I was in the same place when I tried to get my head around the theory. We'll dissect it together.

The first thing we can do is split the units in two: the ones related to a surface (nits and lux) and the others (cd and lm).

Let's start with the candela (cd) which measures the luminous intensity and is the base unit in the international system of units. You'll find it in all the units mentioned above as it's the foundation on which everything is built. A candela takes into consideration the angle of diffusion, this means that it will be inversely proportional to how much the light spreads. A narrow beam like a pencil torch light will have a higher candela value than a classic incandescent bulb spreading almost at 360°. Although being the base unit, you will mostly rarely see cds in your day-to-day life. 

The lumen on the other side is the unit you'll find on every light fixture or bulb you buy. It is the quantity of light emitted by a source, the brightness of it. It is the most easy unit to understand as it doesn’t care about angle or surface.

lm=cd*sr

cd=lm/sr

The relationship between lumen and candela states that 1 lumen equals 1 candela times steradian. If you don't know about it, a steradian is just an angle but in 3D, in other words, a cone. If you consider a spherical diffusion of light ("at 360°"), a candela would be equal to 12,56 lumens.

Practically, most of the lights in Unreal have the possibility to be set in candela or lumen but I would suggest using lumen as you can find references in that unit all over the internet.

Luminance and Illuminance both introduce surface into the equation. I found their Wikipedia definition confusing and hard to differentiate so I focused on the practical application of the units and here's what I concluded:

Luminance is calculated in nits (nt) and is based on candela which means it both takes into account the angle of diffusion and the surface from which the light is emitted. It is the measure of light that is emitted from a surface. In your day-to-day life, you will find it on the packaging of emitting devices such as screens. From a practical point of view, luminance is calculated by pointing a device called a spot meter directly at the source of light.

Illuminance is calculated in lux (lx) and based on lumen which means it only takes into account the "receiving" surface and brightness from the source. It is the measure of the quantity of light that hits a surface. In your day-to-day life, you can find it on the camera's specifications as the subject illumination range (the range at which the camera can record a satisfactory image).

From a practical point of view, illuminance is calculated by placing a device called an incident meter or lux meter where you want to measure it. It is used by photographers and cinematographers to figure out the right amount of light needed for portrait or product photography. Here's the use of a lightmeter on the set of Inception:

nt=cd/m²

lx=lm/m²

To sum up:

• Candela (cd) is the base unit that takes the angle of diffusion into account;
• Lumen (lm) is the measure of brightness of a light;
• Luminance (nits) is calculated from the source;
• Illuminance (lux) is calculated from the environment lit by the source;
• We will mostly use lumen and illuminance.

Units in Unreal Engine

Lighting in Unreal is pretty contained. You only have to tweak the light themselves, the Post Process Volume, and the Project Settings. While local lights can be set to lumen or candela, HDR Backdrop (Skylight), emissive surfaces, and Directional Light can't be changed. The first ones are set in nits (cd/m²) and the last one in lux (lm/m²).

If you want to check by yourself, Epic made a pretty extensive doc about it.

Before any lighting work, you should always check your Project Settings. "Extend default luminance range in Auto Exposure settings" should be checked. It is located in the Project Settings > Engine > Rendering > Default Settings. As of today, in the 5.4 version of Unreal, it is deprecated which means it'll already be checked. Nonetheless, it could not be the case if you have an older version of the engine. The other thing I usually do is set the lighting base unit to lumen instead of candela. You can find this setting in the same section as the previous one.

Additionally, each light's unit can be tweaked individually in the Details panel of the actor:

Rules

EV100 & Stops

As I mentioned above, EV is the combination of its components: the ISO, shutter speed, and aperture. I won't go over the basics of the exposure triangle but if you're lost, I would strongly suggest reading the Ultimate Guide from Studio Binder. It's a very good place to start.

So how does EV gets affected by its components?

In their own range of values, each of them can be increased or decreased by one stop. A stop is just the difference between two adjacent values. For instance, if you want to add one stop to the shutter speed at 1/125, you will get 1/250. This change in value will affect the overall exposure by a factor of 2. Each time you change the values mentioned above, you double or divide by two the exposure.

To make things easier, we usually lock one or two components and adjust the other one(s). The most common thing to do is to keep the ISO at 100 and change the shutter speed and aperture. From now on, we'll only be using EV100 which indicates we locked the ISO at 100.

EV100 also has its own value that matches several combinations of shutter speed and aperture (also called f-stop). In the following chart, you can see all the possible combinations for a specific EV100 value (on the left column).  For instance, if we take EV100=5, we can achieve it through the combination of 1/30 and 1.0 or 1/2 and 4.0.

How is it useful?

From those numbers, we figured out the rules of thumb:

1. The ideal EV100 for any lighting condition;
2. The lighting contrast ratio rule;
3. The Sunny 16 rule and its variants.

EV100 and lighting conditions

Each EV100 has its own ideal lighting condition. The brighter the scene, the bigger the EV100. You can find some of the main examples down below but I've listed them all in the database.

Lighting Contrast Ratio Rule

The lighting contrast ratio is the difference in stops between the bright (key) and dark (shadow). On a sunlit outside scene, the contrast ratio will vary with the time of day.

• In midday, the sunny side of an object is 4 stops (16:1) brighter than the shadows;
• In the afternoon, the sunny side of an object is 3 stops (8:1) brighter than the shadows;
• In the evening, the sunny side of an object is 2 stops (4:1) brighter than the shadows.

Keep in mind that those values highly depend on the sky and your surroundings. To validate this assumption, I sampled a few data myself (see Values chapter) and realized that the white wall next to me affected the result by a factor of 2 stops. In Unreal, Lumen will lighten your shadows so those numbers should be taken with a pinch of salt.

In cinema, the lighting contrast ratio mostly varies depending on the art direction of the movie. In this video of Justin Phillip, we can see how contrast differs from film to film regardless of the lighting conditions:

The Sunny 16 Rule

The Sunny 16 rule is a very simple rule in photography that can help you estimate the correct exposure settings for daylight. The rule is the following: On a sunny day, set the aperture to f/16 and the shutter speed to the inverse of the ISO value for a subject in direct sunlight. If we're shooting at 100 ISO, those numbers will be f/16, 1/100, and 100.

There are derivatives of this rule for a few lighting conditions that are listed below:

Values

When starting my studies for this project, I was struck by the lack of publicly available data. After a little research on the subject, I decided to gather every bit of information I could find in my own database.

By comparing several websites, I could validate my data and eliminate what seemed divergent. Nonetheless, I was still missing some scenarios and I wanted to take part in the process, to understand how weather affects light, and to know how bright are my surroundings. So, I bought a light meter and took it everywhere with me.

One thing I learned is that there are a ton of parameters that affect the equation. Numbers depend on the weather, the pollution, the time of day, the season, the location, the surroundings, and the clouds. If we compare my own data to the web we have 58.000 lx against 100.000 lx in bright sunlight.

Additionally, a few other parameters, related to our own human perception and the limitation of our tools, alter the result. Due to the way our cones work, perceived brightness depends on the color of light. Each of us sees colors differently and sometimes more drastically (color blindness). Depending on where we sample our data of the sky, the numbers might change even if it is uniform and cloudless.

This website highlights the distribution of the sky's luminance depending on the weather conditions:

So why do we use that data if it varies so much?

It's a matter of scale. While a sunlit outdoor is at 100.000 lx, a home kitchen can be at 300 lx. We can define a range of values for each scenario and adapt them based on our scenes. An office for instance has an ideal illuminance range between 300 and 500 lux, it will never be above 5000 lux.

If you take the appropriate EV100 as a starting point and work in the range of each data, you already have a correctly exposed scene that you can iterate on.

Workflow

Now that we learned the theory, we can finally see how to apply this in Unreal Engine. 

Exposure can be set in two places: the post-process volume (PPV) and the camera. As I usually like to move around in my level to see the lighting through several POVs, I would suggest inputting the data right in the PPV. If you make several lighting scenarios, create one PPV per scenario and name it properly.

There are two ways to approach exposure in Unreal: the camera settings or the EV100 value. For the first one, you use the Sunny 16 rule and its derivatives and for the second one, you use the EV100 chart.

I personally like the EV100 approach because you only need to take care of 1 value that will drive the whole exposure, it's quick and simple. You can then tweak it slightly by changing the Exposure Compensation value.

How to Measure Luminance & Illuminance in Unreal Engine

We saw that light intensity can be set in the Details panel in lumen or candela for local lights and in nits and lux, respectively, for the Skylight and Directional Light. But how do we validate our data? Similarly to a light meter, how do we sample the luminance or illuminance of a scene?

The AutoExposure view mode is a very helpful tool that answers all those questions. It includes two little squares that sample illuminance (above) and luminance (below). The first one comes with RGB values that point out the color distribution of your lighting. The second one features an EV100 ideal value for the current sample.

You also have a ton of information related to the AutoExposure system but as we locked it in the PPV (Min EV100=Max EV100), it's not that important. I usually lock AutoExposure when working on a specific shot but this feature is really interesting for games and cinematics.

Now let's see how to create a physically accurate lighting step by step. If there's one thing you should keep from this article, it's this workflow:

Steps

1. Set up your EV100 (in the PPV) to the right approximation based on the EV100 chart or the camera settings (in the PPV) based on the Sunny 16 rule & co;
2. Set up your HDRI based on the database and validate the luminance (rough estimate through AutoExposure view mode);
3. Set up your Directional Light based on the database and validate the illuminance (rough estimate through the AutoExposure view mode);
4. Set up your Local Lights (if any) based on the database;
5. Optional: Tweak the values based on the target Light Contrast Ratio;
6. Tweak the lighting to match your art direction/references.
   1. Use Exposure Compensation to adjust slightly the exposure in the PPV
   2. Play with light placement, flags, and reflectors
   3. Tweak lumen and light intensity

Tips & Tricks

• With all those rules and data, we quite often try to compare one method with the others and don't get the same result. This is something I encountered a few times during my studies. As explained above it's due to the amount of parameters and you will get nowhere if you try to match a value to all the rules. Sometimes it doesn't match so pick the one that works the best visually.
• Depending on where you sample your HDR, the result might vary a lot. As we saw above, the sky is not uniform in terms of light intensity. Try to find a sweet spot that is not too close to the sun or the horizon.
• As Unreal default values are not physically correct (i.e. Directional Light intensity is at 10 lux), your scene will probably be pitch black when setting your EV100. Don't worry, it's normal and you'll get decent lighting when tweaking your HDR and lights.
• If you use emissive materials, don't forget to take lumen into consideration. GI will lift your shadows thus decreasing the lighting contrast ratio.
• Even if you’re EV100 is set, don’t hesitate to tweak it later in the process.

Tool

I'm currently working on a toolset to simplify the physical lighting process. The following prototype allows you to pick a lighting scenario and retrieve the numbers from my database without leaving the engine. A few additional tools like an EV calculator or a kelvin convertor are also available.

The structure is there but I want to improve this by:

• Adding the basic theory about physical lighting (to refresh your memory when you have a doubt);
• Feed the database with my own samples I've been collecting for a few months (for accuracy);
• Improve the UI and UX;
• Adding a few other tools;
• Creating a debug view mode.

If you want this tool or have additional features in mind, don't hesitate to reach me.

Conclusion

This whole process can be overwhelming and feel too technical but once you do it a couple of times, it gets way more organic. I really think that knowing what's happening under the hood can elevate your work because you have a deeper understanding and connection to the process. Like in any art discipline, once you know the rules, you can bend them or completely break them. However, keep in mind that it’s not the only way to light, choose what suits you the most.

We come to the end of this long journey, I hope you liked the overview and that you learned something. If you want to reach me, you can connect through LinkedIn or send me an email.

I also write about visual theory on my own website. My last blog post was an analysis of The French Dispatch.