Learn How to Create a Magical Landscape in Substance 3D and UE5

Introduction

Hi, my name is Robert Redecki. I'm a 3D Environment Artist living in Germany. I have been working in the gaming industry for about six and a half years. In this article, I'll describe some of the individual steps of this project in more detail, and I hope that it will be helpful for some of the readers.

Inspiration

This project started as a gift for a friend about two years ago. I sculpted a little platform with a tree that should hold a tablet pencil later. Unfortunately, we never printed it with a 3D printer. I liked the sculpted tree and the platform so much that the idea of creating a complete environment with the tree as a focus point was in my head for a long time. Finally, I started to extend this project to have a new portfolio piece.

Planning

The first idea was to stick to the platform with the tree, add two to three small plants, and render it with a clean background later. But the friend I made the gift for told me that I could make a whole environment out of it.

The basic idea of a great cave with the tree as a focus point was the first thing that came to my mind. Before starting the project's expansion, I created a small list of things that would fit the project's scope. I decided to make a group of rocks from large to small scale and two to three small plants, grass, and lianas to have a good transition between the plants and the overgrowth rocks.

The butterfly idea and the stairs in the background weren't there from the start. When the project was about 75% finished, additional ideas came to my mind because something was missing in the scene. It didn't feel complete at that moment. Then, I created the butterflies and the fire, lighting the stairs in the background to have more interesting spots in the scene.

Tree Creation

For the blockout of the tree, I extruded edges until I was satisfied with the shape in Blender. These edges became a full polygonal body with the help of Blender's Skin Modifier. This workflow became really handy for me when I created trees or branches as an example.

When the tree's sculpt was finished, I used ZRemesher as a base for the low poly. I made the ZRemesher mesh more low poly and deleted unnecessary loops by hand. It was important for me that the tree's overall silhouette remained. In terms of the UVs, I placed the seams in concave areas. I also tried to put them in unseen areas. 

I created the tree's texture in Substance 3D Painter. When the low poly was ready, I imported the high and low poly into Painter and baked all the needed Mesh Maps. In the image above, you can see the progress of the texturing. Keep in mind that I fine-tuned all the steps, and the picture only shows the overall approach.

I blocked out four different leaves in Blender for the treetop and sculpted them in ZBrush. I used soft brushes like Standard, ClayBuildup, and Smooth for the sculpt. The branches were created similarly to the big tree with the Skin Modifier workflow. Afterward, I assembled multiple leaf clusters inside the atlas with the leaves and branches. I also placed the four unique leaves in the atlas and cut them out as single pieces later.

I assigned different Vertex Colors to the pieces. Later, these Vertex Colors were baked out as a Material ID texture so that I could control the texturing progress. When the atlas was finished, I cut out different cards and assembled them into big, medium, and small branches. These were placed on top of the tree by hand. I also used this atlas to create the small bush.

Foliage Creation

The plant blockouts were created with simple planes. I subdivided them and varied them in shape, size, and thickness. I wanted to make several of the assets look clean and without much noise. When I start to block out the meshes, I always have a reference body in my scene to get a good feel for the scale of the objects compared to the player.

Most of the leaves were created with subdivision workflow in Blender. Only the tree leaves and one of the plant leaves were sculpted in ZBrush. Once the parts were assembled in the atlas, I did a quick bake in Substance 3D Painter without focusing too much on the texture in this step.

I imported the baked textures back to Blender, cut out several pieces from the texture atlas, gave them more polygons, and deformed them until I was happy with the forms.

Sometimes, I used Blender's Simple Deform Modifier to change the shape of the leaves non-destructively. With the deformed pieces, I built the final meshes, parented to an empty as pivot points.

For the texturing of the atlas, which included small flowers, plants, and the butterfly, I used Material IDs to control the individual parts later in Substance 3D Painter. I created a main folder for each part of this atlas to keep the file organized and clean and added a color selection. The image above shows the overall approach of the atlas texturing with the blue flower as an example.

Liana Creation

I also used the Skin modifier workflow to create the lianas. I created two separate shapes, formed them into the desired look, and gave them variations in size.

The bark of the lianas was sculpted in ZBrush. I used a previously created smart material of the tree texture and changed the values slightly to texture the bark.

Rock Creation

Overall, five rocks were used for the scene, including the rock platform for the tree, three smaller rocks, and one large rock for building the cave. I also created two pebbles out of the small rocks.

I usually use DynaMesh when I start sculpting rocks because the Trim Smooth Border brush slightly crushes the polygons. DynaMesh converts these kinds of destroyed areas into clean polygons. For this project, I applied ClayPolish to all the rocks in the end.

The low poly rocks were created by using Decimation Master in ZBrush. I loaded the low poly meshes in Blender and applied the Clean Up tool. Sometimes Decimation Master weirdly flips edges, so for this cleanup task, I searched for these edges and fixed them. In addition, I looked at the overall shading of the mesh and fixed shading stretches as an example. The UV seams were mostly placed in concave areas. I put every rock except the big one into one texture.

The rocks' texture was created in Substance 3D Painter. The image above shows the overall approach. I textured the big rock in the same way.

Workflow in Unreal Engine

When building the scene, I started with the landscape and large rocks. The composition took me a little time. I tried different lighting angles until I was satisfied with it. Then, I fine-tuned where the light should shine through and deleted some of the rocks that formed the cave.

I used the Foliage tool in UE5 to scatter most plants and small rocks. The trees, big rocks, lianas, and the stairs in the background were placed by hand.

The lighting setup contains one Directional Light source and a few other Rectangle Lights as a support. I placed these support lights, which weren't strong in intensity, in specific areas that I wanted to be brighter. 

Initially, the butterflies were animated in Blender with keyframes, but I decided to use a particle system approach instead of the keyframe animation because I didn't like its outcome. The particle system for the butterflies is a simplified version of a great tutorial by Pierrick Picaut on YouTube.

Advice for Beginner Artists

As advice for beginners, I recommend taking enough time in the beginning to plan the project. Of course, sometimes there will be "in-between ideas," for example, the butterflies and the stairs in the background in my project. These ideas complete the project in the end, but it helps to write down most of the things you want to create in the planning phase.

Additionally, I recommend spending enough time in the blockout phase of the project and bringing objects soon to the engine so you'll be able to get a feeling for the overall composition and how the several objects fit together in terms of size and silhouettes.

Make sure to use necessary real-life reference images of the objects you want to create and search for scientific names to get more precise results (e.g., instead of searching "Dandelion," search for "Taraxacum officinale."). It can also be helpful to look at other artists' work and figure out how they built the objects or their approaches.

My last tip is that you should get feedback from your friends and colleagues. This feedback is very important during the creation process; you can learn much from your surroundings.

Conclusion

Overall, for this project, I used Unreal Engine 5 for the scene building, ZBrush for high poly sculpting, Substance 3D Designer/Painter for baking and texturing, Blender for arranging texture atlas, modeling, and making UVs, and Photoshop for creating breakdowns.

Since I created the pencil holder about two years ago, I can't tell how long this task took me. I made the rest of the project in about four to five weeks. The biggest challenges in this project for me were the overall coloring of the final scene and the lighting.

I want to thank 80 Level for the opportunity to write this article. It's a great honor for me that you reached out to me. Additionally, I want to thank my friend Nicolas Klug for his constant feedback.

Robert Redecki, 3D Environment Artist

Interview conducted by Gloria Levine