3D Generalist Marcin Solarz has shared a comprehensive breakdown of the Black Chasuble project, explaining how the asset was made using ZBrush, RealityCapture, and Substance 3D Painter.
Introduction
Photogrammetry enables the creation of accurate 3D models from two-dimensional images. It's a 3D scanning technique that uses multiple images taken from different angles to create a 3D model of an object or scene. It is a versatile tool that can be used in many fields, such as surveying, engineering, architecture, 3D printing and video game development.
Photogrammetry is also a method used in the preservation of cultural heritage sites, museum objects, or architectural monuments. Photogrammetry offers many advantages over traditional 3D modeling methods, including accuracy, precision, cost savings, and speed. If you are looking for an efficient and cost-effective way to create accurate 3D models, then photogrammetry is the perfect solution for you.
My name is Marcin Solarz, and I have been working in digital art for more than 20 years. Although my formal education is as an engineer, I have never worked in this profession. At the beginning of my career, I did 2D design for several years – book covers, folders, and secure bank documents – but I was always fascinated by 3D creation. With the rise in popularity of video games, I was inspired to explore 3D modeling. My adventure began with programs for the Amiga, then came 3ds Max. I expanded my skills from modeling to texturing, shading and rendering in various engines (Brazil, Mental Ray, VRay, Arnold). Then came physics simulations and particle systems. I improved my skills when I took a job at a computer game company as a Modeler and Animator; here I learned a lot about animation and game production.
Together with a group of friends, we decided to create a turn-based strategy game called Soul Quest. It was an intense three years of learning for me, as our team consisted of only three graphic designers (including me) and three programmers. Everyone had to multitask and cover different aspects, such as gameplay design, level building, balancing, narrative development, and much more. I worked on ambient elements, character models and animations, while taking care of special effects. In the end, the game was not released due to lack of marketing support; however, it is still an amazing testament to what we accomplished.
After that, I worked mainly as a freelancer in film and advertising production companies and architectural visualization companies, (including Platige Image, AES+F and many others). I worked as a freelancer for nearly 10 years. The only thing missing in this adventure was mastering photogrammetric visualization.
Taking advantage of my free time, I try to develop my skills in traditional art techniques such as wood and clay carving and watercolor painting. These activities are incredibly rewarding and allow me to add to my existing knowledge and skills. You can check out my works on my Instagram page.
Getting Started With Photogrammetry
For several years, I have been working on one of the world's largest digitization projects – the Virtual Museums of Malopolska. We have several thousand digitized (processed) museum objects from 47 museums in the region. My main responsibilities include the development of cultural heritage objects using photogrammetry and structured light scanning. My visualizations have a set of PBR (Physically Based Rendering) textures. The resulting digital material virtually simulates any type of physical material and enhances the details of the 3D model. All of our models are available under open licenses (CC0) for free download and use in any project. My models have recently been used in Unity teaser (anatomical models – écorché), promotional videos for Unity and Unreal Engine.
My models used in animation:
So far, the 3D replicas prepared by our team have been viewed on Sketchfab more than 1,200,000 times. What I like about photogrammetry is that we work on very large grids (often the raw model is 60-100 Million poly each). The data downloaded by my team is often hundreds of photos and dozens of scans, from which we create visualization.
Increasingly, my objects are appearing – used directly or as part of a scene composition – in the works of visual artists working in the digital domain. Our materials are reused and used in a completely new context – they are given a second life. An example is the animation of the Master Twardowski figurine designed by Zofia Stryjeńska. In my version, the toy comes to life and performs a choreographed routine from world-famous hit songs.
My animation of Master Twardowski toy from Ethnographic Museum in Kraków:
We are publishing the results of our work on the Sketchfab portal. The technology has been used to create virtual museums of Malopolska, which are hosted online and accessible to anyone with Internet access. Through these virtual museums, users can immerse themselves in the history and culture of Malopolska, learning about different aspects of the region's history and visiting historical sites. Using photogrammetry technology, the virtual museums are more detailed than ever before, providing viewers with a realistic representation of places that would not be possible without it. Combining advanced technology with traditional cultural practices, the project allows viewers to learn about Poland's history and culture without leaving home.
How to Make a Good 3D Model With Photogrammetry?
Representing a 3D model and highlighting details is one of the most important parts of the photogrammetry process. Getting the lighting and perspective right is key to getting a good end result. Post-production can also help to crank up the settings and highlight the most important elements. There are many ways to use photogrammetry to create three-dimensional models. One of the most popular is the so-called photogrammetric method, which involves transforming photos into three-dimensional spatial models.
Photogrammetry is the process of using photography to create 3D models from 2D images. The technique is increasingly used in cultural heritage preservation because it provides an efficient and accurate way to capture and preserve information about historic structures and artifacts. The process involves taking multiple images of an object or structure, usually from different angles, and then combining them to create a 3D representation. The resulting model can be used to generate detailed measurements and a more accurate representation of the object or structure than would be possible with just one photo. Photogrammetry is particularly useful for capturing complex shapes and intricate details, which makes it ideal for preserving and studying cultural heritage. In addition, photogrammetry can be used to document buildings and artifacts that are destroyed or no longer exist, allowing us to create a digital record of their history. With this type of documentation, we can reconstruct physical elements that may have been lost due to natural disasters or human destruction. In this way, our understanding of cultural heritage remains intact, even if the physical objects have been destroyed. Additionally, photogrammetry allows us to create three-dimensional replicas of artifacts and monuments for use in virtual environments, such as augmented reality applications or interactive museum displays.
I will discuss the process of creating the 3D model using the example of the Black Ornate object from the Miechowska Land Museum. The chasuble dates back to the 18th century, when the technique of flat, colored and shaded embroidery prevailed in Polish embroidery manufacturing. Chasubles in black were used during funeral masses. The black color symbolizes penance, mourning, sorrow and the absolute. It is the color associated with death and resurrection.
The Working Process
Creating 3D models of cultural heritage based on photogrammetry is a process that begins with taking photos of the site, usually using a digital camera with a suitable lens. The photos should be taken from multiple angles, with the same lighting and with as much overlap as possible.
Once the photos are taken, they are uploaded to specialized software such as Agisoft Photoscan or RealityCapture. This software stitches all the photos together to create a mesh, which is then exported to another 3D software package such as ZBrush or Blender. Here, the mesh is refined, unwrapped using RizomUV, and its number of polys reduced to prepare the model for texturing.
It happens very often that I am forced to fully rebuild the 3D object mesh. This happens if the raw model from a photogrammetry program does not meet quality expectations, has a lot of noise or is simply too complex for photogrammetry. This is what happened, for example, in the case of a camera model that I modeled from scratch in ZBrush. I would also like to add that ZModeler, once I became more familiar with it, is a very convenient and fast modeling tool. Now I mainly model just in ZBrush, and for more complex objects I use Blender, which as a versatile 3D program has no equal.
Fully remodeled object of speed graphic camera:
PBR texturing is then performed in dedicated texturing software, such as Substance 3D Painter or Quixel Mixer. The process involves applying materials to the model, adjusting albedo, roughness and normal maps until the desired look is achieved. Once texturing is complete, the model can be loaded into a real-time engine, such as Unity or Unreal Engine, for use in interactive applications such as games, virtual reality or augmented reality. The final step is to upload the model to the Sketchfab service so it can be viewed by anyone with an Internet connection.
By following this process, 3D models can be made that accurately capture the details and texture of the original object. The result is a digital representation that can be used in a variety of ways to help preserve and share important elements of our cultural heritage with the world. The power of PBR texturing comes from its ability to recreate the intricate details of an object's surface, allowing them to look realistic when viewed in a real-time engine. In addition to making models look more realistic, PBR textures also allow them to be used in immersive experiences, such as VR and AR.
Photo preparation
To create accurate 3D models of cultural heritage using photogrammetry, you must have the right equipment. The most important pieces of equipment you will need is a camera. A structured light scanner and a colorimeter may also be useful. The camera should be able to take high-resolution photos. A digital SLR camera will be ideal, as they produce detailed photos with minimal noise and high sharpness.
Start by setting the camera on a tripod and position it so that it is pointed directly at the heritage object. Once positioned correctly, use the scanner to create a scan of the object. Then use the color checker tool to accurately measure the colors on the object. Finally, use the camera to take multiple photos of the object from different angles. These photos can then be used to create an accurate 3D model of the heritage object using specialized software. First, the software identifies the features in each photo and combines them into a unified 3D point cloud. The software then creates a mesh based on these points, which gives the model its shape and structure. Finally, textures are applied to give the model color and detail. With this information, the software is able to generate an accurate 3D model that can be viewed from any angle or exported for further manipulation.
Color Checker is used to measure the colors of an object. This helps ensure that the colors in the 3D model match the colors of the original object. Color checkers come in different sizes and shapes and can be used for both indoor and outdoor photography. It is important to choose a color checker that is designed for the environment in which you will be shooting. For example, if you plan to photograph outdoors, then you should get a color checker designed for the outdoors. Similarly, if you plan to shoot indoors, then you should get a color checker made specifically for indoors.
Making a color profile:
It is necessary to take a photo of the object (.cr2 files) together with the color pattern under stable lighting conditions, so that later we can correctly interpret the color information. After opening the raw photo in camera raw (or Lightroom), we save it in .dng format. Then, using ColorChecker Camera Calibration, we save the color profile (.dcp). Now we assign this profile to all photos and set the white balance. We save the photos in a lossless .tiff format. Now we have ready material for further work in the photogrammetric program.
What software should I use to develop the object?
The most popular software is RealityCapture by Capturing Reality. In my experience, it is easy to use and provides very detailed results in the shortest time.
The combination of modeling software such as ZBrush and Blender, along with UV mapping tools such as RizomUV, makes it easier than ever to create complex 3D models of heritage objects. With these tools, even novice users can quickly create high-quality models that can be incorporated into projects of any scale. Texturing these models requires skill and patience, but if done correctly, it will really bring them to life. Knowing how to work with PBR textures is essential if you want to create convincing models of heritage objects. Programs such as RizomUV will help you get the best results when unwrapping a model's UV, while programs such as ZBrush and Blender will give you plenty of options when it comes to modeling and refining meshes.
Familiarity with texturing software such as Substance 3D Painter and Quixel Mixer is also important if you want to apply realistic materials to your models. However, once you have mastered all the techniques, creating convincing 3D models of heritage objects becomes much easier and faster! Tutorials and workshops are now available to teach people the skills they need to create their own 3D models. With the right knowledge, anyone can start creating impressive 3D representations of objects from the past. What's more, sharing the resulting creations online allows everyone to benefit from them, spreading appreciation and understanding of different cultures around the world!
I used 120 images to prepare the chasuble model. RealityCapture generated a very dense having many millions of poly 3D model along with vertex color. This provided the basis for further work on the mesh and color texture. I exported the object to ZBrush. This is where the mesh errors are smoothed and artifacts are removed.
In ZBrush, I prepare two versions of the mesh: high poly and low poly. The dense version will be used later in my work to generate the Normal Bump and Ambient Occlusion maps. The low poly version, consisting of 250,000 faces, I prepare using ZRemesher. It is a tool for automatic retopology of the mesh, which guarantees a uniform structure of the walls and quads version.
So I have the model almost ready for texturing. The UV channel still needs to be prepared. Here RizomUV, in which I generate it, will be helpful. I use a script (.lua) prepared by me with the settings saved. The program guarantees me full control over the process and informs me of any errors, which I remove manually.
With the low poly model thus prepared, I return back to RealityCapture, where the final texturing of the object takes place. Of course, not all parts of the model will be perfectly textured. There remain areas that have not been photographed sufficiently. There, errors will appear, which will have to be corrected and sometimes redrawn. I do all the work on the textures in Substance 3D Painter.
I start texture development by generating the necessary channels (Word Space Normal, AO, Position) from the high poly object. Substance 3DPainter gives you the ability to paint the texture directly on the 3D object. I fill in the missing parts of the Base Color texture by cloning or using standard brushes. Sometimes I use projected images of the object from different angles. The program gives the possibility to use advanced masks, which I often use to make visualizations of the model. At the same time, I prepare Metallic and Roughness maps. PBR (Physically Based Rendering) textures are now a very popular technique in the creation of 3D models, which allows to obtain very realistic effects. By using this technique, the digital material virtually simulates any type of physical material, such as wood, metal, leather, or stone.
PBR textures allow for highly detailed 3D models that look like their physical counterparts. This is possible because PBR textures take into account many factors, such as lighting, viewing angle, material to be simulated, etc. This makes the digital material look very realistic, allowing for impressive visual effects.
It is worth noting that PBR textures are particularly useful in creating 3D models of heritage objects, as they allow for very realistic effects that capture the beauty and character of these objects.
Why publish 3D models on Sketchfab?
Sketchfab is a great platform for visualizing 3D models. It provides users with an interactive 3D view of an object and allows them to explore it from all angles. Sketchfab also offers many tools to customize the model's appearance and interactivity, so people can more easily appreciate the object's details and appreciate its beauty.
When using Sketchfab to display a 3D model, it is important to configure the scene correctly to get the desired viewing experience. Sketchfab also offers various tools for making adjustments to the scene and the model itself. For example, users can change the background color and texture of the scene and add annotations, text, or links to external sources. Finally, users can save their work and share it with others on social media platforms.
Final visualization in Sketchfab:
The technology has been used to create virtual museums of Malopolska, which are hosted online and accessible to anyone with Internet access. Through these virtual museums, users can immerse themselves in the history and culture of Malopolska, learning about different aspects of the region's history and visiting historical sites. Thanks to the use of photogrammetry technology, virtual museums are more detailed than ever before, providing viewers with a realistic representation of places that would not be possible without it. In addition to creating virtual museums of Malopolska, the technology has also been used to create virtual tours of other historical sites and objects across Poland. Combining advanced technology with traditional cultural practices, the project allows viewers to learn about Poland's history and culture without leaving home.
Summary
Creating heritage models in 3D is an exciting way to preserve history and share it with others. Photogrammetry is a technique that uses photographs to create 3D models of objects and landscapes. To do this, you need the right equipment, software and a good understanding of the process. You will need to take multiple photos of an object from different angles to capture enough data for the photogrammetry software to generate a 3D model. Once the 3D model is created, you can share it with the world using the Sketchfab visualization platform. With a little practice and patience, you can create stunning 3D models of cultural heritage using photogrammetry. Once you've taken the photos, import them into the photogrammetry software of your choice. This software will use various algorithms to stitch the photos into a single 3D mesh. Depending on the complexity of the object being modeled, more than one round of processing may be necessary. In addition, the availability of certain features in photogrammetric software can affect the quality of the final result. For example, if texture mapping or texture baking is not available, some details may be lost when generating the 3D model. Fortunately, there are many software options to choose from, ranging from basic to advanced capabilities. Once you've found the perfect software for your project, there are no limitations when it comes to creating detailed 3D models. At a glance, my workflow is RealityCapture, ZBrush/Blender, RizomUV, Substance 3D Painter, Sketchfab. This is the optimal process path to success.
The force will be with you. Always.