Ivan Savić discussed his Assetto Corsa experiment, explained how he captured initial data and used it in RealityCapture, and shared what challenges he faced while recreating the entire city.
Introduction
My name is Ivan Savić, I live in a small town in Serbia. I mostly do 3d modeling as a freelancer and try to make some of my passion projects along the way.
The Assetto Corsa Experiment
I always loved video games and sim racing. As a kid I wondered what would be the layout for an F1 track in my city, I was imagining some routes. There is also this hill climb race near the town that used to be held, I always wanted to make that road in a game. When Assetto Corsa came out, it was perfect for my passion. I tried to make this Hill Climb road by modeling it manually, but I was never satisfied with how it looked and never finished it.
Screenshot of Ivan Savić's first mod of a Hill Climb track
The next track modding I did was the NAVAK track near Belgrade, there is no mod for it so I wanted to make it. That one turned out quite well.
Screenshot of the NAVAK track
After that, I tried to manually model my city and make a race track in it, but that turned out to be a lot of work, and I stopped working on it soon enough. Also, the way I placed buildings as boxes with textures copied from street view didn't look nearly good enough. Modeling everything in more detail would take an enormous amount of time. Actually, I was thinking of making only buildings with photogrammetry, but I eventually figured out it would be even easier to reconstruct everything.
Screenshot of the manually modeld city track
Screenshot of the manually modeld city track
When I discovered photogrammetry, I was amazed by the technology. I scanned some small objects and really enjoyed doing this. At some point, I realized that I could try to scan larger areas, I could actually scan roads for track modding. After some experiments on how to take the images to have good reconstruction, I had some good results, which I shared in my first video of my neighborhood.
The neighborhood track made using photogrammetry
Of course, I was always planning for a bigger take, a track through a whole city. That was shown in the second video which got bigger attention. I actually took images for a couple of more streets to create a larger layout throughout the city, but I still didn't manage to make it finished.
Screenshot of an uncleaned mesh of one section Ivan scanned to extend the track
Screenshot of an uncleaned mesh of one section Ivan scanned to extend the track
As for the tutorials, I don't remember if I planned to make them, but people were asking so I made a few to explain the process.
Capturing the Initial Data
For the first experiments, I only used an action camera mounted to a bike. When I had a successful reconstruction in my neighborhood, it looked quite well, but there were a lot of holes in the mesh of the areas that were not visible to the camera, like behind the fences, walls, etc.
The textures you can see behind these fences in the images below are google mesh, not photogrammetry mesh.
I realized that I should have added drone images to cover those areas. So with both images from the ground and air, I covered most of the environment. After that, I tried using only a drone to scan open roads. It is much faster to use drone scans only but this technique can be used only if there are not too many trees above the road – otherwise, this can obstruct the drone recording environments.
Hill Climb photogrammetry track
Hill Climb photogrammetry track
Using RealityCapture
When I discovered photogrammetry, I tried a number of programs as I wanted to try how each of them worked. For my track experiments, I also tried to reconstruct tracks in a couple of programs, but I found that RealityCapture worked the best for me. All of the photogrammetry programs work similarly and are not too complex to use, so there is not too difficult to learn them.
Using the Captured Data in RealityCapture
I used low-budget equipment and the images were taken in a kind of a quick way, so my input data was not that good. When I input all of the images into the software, it could not reconstruct the whole track seamlessly. I had to find some workaround to solve that. I separated the track into smaller sections and reconstructed each of them separately.
That made it much easier and faster to work. After that, I input all of the sections in a new project and reconstructed the whole track like that. But that didn't work seamlessly as well – there were a lot of errors in image alignment so I had to manually make a lot of control points to guide the software on what parts should be where.
Photogrammetry is a technique that requires high-quality images, the better the image quality and the better pattern of taking them the easier it is for the program to create a 3D model of it. However, I was surely not in a position to close the traffic in the city for a day or so and couldn't make scans slowly and thoroughly. Besides, I didn't want to spend too much time doing it, so decided on taking a video, which anyway took about 10 hours, because I had to do multiple passes. So taking individual images would probably take forever.
Cleaning the Data and Preparing the Final Track
With this quick method of recording a video, a lot of errors and inaccuracies appear in the reconstructed mesh. In my test, the road was never smooth enough and had to be adjusted or even remodeled entirely. Thin objects like poles and fences were not reconstructed well enough or could not be reconstructed at all.
Example of noise in the road
Example of fences that were not reconstructed well enough
Photogrammetry can surely create a fence from images, but not from the distance I have recorded and from so many images I had when passing by that fence. Taking the images of those objects separately would take too much time, increase the difficulty of reconstruction and the overall size of the mesh, and I didn't even consider it. It was much easier to make them manually.
Some objects had to be modeled manually
The other parts of the mesh also weren't created perfectly. For example, the walls were not flat, there were bumps in them, and so on. But for me, it didn't have to be perfect. When you drive, especially if you go fast, you don't pay attention to all the small details. I only had to capture basic large shapes correctly, make the colors and texture feel realistic as well as make the proportion and sizes be somewhat accurate. And for me and my friends driving the track, this is good enough to associate it with the real world. And it is awesome to drive an F1 downtown.
Turning the Environment Into a Playable Track
Assetto Corsa is made with modding in mind, so it is very easy to add content to it. Although not that easy to create it. In a 3D program, you have to define the names of objects so the Assetto Corsa engine knows which polygons are roads, grass, walls, etc. Also, you need to add some objects to define where the cars' starting positions are, where the starting line is, etc. Assetto Corsa comes with an editor for modding, which can import these 3D files and if everything is correct make a track or a car to be run in the game.
Conclusion
The whole project took me 240 hours over several months. For me, every step was more or less time-consuming, I can't point to a particular thing. Anyway, making tracks with photogrammetry turned out to be much, much faster than creating them manually. You could never replicate this amount of geometry and textures by hand. Yes, my results aren't perfect, but still.
If someone wants to try something like this, make some experiments first on a short part of the road, like 100 meters long. Get familiar with the process and see what results you can get, then plan something bigger. The most important part of photogrammetry is to have good images and enough of them so every part of the scanned environment is visible.