LogInDr. Ethan Hu, the Co-Founder and CEO of Meshy AI, delved into the industry's impending transformation under the influence of AI, and shed light on the benefits of this emerging technology that artists and developers can benefit from.
Introduction
Hello there, I'm Ethan, Co-Founder and CEO of Meshy AI, a coder and product builder. My adventure with computer graphics started when I was very young, sparking a lifelong passion for this field. This early interest led me to MIT, where I fully immersed myself in CG research, particularly fascinated by VFX and simulations. At MIT, I had the honor of contributing to cutting-edge projects, including the Moving Least Squares Material Point Method (MLS-MPM), which enabled faster simulations of materials like snow, sand, and mud. My doctoral thesis on a GPU programming language for physical simulation and AI, earned an honorable mention for the outstanding doctoral dissertation at SIGGRAPH 2022. Now, I moved to the West Coast in Silicon Valley, leading Meshy AI to new frontiers in 3D content creation.
Overview of Meshy
Meshy stands out in the realm of generative AI tools for 3D models, thanks to its unique features and exceptional speed. Our core offerings include Text to Texture, Text to 3D, and an experimental Image to 3D feature.
The Text to Texture feature of Meshy is particularly beneficial for 3D artists seeking to create high-quality textures quickly. By simply providing a model and a text prompt, users can generate highly detailed textures with Meshy. This feature is currently leading the market in AI texture generation, supporting 4K resolution and offering extensive style control.
Our Text to 3D functionality is designed to produce diverse, fast, high-quality 3D models. This feature is especially helpful for creating environment art (especially props), streamlining the creative process for artists and designers.
We place a strong emphasis on workflow integration, supporting various formats like glb, usdz, and fbx, and providing plugins for popular platforms like Unity and Blender. Our standout feature is the web UI system, which efficiently manages all generated content, allowing users to iterate easily and streamline their creative workflow.
Inspiration, Challenges or Opportunities
Time and again, my friends in the CG artist community have expressed a need for a tool that could not only accelerate the creation of high-quality 3D models but also enable them to swiftly bring their 3D visions to life, enhancing their creative freedom. AI, in my view, is the perfect catalyst for these aspirations.
Moreover, there's a whole group of individuals fascinated by 3D content creation, like myself, who lack traditional 3D modeling skills. Meshy is designed to lower this barrier, democratizing 3D content creation for a broader audience.
Meshy has just started and still has limitations in quality and control in AI-generated 3D models. While we've achieved a decent level of proficiency in Text to Texture, the realm of AI-driven 3D modeling (mesh generation) still lags behind human standards. However, I remain optimistic. Considering the rapid advancements in this field, I believe that within the next two to three years, we could see a leap in quality akin to what Midjourney has achieved in 2D. We're moving fast, and the future looks promising for Meshy and 3D content creation using AI.
Exploring the Impact of GenAI on the Game and XR Industry
GenAI's influence on the game and XR industry can be best understood by considering its impact on both professional creators and hobbyists engaged in User Generated Content (UGC).
For professionals, Meshy is already making significant strides. Our Text to Texture feature enables artists to rapidly generate textures, revolutionizing how quickly a Blender model can be textured – we're talking about a process that takes just a minute. Additionally, Meshy's Text to 3D feature is used for creating non-hero environment props, especially those that don't require close-up shots. As Meshy's quality continues to evolve, I anticipate even more groundbreaking applications in professional CG production.
On the UGC front, the impact is more focused on creativity and fun. We've forged partnerships with companies like Snap and various game developers to empower users to use text-based inputs for creating in 3D. This collaboration not only democratizes 3D content creation but also opens up a world of creativity and interactivity for users who might not have professional skills in 3D modeling.
The Future of GenAI
While Meshy is in its early stages, I am confident that 3D GenAI will rapidly evolve, enhancing control, mesh and texture quality, speed, and style consistency.
The whole academia and industry are just scratching the surface of 3D GenAI, having solved only about 10% of the challenges in 3D generation. However, given the incredible pace at which generative AI is advancing in other modalities like text, images, videos, and speech, I'm optimistic that the remaining 90% will be addressed swiftly.
Long-standing challenges like effective UV unwrapping and topology in AI-generated models remain unsolved, and these are areas we recognize as crucial. While improving UV and topology in 3D generative AI systems won't be easy, it's a goal we're committed to at Meshy.
As mesh and texture quality continue to improve rapidly, the prospect of AI assisting 3D artists in quickly realizing their visions and consumers freely creating stunning 3D models becomes more tangible.
Moreover, the widespread adoption of XR devices like Vision Pro and Quest 3 will create a substantial demand for 3D assets. In such a scenario, generative AI for 3D, along with 3D scanning, will likely be key methods for meeting this demand.
Ultimately, 3D Generative AI is set to unlock a new realm of creativity in the 3D world, transforming how we create, interact with, and experience 3D content.
Success Stories & Notable Projects
We have seen some incredible results from our users, showcasing the transformative power of Meshy. One particularly inspiring story comes from a user known as rolfy on Discord. He has been leveraging our Text to Texture feature to automatically generate highly detailed textures for armor sets in his game, subsequently integrating these creations into Unity. Rolfy reports that Meshy's Text to Texture has been a game-changer, saving him countless hours in the development process.
AI-textured armor sets from rolfy, a 3D artist creating an indie game called Madrigal.
Another area where Meshy shines is in the creation of environmental art. The work done by RenderMan using our Text to 3D feature stands out as a prime example. His project, featuring an ancient temple, showcases the level of detail and realism that can be achieved with Meshy, transforming the workflow and output quality for 3D artists.
These examples underscore how Meshy isn't just a tool; it's a catalyst for creativity and efficiency, enabling developers and creators to push the boundaries of their craft and achieve their vision with greater speed and less effort.
Next Steps and Upcoming Projects
At Meshy, we are actively forging collaborations with various applications that utilize 3D models. This includes professional 3D modeling and rendering apps, as well as more casual platforms like games and user-generated content apps, such as Snapchat's Lens Studio. Our team is focused on developing and upgrading plugins for Blender, Unity, and Unreal Engine (UE), aiming to seamlessly integrate Meshy into these tools and streamline the 3D content creation process.
I envision the future of 3D content creation as a harmonious blend of traditional modeling techniques and AI-assisted asset generation. Achieving this vision requires effective integration of Meshy into users' existing workflows, particularly in widely used platforms such as Blender and UE. This integration is key to unlocking the full potential of AI in enhancing and simplifying the 3D design process.
Conclusion
For those interested in exploring Meshy, you can visit our website or join our Discord community. Our web app offers 40 free generations monthly, with additional capabilities like task queuing and style control. Meshy is also available on Discord, providing a platform for unlimited free generations. We invite everyone to try Meshy, experience its capabilities firsthand, and let us know their valuable feedback.