LogInProject TH developer EVR Studio told us about Project EDEN, a new AI-driven digital human platform, discussing the tech behind it and explaining how they managed to assemble the first facial-only volumetric capture system in Asia.
Project EDEN & The Idea Behind It
EDEN presents a fresh and innovative interface tailored for the AI era, departing from conventional UX and UI frameworks to prioritize engaging dialogues with digital humans.
Its origins can be traced back to a VR project named Project M, which was initiated during the early years of EVR Studio's establishment back in 2016. Project M was a VR content that allowed interactive engagement with users, effectively remembering their actions and choices.
Over time, the roles of the digital human have evolved into what we now see in EDEN, focusing on nurturing relationships through emotional interaction and retaining user information.
The Target Audience
Project EDEN's ultimate goal is to cater to all interactive AI users. At present, conversational AI has not yet achieved widespread usability and is predominantly utilized by specific user segments. To bring large language model-based conversational AI into mainstream adoption, there are various aspects that require further development, with AI interfaces standing out as one of the most pivotal elements.
While ChatGPT's core interface primarily relies on text-based input and output, there is a small subset of users who engage with ChatGPT through voice interfaces. Looking forward, the evolution of conversational AI is expected to encompass not only auditory input and output but also the capability to engage in visual interactions with a tangible representation. In this context, EDEN's digital human serves as an interface that seamlessly integrates both auditory and visual elements, providing users with a familiar and holistic conversational experience.
EDEN's AI digital human interface will possess the ability to remember users' preferences and tastes, offering a wealth of information and engaging in emotional interactions, akin to conversing with a genuine friend.
Assembling the Facial-Only Volumetric Capture System
We have devoted a substantial amount of time to tackling several challenges, with a primary focus on minimizing noise in the final output. One notable challenge arises when the shutter synchronization isn't aligned correctly, resulting in substantial noise issues. To tackle this challenge, we have implemented a synchronized operation of all cameras, triggered simultaneously by the signal from the sync generator, ensuring precise synchronization.
Additionally, we've encountered issues related to skin reflections, which can introduce significant noise into the 3D geometry of the captured results. To completely eradicate these undesirable skin reflections, we took a comprehensive approach by designing and manufacturing the PCB (Printed Circuit Board) of the light module from the ground up.
To build an AI generation model capable of creating digital human facial expressions and voices, a sound studio was constructed by integrating both systems to ensure seamless synchronization with volumetric capture. Through these steps, we have set up Asia's first dedicated facial volumetric capture system and a sound studio for AI training.
The Tech Behind EDEN
In order to achieve real-time lip synchronization and lifelike facial expressions for our 3D real-time Digital Human production, the development of a generative model was imperative. To achieve this goal, we are actively engaged in the creation of a generative model through the training of high-quality volumetric capture data. Furthermore, we have established collaborations with the world's foremost AI companies within Project EDEN, with the overarching objective of crafting a multilingual model that can comprehend and communicate in excess of 40 languages, effectively eliminating language barriers.
Conversational AI
Large language models like ChatGPT and Bard are poised to expand their capabilities by leveraging specialized databases tailored to specific regions. This will empower users to select conversational AIs that align with their location and unique preferences. This evolution will result in a more natural integration of AI into our daily lives, enabling continuous interaction without being constrained by geographical boundaries or current circumstances.
In the near future, it is anticipated that 3 out of every 10 individuals walking down the street will engage in conversations with AI. Project EDEN is the platform designed to transform the AI-enhanced lifestyle we've all envisioned into a tangible reality. It will be accessible through displays seamlessly integrated into our everyday environments, including our homes, workplaces, and vehicles.
As AI becomes increasingly integrated into our daily routines, the importance of emotional interaction between humans and AI is growing. To cultivate a sense of closeness and connection with users, AI can either adopt human-like appearances or resemble entities like pets, which have historically formed strong bonds with people.
In this context, EDEN's ultimate objective is to facilitate the development of relationships with users through emotional interactions. It will accomplish this by providing a digital human interface infused with the appearance, voice, and personality of users' favorite artists as an AI companion.
Conclusion
We are currently in the active development phase of crafting digital representations for renowned artists within the K-pop and K-drama realms. Although there remain specific business and ethical aspects that demand attention, it's crucial to highlight that all stakeholders in the EDEN project are aligned with a shared vision for the upcoming AI era. Consequently, we foresee the introduction of these participating artists in the near future.
Project EDEN is presently in the technical validation stage of its development. As mentioned earlier, we have been engaged in extensive collaborations with distinguished AI firms on a global scale to create generative models custom-tailored for EDEN. We are thrilled to announce that, within the upcoming year, we will publicly reveal EDEN's integration with interactive AI systems like ChatGPT and Bard.