Nvidia has made its AI-powered Audio2Face technology open source. This step will enable developers, students, and researchers to more easily use this technology and create new projects.
Audio2Face technology has been made open source
Audio2Face is an artificial intelligence model that analyzes audio data to create realistic facial expressions and lip syncs. This tool generates animations using tone and acoustic data. This allows avatars capable of natural and emotional responses in a variety of settings, from game characters to virtual customer representatives.
Shiftdelete.netNvidia has made not only the model but also the Audio2Face SDK and training framework available to the open source community. This allows developers to customize existing models to their own needs.
Audio2Face is already used by many game studios and software developers. Game developers such as Codemasters, GSC Game World, NetEase, and Perfect World Games, as well as software providers such as Convai, Inworld AI, Reallusion, Streamlabs, and UneeQ, are using this technology in their projects.
For example, The Farm 51 claims to have saved significant time on animations in Chernobylite 2: Exclusion Zone thanks to this technology.
Along with this development, Nvidia also announced significant updates to its RTX Kit. This package, which offers AI-powered ray tracing and realistic character generation, has been enhanced with new optimizations. The RTX Neural Texture Compression SDK reduces memory usage for large textures without sacrificing visual quality.
