digital human technology

At Computex 2024, NVIDIA demonstrated its latest digital human technology through the Covert Protocol tech and game demo, in collaboration with Inworld AI. This technology creates lifelike in-game characters with unique personalities, allowing players to interact using text or voice commands. The demo showcased a detective-style adventure powered by a GeForce RTX 40 Series GPU, emphasizing player engagement through puzzle-like interactions. NVIDIA plans to unveil an in-game version of this technology at Gamescom 2024 in the multiplayer mech game, Mecha BREAK, developed by Amazing Seasun Games. This game utilizes NVIDIA's Nemotron-4 4B NIM language model and features a Digital Human character named Martel, who provides insights and customization advice. Martel responds to voice commands, enhancing the gameplay experience. The technology integrates Audio2Face-3D NIM for facial animations, Whisper for speech recognition, and Elevenlabs for voice synthesis. Real-time performance requires robust hardware, but the impact on frame rate is minimal, around 5%, when using Digital Humans in Mecha BREAK. This performance is demonstrated on a GeForce RTX 4090-powered rig. Additional details about Mecha BREAK are available on the game's Steam page.