MULTIMODAL INTELLIGENCE · DIGITAL HUMANS
Building multimodal intelligence that understands people and grounds digital humans in the real world.
Human Behavior Modeling Motion Capture Character Animation Physics Simulation Human Pose Estimation Human Reconstruction Digital Humans Computer Vision Graphics Machine Learning Conversational AI Multimodal Models Embodied Agents Self-Supervised Reasoning
Embodied agents
Perception, memory, and language connect to digital humans that can move, speak, and respond.
- Transformers
- LLMs
- JEPA
- World Models
- Multimodal Learning
Controllable speech
Text, a reference voice, and delivery controls condition an autoregressive diffusion model to generate personalized speech.
- Audio Diffusion
- Speech Synthesis
- LoRA
- Post-Training
- Voice Personalization
Human reconstruction
Multiview observations initialize an MHR body model, then flow through inverse rendering and SDF optimization into a persistent 3D human.
- Body Model Definition
- Inverse Rendering
- SDFs
- Splat Rendering
- Computer Vision
- 3D Reconstruction
- MHR paper
Sparse tracking
A headset and two controllers become coherent full-body motion through an MHR prior and optimized Gauss–Newton solver.
- Human Pose Estimation
- Motion Capture
- Body Models
- Gauss–Newton Optimization
Animation + physics
Articulated collision bodies feed procedural animation, blending, and draggable IK targets.
- Physics Simulation
- Ragdoll Dynamics
- Character Animation
- Inverse Kinematics