ByteDance Developing Real-Time Spatial Video Model
ByteDance is preparing a real-time spatial video model under the supervision of its founder, Zhang Yiming, according to Bloomberg [^1]. Built on their existing system, Seedance, this model aims to render interactive 3D worlds in the cloud with minimal latency.
Key Features:
- Low Latency: The model is designed for approximately 50 milliseconds of latency, enabling smooth and responsive user interactions.
- Cloud Rendering: Content generation will occur in the cloud, offloading computational tasks from devices, which reduces hardware costs.
- Voice and Motion Interaction: The model is reportedly integrated with Pico headsets, allowing users to create and interact with virtual worlds through voice and movements.
Background:
ByteDance’s interest in AI models for spatial video generation isn’t new. This year, 36Kr reported [^2] that the company had identified four AI priorities for 2026, with world models as their top priority, followed by video enhancements, coding improvements, and commercializing Doubao. They planned to invest heavily in data for world model development, targeting a significant lead over competitors.
Strategy:
By focusing on models, cloud capacity, and content distribution, ByteDance aims to revolutionize the XR (Extended Reality) space. Their existing infrastructure, including Seedance and Pico headsets, provides a strong foundation for this venture.
Technical Approach:
The spatial video model is part of two parallel strategies:
- Embodied Intelligence and Robotics: Using a vision-language-action approach to develop intelligent systems that understand and interact with their environment.
- Entertainment and Gaming: Leveraging 3D simulations for interactive gaming experiences, where the existing user base plays a significant role.
A world model, in this context, refers to a system capable of rendering environments that respond coherently to user actions. ByteDance’s unique advantage lies in their extensive experience in video compression and processing, built over a decade of developing systems for a different purpose.
[^1]: Bloomberg Report
[^2]: 36Kr Report