Nvidia’s Open Model Initiative
Nvidia is making waves with its ambitious open model project, aiming to create a trillion-parameter model—a significant step forward. However, even with this massive effort, it pales in comparison to China’s leading open models.
NVIDIA’s Latest Release: Nemotron 3.5 Lightning
Nvidia recently introduced Nemotron 3.5 Lightning, a 30 billion parameter mixture-of-experts model with 3 billion active parameters at any given time. This hybrid architecture combines elements from Mamba-2, MoE, and attention layers, offering a one million token context window. The weights are accessible on Hugging Face and ModelScope, and the training data and recipes are publicly available under the OpenMDW-1.1 license, allowing commercial use without restrictions.
Speed and Performance Claims
Nvidia boasts remarkable speed, claiming up to four times faster output compared to similar models. However, a closer look reveals a more modest figure: on PinchBench, it achieved 86% accuracy while completing 10,000 tasks 30% faster than Alibaba’s Qwen 3.6 35B at a similar accuracy level.
Benchmark scores are impressive but not extraordinary. Nemotron 3.5 posts respectable numbers on MMLU Pro (81.94), GPQA Diamond (75.44), and SWE-bench Verified (51.56). Pre-training involved processing over 20 trillion tokens.
NeMo Switchyard: Routing for Efficiency
Alongside Nemotron 3.5, Nvidia released NeMo Switchyard, an open-source library that routes workflow steps to the most suitable models. This approach optimizes performance by directing tasks to either Lightning or Claude Opus 4.8, resulting in significant cost savings (74%) compared to using only the frontier model.
Strategizing for Inference and Hardware Sales
Nvidia’s strategy is clear: promoting freely available models will drive inference demand, ultimately benefiting GPU sales. By commoditizing the software, they aim to capture a larger market share for their hardware. This approach aligns with their previous efforts to tie safety work to chip demand.
The Trillion-Parameter Model and China’s Lead
The Information reports that Nvidia is developing Nemotron 4, a model exceeding one trillion parameters, building upon the success of Nemotron 3 Ultra‘s 550 billion parameters. Aiming to compete with top open models globally, Nemotron 4 still lags behind China’s leading open models, such as Moonshot’s Kimi K3, currently holding the record for the largest open model.
Nvidia acknowledges China’s pace and aims to catch up rather than lead, demonstrating a strategic shift in the open-source landscape.