OpenAI’s chip chief shares insights with More Than Moore
Richard Ho, OpenAI’s chip chief, recently sat down with Ian Cutress from More Than Moore to discuss the development of OpenAI’s Jalapeño chip. Ho expressed a degree of skepticism regarding the full trustworthiness of AI design, despite its advancements.
Behind the Scenes of Jalapeño Chip Development
OpenAI revealed its custom AI chip in June, built in collaboration with Broadcom for running their models, not training them. Ho, who joined the team in 2023, shared details about the chip's creation, highlighting the challenges and innovative solutions employed.
A Closer Look at the Design Process
The team encountered logic fitting issues early on, leading them to involve their AI models in the design process. Ho explained that these models, though raw at the time, proved more adept at handling software than traditional Verilog, prompting the use of XLS, a tool that translates high-level code into chip logic, initially released as open source.
Unveiling the Enigma
Some model adjustments proved enigmatic, as the team raced against time. Ho acknowledged their cautious approach, ensuring every change went through rigorous validation, even when the models' logic wasn't always clear.
Unifying Inference Hardware
Ho defended OpenAI’s decision to design a single chip for all inference stages, a choice that contrasts with industry norms. He attributed this to the risks associated with locking down data center capacity with fixed ratios, especially as work demands fluctuate. He also predicted that this design would be adopted by rivals, including GPU makers.
Powering Ahead
According to Ho, the industry’s limiting factor isn't power, memory, or packaging; it's the inability of executives and investors to envision the immense potential of AI, leading to conservative bets. OpenAI (find out more) is now preparing for volume production with the second version of the Jalapeño chip.