OpenAI Publishes 722 Math Papers from Unreleased Model
OpenAI has published 722 math manuscripts written by an internal model it has not released. The papers, categorized into 372 families of results, were made publicly available on GitHub on Tuesday. This same model was behind OpenAI's Navier-Stokes proof last month.
During the evaluation, the model was given approximately 4,000 problems, with each result requiring an average of three hours of ChatGPT Pro thinking compute.
What's in the Repository?
Many proofs include formal versions in Lean, a programming language designed for computer-checked proofs. Some results lack formal versions and OpenAI acknowledges these might have issues, promising to address them promptly. All previous versions will remain public, with each paper containing its own citation block.
The repository also includes ten summaries of the model's reasoning. Notably, two results deviated from standard procedure: one concerning a zero-free region for the Riemann zeta function and another proving the Hodge conjecture for CM abelian varieties.
According to OpenAI, nearly every paper originated from a single prompt to a single AI agent. The company plans to fund workshops, conferences, and special programs centered around these AI-generated results and is committed to responsible model release.
Advisors: A New Beginning
Mathematicians advising OpenAI view this as the beginning, not the end, of the process. In its guidelines dated 29 September, the Advisory Group on Mathematics and Artificial Intelligence (AGMAI) had urged AI labs to stop using private models for testing hard math problems and instead publish prompts, time spent, and compute cost behind each result.
While AGMAI did not endorse the OpenAI results or their methodology, they noted that "this release is the beginning, not the completion, of the process of human understanding."
Dan Roberts, OpenAI's research lead, justified testing internal models for improved tool development, emphasizing that proofs were a byproduct.
However, Tristan Buckmaster, a mathematician from New York University who had been working on the Navier-Stokes problem before OpenAI, expressed doubt about the thoroughness of their checks, given the large number of results released simultaneously.
MIT mathematician Andrew Sutherland, likewise, advised treating single-agent claims as unverified until others can replicate the results by running the model themselves.